Towards Efficient Scheduling of Transformer Neural Network Computation for Edge AI Deployment

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F26%3A0197537" target="_blank" >RIV/00216305:26230/26:0197537 - isvavai.cz</a>

Result on the web

<a href="http://dx.doi.org/10.1145/3712255.3734345" target="_blank" >http://dx.doi.org/10.1145/3712255.3734345</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1145/3712255.3734345" target="_blank" >10.1145/3712255.3734345</a>

Result language

angličtina

Original language name

Towards Efficient Scheduling of Transformer Neural Network Computation for Edge AI Deployment

Original language description

Transformer neural networks have gained popularity in recent years, demonstrating remarkable performance across many application domains. However, inference on resource-constrained embedded hardware remains challenging due to Transformers' substantial computational demands. We aim to address this problem by focusing on exploiting the inherent parallelism opportunities presented by the multi-head self attention operations of Transformers, to achieve a speedup in processing on embedded hardware. In this paper, we present an evolutionary-based scheduling approach for distribution and allocation of Transformer operations across systolic array-based hardware accelerators used for execution. Our methodology takes as input specifications of the Transformer workload and the target systolic array architecture and explores the large mapping space to identify an efficient plan of operation-to-array assignments. The plans are evaluated against a hardware-aware cost model, capturing the cost of computational cycles for a given operation and systolic array, with the objective to minimize the total sum across all operations. Through extensive experimental evaluations across diverse systolic array dimensions, we demonstrate that our evolutionary-based scheduler surpasses conventional heuristics and is able to find plans offering up to 33.8% average reduction in overall cycle count.

Czech name

—

Czech description

—

Type

O - Miscellaneous

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/GA25-15490S" target="_blank" >GA25-15490S: LEDNeCo: Low Energy Deep Neurocomputing</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)<br>S - Specificky vyzkum na vysokych skolach

Publication year

2025

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů