TransInferSim: Toward Fast and Accurate Evaluation of Embedded Hardware Accelerators for Transformer Networks

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F26%3A0193349" target="_blank" >RIV/00216305:26230/26:0193349 - isvavai.cz</a>

Výsledek na webu

<a href="https://ieeexplore.ieee.org/document/11202474" target="_blank" >https://ieeexplore.ieee.org/document/11202474</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/ACCESS.2025.3621062" target="_blank" >10.1109/ACCESS.2025.3621062</a>

Jazyk výsledku

angličtina

Název v původním jazyce

TransInferSim: Toward Fast and Accurate Evaluation of Embedded Hardware Accelerators for Transformer Networks

Popis výsledku v původním jazyce

Transformers are neural network models that have gained popularity in various advanced AI systems including embedded/Edge-AI. Due to their architecture, hardware accelerators can leverage massive parallelism, especially when processing attention head operations. While accelerators for Transformers are being discussed in the literature, efficient scheduling of cache operations and detailed modeling of inference dynamics has not yet been addressed comprehensively. In this paper, we introduce TransInferSim, a novel tool that combines cycle-accurate simulation for performance estimation (including latency, memory usage, memory access counts, and computation counts) with a discrete-event-based scheduler that determines the execution order of compute and memory operations. By combining this tool with the Accelergy tool, the simulator enables accurate estimation of energy consumption and on-chip area, leveraging pre-characterized hardware parameters. The proposed tool allows for the accurate determination of cache misses at different levels and with different victim selection configurations. It supports different memory hierarchies and offers several strategies for scheduling operations on compute units. In addition, TransInferSim can extract the full execution plan generated during simulation, enabling its further use for behavioral Register Transfer Level validation or for deployment in real hardware implementations. This makes the tool applicable not only for high-level design space exploration, but also as a software front-end for hardware execution mapping. Finally, we can optimize the architecture for a particular network, as demonstrated through multiobjective design space exploration to adjust the size of processing arrays. In our experiments, the introduction of an on-chip memory hierarchy improved the inference speed by ∼3.5× and reduced energy by ∼1.9× for the RoBERTaBase Transformer model, while design space exploration achieved up to 10× latency reduction and 6× area savings for the ViTTiny vision Transformer. The tool is available online at https://github.com/ehw-fit/TransInferSim.

Název v anglickém jazyce

TransInferSim: Toward Fast and Accurate Evaluation of Embedded Hardware Accelerators for Transformer Networks

Popis výsledku anglicky

Transformers are neural network models that have gained popularity in various advanced AI systems including embedded/Edge-AI. Due to their architecture, hardware accelerators can leverage massive parallelism, especially when processing attention head operations. While accelerators for Transformers are being discussed in the literature, efficient scheduling of cache operations and detailed modeling of inference dynamics has not yet been addressed comprehensively. In this paper, we introduce TransInferSim, a novel tool that combines cycle-accurate simulation for performance estimation (including latency, memory usage, memory access counts, and computation counts) with a discrete-event-based scheduler that determines the execution order of compute and memory operations. By combining this tool with the Accelergy tool, the simulator enables accurate estimation of energy consumption and on-chip area, leveraging pre-characterized hardware parameters. The proposed tool allows for the accurate determination of cache misses at different levels and with different victim selection configurations. It supports different memory hierarchies and offers several strategies for scheduling operations on compute units. In addition, TransInferSim can extract the full execution plan generated during simulation, enabling its further use for behavioral Register Transfer Level validation or for deployment in real hardware implementations. This makes the tool applicable not only for high-level design space exploration, but also as a software front-end for hardware execution mapping. Finally, we can optimize the architecture for a particular network, as demonstrated through multiobjective design space exploration to adjust the size of processing arrays. In our experiments, the introduction of an on-chip memory hierarchy improved the inference speed by ∼3.5× and reduced energy by ∼1.9× for the RoBERTaBase Transformer model, while design space exploration achieved up to 10× latency reduction and 6× area savings for the ViTTiny vision Transformer. The tool is available online at https://github.com/ehw-fit/TransInferSim.

Druh

J_imp - Článek v periodiku v databázi Web of Science

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/GA25-15490S" target="_blank" >GA25-15490S: LEDNeCo: Nízkoenergetické hluboké neurovýpočty</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

IEEE Access

ISSN

2169-3536

e-ISSN

—

Svazek periodika

13

Číslo periodika v rámci svazku

October

Stát vydavatele periodika

US - Spojené státy americké

Počet stran výsledku

12

Strana od-do

177215-177226

Kód UT WoS článku

001596848900005

EID výsledku v databázi Scopus

2-s2.0-105019806172