Neural Architecture Search and Hardware Accelerator Co-Search: A Survey

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F21%3APU142976" target="_blank" >RIV/00216305:26230/21:PU142976 - isvavai.cz</a>

Result on the web

<a href="https://ieeexplore.ieee.org/document/9606893" target="_blank" >https://ieeexplore.ieee.org/document/9606893</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/ACCESS.2021.3126685" target="_blank" >10.1109/ACCESS.2021.3126685</a>

Result language

angličtina

Original language name

Neural Architecture Search and Hardware Accelerator Co-Search: A Survey

Original language description

Deep neural networks (DNN) are now dominating in the most challenging applications of machine learning. As DNNs can have complex architectures with millions of trainable parameters (the so-called weights), their design and training are difficult even for highly qualified experts.  In order to reduce human effort, neural architecture search (NAS) methods have been developed to automate the entire design process. The NAS methods typically combine searching in the space of candidate architectures and optimizing (learning) the weights using a gradient method. In this paper, we survey the key elements of NAS methods that -- to various extents -- consider hardware implementation of the resulting DNNs. We classified these methods into three major classes: single-objective NAS (no hardware is considered), hardware-aware NAS (DNN is optimized for a particular hardware platform), and NAS with hardware co-optimization (hardware is directly co-optimized with DNN as a part of NAS). Compared to previous surveys, we emphasize the multi-objective design approach that must be adopted in NAS and focus on co-design algorithms developed for concurrent optimization of DNN architectures and hardware platforms. As most research in this area deals with NAS for image classification using convolutional neural networks, we follow this trajectory in our paper. After reading the paper, the reader should understand why and how NAS and hardware co-optimization are currently used to build cutting-edge implementations of DNNs.

Czech name

—

Czech description

—

Type

J_imp - Article in a specialist periodical, which is included in the Web of Science database

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/GA21-13001S" target="_blank" >GA21-13001S: Automated design of hardware accelerators for resource-aware machine learning</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2021

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Name of the periodical

IEEE Access

ISSN

2169-3536

e-ISSN

—

Volume of the periodical

9

Issue of the periodical within the volume

9

Country of publishing house

US - UNITED STATES

Number of pages

26

Pages from-to

151337-151362

UT code for WoS article

000719556200001

EID of the result in the Scopus database

2-s2.0-85119718262