ApproxDARTS: Differentiable Neural Architecture Search with Approximate Multipliers

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F24%3APU151218" target="_blank" >RIV/00216305:26230/24:PU151218 - isvavai.cz</a>

Výsledek na webu

<a href="https://ieeexplore.ieee.org/document/10650823" target="_blank" >https://ieeexplore.ieee.org/document/10650823</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/IJCNN60899.2024.10650823" target="_blank" >10.1109/IJCNN60899.2024.10650823</a>

Jazyk výsledku

angličtina

Název v původním jazyce

ApproxDARTS: Differentiable Neural Architecture Search with Approximate Multipliers

Popis výsledku v původním jazyce

Integrating the principles of approximate computing into the design of hardware-aware deep neural networks (DNN) has led to DNNs implementations showing good output quality and highly optimized hardware parameters such as low latency or inference energy. In this work, we present ApproxDARTS, a neural architecture search (NAS) method enabling the popular differentiable neural architecture search method called DARTS to exploit approximate multipliers and thus reduce the power consumption of generated neural networks. We showed on the CIFAR-10 data set that the ApproxDARTS is able to perform a complete architecture search within less than 10 GPU hours and produce competitive convolutional neural networks (CNN) containing approximate multipliers in convolutional layers. For example, ApproxDARTS created a CNN showing an energy consumption reduction of (a) 53.84% in the arithmetic operations of the inference phase compared to the CNN utilizing the native 32-bit floating-point multipliers and (b) 5.97% compared to the CNN utilizing the exact 8-bit fixed-point multipliers, in both cases with a negligible accuracy drop. Moreover, the ApproxDARTS is 2.3 times faster than a similar but evolutionary algorithm-based method called EvoApproxNAS.

Název v anglickém jazyce

ApproxDARTS: Differentiable Neural Architecture Search with Approximate Multipliers

Popis výsledku anglicky

Integrating the principles of approximate computing into the design of hardware-aware deep neural networks (DNN) has led to DNNs implementations showing good output quality and highly optimized hardware parameters such as low latency or inference energy. In this work, we present ApproxDARTS, a neural architecture search (NAS) method enabling the popular differentiable neural architecture search method called DARTS to exploit approximate multipliers and thus reduce the power consumption of generated neural networks. We showed on the CIFAR-10 data set that the ApproxDARTS is able to perform a complete architecture search within less than 10 GPU hours and produce competitive convolutional neural networks (CNN) containing approximate multipliers in convolutional layers. For example, ApproxDARTS created a CNN showing an energy consumption reduction of (a) 53.84% in the arithmetic operations of the inference phase compared to the CNN utilizing the native 32-bit floating-point multipliers and (b) 5.97% compared to the CNN utilizing the exact 8-bit fixed-point multipliers, in both cases with a negligible accuracy drop. Moreover, the ApproxDARTS is 2.3 times faster than a similar but evolutionary algorithm-based method called EvoApproxNAS.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/GA24-10990S" target="_blank" >GA24-10990S: Strojové učení zohledňující hardware: Od automatizovaného návrhu k inovativním a vysvětlitelným řešením</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2024

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

2024 The International Joint Conference on Neural Networks (IJCNN)

ISBN

979-8-3503-5931-2

ISSN

—

e-ISSN

—

Počet stran výsledku

8

Strana od-do

1-8

Název nakladatele

Institute of Electrical and Electronics Engineers

Místo vydání

Yokohama

Místo konání akce

Yokohama

Datum konání akce

30. 6. 2024

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—