Genetic Programming with Memory for Approximate Data Reconstruction

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F26%3A0193318" target="_blank" >RIV/00216305:26230/26:0193318 - isvavai.cz</a>

Výsledek na webu

<a href="https://link.springer.com/chapter/10.1007/978-981-96-0077-9_10" target="_blank" >https://link.springer.com/chapter/10.1007/978-981-96-0077-9_10</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/978-981-96-0077-9_10" target="_blank" >10.1007/978-981-96-0077-9_10</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Genetic Programming with Memory for Approximate Data Reconstruction

Popis výsledku v původním jazyce

This chapter addresses the computation-memorization trade-offs in the context of genetic programming (GP). We introduce genetic programming with memory (GPM) in which GP evolves not only the expression but also the content of a small local memory to better approximate the original data set. In particular, we evolved expression-memory pairs that can serve as weight generators and thus approximate the weights associated with convolutional layers of some convolutional neural networks (CNNs). This is potentially interesting for the efficient implementations of hardware accelerators of CNNs in which memory access is significantly more energy-demanding than arithmetic operations. In our approach, most of the weights are approximated using an evolved expression; only some fraction of them must be read from memory. For example, if memory contains 10% of the original weights, the weight generator evolved for a convolutional layer can approximate the original weights such that the CNN utilizing the generated weights shows less than a 1% drop in the classification accuracy on the MNIST data set. The memory requirements are reduced 3.1x or 12.6x for 8-bit or 32-bit weights, respectively. Additional experiments conducted for more complex CNNs and challenging image classification benchmarks show various impacts of weights' approximation on classification accuracy.

Název v anglickém jazyce

Genetic Programming with Memory for Approximate Data Reconstruction

Popis výsledku anglicky

This chapter addresses the computation-memorization trade-offs in the context of genetic programming (GP). We introduce genetic programming with memory (GPM) in which GP evolves not only the expression but also the content of a small local memory to better approximate the original data set. In particular, we evolved expression-memory pairs that can serve as weight generators and thus approximate the weights associated with convolutional layers of some convolutional neural networks (CNNs). This is potentially interesting for the efficient implementations of hardware accelerators of CNNs in which memory access is significantly more energy-demanding than arithmetic operations. In our approach, most of the weights are approximated using an evolved expression; only some fraction of them must be read from memory. For example, if memory contains 10% of the original weights, the weight generator evolved for a convolutional layer can approximate the original weights such that the CNN utilizing the generated weights shows less than a 1% drop in the classification accuracy on the MNIST data set. The memory requirements are reduced 3.1x or 12.6x for 8-bit or 32-bit weights, respectively. Additional experiments conducted for more complex CNNs and challenging image classification benchmarks show various impacts of weights' approximation on classification accuracy.

Druh

C - Kapitola v odborné knize

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/GA24-10990S" target="_blank" >GA24-10990S: Strojové učení zohledňující hardware: Od automatizovaného návrhu k inovativním a vysvětlitelným řešením</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název knihy nebo sborníku

Genetic Programming Theory and Practice XXI

ISBN

978-981-9600-76-2

Počet stran výsledku

20

Strana od-do

199-218

Počet stran knihy

417

Název nakladatele

Springer Nature Singapore

Místo vydání

Singapore

Kód UT WoS kapitoly

—