Reinforcement Learning for Mathematical Reasoning in Small-Scale Language Models with Structured Policy Optimization

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26220%2F26%3A0200028" target="_blank" >RIV/00216305:26220/26:0200028 - isvavai.cz</a>

Result on the web

<a href="https://ieeexplore.ieee.org/document/11268643" target="_blank" >https://ieeexplore.ieee.org/document/11268643</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/ICUMT67815.2025.11268643" target="_blank" >10.1109/ICUMT67815.2025.11268643</a>

Result language

angličtina

Original language name

Reinforcement Learning for Mathematical Reasoning in Small-Scale Language Models with Structured Policy Optimization

Original language description

Advancing mathematical reasoning in small-scale language models remains a challenge due to their limited capacity and the high computational demands of standard reinforcement learning methods like Proximal Policy Optimization (PPO). To address this, a resource-efficient training pipeline based on Group Relative Policy Optimization (GRPO) is proposed, enabling fine-tuning of compact models under strict memory constraints. The proposed method introduces structured prompting to explicitly separate reasoning steps from final answers and applies a dual reward system to jointly optimize for format adherence and mathematical correctness. The training incorporates low-overhead techniques such as 8-bit optimization, mixed-precision training, gradient checkpointing, and accelerated decoding for efficient rollout and policy updates. Experimental results show a 50.95% accuracy on a benchmark reasoning dataset—GSM8K, outperforming several larger models—while training entirely on a single GPU. These findings demonstrate that small-scale models, when trained with structured reinforcement learning, can achieve competitive performance in mathematical reasoning tasks. The approach offers a practical pathway for deploying interpretable, reasoning-capable models in low-resource environments.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

20203 - Telecommunications

Project

<a href="/en/project/FW07010015" target="_blank" >FW07010015: Indoor environment quality monitoring of buildings using odor sensors and artificial intelligence</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2025

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

2025 17th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT)

ISBN

979-8-3315-7675-2

ISSN

—

e-ISSN

2157-023X

Number of pages

6

Pages from-to

272-277

Publisher name

IEEE

Place of publication

Florence, Italy

Event location

Florence, Italy

Event date

Nov 3, 2025

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—