Large-scale Ultrasound Simulations Using the Hybrid OpenMP/MPI Decomposition

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F15%3APU116961" target="_blank" >RIV/00216305:26230/15:PU116961 - isvavai.cz</a>

Result on the web

<a href="http://www.easc2015.ed.ac.uk/proceedings" target="_blank" >http://www.easc2015.ed.ac.uk/proceedings</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

Large-scale Ultrasound Simulations Using the Hybrid OpenMP/MPI Decomposition

Original language description

The simulation of ultrasound wave propagation through biological tissue has a wide range of practical applications including planning therapeutic ultrasound treatments of various brain disorders such as brain tumours, essential tremor, and Parkinson's disease. The major challenge is to ensure the ultrasound focus is accurately placed at the desired target within the brain because the skull can significantly distort it. Performing accurate ultrasound simulations, however, requires the simulation code to be able to exploit several thousands of processor cores and work with datasets on the order of tens of TB.We have recently developed an efficient full-wave ultrasound model based on the pseudospectral method using pure-MPI with 1D slab domain decomposition that allows simulations to be performed  using up to 1024 compute cores. However, the slab decomposition limits the number of compute cores to be less or equal to the size of the longest dimension, which is usually below 1024. This paper presents an improved implementation that exploits 2D hybrid OpenMP/MPI decomposition. The 3D grid is first decomposed by MPI processes into slabs. The slabs are further partitioned into pencils assigned to threads on demand. This allows 8 to 16 times more compute cores to be employed compared to the pure-MPI code, while also reducing the amount of communication among processes due to the efficient use of shared memory within compute nodes. The hybrid code was tested on the Anselm Supercomputer (IT4Innovations, Czech Republic) with up to 2048 compute cores and the SuperMUC supercomputer (LRZ, Germany) with up to 8192 compute cores. The simulation domain sizes ranged from 256^3 to 1024^3 grid points. The experimental results show that the hybrid decomposition can significantly outperform the pure-MPI one for large simulation domains and high core counts, where the efficiency remains slightly below 50%. For a domain size of 1024^3, the hybrid code using 81

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

V - Vyzkumna aktivita podporovana z jinych verejnych zdroju

Publication year

2015

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Proceedings of the 3rd International Conference on Exascale Applications and Software

ISBN

978-0-9926615-1-9

ISSN

—

e-ISSN

—

Number of pages

5

Pages from-to

115-119

Publisher name

Association for Computing Machinery

Place of publication

Edinburgh

Event location

John McIntyre Centre, Pollock Halls, Edinburgh.

Event date

Apr 21, 2015

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—