Memory Access Optimization for Particle Operations in Computational Fluid Dynamics-Discrete Element Method Simulations

Deepthi Vaidhynathan, Hariswaran Sitaraman, Ray Grout, Thomas Hauser, Christine Hrenya, Jordan Musser

Research output: Contribution to journalArticlepeer-review


Computational Fluid Dynamics – Discrete Element Method is used to model gas-solid systems in several applications in energy, pharmaceutical and petrochemical industries. Computational performance bottlenecks often limit the problem sizes that can be simulated at industrial scale. The data structures used to store several millions of particles in such large-scale simulations have a large memory footprint that does not fit into the processor cache hierarchies on current high-performance-computing platforms, leading to reduced computational performance. This paper specifically addresses this aspect of memory access bottlenecks in industrial scale simulations. The use of space-filling curves to improve memory access patterns is described and their impact on computational performance is quantified in both shared and distributed memory parallelization paradigms. The Morton space filling curve applied to uniform grids and k-dimensional tree partitions are used to reorder the particle data-structure thus improving spatial and temporal locality in memory. The performance impact of these techniques when applied to two benchmark problems, namely the homogeneous-cooling-system and a fluidized-bed, are presented. These optimization techniques lead to approximately two-fold performance improvement in particle focused operations such as neighbor-list creation and data-exchange, with ∼ 1.5 times overall improvement in a fluidization simulation with 1.27 million particles.

Original languageAmerican English
Pages (from-to)97-110
Number of pages14
StatePublished - 2023

Bibliographical note

Publisher Copyright:
© 2022 Chinese Society of Particuology and Institute of Process Engineering, Chinese Academy of Sciences

NREL Publication Number

  • NREL/JA-2C00-79825


  • Memory access optimization
  • Performance optimization
  • Spatial reordering


Dive into the research topics of 'Memory Access Optimization for Particle Operations in Computational Fluid Dynamics-Discrete Element Method Simulations'. Together they form a unique fingerprint.

Cite this