Fast Multiresolution Reads of Massive Simulation Datasets

Ray Grout, Sidharth Kumar, Cameron Christensen, John Schmidt, Peer-Timo Bremer, Eric Brugger, Venkatram Vishwanath, Philip Carns, Hemanth Kolla, Jacqueline Chen, Martin Berzins, Giorgio Scorzelli, Valerio Pascucci

Research output: Contribution to conferencePaperpeer-review

7 Scopus Citations


Today's massively parallel simulation codes can produce output ranging up to many terabytes of data. Utilizing this data to support scientific inquiry requires analysis and visualization, yet the sheer size of the data makes it cumbersome or impossible to read without computational resources similar to the original simulation. We identify two broad classes of problems for reading data and present effective solutions for both. The first class of data reads depends on user requirements and available resources. Tasks such as visualization and user-guided analysis may be accomplished using only a subset of variables with a restricted spatial extent at a reduced resolution. The other class of reads requires full resolution multivariate data to be loaded, for example to restart a simulation. We show that utilizing the hierarchical multiresolution IDX data format enables scalable and efficient serial and parallel read access on a variety of hardware from supercomputers down to portable devices. We demonstrate interactive view-dependent visualization and analysis of massive scientific datasets using low-power commodity hardware, and we compare read performance with other parallel file formats for both full and partial resolution data.

Original languageAmerican English
Number of pages17
StatePublished - 2014
Event29th International Supercomputing Conference, ISC 2014 - Leipzig, Germany
Duration: 22 Jun 201426 Jun 2014


Conference29th International Supercomputing Conference, ISC 2014

NREL Publication Number

  • NREL/CP-2C00-62475


  • interactive visualization
  • multiresolution
  • parallel I/O
  • PIDX
  • read performance
  • S3D
  • Uintah
  • VisIt


Dive into the research topics of 'Fast Multiresolution Reads of Massive Simulation Datasets'. Together they form a unique fingerprint.

Cite this