Characterization and Modeling of PIDX Parallel I/O for Performance Optimization

Sidharth Kumar; Avishek Saha; Venkatram Vishwanath; Philip Carns; John A. Schmidt; Giorgio Scorzelli; Hemanth Kolla; Ray Grout; Robert Latham; Robert Ross; Michael E. Papka; Jacqueline Chen; Valerio Pascucci

doi:10.1145/2503210.2503252

Characterization and Modeling of PIDX Parallel I/O for Performance Optimization

Sidharth Kumar, Avishek Saha, Venkatram Vishwanath, Philip Carns, John A. Schmidt, Giorgio Scorzelli, Hemanth Kolla, Ray Grout, Robert Latham, Robert Ross, Michael E. Papka, Jacqueline Chen, Valerio Pascucci

Computational Science

Research output: Contribution to conference › Paper › peer-review

19 Scopus Citations

Abstract

Parallel I/O library performance can vary greatly in re-sponse to user-tunable parameter values such as aggrega-tor count, file count, and aggregation strategy. Unfortu-nately, manual selection of these values is time consuming and dependent on characteristics of the target machine, the underlying file system, and the dataset itself. Some charac-teristics, such as the amount of memory per core, can also impose hard constraints on the range of viable parameter values. In this work we address these problems by using machine learning techniques to model the performance of the PIDX parallel I/O library and select appropriate tun-able parameter values. We characterize both the network and I/O phases of PIDX on a Cray XE6 as well as an IBM Blue Gene/P system. We use the results of this study to develop a machine learning model for parameter space ex-ploration and performance prediction.

Original language	American English
DOIs	https://doi.org/10.1145/2503210.2503252 https://doi.org/10.1145/2503210.2503252
State	Published - 2013
Event	2013 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2013 - Denver, CO, United States Duration: 17 Nov 2013 → 22 Nov 2013

Conference

Conference	2013 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2013
Country/Territory	United States
City	Denver, CO
Period	17/11/13 → 22/11/13

NREL Publication Number

NREL/CP-2C00-62250

Other Report Number

Article No. 67

Keywords

I/O & Network Characterization
Performance Modeling

Access to Document

Cite this

Kumar, S., Saha, A., Vishwanath, V., Carns, P., Schmidt, J. A., Scorzelli, G., Kolla, H., Grout, R., Latham, R., Ross, R., Papka, M. E., Chen, J., & Pascucci, V. (2013). Characterization and Modeling of PIDX Parallel I/O for Performance Optimization. Paper presented at 2013 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2013, Denver, CO, United States. https://doi.org/10.1145/2503210.2503252, https://doi.org/10.1145/2503210.2503252

@conference{9a51730283d34ec19ddd091ffefadbbb,

title = "Characterization and Modeling of PIDX Parallel I/O for Performance Optimization",

abstract = "Parallel I/O library performance can vary greatly in re-sponse to user-tunable parameter values such as aggrega-tor count, file count, and aggregation strategy. Unfortu-nately, manual selection of these values is time consuming and dependent on characteristics of the target machine, the underlying file system, and the dataset itself. Some charac-teristics, such as the amount of memory per core, can also impose hard constraints on the range of viable parameter values. In this work we address these problems by using machine learning techniques to model the performance of the PIDX parallel I/O library and select appropriate tun-able parameter values. We characterize both the network and I/O phases of PIDX on a Cray XE6 as well as an IBM Blue Gene/P system. We use the results of this study to develop a machine learning model for parameter space ex-ploration and performance prediction.",

keywords = "I/O & Network Characterization, Performance Modeling",

author = "Sidharth Kumar and Avishek Saha and Venkatram Vishwanath and Philip Carns and Schmidt, {John A.} and Giorgio Scorzelli and Hemanth Kolla and Ray Grout and Robert Latham and Robert Ross and Papka, {Michael E.} and Jacqueline Chen and Valerio Pascucci",

year = "2013",

doi = "10.1145/2503210.2503252",

language = "American English",

note = "2013 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2013 ; Conference date: 17-11-2013 Through 22-11-2013",

}

Kumar, S, Saha, A, Vishwanath, V, Carns, P, Schmidt, JA, Scorzelli, G, Kolla, H, Grout, R, Latham, R, Ross, R, Papka, ME, Chen, J & Pascucci, V 2013, 'Characterization and Modeling of PIDX Parallel I/O for Performance Optimization', Paper presented at 2013 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2013, Denver, CO, United States, 17/11/13 - 22/11/13. https://doi.org/10.1145/2503210.2503252, https://doi.org/10.1145/2503210.2503252

TY - CONF

T1 - Characterization and Modeling of PIDX Parallel I/O for Performance Optimization

AU - Kumar, Sidharth

AU - Saha, Avishek

AU - Vishwanath, Venkatram

AU - Carns, Philip

AU - Schmidt, John A.

AU - Scorzelli, Giorgio

AU - Kolla, Hemanth

AU - Grout, Ray

AU - Latham, Robert

AU - Ross, Robert

AU - Papka, Michael E.

AU - Chen, Jacqueline

AU - Pascucci, Valerio

PY - 2013

Y1 - 2013

N2 - Parallel I/O library performance can vary greatly in re-sponse to user-tunable parameter values such as aggrega-tor count, file count, and aggregation strategy. Unfortu-nately, manual selection of these values is time consuming and dependent on characteristics of the target machine, the underlying file system, and the dataset itself. Some charac-teristics, such as the amount of memory per core, can also impose hard constraints on the range of viable parameter values. In this work we address these problems by using machine learning techniques to model the performance of the PIDX parallel I/O library and select appropriate tun-able parameter values. We characterize both the network and I/O phases of PIDX on a Cray XE6 as well as an IBM Blue Gene/P system. We use the results of this study to develop a machine learning model for parameter space ex-ploration and performance prediction.

AB - Parallel I/O library performance can vary greatly in re-sponse to user-tunable parameter values such as aggrega-tor count, file count, and aggregation strategy. Unfortu-nately, manual selection of these values is time consuming and dependent on characteristics of the target machine, the underlying file system, and the dataset itself. Some charac-teristics, such as the amount of memory per core, can also impose hard constraints on the range of viable parameter values. In this work we address these problems by using machine learning techniques to model the performance of the PIDX parallel I/O library and select appropriate tun-able parameter values. We characterize both the network and I/O phases of PIDX on a Cray XE6 as well as an IBM Blue Gene/P system. We use the results of this study to develop a machine learning model for parameter space ex-ploration and performance prediction.

KW - I/O & Network Characterization

KW - Performance Modeling

UR - http://www.scopus.com/inward/record.url?scp=84899675882&partnerID=8YFLogxK

U2 - 10.1145/2503210.2503252

DO - 10.1145/2503210.2503252

M3 - Paper

AN - SCOPUS:84899675882

T2 - 2013 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2013

Y2 - 17 November 2013 through 22 November 2013

ER -

Characterization and Modeling of PIDX Parallel I/O for Performance Optimization

Abstract

Conference

NREL Publication Number

Other Report Number

Keywords

Access to Document

Other files and links

Fingerprint

Cite this