Uniform-in-Phase-Space Data Selection with Iterative Normalizing Flows: Article No. e11

Research output: Contribution to journalArticlepeer-review

Abstract

Improvements in computational and experimental capabilities are rapidly increasing the amount of scientific data that are routinely generated. In applications that are constrained by memory and computational intensity, excessively large datasets may hinder scientific discovery, making data reduction a critical component of data-driven methods. Datasets are growing in two directions: the number of data points and their dimensionality. Whereas dimension reduction typically aims at describing each data sample on lower-dimensional space, the focus here is on reducing the number of data points. A strategy is proposed to select data points such that they uniformly span the phase-space of the data. The algorithm proposed relies on estimating the probability map of the data and using it to construct an acceptance probability. An iterative method is used to accurately estimate the probability of the rare data points when only a small subset of the dataset is used to construct the probability map. Instead of binning the phase-space to estimate the probability map, its functional form is approximated with a normalizing flow. Therefore, the method naturally extends to high-dimensional datasets. The proposed framework is demonstrated as a viable pathway to enable data-efficient machine learning when abundant data are available.

Original languageAmerican English
Article numbere11
Number of pages28
JournalData-Centric Engineering
Volume4
Issue number7
DOIs
StatePublished - 25 Apr 2023

Bibliographical note

See NREL/JA-2C00-81433 for preprint

NREL Publication Number

  • NREL/JA-2C00-86434

Keywords

  • Data reduction
  • instance selection
  • normalizing flows

Fingerprint

Dive into the research topics of 'Uniform-in-Phase-Space Data Selection with Iterative Normalizing Flows: Article No. e11'. Together they form a unique fingerprint.

Cite this