An Open Combinatorial Diffraction Dataset Including Consensus Human and Machine Learning Labels with Quantified Uncertainty for Training New Machine Learning Models

Jason R. Hattrick-Simpers, Brian DeCost, A. Gilad Kusne, Howie Joress, Winnie Wong-Ng, Debra L. Kaiser, Andriy Zakutayev, Caleb Phillips, Shijing Sun, Janak Thapa, Heshan Yu, Ichiro Takeuchi, Tonio Buonassisi

Research output: Contribution to journalArticlepeer-review

5 Scopus Citations

Abstract

Modern machine learning and autonomous experimentation schemes in materials science rely on accurate analysis of the data ingested by these models. Unfortunately, accurate analysis of the underlying data can be difficult, even for domain experts, complicating the training of the models intended to drive experiments. This is especially true when the goal is to identify the presence of weak signatures in diffraction or spectroscopic datasets. In this work, we examine a set of as-obtained diffraction data that track the phase transition from monoclinic to tetragonal in a Nb-doped VO2 film as a function of temperature and dopant concentration. We then task a set of domain experts and a set of machine learning experts with identifying which phase is present in each diffraction pattern manually and algorithmically, respectively; in both cases, the labels can vary dramatically, especially at the phase boundaries. We use the mode of the labels and the Shannon entropy as a method to capture, preserve and propagate consensus labels and their variance. Further we use the expert labels as a benchmark and demonstrate the use of Shannon entropy weighted scoring to test the performance of machine learning generated labels. Finally, we propose a material data challenge centered around generating improved labeling algorithms. This real-world dataset curated with expert labels can act as test bed for new algorithms. The raw data, annotations and code used in this study are all available online at data.gov

Original languageAmerican English
Pages (from-to)311-318
Number of pages8
JournalIntegrating Materials and Manufacturing Innovation
Volume10
Issue number2
DOIs
StatePublished - Jun 2021

Bibliographical note

Publisher Copyright:
© 2021, This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply.

NREL Publication Number

  • NREL/JA-5K00-78444

Keywords

  • Artificial intelligence
  • Combinatorial materials science
  • Open data
  • Trust

Fingerprint

Dive into the research topics of 'An Open Combinatorial Diffraction Dataset Including Consensus Human and Machine Learning Labels with Quantified Uncertainty for Training New Machine Learning Models'. Together they form a unique fingerprint.

Cite this