A Survey of Numerical Linear Algebra Methods Utilizing Mixed-Precision Arithmetic

Ahmad Abdelfattah; Hartwig Anzt; Erik Boman; Erin Carson; Terry Cojean; Jack Dongarra; Alyson Fox; Mark Gates; Nicholas Higham; Xiaoye Li; Jennifer Loe; Piotr Luszczek; Srikara Pranesh; Siva Rajamanickam; Tobias Ribizel; Barry Smith; Kasia Swirydowicz; Stephen Thomas; Stanimire Tomov; Yaohung Tsai; Ulrike Yang

doi:10.1177/10943420211003313

A Survey of Numerical Linear Algebra Methods Utilizing Mixed-Precision Arithmetic

Ahmad Abdelfattah, Hartwig Anzt, Erik Boman, Erin Carson, Terry Cojean, Jack Dongarra, Alyson Fox, Mark Gates, Nicholas Higham, Xiaoye Li, Jennifer Loe, Piotr Luszczek, Srikara Pranesh, Siva Rajamanickam, Tobias Ribizel, Barry Smith, Kasia Swirydowicz, Stephen Thomas, Stanimire Tomov, Yaohung TsaiUlrike Yang

Computational Science

Research output: Contribution to journal › Article › peer-review

83 Scopus Citations

Abstract

The efficient utilization of mixed-precision numerical linear algebra algorithms can offer attractive acceleration to scientific computing applications. Especially with the hardware integration of low-precision special-function units designed for machine learning applications, the traditional numerical algorithms community urgently needs to reconsider the floating point formats used in the distinct operations to efficiently leverage the available compute power. In this work, we provide a comprehensive survey of mixed-precision numerical linear algebra routines, including the underlying concepts, theoretical background, and experimental results for both dense and sparse linear algebra problems.

Original language	American English
Pages (from-to)	344-369
Number of pages	26
Journal	International Journal of High Performance Computing Applications
Volume	35
Issue number	4
DOIs	https://doi.org/10.1177/10943420211003313
State	Published - 2021

NREL Publication Number

NREL/JA-2C00-77220

Keywords

GPUs
high performance computing
linear algebra
mixed precision arithmetic
numerical mathematics

Access to Document

10.1177/10943420211003313

Cite this

Abdelfattah, A., Anzt, H., Boman, E., Carson, E., Cojean, T., Dongarra, J., Fox, A., Gates, M., Higham, N., Li, X., Loe, J., Luszczek, P., Pranesh, S., Rajamanickam, S., Ribizel, T., Smith, B., Swirydowicz, K., Thomas, S., Tomov, S., ... Yang, U. (2021). A Survey of Numerical Linear Algebra Methods Utilizing Mixed-Precision Arithmetic. International Journal of High Performance Computing Applications, 35(4), 344-369. https://doi.org/10.1177/10943420211003313

@article{d5f047827e3743d4970805835673125b,

title = "A Survey of Numerical Linear Algebra Methods Utilizing Mixed-Precision Arithmetic",

abstract = "The efficient utilization of mixed-precision numerical linear algebra algorithms can offer attractive acceleration to scientific computing applications. Especially with the hardware integration of low-precision special-function units designed for machine learning applications, the traditional numerical algorithms community urgently needs to reconsider the floating point formats used in the distinct operations to efficiently leverage the available compute power. In this work, we provide a comprehensive survey of mixed-precision numerical linear algebra routines, including the underlying concepts, theoretical background, and experimental results for both dense and sparse linear algebra problems.",

keywords = "GPUs, high performance computing, linear algebra, mixed precision arithmetic, numerical mathematics",

author = "Ahmad Abdelfattah and Hartwig Anzt and Erik Boman and Erin Carson and Terry Cojean and Jack Dongarra and Alyson Fox and Mark Gates and Nicholas Higham and Xiaoye Li and Jennifer Loe and Piotr Luszczek and Srikara Pranesh and Siva Rajamanickam and Tobias Ribizel and Barry Smith and Kasia Swirydowicz and Stephen Thomas and Stanimire Tomov and Yaohung Tsai and Ulrike Yang",

year = "2021",

doi = "10.1177/10943420211003313",

language = "American English",

volume = "35",

pages = "344--369",

journal = "International Journal of High Performance Computing Applications",

issn = "1094-3420",

publisher = "SAGE Publications Inc.",

number = "4",

}

Abdelfattah, A, Anzt, H, Boman, E, Carson, E, Cojean, T, Dongarra, J, Fox, A, Gates, M, Higham, N, Li, X, Loe, J, Luszczek, P, Pranesh, S, Rajamanickam, S, Ribizel, T, Smith, B, Swirydowicz, K, Thomas, S, Tomov, S, Tsai, Y & Yang, U 2021, 'A Survey of Numerical Linear Algebra Methods Utilizing Mixed-Precision Arithmetic', International Journal of High Performance Computing Applications, vol. 35, no. 4, pp. 344-369. https://doi.org/10.1177/10943420211003313

TY - JOUR

T1 - A Survey of Numerical Linear Algebra Methods Utilizing Mixed-Precision Arithmetic

AU - Abdelfattah, Ahmad

AU - Anzt, Hartwig

AU - Boman, Erik

AU - Carson, Erin

AU - Cojean, Terry

AU - Dongarra, Jack

AU - Fox, Alyson

AU - Gates, Mark

AU - Higham, Nicholas

AU - Li, Xiaoye

AU - Loe, Jennifer

AU - Luszczek, Piotr

AU - Pranesh, Srikara

AU - Rajamanickam, Siva

AU - Ribizel, Tobias

AU - Smith, Barry

AU - Swirydowicz, Kasia

AU - Thomas, Stephen

AU - Tomov, Stanimire

AU - Tsai, Yaohung

AU - Yang, Ulrike

PY - 2021

Y1 - 2021

N2 - The efficient utilization of mixed-precision numerical linear algebra algorithms can offer attractive acceleration to scientific computing applications. Especially with the hardware integration of low-precision special-function units designed for machine learning applications, the traditional numerical algorithms community urgently needs to reconsider the floating point formats used in the distinct operations to efficiently leverage the available compute power. In this work, we provide a comprehensive survey of mixed-precision numerical linear algebra routines, including the underlying concepts, theoretical background, and experimental results for both dense and sparse linear algebra problems.

AB - The efficient utilization of mixed-precision numerical linear algebra algorithms can offer attractive acceleration to scientific computing applications. Especially with the hardware integration of low-precision special-function units designed for machine learning applications, the traditional numerical algorithms community urgently needs to reconsider the floating point formats used in the distinct operations to efficiently leverage the available compute power. In this work, we provide a comprehensive survey of mixed-precision numerical linear algebra routines, including the underlying concepts, theoretical background, and experimental results for both dense and sparse linear algebra problems.

KW - GPUs

KW - high performance computing

KW - linear algebra

KW - mixed precision arithmetic

KW - numerical mathematics

U2 - 10.1177/10943420211003313

DO - 10.1177/10943420211003313

M3 - Article

SN - 1094-3420

VL - 35

SP - 344

EP - 369

JO - International Journal of High Performance Computing Applications

JF - International Journal of High Performance Computing Applications

IS - 4

ER -

A Survey of Numerical Linear Algebra Methods Utilizing Mixed-Precision Arithmetic

Abstract

NREL Publication Number

Keywords

Access to Document

Fingerprint

Cite this