Benchmarking SIG Overview
Benchmarking novel architectures has been a key focus on research into post-Moore technologies as well as recent exascale architecture explorations. However, there is limited information as to what makes a benchmark "good" for a particular architecture or what benchmarks can appropriately determine if a new type of system is a successful platform for the post-Moore era. This SIG is focused on determining the state-of-the-art in the benchmarking space and on developing new best practices for benchmarking unique architectures in ways that are meaningful.
Goals of this SIG
The Benchmarking SIG is focused on the following goals:
- Work with partners in the DoE, industry, and goverment to collect and curate benchmarks useful for different classes of post-Moore systems.
- Create an association of advanced architecture testbeds focused around tools and testing they use related to benchmarks.
- Make available related benchmarks for hosted hardware in the Rogues Gallery testbed.
- Discuss what makes benchmarks and proxy applications/kernels "good" and how we can improve the usefulness of these tools.
Who is involved?
SIG Leads: Richard Vuduc, Jeffrey Young
Members: TBD
What are some related projects and efforts?
This SIG is being initiated in Fall 2020. If you are interested to join, please reach out to one of the leads of the SIG for more information.
