Abstract
Since 1992, SPEC has used multiple identical benchmarks to measure multi-processor performance. This “Homogeneous Capacity Method” (aka “SPECrate”) has been criticized on the grounds that real workloads are not homogeneous. Nevertheless, SPECrate provides a useful window into how systems perform when stressed by multiple requests for similar resources. This paper reviews SPECrate’s history, and several performance lessons learned using it: (1) a 4:1 performance gain for startup of a benchmark when I/O was reconfigured; (2) a benchmark that improved up to 2:1 when a TLB data structure was re-sized; and (3) a benchmark that improved by 52% after a change to NUMA page allocation. The SPEC CPU workloads usefully exposed several opportunities for performance improvement.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Carlton, A.: CINT92 and CFP 92 Homogeneous Capacity Method Offers Fair Measure of Processing Capacity, http://www.spec.org/cpu92/specrate.txt
Chew, J.: Memory Placement Optimization (MPO), http://opensolaris.org/os/community/performance/mpo_overview.pdf
Gove, D.: CPU2006 Working Set Size. ACM SIGARCH Computer Architecture News 35(1), 90–96 (2007), http://www.spec.org/cpu2006/publications/
Henning, J.L.: SPEC CPU Suite Growth: An Historical Perspective. ACM SIGARCH Computer Architecture News 35(1), 65–68 (2007), http://www.spec.org/cpu2006/publications/
McGhan, H.: Niagara 2 Opens the Floodgates. Microprocessor Report (November 6, 2006), http://www.sun.com/processors/niagara/M45_MPFNiagara2_reprint.pdf
SPEC CPU2000 published results, http://www.spec.org/osg/cpu2000/results/res2000q2/cpu2000-20000511-00104.html , http://www.spec.org/osg/cpu2000/results/res2000q2/cpu2000-20000511-00105.html
SPEC CPU2000 published results, http://www.spec.org/osg/cpu2000/results/res2002q2/cpu2000-20020422-01329.html , http://www.spec.org/osg/cpu2000/results/res2002q1/cpu2000-20020211-01256.html
SPEC CPU2006 published results, http://www.spec.org/cpu2006/results/res2008q2/cpu2006-20080408-04064.html
SPEC CPU2006 published results, http://www.spec.org/cpu2006/results/res2008q4/cpu2006-20080929-05409.html
SPEC CPU2006 Documentation, http://www.spec.org/cpu2006/docs/utility.html#convert_to_development
SPEC CPU2006 Documentation, http://www.spec.org/cpu2006/docs/utility.html#specinvoke
Sun Microsystems, UltraSPARC T2 Processor, http://www.sun.com/processors/UltraSPARC-T2/datasheet.pdf
Sun Microsystems, UltraSPARCT2 Supplement to the UltraSPARC Architecture 2007, section 12.2 (2007), http://opensparc-t2.sunsource.net/specs/UST2-UASuppl-current-draft-P-EXT.pdf
Sun Microsystems, Solaris 10 Reference Manual Collection, http://docs.sun.com/app/docs/doc/816-5166/trapstat-1m?a=view
Sun Microsystems, NUMA Observability, http://www.opensolaris.org/os/community/performance/numa/observability/
Sun Microsystems, Solaris Tunable Parameters Reference Manual, http://docs.sun.com/app/docs/doc/817-0404
STREAM: Sustainable Memory Bandwidth in High Performance Computers, http://www.cs.virginia.edu/stream/
Tufte, E.R.: The Visual Display of Quantitative Information, pp. 107–121. Graphics Press, Chesire (1983)
Weicker, R.P., Henning, J.L.: Subroutine Profiling Results for the CPU2006 Benchmarks. ACM SIGARCH Computer Architecture News 35(1), 102–111 (2007), http://www.spec.org/cpu2006/publications/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Henning, J.L. (2009). SPECrate2006: Alternatives Considered, Lessons Learned. In: Kaeli, D., Sachs, K. (eds) Computer Performance Evaluation and Benchmarking. SBW 2009. Lecture Notes in Computer Science, vol 5419. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-93799-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-540-93799-9_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-93798-2
Online ISBN: 978-3-540-93799-9
eBook Packages: Computer ScienceComputer Science (R0)