PerfKit Benchmarker is an open effort to define a canonical set of benchmarks to measure and compare cloud offerings. It’s designed to operate via vendor provided command line tools. The benchmarks are not tuned (ie the defaults) because this is what most users will use. This should help us drive to great defaults. Only in the rare cause where there is a common practice like setting the buffer pool size of a database do we change any settings.