BenchmarkHQ - OpenSourceMark

OpenSourceMark 1.0 beta 10

By Van Smith
OS: Windows 2k/XP/2003

As you know, benchmark results can make or break the launch of a new CPU. This means that billions of dollars are at risk to reviewers who might expose a new product's embarrassing Achilles' Heel by simply running a benchmark. Because of this, benchmark development is a dirty business with certain CPU vendors constantly attempting to "influence" the various benchmark organizations.
The problem of corruption is made possible because nearly all commercial benchmarks are closed source and are therefore innately unverifiable. This problem is exacerbated because many of these benchmarks also obfuscate what is exactly being tested. Instead, many benchmarks simply produce a few magic numbers that are meaningless without context.
The problem is analogous to Douglas Adams' famous joke of "42" as the ultimate "answer to Life, the Universe and everything" in his humorous Hitchhiker's Guide to the Galaxy. Without a much more specific question, Adams' answer is meaningless, just like a "642" on some benchmark suite is meaningless without disclosing specifically what is being measured and how it is being tested.
OpenSourceMark is an attempt to address the inherent verifiability problem posed by closed source benchmarks. Not only is OpenSourceMark a completely Open Source system level benchmark, but the entire development process is conducted publicly on the COSBI forums. "COSBI" is the acronym for "Comprehensive Open Source Benchmarking Initiative" and OpenSourceMark is the primary container for all developed tests.
OpenSourceMark is also completely inclusive since it is intended to be an ever growing warehouse of tools. All meaningful, unique tests will be accepted and incorporated into OpenSourceMark. Additionally, groups of tests can be run in suites that typify different usage profiles like desktop applications, gaming, etc. Moreover, custom suites, where users can pick and choose which test most interests them, are easy to configure and run.
OpenSourceMark empowers the user to decide what is important to benchmark rather than relying on the nebulous, blackbox decisions of quasi-secret cabals having shady connections with CPU vendors.

Click to enlarge

Download OpenSourceMark 1.0 beta 10! (79Mb)

hosted by spaceweb