Pendragon Software Home Page

CaffeineMark Home Page

Run the CaffeineMark

Benchmark Information

CaffeineMark Results

License and Results Publishing Guidelines

Why do some VM's obtain astronomical scores on CaffeineMark 2.5?

Some Java Virtual Machines (VM's) have begun to post extremely high scores on the the CaffeineMark 2.5 benchmark. In some cases, individual scores were as high as 1 million!

The reason for these high scores is that the Loop and Method tests were too optimizable. The Just-In-Time compilers realize that the calculations performed in these tests are not used, so the VM's don't execute the computations. While the high scores are an indicator of superior performance, the benchmark is misleading in these cases because real world performance is not 10,000 times better than it was a year ago.

The optimizability problems of CaffeineMark 2.5 have been fixed in the new CaffeineMark 3.0. Also, the overall score is calculated using a geometric mean instead of a weighted average. This results in performance numbers that correlate with typical user experience.

© Copyright 1997 Pendragon Software Corporation. All rights reserved. CaffeineMark is a trademark of Pendragon Software. Java and HotJava are trademarks of Sun Microsystems, Inc., and refer to Sun's Java programming language and HotJava browser technologies.