Of Lies, Damned Lies, And Benchmarks – Redux!

Upon request of the PHP community, Joe Stagner conducted additional tests of PHP, using both PHP 5.3 and APC. In his article he also revisits the questions raised by myself and Andi Gutmans about the fairness of his tests.

The new tests reveal that APC doesn’t do much to improve the performance of his applications. This wasn’t a significant surprise to me; after reviewing his code I realized that the script wouldn’t be called again and thus APC wouldn’t provide a benefit. And though some of his tests (MySQL, file system tests, and Postgres) are dependent on external factors (like how well MySQL performs on Linux versus Windows), I think they paint a fair picture. Joe’s tests were fair.

(more…)

Thursday, August 13th, 2009 @ 11:04 am | Comment (9) | Categories: Community, PHP 5
Tags: ,

Of Lies, Damned Lies, and Benchmarks

It’s no secret that we like statistics. Human beings like being able to compare two things, apples to apples. We like the ability to say “this thing is better because…” and being able to back it up with a seemingly solid fact, rather than just our opinion. We build charts to show how things stack up side-by-side, and we even build software to make those charts, which are then compared using the charts, and on we go.

But benchmarks, for all their decision-making aid, fail under the best of circumstances for one simple reason: they’re not real life.

(more…)

Tuesday, August 11th, 2009 @ 8:03 am | Comment (20) | Categories: PHP 5
Tags: , , ,


Copyright © 2022 by Brandon Savage. All rights reserved.