Comments on: Of Lies, Damned Lies, And Benchmarks – Redux! http://www.brandonsavage.net/of-lies-damned-lies-and-benchmarks-redux/ The personal blog of Brandon Savage. Contains entries of a personal and professional nature focusing on PHP, Apple, LAMP, MySQL and Washington, DC. Fri, 03 Feb 2012 19:36:33 -0500 hourly 1 http://wordpress.org/?v=3.3.1 By: Andrei Rinea http://www.brandonsavage.net/of-lies-damned-lies-and-benchmarks-redux/#comment-1036 Andrei Rinea Sun, 30 Aug 2009 08:47:05 +0000 http://www.brandonsavage.net/?p=548#comment-1036 I must back up Scott Mitchel's comment in that we (and you!) have a free tool, called reflector which you can download here : http://www.red-gate.com/products/reflector/ since like 2001 and you can dissamble *ANY* .NET compiled code. Including the framework itself. It is a powerful, easy-to-use and day-to-day helpful decompiler which I suggest you should use it to inspect "why [ASP.NET] it’s faster". I must back up Scott Mitchel’s comment in that we (and you!) have a free tool, called reflector which you can download here :

http://www.red-gate.com/products/reflector/

since like 2001 and you can dissamble *ANY* .NET compiled code. Including the framework itself. It is a powerful, easy-to-use and day-to-day helpful decompiler which I suggest you should use it to inspect “why [ASP.NET] it’s faster”.

]]> By: Manuel Lemos http://www.brandonsavage.net/of-lies-damned-lies-and-benchmarks-redux/#comment-950 Manuel Lemos Mon, 17 Aug 2009 07:38:35 +0000 http://www.brandonsavage.net/?p=548#comment-950 Brandon, PHP/Zend engine can be better if you use an opcode optimizer. There are several optimizer extensions from Zend and other developers for PHP but for obvious business reasons Zend guys kept such extensions out of the main PHP distributions. So it is not the PHP community's fault that PHP official distribution is not better. Brandon, PHP/Zend engine can be better if you use an opcode optimizer. There are several optimizer extensions from Zend and other developers for PHP but for obvious business reasons Zend guys kept such extensions out of the main PHP distributions.

So it is not the PHP community’s fault that PHP official distribution is not better.

]]> By: Scott Mitchell http://www.brandonsavage.net/of-lies-damned-lies-and-benchmarks-redux/#comment-942 Scott Mitchell Sat, 15 Aug 2009 16:50:38 +0000 http://www.brandonsavage.net/?p=548#comment-942 Just one small correction in your post. You said: "We can’t examine the ASP.NET core to determine why it’s faster." That's not true. You can view the .NET Framework code, by using disassemblers like Reflector. Also, Microsoft has made most of the .NET Framework (including the ASP.NET bits in System.Web) shared source. See: http://referencesource.microsoft.com/netframework.aspx Granted, Microsoft won't accept patches from the community, but you can certainly examine how ASP.NET's internals work. Thanks Just one small correction in your post. You said: “We can’t examine the ASP.NET core to determine why it’s faster.”

That’s not true. You can view the .NET Framework code, by using disassemblers like Reflector. Also, Microsoft has made most of the .NET Framework (including the ASP.NET bits in System.Web) shared source.

See: http://referencesource.microsoft.com/netframework.aspx

Granted, Microsoft won’t accept patches from the community, but you can certainly examine how ASP.NET’s internals work.

Thanks

]]> By: Luis Morales http://www.brandonsavage.net/of-lies-damned-lies-and-benchmarks-redux/#comment-936 Luis Morales Sat, 15 Aug 2009 02:40:26 +0000 http://www.brandonsavage.net/?p=548#comment-936 What I said at www.misfitgeek.com: Luis Morales wrote on Friday, August 14 2009 There are lots of things to consider, besides benchmark results. My comparison includes: 1- Platforms supporting the technology: - ASP.NET: Microsoft Server and Apache (Mono Castle) - PHP: All known commercial and community platforms. 2- Corporations Backing or Contributing to the technology: - ASP.NET: Microsoft - PHP: Microsoft, Zend, IBM, Oracle, Sun, Google, etc. 3- How intuitive is the technology: - ASP.NET: Good - PHP: Excellent 4- How flexible is the technology: - ASP.NET: Debatable. - PHP: Flexible by nature, not debatable, almost an absolute fact. 5- Performance: - ASP.NET: Excellent on Windows Server. The Mono / Castle project is still working on performance. - PHP: Good on most platforms. Results may vary depending on a lot of settings out of the box or included by extensions. But given to an expert on PHP and Apache, can be as fast as ASP.NET on a Windows Server. 6- How expensive would be implementing the technology on a large scale: - ASP.NET: ask MySpace and Monster (very expensive but works good out of the box = good for Fortune 500). - PHP: ask Facebook, Flickr and others (considerably cheap and there are no license issues = good for startups.) What I said at http://www.misfitgeek.com:

Luis Morales wrote on Friday, August 14 2009

There are lots of things to consider, besides benchmark results. My comparison includes:

1- Platforms supporting the technology:

- ASP.NET: Microsoft Server and Apache (Mono Castle)

- PHP: All known commercial and community platforms.

2- Corporations Backing or Contributing to the technology:

- ASP.NET: Microsoft

- PHP: Microsoft, Zend, IBM, Oracle, Sun, Google, etc.

3- How intuitive is the technology:

- ASP.NET: Good

- PHP: Excellent

4- How flexible is the technology:

- ASP.NET: Debatable.

- PHP: Flexible by nature, not debatable, almost an absolute fact.

5- Performance:

- ASP.NET: Excellent on Windows Server. The Mono / Castle project is still working on performance.

- PHP: Good on most platforms. Results may vary depending on a lot of settings out of the box or included by extensions. But given to an expert on PHP and Apache, can be as fast as ASP.NET on a Windows Server.

6- How expensive would be implementing the technology on a large scale:

- ASP.NET: ask MySpace and Monster (very expensive but works good out of the box = good for Fortune 500).

- PHP: ask Facebook, Flickr and others (considerably cheap and there are no license issues = good for startups.)

]]> By: Luis Morales http://www.brandonsavage.net/of-lies-damned-lies-and-benchmarks-redux/#comment-931 Luis Morales Fri, 14 Aug 2009 11:42:31 +0000 http://www.brandonsavage.net/?p=548#comment-931 @Brandon I agree with you. Libraries of common functionality are good. But I think the current frameworks have too many inheritance and includes in their trees. I like the DRY principle, but we could hard code some things for sake of performance. Did you see Rasmus presentation at Drupalcon on the subject of php frameworks? @Brandon

I agree with you. Libraries of common functionality are good. But I think the current frameworks have too many inheritance and includes in their trees. I like the DRY principle, but we could hard code some things for sake of performance. Did you see Rasmus presentation at Drupalcon on the subject of php frameworks?

]]> By: Brandon Savage http://www.brandonsavage.net/of-lies-damned-lies-and-benchmarks-redux/#comment-929 Brandon Savage Fri, 14 Aug 2009 11:21:59 +0000 http://www.brandonsavage.net/?p=548#comment-929 I think that PHP frameworks have their place. Certainly due to PHP's "share nothing" structure and the fact that everything is garbage collected at the end of a request we have some performance issues with intense frameworks; this can't be avoided easily. Still, I think that frameworks are good, and writing libraries of common functionality is the next evolution in any language. I'd like to see frameworks make use of caching, placing objects into memory and some improvement in the request handling abilities. I think there's a lot that can be done for PHP to resolve this performance issue. I think that PHP frameworks have their place. Certainly due to PHP’s “share nothing” structure and the fact that everything is garbage collected at the end of a request we have some performance issues with intense frameworks; this can’t be avoided easily. Still, I think that frameworks are good, and writing libraries of common functionality is the next evolution in any language.

I’d like to see frameworks make use of caching, placing objects into memory and some improvement in the request handling abilities. I think there’s a lot that can be done for PHP to resolve this performance issue.

]]> By: Luis Morales http://www.brandonsavage.net/of-lies-damned-lies-and-benchmarks-redux/#comment-928 Luis Morales Fri, 14 Aug 2009 11:14:52 +0000 http://www.brandonsavage.net/?p=548#comment-928 I'm following this subject too. My humble suggestion of what someone in the php community should do: a C++ compiled dll extension MVC framework to php. The actual frameworks are killing php; making it slower, abstracting it and even changing its default behavior. Agree? I’m following this subject too. My humble suggestion of what someone in the php community should do: a C++ compiled dll extension MVC framework to php. The actual frameworks are killing php; making it slower, abstracting it and even changing its default behavior. Agree?

]]> By: Janier Davila http://www.brandonsavage.net/of-lies-damned-lies-and-benchmarks-redux/#comment-919 Janier Davila Thu, 13 Aug 2009 21:43:32 +0000 http://www.brandonsavage.net/?p=548#comment-919 Very encouraging words: "The difference in PHP is you." Good job. I personally see both technologies as tools. It doesn't hurt knowing both, like a carpenter would know how to use a hammer or any other tool. That is my own karma, my own statement to make up my mind. And...I love them both and I use them both :). Programming languages don't make good programmers... Anyways, good article! Keep it up! Very encouraging words: “The difference in PHP is you.” Good job.

I personally see both technologies as tools. It doesn’t hurt knowing both, like a carpenter would know how to use a hammer or any other tool. That is my own karma, my own statement to make up my mind. And…I love them both and I use them both :). Programming languages don’t make good programmers…

Anyways, good article! Keep it up!

]]> By: Gaetano http://www.brandonsavage.net/of-lies-damned-lies-and-benchmarks-redux/#comment-915 Gaetano Thu, 13 Aug 2009 17:11:20 +0000 http://www.brandonsavage.net/?p=548#comment-915 I did not yet read the update on the tests, but a quick glance at the php source code was revealing enough to tell me that adding an opcode cache would be of little use: the test stress execution of empty loops N times in a row, in a single page. While it is formally correct, that metric is of absolutely zero interest to any web developer. The overall time spent by an application executing the branches will be about 1/1000th,1/100th of a second out of a 2,3 secs total time in any real application. Adding an opcode cache for a single page run might even give you poorer results, since if you omit a cache-warmup pass you will now include in the response the time spent stocking the opcode list to cache before execution. A more interesting metric would be using AB, Jmeter or some other tool that measures pages-per-second, in sets where 1,10,100 parallel clients are used to pound on the server. I did not yet read the update on the tests, but a quick glance at the php source code was revealing enough to tell me that adding an opcode cache would be of little use: the test stress execution of empty loops N times in a row, in a single page.

While it is formally correct, that metric is of absolutely zero interest to any web developer. The overall time spent by an application executing the branches will be about 1/1000th,1/100th of a second out of a 2,3 secs total time in any real application.

Adding an opcode cache for a single page run might even give you poorer results, since if you omit a cache-warmup pass you will now include in the response the time spent stocking the opcode list to cache before execution.

A more interesting metric would be using AB, Jmeter or some other tool that measures pages-per-second, in sets where 1,10,100 parallel clients are used to pound on the server.

]]>