update: The promised follow-up article is available at Weblog software benchmark II.
Wordpress and Textpattern are both roughly comparable software applications for managing small sites, usually – but not necessarily – weblogs. You may remember that I have written about them before. They do a lot of things differently, but can often be used to accomplish the same thing. So I think it’s interesting to compare and see how well they perform relative to each other.
Some of you may have looked at the page-footer of either software, there is a comment in the html that shows the page generation time in both applications. Those however are not very reliable, so I set up a quick test-installation of both apps and ran apache bench on them. Apache bench is a tool that makes requests to a webserver (like a browser) and measures the time that elapses.
I installed Textpattern’s most recent development version (revision 235 at the time of testing) and Wordpress version 1.5. I had an old b2-dump with > 1000 posts that I imported into both applications to have a comparable set of posts. Since I want to move a b2-site to Textpattern I tested and patched the import-routine for Textpattern just recently and so it worked as expected. I had some problems importing the dump with WP, but in the end I had the posts imported (I had to install WP, drop the new wpposts table and rename the old b2posts table to the wpposts table and then run the included importb2-script. Is this how it is supposed to be?). Since I wanted the two default-pages of both applications to be comparable, I emptied most of the sidebar of Wordpress, where it by defaults would otherwise show a blogroll and monthly archives-links etc. I also reduced the page-size by throwing out some RDF, comments etc. In the end both sites were only showing the excerpts of the 10 most recent articles and had comparable page-size. (Note that apache bench only makes a request for a single file; it does not load images, external css files etc.)
I ran everything locally: Apache 1.3.x, PHP 4.3.10 and MySql 4.0 for running the applications as well as Apache Bench 2.0.40-dev for generating the benchmark. My hardware is a bit older (PIII-Notebook, 256MB – hey I’m only a student;) ) but the resulting numbers are not meant to be absolute numbers, they are only for comparing the two applications relative to each other. I simluated 500 Requests and turned keep-alive on:
./ab.exe -n500 -k http://localhost/app/
|Requests per second:||3.63 [#/sec] (mean)||1.34 [#/sec] (mean)|
|Time per request:||275.456 [ms] (mean)||743.910 [ms] (mean)|
|Percentage of the requests served within a certain time (ms)||–|
|100%||510 (longest request)||981 (longest request)|
Textpattern could handle 3.63 requests per second, while Wordpress handled 1.34 requests per second (higher is better) or the other way around: A single request for TXP was finished in ~0.28 seconds and for WP in ~0.74 seconds. So in this test run, Textpattern was on average more than twice (up to 2,7 times) as fast as Wordpress. Remember that this was only a very simple Testpage that listed 10 excerpts and nothing else. Usually there is more going on on a page (recent articles, linkblog etc.), and that will likely affect the difference quiet a bit. One thing that struck me as strange, and that someone with more Wordpress-Knowledge can hopefully clarify, is that Wordpress claimed to make 39 queries, although it only displayed one list of posts. I remember that b2 was around 8 queries on a simple list of posts. Of course query count is not necessarily proportional to execution time or performance, but in this case it seems a very good candidate to investigate.
A reason why textpattern might have performed so well, is that it stores a parsed version of the posts in the DB, so that parsing of posts only happens upon posting and editing, and not upon every pageview. I remember when I profiled b2 a long time ago, that the parsing of the page was the major bottleneck. But from the casual look I took, Wordpress has already improved a quiet a bit in that area.
update: I missed something the first time around: While Wordpress always uses static files for the css, Textpattern by default stores css in the database – if you make use of this feature (which most of the time will be the case), then on txp you will receive an additional hit on the CSS-File (php-script w/ db-access) which will further close the gap between the two, but the css-request is a lot lighter than a regular page. It is of course debatable wether to include this in the comparison or not.
This is certainly no reason for WP-users to worry – for 98% of people performance is not relevant. And for the remaining 2% there are probably plugins that can do things like caching which will reduce the difference.
If there is interest in more investigation in this, a good first step might be to make a more realistic blog-page that includes more elements than just a list of posts.
Also this test only checked for the execution time with consecutive requests. For scalability it is also very interesting to see memory consumption, because often high load means concurrent requests. And even though one script (with high memory needs) may execute faster, it will slow to a crawl earlier than a slower script, which uses less memory: because at some point all the available memory will be filled and disk-swapping kicks in, which is a looooot slooowweeeer than RAM. So this is certainly something that will be interesting to investigate. Apache bench also has an option (-c) to simulate an arbitrary number of concurrent requests.