Labs Roundup: Performance Analysis

Ten systems entered, and when the Labs team was finished with them, only two managed to walk away with awards. Read on to find out which ones.

Benchmarking PCs is a tricky business. You need to ensure that you stress the systems, but also that the results you get will be relevant in the real world. Previously we have relied heavily on BAPCo's SYSmark2002 for simulating home and office use, with programs such as Excel, Word, Outlook, and web content through Adobe Premiere and Macromedia Dreamweaver. However, due to SYSmark's history of instability as well as the sheer length of time it takes to complete a run, we have put it on the backburner in favour of PCMark04 for this Labs. The applications in PCMark04 include open source media encoders/decoders such as Zlib, OGG Vorbis and DivX compression, as well as physics and polygon throughput calculations, so it's sure to push the system to its limits.
We also used our staple graphical testing in 3DMark2001SE Pro and Quake 3: Arena with the 1.17 point release and our custom demo to gauge 3D performance. We ran both standard 1,024 x 768 and 1,600 x 1,200 at 32-bit colour with high geometry and max texture rendering to test high resolution rendering performance in each of the graphics cards. We also ran with a lower 640 x 480 32-bit resolution which is used to stress the CPU by eliminating the graphics card as the bottleneck.

The results

As you can see from the specifications, the PCs we received this month are all fairly entertainment-focused, with features like high-end 3D cards, plenty of RAM and reasonable speaker sets. Most systems sport the now commonplace RADEON series of graphics cards and only a couple opt for NVIDIA cards. While the FX 5700 found in the IntelliPro 3000GX system performed quite well, Modtech's Civic 3200.dX still outperformed it by a gap in excess of 3,600 points.

The Civic was the strongest performer, managing to top each of the tests and take out our coveted Labs Winner Award this month.

Hallmark bucked the trend by using a significantly lower spec graphics card, in a GeForce FX 5200, and using the extra money to bundle a 600 x 1,200 dpi scanner and colour inkjet printer. Its performance figures suffered as a result, but the overall package is a fantastic bundle for students and home users.

Although the majority of the processors were Pentium 4 Northwood CPUs, we did receive three of the latest Prescott core chips, all three of which performed quite well, taking out three of the top four PCMark04 scores, if only by a narrow margin. Check out all the Prescott details in Tim Dean's review on page 49.

This Labs really did disprove the old 'you get what you pay for' theory, as we got more than what we paid for in most of these systems, with some offering astounding bargains.
Unfortunately as we don't review a lot of non PC hardware, our standard PC Authority benchmark suite is ill equipped to cater to benching the Apple eMac system. We instead opted for a feature-based review, as that was the real focus of our roundup, the test results only re-enforcing our opinions with tangible and quantifiable results, especially in regards to referencing other machines. Secondly there is no direct comparative benchmark for cross-platform testing, leaving room for speculation and inconsistency.

We're not prepared to sacrifice our position as a trusted buyer's guide to have results for the sake of results. Previously we have benchmarked Apple systems such as the G5 (December 2003, page 47) using timed Adobe Photoshop 7.0.1 filter tests and Quake 3: Arena custom map demos. PCMark04 however is a comprehensive component level benchmark utility which covers a weighted task model for productivity, internet usage, entertainment and general usage. PCMark04 represents an average normal workload, covering anti-viral scanning, text editing and spreadsheet calculations in addition to its wide range of media encoders and decoders for formats such as DivX, WMV compression, audio conversion through the xiph encoder for the Vorbid OGG format. We also find it to be a better representative of the common use of PCs, especially media capable units. Benchmarking Excel spreadsheet calculations or web page creation is nice to know, but we find they're not as relevant for systems with an entertainment focus.

Given the price ceiling we imposed on participants and our focus on value for money over specifications, we were more than pleasantly surprised with the calibre of the machines in the marketplace for under $2,000. Ultimately it's about finding a unit which performs all your intended functions to the best of its ability, we're sure you'll find at least one of those in the pages ahead.

This month while calculating our Overall score we used our normal scoring system where 100 is the average. As with these systems raw speed is not the most important factor, the Overall score was weighted at 20% performance, 30% value for money and 50% for features.