Opinion: Testing in the post-CPU era (updated)

Opinion: Testing in the post-CPU era (updated)

AMD, VIA and NVIDIA have jumped ship from BaPCo and its SYSmark12 benchmark, indicating that CPU focused benchmarks are a thing of the past.

BaPCo and its industry standard SYSmark benchmark has a tough struggle ahead, with an exodus of member companies leaving Intel as the last semiconductor producer in the consortium. NVIDIA, VIA and AMD have all departed BaPCo, under concerns that the latest version of the benchmark is biased towards Intel’s products.

The concerns seem to largely focus around the choice of tests within SYSMark, which neglect the rising number of applications that use GPU acceleration. AMD has been quite vocal over the past year about its concerns that SYSmark is too CPU focused, and in an overnight blog post its CMO Nigel Dessau outlined the concerns.

It is well worth a read, and outlines just why AMD sees SYSmark to be non representative. In it are some quite valid points about SYSmark testing extremes rather than average workloads SYSmark is predominantely used to decide on corporate and government system purchasing (and hence is responsible for deciding where the money ends up), and while there are some areas where heavy CPU workloads emerge, most of the time people’s work PCs don’t need to be capable of enormous feats of CPU heavy lifting.

In reality most people will be running applications like Office, Web browsers and Outlook. All are capable of taxing a CPU, but very few people will ever be working with the kind of datasets that will. Even more curiously, despite the fact that all major browsers now feature GPU acceleration SYSmark 2012 uses versions from just before that came into effect.

One doesn’t even have to imagine the nefarious hand of Intel in this. Design by committee is almost always a poor strategy and in an organisation heavily focused on the CPU based workloads of past decades one could assume tunnel vision would set in. But this isn’t the first time that SYSmark has been slapped with a perceived Intel bias, and we have moved well beyond the raw megahertz race that once determined whether Intel or AMD was top dog.

The issue is also a barometer of sorts about where processors are headed. We are at a pretty crucial turning point where the CPU is less important than it once was and the GPU is less important than it will become. It is an issue that we have been facing in the PC & Tech Authority Labs as we test out the new Fusion APUs from AMD and try to put them in context with Intel’s Sandy Bridge CPUs.

No matter how you test you can always find an angle that makes one or the other processor look good. Traditional benchmarks are still a lot like SYSmark, focusing on hammering the CPU with extreme situations and seeing how it fares. Games and other graphically focused benchmarks absolutely shine on Fusion, making Sandy Bridge seem kinda lame (due largely to the relatively lower complexity and performance of Intel's processor graphics).

But this isn’t just a reviewing issue, it is a reflection of the computing market as a whole. For years now processing has been Intel from top to bottom but with the C, E and A series APUS AMD has delivered products that shine in certain market segments. If anything AMD is deftly creeping upwards in performance, taking out Atom in the netbook space and moving into the mainstream laptop space with the A series.

We are still skeptical of whether AMD can compete with Intel at the CPU high end again – rumours are that while the upcoming ‘Zambezi’ CPU should impress, it can’t quite hit the heights seen by Core i7, especially the upcoming ‘Sandy Bridge-E’ socket 2011 processors. But when it comes to mainstream laptops AMD has a phenomenally strong product in the form of the A series. These APUs combine great battery life, gaming, video playback and light productivity (or as AMD puts it 100-2000 row spreadsheets instead of the 35,000 row ones seen in SYSmark).

It is likely that we’ll never see AMD snatch the raw CPU performance crown from Intel again. But it is clear that this is something AMD doesn’t really care about anymore. With its departure (along with the other GPU manufacturers) from BaPCo it has signaled it belief that we have moved to a post-CPU world, an environment that needs new standard benchmarks and rethought assumptions about what makes a ‘good’ processor.

Update: It has somewhat forcibly been brought to our attention that the news of NVIDIA and VIA departing BaPCo was first broken by Charlie at Semiaccurate.com. That article can be found here.

Copyright © PC & Tech Authority. All rights reserved.

Most Read Articles

Review: WD Blue 3D &  Sandisk Ultra 3D 1TB SSDs

Review: WD Blue 3D & Sandisk Ultra 3D 1TB SSDs

Explainer: Who was Ada Lovelace?

Explainer: Who was Ada Lovelace?

The 31 best podcasts of 2017

The 31 best podcasts of 2017

Are these the biggest gaming franchises in history?

Are these the biggest gaming franchises in history?

Would you like to receive

Our Newsletter?