Testing scanners is a tricky business. Unlike PCs or graphics cards, which can be benchmarked to show performance, scanners are inherently more subjective. As such, we've developed our own in-house tests for scanners that remove as much subjectivity from the process as possible. Our tests were designed to highlight all the individual aspects of the scanners that relate to image quality and speed, and give a reliable indication of real world performance.

We also had all the test images scanned by our bureau on their professional drum scanner for reference.

The Tests

Each scanner was freshly installed with the latest drivers on the PC Authority testbench, consisting of a 3GHz Pentium 4 with 512MB of DDR RAM, and connected through the fastest connection option, whether that be USB 2.0, USB 1.1, FireWire or Parallel.

First we tested colour accuracy - important if you want your scanner to accurately reproduce colour photos or graphics. We scanned a colour target from AGFA, which featured multiple levels of red, green, blue and black using the scanner's best colour settings. We then imported the image into Adobe Photoshop and used the eye dropper tool to take a colour sample to see how close to the true colour it was.

Each scanner's results for red, green and blue were then compared to the drum reference scan, the deviation from the correct colour was determined as a percentage then averaged to give a single result.

Next we scanned a large photographic print, which contained many colours and fine detail. This scan was performed at the scanner's maximum optical resolution and at top detail settings. The maximum optical resolution is the actual resolution of the optics of the scanner, and is usually around 1,200 or 2,400dpi these days.

Many scanners also feature interpolated resolution settings, which can be as high as 65,000dpi or more, although these settings are practically useless and only used when performing a huge enlargement. In the real world, 300dpi, or 600dpi at the most, is more than adequate for most applications.

We also scanned the same photo at default settings and resolution to see how they cope with day to day use. Each image was then compared to the reference drum scan and marked for colour, detail and resolution.

Another important measure of quality in scanners is tonal resolution. This represents the ability of the scanner to distinguish between fine variations in the brightness of an image. For this test we used a black and white photo by Miriam Stirling that had very fine gradients of grey, including 100 percent white and 100 percent black. This image was scanned using the scanner's best greyscale settings, but with no auto-adjustment to gamma so we could evaluate the scanner's raw capability. The image was then loaded into Photoshop where we took a histogram reading, which showed how many pixels of each level of grey appeared in the image. If the scanner didn't accurately capture the full range of tones in the photo, the histogram was cropped at one or both ends. If the tonal resolution of the scanner was poor, then there were no readings at every grey level, resulting in a 'spiked' histogram. The numeric values for each scanner's histogram was compared to the reference image, and the deviation determined.

It's sometimes tricky for a scanner, which is RGB by nature, to scan a CMYK print. If the scanner is not set to the right resolution, or the scanner's optics aren't uniform, the result can be an interference, or moiré, pattern. To test this, we scanned a 300dpi print of a past PC Authority cover at 600dpi to see if any anomalies arose, and rated each scanner accordingly.

Our final quality test was of the scanner's provided optical character recognition (OCR) software. We scanned the PC Authority OCR test document, which contained a variety of fonts and styles, along with a tricky watermark in the background that the software needed to distinguish from text. The scan was done at 300dpi, and run through the software, then checked in Word for errors.

We also tested for speed, by timing the colour photo scan on default settings, the A4 text document as well as two 4 x 3 photo prints – one done at default settings, the other done on maximum quality settings.

Once all the results were in, we tallied them together, weighted them and had a final result for each scanner that was used to determine the Quality star rating.

We also rated each scanner on its included features, which contributed to the Features star rating. Finally, the two results were weighted together, with 70 percent Quality, and 30 percent Features, with the price factored in to derive the Value Star rating – with Overall being an average of the three.

Awards were given to the scanner with the best quality, and the one with the best overall value.