How we count

We do not test. We count.

Testing a field of products properly costs money, takes weeks, and produces one person's opinion at the end of it. Plenty of people have already done that work, and plenty more have lived with the results for years. Our job is to read what they concluded and add it up.

So there is nothing on this site that is our judgement of a product. There is a count, and there is what the count came out as. If you disagree with a pick, the thing to argue with is the arithmetic.

Two kinds of source, kept apart

People who put the products side by side. They bought a field of them, used them, and published an order of merit. They are the ones who did the testing, and their votes are counted first, because putting nine things next to each other is the only way to learn which is best.

Owners. People who bought one and rated it on the shop. One owner knows one product very well and the field not at all. Thousands of ratings, added up, know something no single person who tested it does: what the thing is like in the sixth month. Owners never pick a product. They can keep one out: a product whose owners are markedly less happy than owners across the rest of its category does not reach a page.

Some sites publish an order of merit without anybody having touched a product. No code can tell those apart from the real thing — both publish a list with a rejection in it — so they are ruled out by hand, once, for a whole category, and the ruling is written down. Content farms, deal sites, and shops or brands dressed up as reviewers.

Small specialists are deliberately kept. On a narrow product, somebody who owns forty of them is worth more than a large newspaper, and a rule about being well known would have thrown those people out along with the junk.

What "comments read" means, exactly

When a page says a number of comments was read, this is what that number counts: everything written in a conversation about the product. A remark counts if it is about the product itself, or if it sits under an opening post that is.

The older rule made a remark count only if it repeated the product's name, which threw away somebody answering "that one, every time" under a question about which one to buy. That is a real opinion, discarded for not repeating a word.

One caution, and we would rather write it ourselves than have somebody else write it for us. A number of comments read is not a number of people who chose the winner. Far fewer of them name any product at all, and the people who did the testing do most of the deciding. The two sentences are one word apart and only one of them is true.

How reviews become a pick

We read every hands-on review of a product we can find. A publication only counts if it tested the products itself and prints what it saw in those tests. A maker's or a shop's own page never counts. Nor does a guide that only compares one brand's models.

Each review gives points to the products it picks: 3 for its single best pick, 2 for a clearly named runner-up, and 1 for any other pick or a plain "buy this". A publication counts once per product, however many pages it writes. A product needs at least three separate publications behind it before it can be picked.

The best-backed products are sorted by price into three levels. Price only decides which level a product sits in. It never decides which one wins. In Essential, the winner is the product with the most backing for what it costs. In Upgrade and Min-Max, the winner is simply the one with the most backing.

Before a pick reaches a page, it must pass four tests. No review says it has been replaced or is no longer made. Its owners aren't markedly less happy than owners across the rest of the category. It still shows on the page if any one publication is taken away. And a separate researcher, who isn't told our picks, looks for a better-backed rival at the same level. Any rival found is counted like everything else, and the count decides. A pick that passes, but only just, is marked a close call, with its nearest rival shown beside it.

When two products are too close to separate, the level shows both, side by side. We'd rather say so than pretend one is better.

A product is priced where it's sold new today. One nobody sells new can't be picked, because a pick you can't buy isn't a pick.

Two or three picks, always in this order

Essential. For something you do now and then. The product that does the job properly and asks nothing more of you: nothing to learn, no fussy upkeep, no weak spot that lets you down.

Upgrade. For something you do often enough to feel the difference. Better control, better results, a tool that makes the work more pleasant, and only where that difference is real.

Min-Max. For something you take seriously. The most capable product that still earns its place. Beyond it, the extras stop making you better at the job.

Every product page has these three, in this order, on every page of this site. Learn the shape once and you never have to learn a page again.

When there is no honest winner, the band stays empty

Sometimes a level has no honest winner: too few publications tested products at that level, or nothing there passes the four tests. Before giving up, we search for more reviews, up to six more rounds. If that still turns up nothing solid, the level stays empty and the page says why.

When that happens the band is left empty and the page says so. We do not promote the next thing along to fill the hole, and we do not soften the language until a weak pick sounds like a strong one. A band you cannot trust is worse than a band with nothing in it, because it quietly poisons the two beside it.

Why a page does not name its sources

It was considered properly, and turned down. Naming a source hands you a thread to pull, and pulling it takes you off this page to read somebody else's — which is a fine outcome for them and a bad one for you, because you now have to do the adding up yourself. The method is the product here. The sources are its ingredients.

What that costs us is the obvious check: you cannot look over our shoulder. What it does not cost is the record. Every source read for a product is written down, with the date it was read, and it is what any answer here is checked against before it is published. The page gives you the answer. The proof is kept, and it is not on the page.

That is the weakest point of the way we work, so it is written down here rather than left for somebody else to find.

A price was true on a day, and that day is printed

Every price on this site carries the date it was checked. Shops move prices hourly, so treat the number as the size of the thing rather than as a quote: it tells you which band a product sits in, and the shop tells you what it costs today.

A number is only printed when it came from the shop's own official feed and was refreshed within the hour. If it cannot pass that, no number is printed and the button simply sends you to go and look. An old price shown as a current one is a lie with a decimal point in it.

The maker's own picture, cut out by us

Every photograph on this site is the maker's own — the picture the maker chose to represent that product. It is downloaded once, its background is cut off here on our own machine, and we serve our own copy. Nothing is pulled from anybody else's server as you read.

Nothing is invented where a picture is missing. A pick we could not find a photograph for shows no photograph and no stand-in, and the record says why.

Shelves and collections are two different things

Shelves are navigation and nothing else. They are the groups that carry you around inside one category, so you can get to the thing you came for. A shelf holds no meaning on its own.

Collections are where a room leads next: the smaller rooms inside it, and our shopping lists, which gather products because they are bought or used together. A collection is something we made deliberately. A shelf is just a way through.

Both appear on the same page, side by side, which is exactly why they have two different names.