161 comments

[ 3.1 ms ] story [ 199 ms ] thread
90% of the reviews I see online are sponsored, fake, and dishonest. The other 10% are not found by Googling. They're buried deep in Amazon customer reviews of similar products or somewhere on Reddit. I never trusted Wirecutter because their reviews were always in stark contrast to the reviews I'd read on Amazon (by actual people who used the product), so I'm not surprised if Wirecutter is also bought by companies. What better way to promote your product on a website that's supposedly honest because it's NYT after all...
The brand of truth, the paper of record. Not one that would market that brand.

Surely inapplicable to its editorial-like headline and subject choices on the front page.

> I never trusted Wirecutter because their reviews were always in stark contrast to the reviews I'd read on Amazon (by actual people who used the product),

To be fair, I've known multiple people (when my peer group hadn't yet established their veblen side hustles) who got paid to write Amazon reviews for products they had never used.

Yes, some Amazon reviews are even written by ChatGPT. But my point is that there are far better reviews on Amazon than these "review" websites.
That’s true but for a lot of these products, they were negative ones like “broke after a few uses.”

Super sus when Wirecutter is promoting something as a top pick.

But it also makes sense. A review site like Wirecutter is using the product like one time — they have no idea if it’s going to last.

Many of the negative reviews are posted by competitors
Not from my experience.

I’ve used Wirecutter top picks. They suck and actually do break

Can you give a couple examples of Wirecutter top pick items you bought that broke after a short time?
I've used Wirecutter top picks and have never been burnt. This entire thread is people bashing Wirecutter without providing any examples of bad products.
Yes, some stuff does break, and some categories suck more than others, and sometimes Wirecutter gets it wrong. But I have not seen anyone point to any meaningful evidence that Wirecutter is improperly biased or is anything worse than simply a product review site that does it best but sometimes gets it wrong.
I don't find such a review very informative. Especially if it's a popular product sold 10000s times. Even quality things sometimes break. Does it have stupid buttons that are hard to press? That's worse
That's why I only used Wirecutter as a reference. Part of that was also they had different priorities than me. But it was still helpful to find what I wanted. But now their site is rarely updated.
My kneejerk reaction to that is that we need that sort of dishonest behavior, where reviews are fake, or bought (without disclosing) to be policed in some fashion. Of course, I'm sure there's some second or third order effect I haven't thought of, but it seems to me that this type of behavior erodes the trust of consumers and drives the market into lemon territory where consumer expect this asymmetry of information.

I feel that corporations would have an interest in coming together and creating a non-profit organization to be an independent tester of products, to increase the trust that consumer have in high quality products and therefore to help them increase demand on high cost items. If you can get a highly trusted organization to say that your item will last 20 years and be of higher quality, you can probably charge an order of magnitude more than your next competitor and still be profitable.

Wirecutter used to be good, but then it got bought by NYT. I'm surprised at how difficult it is to find legit reviewers in general. I don't trust these larger companies to give us non-biased reviews.
Consumerlabs, consumer reports, rtings, and which.co.uk seem to be the only standardized independent review bodies I'm aware of.
Consumer reports at least is known to be gamed (and they have specific review methodology you need to understand and interpret).

It’s hard

> Consumer reports at least is known to be gamed

I'd be interested to know more about this, but I haven't found much discussion of it when I search. My google fu may be weak today.

Not related to the topic on hand but I’ve felt my google fu diminish significantly over the past years. Not sure if it’s me or my google intuition just not matching the current algorithms
It's not you, it's google. Seriously.
On the whole I think consumer reports is one of the better review organizations out there. but for a hint of reader betrayal read up on what they did to the suzuki samurai.

It is not exactly cut and dried, but it was found that consumer reports was testing to get a specific result. And due to the trust that people put in consumer reports that result killed the market for the samurai in the US.

Any examples after 1988?
Not really, Like I said, I think they are one of the good guys, I am not sure if their testing methodology is any better than any one else. But they do try hard to remain independent from the manufacturers. I think of it as one black mark on their record and a reminder that even our heroes can slip up.

By good methodology I think the best is the person who has a fleet of the things and a deep understanding of how they break, the next best is to take the thing apart and apply some industry specific analysis to it, then test it to destruction to see how it breaks. Ether one is beyond the grasp of the average joe, who only wants to buy one of the things and have it work. And almost no reviewers do this which is why the ones that do are so valuable.

Exactly this - they do try to remain independent but they sometimes obviously have biases towards certain operations of things.

For example, the only thing I care about with toilets are that they don’t break and they always flush and clear. Literally nothing else matters, but to test that you gotta poop in them and that’s hard to reproduce scientifically. So weird substitutes like golf balls and whatnot are used instead.

It’s also an integral problem with new car reviews, because you can’t review the durability of a new car, it’s an unknown. Substitutes end up dinging a car for a body panel alignment as much as a transmission replacement.

> to test that you gotta poop in them and that’s hard to reproduce scientifically. So weird substitutes like golf balls and whatnot are used instead.

Nah, just bring in some large dogs. You'll never be without mighty turds to dispose of.

Beyond that, entire rolls of toilet paper, socks, makeup cassettes, cigarettes, toy cars, bleach wipes, pads, tampons (with and without plastic applicators) and other nonfecal items are the things you should be testing your toilet with, since those are what will go down your pipe in production.

My FIL is the type to insist on buying whichever toilet can suck down the most golf balls. Pooping at his house means never having to sheepishly ask for a plunger. His is like an airplane toilet.

> Consumer reports at least is known to be gamed

I haven't heard of that. Could you tell us what you're referring to?

> and they have specific review methodology you need to understand and interpret

I've ready many Consumer Reports reviews; I don't know what this comment refers to. The style of the graphics in their tables?

Their methodology is simply scientific method.

The one I know best is the toilets, as the “scientific method” has nothing to do with actually using the toilet to eliminate human waste.

And since the testing is known and not realistic, manufacturers can build for the test.

https://seclists.org/politech/2000/Mar/73

You have to understand their methodology because they may be ascribing “points” to things you don’t care about at all - especially if you’re looking for longevity.

> https://seclists.org/politech/2000/Mar/73

The link is a random Internet person's post from 2000 about a single 1995 Consumer Reports review.

> You have to understand their methodology because they may be ascribing “points” to things you don’t care about at all - especially if you’re looking for longevity.

Every review everywhere, unless I hire the reviewer personally, will examine factors I am more and factors I am less interested in. They can't simultaneously, precisely serve everyone's varied interests (ok, I guess they could try some social media feed-type algorithm).

> the “scientific method” has nothing to do with actually using the toilet to eliminate human waste.

That's confusing. The scientific method is to actually do it and report what happens, not talk theoretically or speculatively about it from our basements like the rest of the Internet. CR generally does that (the scientific method) in extensive labs, staffed with experienced scientists and engineers. If you read their toilet reviews, that's what they do - put something in and flush. CR's core product is science, applied to consumer products.

> especially if you’re looking for longevity.

CR looks at longevity for many products, including extensive surveys of product owners.

I've found smaller categories of products with decent reviews on youtube. Project Farm and America's Test Kitchen spring to mind. I'm not sure they're as rigorous, but it's beyond what I'm willing to do at home by myself.
I wish other YouTubers would take note of the information density that Project Farm packs in to his videos. It's really something else.
Similar organizations exist in other countries as well

https://en.wikipedia.org/wiki/Stiftung_Warentest

https://en.wikipedia.org/wiki/International_Consumer_Researc...

I'm subscribed to the former but I found that at least some of my circle of friends and family don't like their reviews because they don't focus on things like "coolness" or aesthetics. Instead they take into account things like safety, labeling and other things.

wow that's a lot. It would be cool if signing up for one would give you access to reviews from the other ones too.
A family member has which.co.uk and I find its marking system wildly inconsistent. A good battery life marked well on several products can arbitrarily lift one of those products to the top for example... and as such I don't read it anymore. And secondly, and this is common across all such review sites and comments, is that you suddenly discover how much bumpf is being written when you happen upon a review for a product or department you own or know well.
The big question is, how would good reviewers get paid? Affiliate relationships are a conflict of interest, but people won’t pay for reviews.
Being a component of a popular media subscription bundle ... like the New York Times. That seems like the right model to me.
Nobody really needs reviews of shitty products. A review site really only needs to write long form reviews of products they recommend due to quality or value. The affiliate stuff then really isn’t that much of a conflict if every product provides equal revenue.
Well yeah but… a lot of the time you don’t know up front that something is shitty, right?
> every product provides equal revenue

Though I’d imagine most places have differing affiliate structures depending on the brand

A lot of people DO pay for reviews. Consumer Reports is an example.

It's also possible to do transparent governance and have firewalls between reviewers and monatization. Few do, but it's a good idea.

Consumer Reports appears to be a shell of its former self though.

I like the firewall idea a lot but modern media are so lacking in credibility that I’m not convinced it would work.

The Consumers' Institute of New Zealand has people sign up for membership, and that funds their reviewing and advocacy. I think that models like that, co-ops, and the like are the only real way to make it work.

As soon as your reviews run on advertising, affiliate links, whatever, you've immediately got pressure to be mindful of revenue streams.

I think part of why I used to trust Wirecutter was all the links (that I saw, at the time) were just Amazon Affiliates - it mitigated the conflict of interest if I knew whichever of the 8 headphones I buy they're getting the same cut from Amazon. Why would they lie about which is best?

But then with per-brand affiliate deals, yeah, I'm much more suspicious.

> Why would they lie about which is best?

If the affiliate cut was a percentage they have an incentive to recommend pricier options (assuming they can't predict the price-volume tradeoff of their readers). If the affiliate is a flat fee per item, they have an incentive to recommend cheaper options, so more people buy. And in either case, they have a disincentive not to recommend a category at all.

None of these are complete trust-killers, I'm just pointing out that getting paid out by Amazon instead of the brands doesn't eliminate incentives. All it does is ensure any incentive is consistent.

I agree. I often wondered if they recommended pricier products before, because some things I bought were expensive.

However, when they just were through amazon, they didn't have to have conversations with the company to get a commission.

What happens if they test something and it's great, and the company making the great product refuses to give them a commission?

If something is a good deal for them, will they not update the review, because the affiliate setup is making a lot of money and they don't want to adjust it?

What happens if they don't like the affiliate system the company's website uses, because it doesn't track cookies well enough? Do they have to call the company after the test and have them setup a specific affiliate system?

It just seems like a terrible idea and there all these conflicts when they are negotiating a side deal with the company they review vs just collecting a consistent cut from a large marketplace like amazon.

I used to think they were good but I think they were only good on paper, even before they were bought by NYT.

I checked out a lot of their top picks and a lot of them had like 2/5 ratings after a few years and it turns out a lot of their picks weren’t really reliable or weren’t that good once you used it a lot.

I feel like Wirecutter just bought a bunch of stuff and compared things superficially.

I only read Wirecutter when Ars Technica would host their occasional article, and came to the same conclusion as you. Any time they covered something that I had experience with, it was painfully obvious that they had barely used the damn thing. If you were in to their house style, they picked things that fit in that sense, but a solid meh for the review side of things.
> I checked out a lot of their top picks and a lot of them had like 2/5 ratings after a few years and it turns out a lot of their picks weren’t really reliable or weren’t that good once you used it a lot.

The other thing that happened more than few times is once the product got reviewed the producer would cut corners and start shipping a vastly inferior product since they knew they could coast on the WC recommendation and increase their profit margin even further. I'm sure some of the corner-cutting was due to increased sales (not necessarily malicious) but a lot of it seemed to come 100% from greed.

They don't seem to get surfaced in Google anymore either. At least, not for me over here in Australia!

Haven't looked at Wirecutter in the longest time!

I dunno, I gave up on Wirecutter for anything I actually cared about almost 10 years ago - I found that for anything that I had actually done even basic research on, the Wirecutter was always wrong (and not just like sort of wrong, but like completely missing the point/out to lunch wrong): https://randomfoo.net/2014/03/22/the-wirecutter-is-always-wr...
I tend to do my own research, often on Reddit or YouTube, rather than trust sites like Wirecutter. I’m often able to find much more individualized information on a wider variety of products.

I think people have gotten smarter and know to treat any listicle with a grain of salt. Although Wirecutter may vet their content better than anyone else, many readers probably assume they’re reading a standard affiliate-link listicle.

If you are doing your own research on Reddit or YouTube do you think you are the first person to think of that? The marketers got to Reddit and YouTube before you did
And your point is I should do what? Of course there are marketers on these channels as well, but they don't dominate.
> they don't dominate

How do you know this? Outside of a few niche interests like mechanical keyboards or PC building, I'm not sure that most of the people on subreddits like vacuumcleaners.reddit.com are huge vacuum cleaner enthusiasts. It's just as likely that the people there have more skin in the game and are either marketers or resellers.

Befriend and talk directly to people who use or service particular items for a living. They have first-hand experience and went through different models. They know their quirks, tradeoffs, workarounds.

It doesn't scale and that's the point. If someone has a scalable audience, the incentives change.

They absolutely dominate - 90%+ of "reviews" on YouTube and Reddit are shilled in some way. They only don't look like marketing because they are explicitly designed to look like legitimate content.
For some products all videos on YouTube are paid promotion. When one company has way more videos over their product than their competitors it's suspicious. The only categories I trust are repair videos. 'I've used this thing for a few years but it stopped working after the kids threw it on the ground, here is how to open it up and oh it was just a loose connector' - they use it so it doesn't suck and it can be easy to repair and I know how to get in
ratings and consumer reports are still okay. Wirecutter has been going downhill since the nyt acquisition
Same approach to reviews as the news it’s all for sale
Quality aside, it's the paywall that grinds my gears.

Wirecutter articles are essentially long, well-researched ads for the products they (affiliate) link to.

I always found these ads useful, at least as a starting point. But putting a paywall in front of the ads rubs me the wrong way, like an unwritten contract was broken.

follow the money. as long as the incentives are to make us buy this product, or that one, this will inherently be biased at some point. I've yet to see a recs website truly aligned with our long-term satisfaction. A few subreddits might be the exception here; but Reddit's model itself causes other type of challenges long-term. we're currently trying with https://objet.cc
I've always wanted a site that--instead of telling which product was the best--told me how to look for quality in the category I'm interested in. What does good look like? What makes a good quality water hose? What are the unsung features and considerations for drying hooks? What does quality look like in jeans? Am I paying for the brand or an actual meticulous craft?

wirecutter might have been that at one point, but I find it doesn't fill that gap for me. As I've gotten older, I'd rather have a few quality things, rather than a whole bunch of good enough things that I'd throw away next year.

The current chat models will attempt to help you do that. Might be sufficient?
Of course they will, they’ll answer whatever you ask. How do you know if what they’re saying is true?
You mean combine sort of related information from all over the internet into new statistically generated phrases that appear to answer the question, but still contain all kinds of hallucinations, fabrications, and wrong statements?

Then yes, they do.

There are some products like that, but the purpose of many of these review articles is to tell you things you can't see for yourself unless you have a bunch of lab equipment and lots of time.

Or in the case of Consumer Reports reliability ratings, a bunch of people you can survey.

I'd say that consumer reports is my.most trustworthy option for the categories it covers.
Yes, although Wirecutter is frequently one of the inputs into my buying decision, I find the one-size-fits-all approach annoying at times--this is the best power strip, because we believe everyone needs at least eight outlets, and twelve is better, but more than twelve is too many. Well maybe I think anything with more than six outlets is too big, or maybe a right-angle plug is my most important criterion.

I can sometimes pick through "The competition" and find something that is slightly less awesome or slightly pricier, but that section reads like you'd be dumb to buy this model instead of our top pick, which also comes in purple.

Your approach to ask specific questions is a good one. I've found that the key to avoiding persuasion by reviewers with undisclosed conflicts of interest—which includes Reddit comments made by people who bought accounts with a posting history to promote a product, and even YouTube video creators who do not disclose their sponsorships—is to pay close attention to the specific details of reviews.

Paid reviews often have lots of emotional arguments (such as "I've been missing out," "I can't imagine life without it," or "I LOVE the one I have") without substance. Even if some of these comments are from people who are genuinely happy, the comments are still uninformative and don't help you understand the strengths and weaknesses of a product, versus alternatives.

More detailed reviews—especially those that include fairly-written comparisons with compelling alternatives—give you the information to think critically and make your own judgement on whether a product or service is worth purchasing.

For product research, I've also found it useful to seek out negative reports or complaints about products breaking over time. That can help you factor in the long-term costs of owning something new: you can learn that you may need to purchase replacement parts for a product over time (adding a recurring maintenance cost), or that certain products cannot be repaired, and will likely require a complete replacement after a few years.

It’s obviously only for the product categories that they sell, but REI’s expert advice posts are close to this. For example, here is their post on downhill skis: https://www.rei.com/learn/expert-advice/downhill-skis.html

They explain all of the parameters that matter when making a decision on what to buy. It’s left to you to evaluate individual products and brands along those parameters for your preferences and needs.

I may not buy everything from REI, but I often do, and these posts are a big reason why.

Tangential, but about REI and their "real enthusiast" nature....

I've been an REI member for.....many many years. I stop by every time I visit the whites in NH, so probs 6-10 times a year.

I have not purchased a single item in years. costs are out of control. "glamour hiking/camping" customers are making REI more off of camping meals and north face t shirts than they ever have off survivalist classes or good quality hiking gear from france.

REI branded "hiking pants" are 100 USD. Made in china, lowest quality possible. My 7 dollar puma running pants are exactly the same and saw me hike through ireland just fine.

There's no reason to buy things at full price there, especially clothes. Between the Outlet, regular sales, and garage sales, there's often deeply discounted gear and out of season clothing available.

If you want really affordable stuff, Walmart and Big 5 offer a lot of cheap outdoor gear that will do the job. REI caters to a different segment, mostly the mid high end. In between, Dick's etc. offer midrange stuff.

Some of the manufacturers (like Patagonia) offer used clothes directly too. But like you said, hiking clothes are often overpriced and any generic synthetic would likely do fine.

Personally I get my gear at REI because I want it to last (and they have a great return policy if it doesn't) but my clothes at Costco or used, and my consumables (backpacking food etc.) direct from manufacturers in small boxes.

Many clothing brands sell various quality of clothing at various price points using the same brand name. For example, I am currently wearing $25 Puma shoes from Costco, but I doubt they sell those on their official website, which is probably just their highest priced stuff.

“Outlet” malls in the US are also dumping grounds for lower priced and probably lower quality goods. They usually put a bigger, uglier logo and have wider fit so it does not compete with their higher priced products.

>REI branded "hiking pants" are 100 USD. Made in china, lowest quality possible. My 7 dollar puma running pants are exactly the same and saw me hike through ireland just fine.

False. A lot of REI brand stuff is made by the same factories that make stuff for other high-end outdoor brands. They still tend to use higher-quality fabrics and construction. They tend to use things like YKK zippers that stand up to years of use. They’ll typically have bar tacks and double stitching in high-stress areas so they won’t fail when you actually use them. They also tend to get really positive reviews on most of their REI-brand items, and their warranty is great. Lastly, there are usually great sales on their REI-branded stuff, so it’s not hard to get 30-50% off the already-lower-than-other-brands price. Yes, they do make some hiking pants that are $100 right now, but they also have others that are $50 and $60.

Do you need high-end hiking-specific pants to hike? No, but I find that they can be much more comfortable and sweat-wicking + fast-drying, in addition to lightweight yet durable with plenty of useful pockets and construction that lasts years.

I find that many folks who make comments like this are the same folks complaining that nothing is made in the US anymore and nothing lasts anymore. Both are untrue, it’s just that the bottom end of the market has proven much more popular with consumers, and many more companies have made ever cheaper and lower-quality products available to satisfy this demand.

A positive take would be that it’s great how we have so much variety available in quality and price points for any given product today. A more negative view is that we’ve made the good stuff even more expensive as now it’s they’re niche luxury items instead of just having the common versions be decent quality because people keep buying the cheapest possible junk regardless of quality because they’re not intelligent enough to understand or appreciate the difference nor the global economics which affect manufacture and distribution of these goods.

I'll ignore the assumptions you made about me as a person and take it as us just not knowing each other.

I'm not exaggerating, and my point is that I don't need high end gear. I want the equivilent of Quechua, an excellent, affordable, mid quality outdoor gear supplier in EU (you can find in surplus stores here occasionally). I have an 18 dollar raincoat from them that I have summited Washington with several times over 8 years now years, that I picked up in ireland after literally forgetting my raincoat.

I cannot walk out of REI with a raincoat that is "lightweight and breathable" (meaning, a cheap shell) for less than 110 dollars.

Would you like me to take a photo of the YKK zipper on my Quechua raincoat that again cost 18 dollars? In fact, I can probably guarantee you, to your point, that REI sources from the same manufacturers as Quechua, at 10x the cost.

I have merino wool socks made for lineme/winter laborers that cost, no joke, 7 dollars for 4 pairs. They absolutely blow REIs out of the water, not even considering cost.They are a no name brand and I got them from the TJ max at the base of a mountain in new hampshire.

Trust me, I want to think like you do, but REI is now an "experiences" company, not a company interested in furthering the middle class's ventures into the great outdoors. I say this as someone who chose to work there in my off time as a "hobby" job before the real sh*t hit the fan when they stopped sending actual checks for the co-op payments.

Thank you for this prime example of shilling behavior.
Responding specifically to the point on glamour hiking/camping:

REI definitely puts a huge focus on growing the outdoors activities market by being attractive to newcomers. And a large part of that is offering fashionable clothes and convenience options, and marketing hiking as a social-media-friendly lifestyle.

On balance, I think this is a good thing for the future of outdoors sports. The downside is an increase in unprepared hikers (dangerous) and overuse of easily accessible trails (unpleasant). But there’s a lot of pluses:

1. I can more easily convince my friends and coworkers from the city to go hiking with me.

2. I’m confident that the parks and trails that are popular and widely used will continue to exist for future generations.

3. This is a subtle point, but I think it’s very important: if you are someone whose parents never took you hiking/camping, and you don’t know how at all, then the combination of a) hiking being fashionable/cool and b) easy to learn about and buy gear for is critical to convince you to try it. Historically, camping in the US has mostly been a white activity. I am white, and to be clear there’s nothing wrong with that, but I like that many of my non-white friends whose parents never took them camping as kids are now interested in hiking and camping.

(I personally think the camping meals and $100 pants are silly, but that’s consumer capitalism for ya. It is what it is)

I’ll also point out amateur radio as a counterpoint, since that’s another activity that I have tried where the story looks very different:

1. There’s no way I could ever convince my friends to try it.

2. There is a very real chance that it will disappear in a generation under pressure by corporate interests who would like to utilize the spectrum.

3. There’s barely any young people joining the community for a variety of reasons.

Amateur radio is a dying activity. I’m grateful that the future of hiking and camping is much brighter, because I can imagine a world where it was also in decline.

https://www.outdoorgearlab.com/ and https://www.rtings.com/ are two more awesome review sites for outdoor gear and electronics, specifically TVs and monitors but now also things like cameras and headphones.

I love that REI largely doesn’t sell any junk — even the cheapest thing in a category that they sell probably still works fine. Backcountry and MooseJaw are similar. We’re lucky to have so many good outdoor brands in the US.

The same cannot be said for home goods and things like appliances. Most of the market is disposable single-use junk. Reddit has r/BIFL (“Buy It For Life”) but there are some things, eg electric toothbrushes, where the best-functioning brand (Sonicare) do things like plastic weld their handles shut, so when the li-ion cell in there dies after 2 years, you’re meant to buy a new handle. Replacement battery kits can be found, though typically they use low-quality cells, require soldering, and have a high likelihood of damaging the case and waterproof seal due to the design of the thing. This is just one example, but I’ve found that more likely than not, “home electronics” fall into this category of unfortunately low-quality and hard-to-repair, so I try to avoid them entirely wherever I can.

Still other good are problematic to shop for and own. Trying to buy any durable goods (and even consumables) on Amazon these days requires a good deal of research. It’s pretty obvious which brands are the “WEKAPO” or “POPCHOSE” or other combination of gibberish (see https://youtu.be/4UrqlMfwUC4 on why), but there’s just SO MANY of these products flooding Amazon that it feels like digging through racks in a thrift shop or something. I wish they had a filter for something like “established brands only” but I know that a ton of their business these days is selling these low-quality products that are typically copies of other products.

The most frustrating result of all the SHNITPWR and VANLEONET for me is probably the impact on rwview sites. If you Google things like “best SUP 2023” you’ll get endless “review” sites that are just Amazon affiliate links with articles that are reworded (likely automated) product descriptions or pathetic paragraphs from pay-per-word authors that have obviously never used any of the products, showing you what amounts to 10 random products in a category from whatever from junk brands. I hope Google finds a way to PageRank these into the ground, and I wish Amazon would likewise do something to combat this behavior, as it’s making the modern shopping experience and the internet in general pretty awful.

rtings.com does a good job of rating things according to use case, like a monitor might be rated of ‘Office’ or ‘Gaming’, and they detail how that scoring works too.

Notebookcheck.com is the only computer review site I’ll go to, simply because of their attention to detail and consistency around things like display specs. I just wish they’d make that data easier to consume.

It’s what annoys me of German darling test.de (Stiftung Warentest, they were set up as a trust by the government to avoid any kind of outside influence, customers pay for their data), their reviews are complete black boxes, no idea why anything was highly rated and there is no real background.

I far prefer ETM-Testmagazin.de, they are a for-profit company and also sell their tests, but they actually give you all their testing & measurement data while still giving recommendations and textual reviews. Maybe they are scummy, and I can’t tell, but considering that I find stiftung warentest mostly useless, they are still far superior.

> their reviews are complete black boxes

They are certainly not "complete black boxes". This page [0], for example, explains in detail their test procedure for vacuum cleaners.

Just an excerpt on durability testing (via deepl):

> Impact test (500 cycles): The hoovers drive over thresholds 10 000 times and bump into posts 1 000 times.

> Drop test: The suction nozzle must survive 1 200 drops from a height of 80 centimetres.

> Cable pull-out test: 6 000 pulls.

> Test of hose attachments: 40 000 swings with the connection nozzle clamped.

> Squeezing the pipes, hoses and auxiliary air adjusters: The testers load them with about 70 kilograms for ten seconds.

[0]: https://www.test.de/Staubsauger-im-Test-1838262-1838266/

That is one of the better ones, most (or at least anything I ever looked at) have far shorter and less detailed "so testen wir" sections. But I guess complete blackbox is incorrect. I’d still want actual data in most cases.
I would presume they are doing it on purpose, so that industry doesn't optimize for their tests when they develop their next product.
Most major product categories have their own niche subreddit and many are excellent for that kind of advice. Here are some I recommend: r/VacuumCleaners, r/OfficeChairs, r/rawdenim/, r/RunningShoeGeeks, r/goodyearwelt, r/Monitors.
Marketers have been aware of this for years and savvy brands have invested heavily in astroturfing enthusiast forums with legitimate-looking content promoting their products. That coupled with weird group-think dynamics leading to cult opinions that aren't well grounded has made the "look it up on reddit" path a lot less reliable in the past 5 years.
> I've always wanted a site that--instead of telling which product was the best--told me how to look for quality in the category I'm interested in. What does good look like? What makes a good quality water hose? What are the unsung features and considerations for drying hooks? What does quality look like in jeans? Am I paying for the brand or an actual meticulous craft?

This would have been very useful for me, even back when Wirecutter was the best option available.

Living in Canada means that I have reduced access to specific models of items; Wirecutter might recommend something great that I can buy on Amazon, or it might recommend something great that literally no retailer in Canada sells, or which costs twice as much in Canada as it does in the US for some reason. Sometimes I was able to grab the runner-up, or they would mention in the review that their pick, the DDW1345-B was 'nearly identical to the DDW1344-A, but with three extra HDMI ports but the same tungsten-carbide drill bits', so I could pick up the DDW1344-A knowing that if I didn't need those ports it was essentially the same as a recommendation.

It was endlessly frustrating, though, when they would recommend exactly what I was looking for (a combination air purifier/humidifier/oscillating fan!) for a reasonable price ($200!) which I either couldn't get or couldn't justify the Canadian price of ($450 plus $88 shipping, arrives in three months). I ended up finding other sites which offered perhaps less rigorous testing but would recommend things I could actually get (shoutout to RTings for even more in-depth testing of home theatre equipment and also being located in Canada).

> I'd rather have a few quality things, rather than a whole bunch of good enough things that I'd throw away next year.

This reminds me of a girl I once dated. She kept criticizing me for spending $8.99 on a water pitcher when I could get one at the dollar store "for like a buck" and "just replace it every year". It blew my mind. Why do I want to go shopping for a new water pitcher every year when I can just buy one and use it for way longer than our relationship is going to last?

With respect to a subscription bundle being the right model: probably so.

But contrast Wirecutter with NYT restaurant reviews. Or movie or theater reviews. Those have permanent staffs, who are free to establish their own brands, e.g. Ruth Reichl, A.O. Scott. If those reviews are coopted by the review subjects, I'm not aware of it.

On the other hand restaurant/movie critics have a vertical. If I want a ukulele I prefer a different critic that the one reviewing toasters, this is not possible with permanent staffs, unless the team is quite big or expert in everything.
That's true, but the only way to cover everything is to be mediocre at all of it. Or corrupt.

So they can have a few people who are expert in broad areas, like Walt Mosberg. No ukuleles, no toasters.

Long-term reliability is not a consideration for a restaurant meal (even if consistency is for the restaurant itself). Given how products churn and the practice of planned obsolescence, by the time you have the long-term durability results, the product is probably no longer available for sale.
This is one of Consumer Reports' advantages. They regularly poll their readers for repair data to get a feel for how well things last, and use that as a factor in their recs on new products.
Yes, the Lindy Effect is a valuable heuristic. But private equity firms love to buy a company with a sterling reputation, then run it into the ground while milking it for profits and saddling it with debts that will never be repaid, so you need to watch out for changes in ownership or management like a hawk.
Wirecutter rarely shows up any more on DDG or Google’s first page for results.

In fact, I don’t remember when I last was directed to WC via DDG at all.

According to Ahrefs, Wirecutter is on page 1 (Google) for 756k keywords, and position 1 for 95k keywords, so it's probably just not appearing for what you search for. While the quality has declined, the latest update from detailed.com suggests Wirecutter is making up to $38M/qtr (revenue not profit) for NYT, not a bad return for the $30M they paid for the business 7 years ago.
Maybe the search results are region specific?

I searched just now for “wireless ANC headphones” and the first hit is tomsguide plus a bunch of Indian shopping sites including Amazon India.

No NYT or WC links on the first page. I came across WC originally via DDG.

They definitely can be region specific. I never get Wirecutter, in fact I don't think I've ever visited the site. What I do tend to get is tomsguide, tweakers.net (which is local), and such.

Though interestingly I just tried your search and I get CNET, what hi-fi, and so on. Oddly google search is insisting in talking to me in English, which it doesn't usually do, so that's probably why I'm not getting local language results. If I write the search in Dutch then I get more results in that, not surprisingly. Typically I get a mix, so I dunno if something has changed on google's end or what.

Still no Wirecutter though.

Typically a query like that will mainly return e-commerce sites. If you prefix with “best” or another superlative, or include any comparison-type words, those are the phrases affiliate sites such as Wirecutter optimize for.
A lot of the reasons for decline cited in the article feel fluffy. "The best washer/dryer depends on who you are"? The Wirecutter always gave alternatives and "upgrade picks". "People prefer influencers"? Are there washer/dryer and toaster influencers now?

What killed the Wirecutter for me was the standing desk referral code scandal, full stop. Their whole thing was "one stop for the best thing", and if there's any hint of bias there, it kills that. I believe that was after the NYT acquisition--I think it's very fair to lay the blame at their feet.

> Are there washer/dryer and toaster influencers now?

Possibly. It might depend on your definition of 'influencer', too.

Vacuum Wars[1] seems to do pretty comprehensive reviews of vacuums.

Watching Project Farm[2] is also interesting, although it's often bulk testing a whole lot of one type of product at once, and is very fast paced.

[1] https://www.youtube.com/@VacuumWars [2] https://www.youtube.com/@ProjectFarm

Project farm is great never been disappointed with his reviews
Reviews?

Feels like it is just, here, tested it once for 2 mins doing one or maybe two things. At best it gives you an idea of what to avoid but I've never come out of one of those videos knowing what I'd actually want.

It also feels like the videos could be a courter in length if they just included a table with product data.

While the reviews can sometimes be superficial, there’s also often cases with a clear winner, or a winner without paying 3x the price. I ended up following his recommendation on automotive scratch remover compound, for example, and was happy with the results.
They do include that I always skip to the summary at the end then back track to any tests I care about
My confidence was recently shook by their garden hose review. I own of the highly rated hoses that he says had zero kinks. Mine kinks 100% of the time. Multiple times in one session.
What I like about Project Farm is he often doesn’t make an absolute recommendation. The way he tests is really useful in identifying the products NOT to buy based on specific use cases.
> What killed the Wirecutter for me was the standing desk referral code scandal, full stop

They responded and pointed out that Nextdesk was wrong, but if you've ever worked for any journal or know someone who has, they will all tell you that the reputable ones, like NYT, have a complete separation between income and reporting. Literally the only common link between them is the CEO.

I don't know the truth of the matter, since there's no elaboration or confirmation, but the article also says:

> A former staffer alleged that in 2019, an employee on the Times business side changed the copy of a post in the Money vertical without telling the editorial team—a major ethical breach in an industry where the separation of Church and state, so to speak, is sacrosanct.

But note that it also says said employee was sacked because of it.
For those paying attention, this is the NYT model. It is not dissimilar from Forbes, though typically more subtle. Similar to Politico.
> Are there washer/dryer and toaster influencers now?

Oh yes. I spend too much time on the social medias and am also a middle-aged white lady, so I'm in the demographic that gets pushed a bunch of domestic and parenting content despite my lack of home and children.

The washer/dryer guys are usually either appliance salesmen or appliance repairmen, and the toaster/small kitchen appliance people are usually momfluencers.

They’re out of date now but John Siracusa’s toaster reviews on the ATP podcast were amazing content.
While I don't recommend blindly buying the top recommendation, I still find Wirecutter reviews useful. Sometimes I ultimately choose a cheaper option they rejected outright or something I can get used. I also consult friends more than I used to. But even in those cases, the article typically doesn't feel like a waste of time.

I do think Wirecutter articles no longer go the extra mile quite like they used to or involve as many out-of-the-box ideas for testing. But I think they remain thorough enough to be useful.

I stopped trusting wire cutter because of their air filter reviews.

They reviewed Molekule air filters as if they were traditional hepa air filters. They didn’t test their ability to remove VOC. They didn’t have the ability to to test that and they didn’t even try to.

I’d be thrilled if they did because no one has tested that outside of labs paid for by Molekule, but writing Molekule tech off as bad for not performing like a traditional hepa filter… when they don’t try to compete in that space…is only really helpful to themselves (engagement) and the traditional air filter industry. When the incentive is to move Amazon affiliate products, it’s not aligned with the average person.

Molekule didn't only advertise their products as VOC filters though. They advertised them as filtering particulates, which they apparently didn't do.
Dynomight was fairly critical of Wirecutter’s coverage of air purifiers as well (specifically their negative review of the inexpensive and decent IKEA air purifier).

I used to love Wirecutter, but it seems to have fallen prey to the inexorable enshittification of the web.

https://dynomight.net/ikea-purifier/

I still think Wirecutter is good for getting a basic understanding of a product category and major brands within it. Often I will go directly to those brands' websites to look at the rest of their products, and to other review sites to see how they compare though.
How do you guys find the "best" stuff? I find it impossible to use product reviews on places like Amazon. Otherwise I rely on familiarity with the category, but am unfamiliar with more product categories than I am familiar.
If I’m not familiar enough with a category to evaluate it, I figure I don’t actually need “the best.”

If my demands on a product will be average, I figure “good enough” is a good enough level of quality for me.

And if it is not, my direct experience of what I didn’t like about the first product will make me more informed and opinionated about buying the next one.

I start by opening a few tabs with various search queries, some of which are usually scoped to Reddit or HN. That usually kicks open a few more tabs, and maybe some dedicated forums or Shopify stores.

Then the hyperfocus takes me, and I stray out of thought and time, and I wander far through hundreds of tabs. I become an expert on the multi-decade rivalries between vacuum cleaner manufacturers, the nuanced multi-dimensional calculus of coffee grinders, and the intricate supply chain dynamics of family-owned towel-makers.

And then I throw down my credit card, the stars wheel overhead, and every day feels as long as a life age of the Earth... until the UPS driver arrives. At last, I have my hard-earned morsel of capitalism.

You look at the shape of the review star distribution histogram, and you read the 1- and 2-star reviews. But yes, Amazon reviews are so games and rife with fraud as to be unusable. Influencers on Reddit or YouTube are on the take for payola.

I suspect what most people do is ask their nerdy friend who seems to be oddly passionate about scissors or cameras (guilty as charged).

It seems like somebody else could probably step up and launch a better version of Wirecutter, following the same format. I had no problem with Wirecutter before NYT bought them, even though affiliate links were their business model. It was understood that their real product was trust, and if they traded that for a quick buck, their days would be numbered (they did; they are).

I assume what would happen is that such a review site be good for a while, and then they'd either decide to scale up, or get bought by someone. At that point, they'd turn evil, and there'd be chatter about it, and I'd start going somewhere else. Eventually, history would produce that rarest of individual, someone who was smart enough to run a lifestyle business reviewing things honestly and not turning greedy, which would put us in a steady state.

> It seems like somebody else could probably step up and launch a better version of Wirecutter, following the same format.

With what revenue model?

Maybe I didn't make it clear enough: affiliate links.
A lot of the comments are asserting outright fraud, which certainly exists in the review community, but my problem has always been that the reviewers don't know what they're reviewing.

Many years ago, I wanted to get a 3D printer, and someone said "just get what Wirecutter tells you to get". I did that and had my printer the next day.

It was TERRIBLE. It didn't take g-code as input, you had to put your model into their slicer, which required registering with their company. (I'll let you guess what language the Terms of Service I agreed to were in.) Their custom software was slow and buggy. The printer itself is designed in a way that the bed can never be level, and so their software requires you to manually level the bed with a piece of paper EVERY SINGLE TIME IT CAN'T SAVE IT OR LET YOU TYPE IT IN YOURSELF, and then it prints a raft that is level, then it prints your thing on top of the raft. Print a 1mm by 1mm cube? 120mm x 120mm raft. Just awful. Did I mention that it's literally incapable of printing PLA or PETG, and just barely eeks out an ABS print before the nozzle clogs because of a horrifyingly bad heatbreak design?

I put it in the box that night and resigned myself to doing my own research. (I ended up with a Prusa i3, which has served me nobly for many years without any problems at all. No, we didn't have Vorons or X1 Carbons back then.)

And no, I didn't know what PLA, PETG, ABS, heat creep or clogs were when I got that printer. That's the years of experience I've accumulated coming into play, explaining why that printer should have never been recommended to anyone for any reason. To be a reviewer, you have to know about that stuff, or you're just misleading people. That's where the Wirecutter failed. They made a cube in the included software. A cube-like thing popped out an hour later. "It's perfect and so easy!" they proclaim. That's not a review, that's "well it didn't blow up the first time I turned it on". Better than nothing but a colossal waste of my time. That was the first and last thing I bought on Wirecutter's recommendation.

(As an aside, I returned the printer but UPS never showed up to pick up the package, so I started at it sadly for many years. One day I decided to rebuild it with an open-source mainboard and a Hemera hotend. Worked muuuuuuuuch better after that. In fact, I love the little thing. So really, it was just cost-cutting on the hotend and "we're going to make our own software instead of just adding our logo to Slic3r" that killed it. Though honestly, the bed design is quite terrible and I haven't been able to fix it. I added auto-leveling, though, so it's bearable to use.)

Making my long comment even longer, I have to say I kind of like Consumer Reports. I bought a subscription a while ago and it was good for building out my new apartment (with working kitchen, a first for me in a decade). I specifically liked that I really wanted to buy a stand mixer, but they basically say they're all bad. KitchenAid apparently replaced all the metal parts with plastic ones, and so while your Mom's Kitchenaid mixer is great, the one you buy on Amazon today is going to break in 2 years. Consumer reports basically said that, and I did a crazy thing in this day and age: I went without! Someday someone will make one that's not a piece of junk, and I'll buy it.

This is an alternative model for quality reviews that's lasted about 60 odd years so far.

www.choice.com.au/about-us

If you can read German, I have come to really appreciate Stiftung Warentest, see test.de. [1]

They conduct rigorous product reviews and try to simulate wear and tear usage of the products. To this end, some reviews take them months. They also come up with their own test suites that are tailored to the product that they are testing.

Even though it is quite costly (sometimes I pay 5€ for a review), I have started to rely on their ratings more and more, as they are completely independent and usually not far off with the end result.

As it stands, currently, their review quality beats internet reviews.

[1] https://en.m.wikipedia.org/wiki/Stiftung_Warentest

When I was in the market for a new washing machine I tried to purchase a review, but the online system wouldn't let me, because I'm not a German resident.

It may be because the foundation recives German public money.

They are a non-profit foundation which can't be bought up, have been around since 1964, don't run ads in any way and buy every item they review in a retail setting.

It's a very nice set-up for high quality results and makes it hard for them to be corrupted.

The closest U.S. equivalent is probably consumer reports

In the UK the Consumer's Association and Which? fulfil the same role, and has been going since the 1950s. Before the internet the magazine was a part of everyday life for many - if my parents were going to buy something like a new washing machine my mother would go to the library and find the relevant issue. Online you can see some of their reviews but I think you need to subscribe to get all the (very rigorous) detail. The CA is a charity and has a sort of NGO status, able to issue a "super-complaint" if some industry is not working for consumers. I suppose I'm a lot less careful with money than my parents were, I never use it.
I'm a fan as well. Their review process can be a bit opaque and sometimes feels like "5 of our people fucked around with these things for a week and here's what they think".

This would be a problem if there was any hint of bias, but (miraculously) there hasn't been so far.

I do wonder whether any of their (anonymous, AFAIK) reviewers have been personally corrupt, even if the organization may not be.

The problem with Wirecutter (eventually - years ago I used it fairly regularly) was that the reviews could not compete with Reddit/Youtube/etc. It worked if I just wanted something generic and half decent, and didn't care too much about the price, but twenty minutes of searching through comment sections usually yielded better and more specific results.

This makes sense - it would be impossibly expensive to pay a team of professional writers and editors to test products in the number and depth of "crowd-sourced" reviews. Wirecutter provided trust and convenience but, for me, not enough to overcome that fundamental disadvantage.

Same Reddit is often where I do research into a product. Just spend a few minutes or even hours reading people talk about products.
I was a subscriber to choice here in Australia. I think the model of paying a subscription was better than the site having to rely on advertising.

I didn't renew though because I just didn't agree with the reviewers on products I knew a lot about, so didn't trust the rest of the reviews.

I also think it is a big oversight not to consider warranty terms when reviewing products. Some produces have standard 1 or 2 year warranty competing with products with "lifetime" warranty. How can that not play a huge factor?

They bizarrely don't review a lot of products that should be included in their reviews, or they quickly dismiss products that are classics. This creates blind spots and makes me not trust anything they write.

Examples:

- High/school college backpacks, but they don't review any North Face backpacks.

- White sneakers, but they've strangely said that the classic Adidas Stan Smiths are bad, despite these shoes being probably the most classic white leather sneakers out there, with 4.5+ star ratings on pretty much every shopping website.

- Best mattresses for 2023, where "Ikea" is mentioned only 4 times in the text, despite an Ikea mattress being a top budget pick. Ikea is probably one of the most popular places to buy mattresses, and especially if an Ikea mattress is a top pick, shouldn't there be much more attention to the brand?

Yeah, the glaring omissions in deciding what to test was the primary way that Wirecutter started its journey to "enshittification". Today I use "The Strategist", Project Farm, America's Test Kitchen, Outdoor Gear Lab, and, uhh, Phallophile Reviews (NSFW).
Another example: under electric razors they barely even test the Panasonic Arc 5 despite sites like Shavercheck praising that one. That review in general is laughable because it used to say that they couldn't get the majority of their testers to agree on a single shaver, but somehow they're recommending one as the "best for most people" anyways?
They seem to be looking at the bag with shoulder straps that is what most of us carried in school. Other than Patagonia I see no outdoor brands in their list--probably because what makes an optimal pack for carrying isn't what makes an optimal pack for school. I consider my Osprey packs (I have a range of sizes) vastly superior at carrying vs anything on that list--but none of them would be practical to toss under my desk in class.
I bought Sennheiser ear buds due to a Wirecutter recommendation, twice. They broke super quickly, which is no surprise since the wires were flimsy as hell.

No more Sennheiser and no more Wirecutter for me.