Generally I've found them to be pretty good with the reviews. The testing is thorough and they explain what to look for when buying something (I've used it for appliances).
Its subscription only (they don't do ads), which means they're not beholden to the companies they're reviewing for money (a good thing)
They generally make a grid of the items you're looking at with dots indicating the ratings. Its very easy to nagivate and intuitive.
You'll have to pay for it (or find someone to give you their old copies, or many libraries have subscriptions).
I've used them for baby equipment, for mattresses, and for cars. They were spot on with cars, baby car seats and cribs, but their mattress ratings could have been better at the time. Overall, I've found them to be pretty high quality, and most importantly, independent. Their reviews will often differ from more public sources. And I could be mistaken, but I think they don't even accept free products from companies. They buy the products they test.
In general, they do well, but there are a few categories which they don't seem to "get". They do a good job of trying to remove a lot of barriers to objectivity, even when useful reviews require subjective knowledge. They also try hard to find equivalent, reproducible tests so that they can use them on multiple products, even if they're not exact equivalents. This means, for instance, that they might end up testing ice-braking tires by measuring the stop distance in an indoor ice rink where they can control the conditions... that makes it easy to measure the tires against other tires, but it's not real-world. They also purchase their products, rather than accepting them from the companies, which helps increase the odds that they're getting the same things off the shelf you do. Again, they seem to be clueless about a few categories, though, such as certain electronics. Just look at the ratings for the Google Home and Amazon Alexa--very poor--because they get lumped in with wireless speakers where sound quality is the utmost importance.
Consumer Reports is pretty good for household products that they have expertise in. For example, I'd trust their dishwasher or toaster recommendation. However, when they test something that's outside of their sphere of expertise, they can miss the mark by failing to focus on criteria that are known to be important by real experts. For example, from time to time they'll review bicycles. Anyone who really knows about bikes will be amused/exasperated by the aspects of the bikes being tested that they make their judgement on. They've been testing computers for long enough that I'm sure they're not completely clueless, but I think their judgement criteria are based more on a casual computer-as-appliance home user, and a power user might feel that they focus on superficial things.
I found them useless for high-end dishwashers (both Mieles they recommended worked poorly), audio equipment, and high-end vacuums. I specify high-end because I am very willing to pay top dollar for appliances that will last many years, not because I'm a snob.
>> I found them useless for high-end dishwashers (both Mieles they recommended worked poorly)
That seems to apply to all of their highly rated dishwashers -- when you read the CR user comments, most of them got a lot of horrible user ratings.
I don't know if it's the dishwasher segment, but it seems like dishwashers in general have gone downhill.
I was also going to buy a higher end dishwasher but it was so hard to choose because of the negative user comments that I ended up getting a lower priced one that got decent reliability ratings. I didn't want to get a Bosch or Miele after reading the user reviews on CR.
Having said all of that, if you've been using dishwashers since the 80s, it turns out your old habits can result in bad washes. I was used to scraping my dishes clean, but according to my research, you actually need your dishes to be soiled enough so the newer detergent pods can clean them properly. I was lucky I found that out or I'd be getting inconsistent wash results.
Also, it appears newer ones have less effective drying hardware which forces them to rely on "rinse aid," so your choice is either still-wet dishes or dishes with residue.
I guess this really just confirms what I've suspected: the only way to get decent battery life out of a Mac is if you are literally using it for nothing but simple web browsing or watching a movie and doing nothing else.
Non-Apple apps are death for battery life. I think I'd be lucky to get two hours of productive development time on my laptops. 1.5 hours is probably closer to reality. I have two BatteryBox devices that add about 45 minutes to 1 hour each to runtime and it's impossible to get productive work done without them.
Perhaps Consumer Reports should try testing battery life by running other apps?
No, as I've been in the Apple ecosystem for about 10 years and haven't shopped around. I used to have a Dell beast of a laptop back then that was 1) very heavy, 2) very hot and 3) would last for hours of development time.
I would actually trade whatever weight would be necessary to get my useful portable development time up to 3-4 hours.
These days I think you'll find that laptops on the thinner and lighter side rely heavily on CPU throttling and other power saving features, as well as efficient software, to achieve good battery life.
You may need to go the "brick" route if you have a heavy workload, whether or not that's due to inefficient software. I've found these days that a lot of software is simply poorly written and drains much more power than it should (the Atom editor is a good example).
Those Razer beasts are exactly what I was thinking. Probably anything with a primary use case of running a discrete GPU on battery power will be a good bet for heavy CPU workloads too.
Sidenote, detachable batteries are also useful in that the overall service lifetime of a battery can easily be shorter than that of the rest of the laptop.
It also means the battery can't have a funky custom-designed shape that devices with integrated batteries can, meaning the battery has a lower capacity. That's a really big reason not to use detachable batteries.
> It also means the battery can't have a funky custom-designed shape that devices with integrated batteries can
... so you're saying that designing for a detachable battery forces a certain finesse when designing the rest of the internals, so that there is not any funky-shaped gap big enough to put an entire LiPo cell.
Also, one can easily make an "extended" model of replacement battery, which protrudes rearward (or downward[1]) of the rest of the case.
It isn't good due to magic or some other vaporous marketing buzzwords. It is good because you can just install big batteries and swap them out in sleep mode.
Get one or more 6 cell 72Wh rear batteries (6,666 mah). Alternatively get one or more Lenovo Power Bank 10,000 mAh (100 Wh give or take). The advantage of the Power Bank is that you don't even have to put the computer to sleep.
One extra 6 cell 72 Wh battery is a lot less heavy than you're imagining (approx 1/2 pound). I'd even go as far as to say that your Macbook's dongles weigh more than doubling the battery life of the Thinkpad T460p.
HP zBooks have 90Wh batteries and you can configure them with a 1080p display and no discrete graphics (just make sure you do spring the $50 for the UWVA; that's HP code for "not a crappy TN panel"); if you have no need for GPGPU and don't go crazy with the CPU/RAM you put in then you can get quite good battery life out of them.
You used to be able to get laptops with batteries that fit in the optical drive-bay so you could expand the battery life, but that appears to be a non-option now that internal optical drives have gone the way of the dodo.
Until the most recent batch, Apple laptops have just had straight up the largest batteries of everything except Thinkpads with batteries that stick out. E.g. the much touted "bigger battery" of the new HP 15" x360 only brings it even with the new Touchbar MBP, which has a 25% smaller battery than the previous MBP. Samsung's current 15" laptop has a battery that's about half the size of what is in the new MBP.
Non-Apple apps don't kill battery life. What kills battery life is shitty apps like Chrome (or anything based on Electron). VSCode for example will cut battery life to 6 hours versus 10+ for Emacs. Just simple stuff like typing and scrolling takes huge amounts of CPU. Even just sitting idle it's using 1% or so CPU.
I put my MacBook Pros through a grueling battery test (basically build and destroy a VM successively, sustaining 100%+ CPU, tons of disk I/O, and maxing out WiFi bandwidth intermittently) for my blog post elsewhere on HN.
Worst-case, the 13" Pro with Touch Bar gets 3.5 hours. The Function Key model gets 4 hours. And the 2015 Pro gets 5 hours.
In light usage (e.g. while writing said blog post, and watching an occasional HTML5 YouTube video, and listening to some music in iTunes through AirPods), I can get 8-10 hours without issue.
1-2 hours sounds like you may have an app that's more buggy than the worst that I'd allow to run on my laptop...
Ughh.. no. I'm on an 6 year old Macbook with SSD which had fresh battery put in a year ago. Running IntelliJ, web server, JS build tools (they run in the background), and over a dozen apps (i.e. slack, browser etc.) lasts me about 3 hours.
This jibes with my own experience. My 2014 Retina MBP averages around three hours when running an Android emulator, the React Native tools, PostgreSQL, and IntelliJ for my backend, plus whatever browsers I'm using along the way.
14 month old rMBP here, I spend all my day with Safari and Firefox open plus Slack and Word/Excel, plus connected to a remote desktop session and I can basically always expect 3-4 hours under those conditions.
If I'm visiting a client and turn off wifi and just take notes in Word, after an 8 hour day I can usually still make it through the evening with some light Netflix.
This isn't a Mac specific issue. Most of the efficiencies in battery life That have taken place in the last decade are all about CPUs and their components lowering their idle power consumption.
I have a circa 2014 MacBook Pro that I use as my daily driver. The battery life is great -- so long as I don't light up VMs, run OneDrive for Business or a few other misbehaving apps.
I'd challenge you to identify another laptop that's better without making other sacrifices. My old ThinkPad T series had great battery life, but you had to deal with the bulk and weight of a huge battery.
So Consumer Reports had Safari Developer Mode enabled which forced full page loads (plus some other bug). And Apple claims "this is not a setting used by customers and does not reflect real-world usage." While I agree most people wouldn't have that option enabled, I would argue that the average user is somewhere between the extremes. For example, most users are not disabling cache and browsing to 20 MB webpages and clicking refresh hundreds of times. But at the same time, neither are users leaving cache on and reloading the same page hundreds of times; they're browsing from site to site, refreshing Facebook feeds, etc. If Apple can make this statement, I'd be curious as to what tests they are performing. I'm almost certain the battery specs released by Apple are not indicative of how average users would use the MacBook (i.e. brigtness 10%, no movement, WiFi off, all background apps disabled, etc.).
EDIT: as a reply mentions, Apple does provide (some) conditions used for their tests. However I'd still imagine that the conditions they do choose are the "most optimal" (for example - who do you know that only browses 25 sites in a single charge?)
>The wireless web test measures battery life by wirelessly browsing 25 popular websites with display brightness set to 12 clicks from bottom or 75 percent. The iTunes movie playback test measures battery life by playing back HD 1080p content with display brightness set to 12 clicks from bottom or 75 percent. The standby test measures battery life by allowing a system, connected to a wireless network and signed in to an iCloud account, to enter standby mode with Safari and Mail applications launched and all system settings left at default. Battery life varies by use and configuration.
Thing is, with dynamic brightness (Macs have long been better at that than PCs), it's often the case that 75% brightness or less is used - at night for me it's more like 25% bright.
>> The CR tests raise more concerns about how CR runs their tests (and specifically how they run this test) as opposed to the MacbookPro.
Why? At some point, all of these battery tests are arbitrary and not 100% reflective of the real world.
If I'm not mistaken, previous Macbook Pro models underwent the exact same test and still got the "Recommended" badge. Nobody was complaining about the test methodology for those models.
>If I'm not mistaken, previous Macbook Pro models underwent the exact same test and still got the "Recommended" badge. Nobody was complaining about the test methodology for those models.
Did Consumer Reports indicate in any previous reviews that this is an aspect of their methodology? It's my understanding that this aspect is just now being revealed.
>> Did Consumer Reports indicate in any previous reviews that this is an aspect of their methodology?
Probably not.
>> It's my understanding that this aspect is just now being revealed.
That's probably because nobody cared to ask before. The new Macbook Pros were already under a microscope even before Consumer Reports did their tests. Any less than stellar review from CR was going to get lots of eyeballs. In any case, the big reveal is not that caching was disabled, it's that there was a bug on Apple's side that is the likely root cause, and I'm guessing that bug didn't exist in the previous versions of Safari that were used to test the previous Macbook Pros that got the "Recommended" badge.
"We turn off caching as part of Consumer Reports' standard laptop test protocol" -- I can only take that at its word. It doesn't say "updated" standard protocol.
"At Consumer Reports, we test every laptop from every manufacturer in a comparable way. Because people use laptops differently and because their usage can vary from day to day, our battery tests are not designed to be a direct simulation of a consumer’s experience. Rather, we look to control as many variables as possible, then perform a test that gives potential users a reasonable expectation of battery life when a computer’s processors, screen, memory, and antennas are under a light to moderate workload. This test has served as a good proxy for battery life on the hundreds of laptops in our ratings."
They don't review hundreds of laptops per year, that's spread out over multiple years. So I don't think it's a huge leap of faith to assume that they turned off the cache on prior Macbook Pro models.
FWIW, John Gruber does not "think it’s fair to say that disabling the caches is unfair or a flawed method" -- which says a lot.
I'm not sure, then, why the lack of scrutiny in previous years is valid criticism; nobody shines a lamp in her doctor's eyes when she gives her a clean bill of health. Of course an unusual review will invite unusual scrutiny.
I just wish CR had been more open with the exact methodology from the start. What other settings do they activate that vary from a normal user's? They want us to take their word that their method sufficiently emulates light to moderate usage, but my own experience has been vastly different from that review.
You can't do every test of every laptop using the popular sites, because content payloads on various sites change over time. You won't get apples to apples comparison tests.
The easiest way to do a repeatable browser test is to turn caching off on the browsers and hit a few web pages that never change and that you control on your own server. By repeating that test over and over again, it simulates hitting new sites even though you're hitting the same pages.
If I was in the business of doing laptop reviews, that's how I'd do it.
Summary: Consumer Reports turned off Safari's page cache to ensure the browsers download fresh content. This hurt performance a lot, presumably because Apple hadn't optimised this usage pattern. Apple has released an update which fixes the "bug".
My opinion: Disabling the cache as Consumer Reports has done is an unrealistic testing pattern. Since Consumer Reports run their tests from local servers [1] they have a better option available. They can use server-generated content to control how much content is served from cache and how much is fresh. They can make the workload 0%–100% fresh by using the servers to generate unique ids for URLs that they want to be served fresh or reusing URLs if they want them to maybe be in the cache.
EDIT: To be clear, I'm not saying the Consumer Reports didn't find something interesting here, I'm just saying they could make a better test for the future.
If a laptop battery capacity is cut in half because you turn off caching in a browser that is important information to know, as it suggests the battery times are the result of carefully sterilized conditions that don't withstand the test of real world use. They didn't disable power management or something, they turned off caching in a single app. Come on.
That's not true. The vast majority of people don't turn off caching. The default caching behavior Safari provides is probably tailored to real-life usage. I could be wrong, but I'm assuming Apple isn't that dense.
That's fine, as long as you don't say your benchmark is just "regular browsing". Enabling developer options put you outside of what the vendor really tested and optimized for.
Also I find quite dubious your claim about `sterilized conditions that don't withstand the test of real world use` based on this.
This PR mess also likely 'justifies' the approach of exposing as little knob as possible to the users.
There was a bug and that seemed to cause variation. But then turning on the cache would invalidate the results making it impossible to compare it with other laptops. So I guess they have to get the new version of Safari before doing a re-run.
Changing a testing approach in my opinion is fine, but should result in the retroactive re-testing of previous models used in any in-article benchmark comparisons.
I think they removed it because the battery life of the new MBPs has been getting a lot of negative attention, deserved or not, and in general people don't seem to be capable of understanding why the estimate would change for different workloads. Maybe the best PR move was just to remove it altogether. I wish they hadn't.
It's not the same machine, but I routinely get > 10 hours on my 2016 12" Macbook with a workflow that includes moderate to heavy browsing in Opera, coding in Tmux/Vim, and various compilers and interpreters invoked periodically. Have you had a look in Activity Monitor to see where all your power is going?
I'm using two non-Apple apps as the core of my workflow (Opera and Alacritty for terminal: https://github.com/jwilm/alacritty) but my battery life is still excellent. However, I chose them carefully on the basis of their efficiency. Things written in Javascript seem to be real resource hogs (Atom comes to mind), and Chrome is notorious for inefficiency in memory, at least.
Click on the battery icon in your menu bar, and see if there are any programs using "Significant Energy". Some programs will switch on the discrete graphics card without needing it, for example.
Chances are they have a script that hits a couple different websites that represent different archetypes: light/heavy text, light/heavy images, light/heavy js, light/heavy ajax, flash, rendering, dependencies, fonts. Instead of using the cache dozens of times, they refetch the pages (which would must more closely represent someone surfing ye olde interwebs). Seems pretty reasonable to me.
Thats correct-the funny bit is they were using a hidden developer feature-not one of the 'Developer' drop down features but an actual hidden flag. They also observed huge internal discrepancies with their testing methodology-but instead of reporting that to Apple or trying to figure out why their test was broken CR reported their results as 'facts'. Thats why people and Apple are irritated about this.
The fact that turning off caching triggered a browser bug is not consumer reports problem nor do they have an obligation to figure out why. They exist to report on existing things even if they have bugs.
The idea that battery life for a browsing workload should be tested from cached sites is silly, as is the idea that not using a check box on the menu makes them out to get Apple.
Well, no battery test truly represents real world, so I don't think it's a big deal.
The same testing methodology in the past gave the "Recommended" badge for Macbook Pros. Why should we question the methodology only when it produces a result that people don't like?
-- edit, just noticed the article I posted this comment on got merged with another by the moderators, so it's a dupe within the overall thread.
> The fact that turning off caching triggered a browser bug is not consumer reports problem nor do they have an obligation to figure out why.
Agree. Unfortunately this makes the test flawed, since it tries to simulate real-world behavior, but they couldn't know that unless they knew there was a bug in the cache thing.
> The test is factual.
Only if the report mentioned their testing methodology, i.e. if it mentions that they turn off caching for all laptops to better simulate browsing patterns. I expected Techcrunch to mention this but they neglected to.
> They exist to report on existing things even if they have bugs.
On one level, sure. They found a legit bug. However, the bug was in a codepath that regular users would never experience. You and I might, but we're not the ones reading Consumer Reports to pick a laptop. My non-technical friends certainly aren't going into hidden settings to turn off browser performance features, but they're the ones who'll see that bad review and worry about it.
> The idea that battery life for a browsing workload should be tested from cached sites is silly
How many first-time-ever websites do you encounter every day? I guarantee that you don't download a new copy of jQuery on every website you visit, or reload every static asset on your favorite sites that you hit daily.
I find it funny that Apple and its users are angry at the company who tested and reported results, not the company which coded and released a browser with bugs.
I guess the way to benchmark browsing to get most realistic numbers would be to record normal browser behavior of a decently-sized sample of people (say 1000) and automatically repeat the behavior of a representative subsample (10-50 may be enough).
This would take forever and cost quite a lot of money, however, so the much more reasonable solution is to simply turn off the cache for everything.
I don't agree with Apple or Techcrunch that CR's methodology is wrong -- pretty much anyone would make that same decision because the alternative is too much work for too little reward.
Websites change over time -- they'd have to download a static copy of the websites and store them locally for repeatable tests (but then it no longer reflects the real world, since websites often tailor themselves for each user).
And since browsing activity changes over time too, they'd have to regularly update the test, which means that tests are not comparable over time.
I don't think that's unreasonable to address. For CPU and GPU benchmarks they often run BenchMark2014 so you can see improvements compared to previous years and BenchMark2017 to see the latest and greatest because hardware architecture, game engine design, and graphics APIs change. You just need to figure out how best to present that to the user.
I dunno here. CR's validation around the fact that users will often visit different sites seems on point to me.
Some users visit the same sites over and over, true. But, a more accurate browsing pattern would be different sites, new content etc.
A more in-depth detail on how the browsing is done (automation) might be relevant. If they hit the same 5 sites 1000 times vs. 1000 sites 5 times would alter whether they disable cache or not. I'd say the former makes sense, not the latter.
They turn off the browser cache for every notebook they test. If their methodology were flawed because of this they would have noticed that long time ago.
Speaking personally, I would want any stress test to be the MOST stressful possible across all laptops for the baseline. Turn off cache (how often do you have the console open), run a VM, sit on a Hangout, all at the same time. Make the fans whirr hard.
If you want an "econo" benchmark go right ahead but I don't know why one would criticize Consumer Reports for being hard on a laptop browser. (It obviously makes sense for hardware makers to dispute the technique as it's harder to hit "X Hours" markers that are tested)
I can see where you're coming from but I disagree. In that case you are testing the battery hardware, while the software actually makes a big difference.
Say you have two phones: one with crap software, and one from a company that spends time on writing proper, efficient software. If you run a stress benchmark, they will both be running at 100% CPU all the time and you'll only be measuring how much mAh is in the battery divided by how much power the CPU draws. All of this is information is already available in the product specification.
The only reason my Galaxy Note 2's battery life is any good (still the original battery which is, what, 5 years old now?) is because I did software tweaks to keep it from holding wake locks when apps are inactive.
That makes sense and you're right, if all you're doing is a head-on stress test then just let the math do the talking.
I think testing heavy use is probably different than a straight up stress test in a number of ways that are more applicable to my daily use and more useful. It's good to know if Safari's caching strategy saves you battery, but as a non-Safari user that often does not refresh from cache I'm not sure I would appreciate buying a laptop advertising X battery and finding it to have changed significantly or be caveated by lots of required settings.
Consumer Reports did nothing different with the MacBook Pro tests than they do with their other tests. So using the benchmarks as a comparison point between Apple and others, such as HP, should still be completely valid.
> Consumer Reports uses a hidden Safari setting for developing web sites which turns off the browser cache. This is not a setting used by customers and does not reflect real-world usage. Their use of this developer setting also triggered an obscure and intermittent bug reloading icons which created inconsistent results in their lab.
This is a good point, which seems to be ignored by other commenters nearby.
Additionally, this seems to be a case where CR (a venerable publication with many good qualities) used results in a home-brew single-point test to characterize a system's quality.
This is not an isolated problem with this particular review. There's a lot of snake oil in audio systems, but CR reviews of audio equipment were a joke for many years because of over-reliance on specific lab measurements. CR has had issues over the years with mistaken testing of child safety seats, pet food, and other products over the years (see their wiki page).
This caveat is apparent after subscribing for a few years.
100 comments
[ 2.8 ms ] story [ 163 ms ] threadIts subscription only (they don't do ads), which means they're not beholden to the companies they're reviewing for money (a good thing)
They generally make a grid of the items you're looking at with dots indicating the ratings. Its very easy to nagivate and intuitive.
You'll have to pay for it (or find someone to give you their old copies, or many libraries have subscriptions).
This became Hacker News fare because lots of people were unhappy with the new MacBook Pros for other reasons.
http://thewirecutter.com/
battery life affects everyone and they had the competency and integrity to call apple out multiple times
That seems to apply to all of their highly rated dishwashers -- when you read the CR user comments, most of them got a lot of horrible user ratings.
I don't know if it's the dishwasher segment, but it seems like dishwashers in general have gone downhill.
I was also going to buy a higher end dishwasher but it was so hard to choose because of the negative user comments that I ended up getting a lower priced one that got decent reliability ratings. I didn't want to get a Bosch or Miele after reading the user reviews on CR.
Having said all of that, if you've been using dishwashers since the 80s, it turns out your old habits can result in bad washes. I was used to scraping my dishes clean, but according to my research, you actually need your dishes to be soiled enough so the newer detergent pods can clean them properly. I was lucky I found that out or I'd be getting inconsistent wash results.
I originally had my eyes set on a European style dishwasher which dries very differently from an American style dishwasher.
I ended up going with an American style dishwasher which has a heating element for drying (compared to passive drying in European models).
Non-Apple apps are death for battery life. I think I'd be lucky to get two hours of productive development time on my laptops. 1.5 hours is probably closer to reality. I have two BatteryBox devices that add about 45 minutes to 1 hour each to runtime and it's impossible to get productive work done without them.
Perhaps Consumer Reports should try testing battery life by running other apps?
I would actually trade whatever weight would be necessary to get my useful portable development time up to 3-4 hours.
You may need to go the "brick" route if you have a heavy workload, whether or not that's due to inefficient software. I've found these days that a lot of software is simply poorly written and drains much more power than it should (the Atom editor is a good example).
... so you're saying that designing for a detachable battery forces a certain finesse when designing the rest of the internals, so that there is not any funky-shaped gap big enough to put an entire LiPo cell.
Also, one can easily make an "extended" model of replacement battery, which protrudes rearward (or downward[1]) of the rest of the case.
[1]: http://www4.pcmag.com/media/images/396005-lenovo-thinkpad-x2...
It isn't good due to magic or some other vaporous marketing buzzwords. It is good because you can just install big batteries and swap them out in sleep mode.
Get one or more 6 cell 72Wh rear batteries (6,666 mah). Alternatively get one or more Lenovo Power Bank 10,000 mAh (100 Wh give or take). The advantage of the Power Bank is that you don't even have to put the computer to sleep.
One extra 6 cell 72 Wh battery is a lot less heavy than you're imagining (approx 1/2 pound). I'd even go as far as to say that your Macbook's dongles weigh more than doubling the battery life of the Thinkpad T460p.
You used to be able to get laptops with batteries that fit in the optical drive-bay so you could expand the battery life, but that appears to be a non-option now that internal optical drives have gone the way of the dodo.
[edit]
A friend pointed me to this as well: https://www.anker.com/deals/powerhouse2 it weighs as much as the first laptop I ever used, but has 400Wh.
Non-Apple apps don't kill battery life. What kills battery life is shitty apps like Chrome (or anything based on Electron). VSCode for example will cut battery life to 6 hours versus 10+ for Emacs. Just simple stuff like typing and scrolling takes huge amounts of CPU. Even just sitting idle it's using 1% or so CPU.
Worst-case, the 13" Pro with Touch Bar gets 3.5 hours. The Function Key model gets 4 hours. And the 2015 Pro gets 5 hours.
In light usage (e.g. while writing said blog post, and watching an occasional HTML5 YouTube video, and listening to some music in iTunes through AirPods), I can get 8-10 hours without issue.
1-2 hours sounds like you may have an app that's more buggy than the worst that I'd allow to run on my laptop...
Edit: Link to said test script: https://github.com/geerlingguy/macbook-pro-battery-test
Is this speculation or have you actually measured it?
Ughh.. no. I'm on an 6 year old Macbook with SSD which had fresh battery put in a year ago. Running IntelliJ, web server, JS build tools (they run in the background), and over a dozen apps (i.e. slack, browser etc.) lasts me about 3 hours.
If I'm visiting a client and turn off wifi and just take notes in Word, after an 8 hour day I can usually still make it through the evening with some light Netflix.
I have a circa 2014 MacBook Pro that I use as my daily driver. The battery life is great -- so long as I don't light up VMs, run OneDrive for Business or a few other misbehaving apps.
I'd challenge you to identify another laptop that's better without making other sacrifices. My old ThinkPad T series had great battery life, but you had to deal with the bulk and weight of a huge battery.
EDIT: as a reply mentions, Apple does provide (some) conditions used for their tests. However I'd still imagine that the conditions they do choose are the "most optimal" (for example - who do you know that only browses 25 sites in a single charge?)
>The wireless web test measures battery life by wirelessly browsing 25 popular websites with display brightness set to 12 clicks from bottom or 75 percent. The iTunes movie playback test measures battery life by playing back HD 1080p content with display brightness set to 12 clicks from bottom or 75 percent. The standby test measures battery life by allowing a system, connected to a wireless network and signed in to an iCloud account, to enter standby mode with Safari and Mail applications launched and all system settings left at default. Battery life varies by use and configuration.
http://www.apple.com/macbook-pro/specs/
Do they do this on other browsers? Browse the same site over and over while disabling cache?
Why did this not cause a problem when they ran Chrome (did they also disable cache there)?
The CR tests raise more concerns about how CR runs their tests (and specifically how they run this test) as opposed to the MacbookPro.
And yes, they disable the cache in every browser and in every PC.
And if you read the Apple answer, the problem was a bug in Safari so no, others browsers are not affected
Why? At some point, all of these battery tests are arbitrary and not 100% reflective of the real world.
If I'm not mistaken, previous Macbook Pro models underwent the exact same test and still got the "Recommended" badge. Nobody was complaining about the test methodology for those models.
Did Consumer Reports indicate in any previous reviews that this is an aspect of their methodology? It's my understanding that this aspect is just now being revealed.
Probably not.
>> It's my understanding that this aspect is just now being revealed.
That's probably because nobody cared to ask before. The new Macbook Pros were already under a microscope even before Consumer Reports did their tests. Any less than stellar review from CR was going to get lots of eyeballs. In any case, the big reveal is not that caching was disabled, it's that there was a bug on Apple's side that is the likely root cause, and I'm guessing that bug didn't exist in the previous versions of Safari that were used to test the previous Macbook Pros that got the "Recommended" badge.
CR's response today: http://www.consumerreports.org/apple/apple-releases-fix-to-m...
Some key quotes:
"We turn off caching as part of Consumer Reports' standard laptop test protocol" -- I can only take that at its word. It doesn't say "updated" standard protocol.
"At Consumer Reports, we test every laptop from every manufacturer in a comparable way. Because people use laptops differently and because their usage can vary from day to day, our battery tests are not designed to be a direct simulation of a consumer’s experience. Rather, we look to control as many variables as possible, then perform a test that gives potential users a reasonable expectation of battery life when a computer’s processors, screen, memory, and antennas are under a light to moderate workload. This test has served as a good proxy for battery life on the hundreds of laptops in our ratings."
They don't review hundreds of laptops per year, that's spread out over multiple years. So I don't think it's a huge leap of faith to assume that they turned off the cache on prior Macbook Pro models.
FWIW, John Gruber does not "think it’s fair to say that disabling the caches is unfair or a flawed method" -- which says a lot.
I just wish CR had been more open with the exact methodology from the start. What other settings do they activate that vary from a normal user's? They want us to take their word that their method sufficiently emulates light to moderate usage, but my own experience has been vastly different from that review.
You can't do every test of every laptop using the popular sites, because content payloads on various sites change over time. You won't get apples to apples comparison tests.
The easiest way to do a repeatable browser test is to turn caching off on the browsers and hit a few web pages that never change and that you control on your own server. By repeating that test over and over again, it simulates hitting new sites even though you're hitting the same pages.
If I was in the business of doing laptop reviews, that's how I'd do it.
-- edit: This blog entry by Marco Arment on the topic is worth reading: https://marco.org/2017/01/10/cr-mbp-battery-update
You're right, most people probably browse fewer than 25 sites.
I always have it enabled and get 8+ hours from my previous model Pro.
My opinion: Disabling the cache as Consumer Reports has done is an unrealistic testing pattern. Since Consumer Reports run their tests from local servers [1] they have a better option available. They can use server-generated content to control how much content is served from cache and how much is fresh. They can make the workload 0%–100% fresh by using the servers to generate unique ids for URLs that they want to be served fresh or reusing URLs if they want them to maybe be in the cache.
1: http://www.consumerreports.org/apple/apple-releases-fix-to-m...:
EDIT: To be clear, I'm not saying the Consumer Reports didn't find something interesting here, I'm just saying they could make a better test for the future.
Also I find quite dubious your claim about `sterilized conditions that don't withstand the test of real world use` based on this.
This PR mess also likely 'justifies' the approach of exposing as little knob as possible to the users.
"the variable battery performance we experienced is a result of a software bug in its Safari web browser that was triggered by our test conditions."
a) That Safari's performance hadn't been tuned for when caching was disabled.
b) That, in general, it's more realistic to test a mixed cached/uncached workload.
Which bit was incorrect?
I've been getting 2-3 hours max out of my new 15" 2016 MBP and I don't even use Safari. I'm just running webpack, vim and Chrome.
It's not the same machine, but I routinely get > 10 hours on my 2016 12" Macbook with a workflow that includes moderate to heavy browsing in Opera, coding in Tmux/Vim, and various compilers and interpreters invoked periodically. Have you had a look in Activity Monitor to see where all your power is going?
I'm using two non-Apple apps as the core of my workflow (Opera and Alacritty for terminal: https://github.com/jwilm/alacritty) but my battery life is still excellent. However, I chose them carefully on the basis of their efficiency. Things written in Javascript seem to be real resource hogs (Atom comes to mind), and Chrome is notorious for inefficiency in memory, at least.
Sierra has wrecked my battery life tho...
The fact that turning off caching triggered a browser bug is not consumer reports problem nor do they have an obligation to figure out why. They exist to report on existing things even if they have bugs.
The idea that battery life for a browsing workload should be tested from cached sites is silly, as is the idea that not using a check box on the menu makes them out to get Apple.
The same testing methodology in the past gave the "Recommended" badge for Macbook Pros. Why should we question the methodology only when it produces a result that people don't like?
-- edit, just noticed the article I posted this comment on got merged with another by the moderators, so it's a dupe within the overall thread.
Agree. Unfortunately this makes the test flawed, since it tries to simulate real-world behavior, but they couldn't know that unless they knew there was a bug in the cache thing.
> The test is factual.
Only if the report mentioned their testing methodology, i.e. if it mentions that they turn off caching for all laptops to better simulate browsing patterns. I expected Techcrunch to mention this but they neglected to.
On one level, sure. They found a legit bug. However, the bug was in a codepath that regular users would never experience. You and I might, but we're not the ones reading Consumer Reports to pick a laptop. My non-technical friends certainly aren't going into hidden settings to turn off browser performance features, but they're the ones who'll see that bad review and worry about it.
> The idea that battery life for a browsing workload should be tested from cached sites is silly
How many first-time-ever websites do you encounter every day? I guarantee that you don't download a new copy of jQuery on every website you visit, or reload every static asset on your favorite sites that you hit daily.
We should aspire to attain such loyalty.
I know when I release bugs, I get the blame!
This would take forever and cost quite a lot of money, however, so the much more reasonable solution is to simply turn off the cache for everything.
I don't agree with Apple or Techcrunch that CR's methodology is wrong -- pretty much anyone would make that same decision because the alternative is too much work for too little reward.
And since browsing activity changes over time too, they'd have to regularly update the test, which means that tests are not comparable over time.
And why it wasn't a problem until now? CR was praised when they acknowledge that Macbooks had almost always the higher battery life
Some users visit the same sites over and over, true. But, a more accurate browsing pattern would be different sites, new content etc.
A more in-depth detail on how the browsing is done (automation) might be relevant. If they hit the same 5 sites 1000 times vs. 1000 sites 5 times would alter whether they disable cache or not. I'd say the former makes sense, not the latter.
CR has disabled the cache for all of their tests in all the computers for years.
And now is bad because there was a bug in Safari?
If you want an "econo" benchmark go right ahead but I don't know why one would criticize Consumer Reports for being hard on a laptop browser. (It obviously makes sense for hardware makers to dispute the technique as it's harder to hit "X Hours" markers that are tested)
Say you have two phones: one with crap software, and one from a company that spends time on writing proper, efficient software. If you run a stress benchmark, they will both be running at 100% CPU all the time and you'll only be measuring how much mAh is in the battery divided by how much power the CPU draws. All of this is information is already available in the product specification.
The only reason my Galaxy Note 2's battery life is any good (still the original battery which is, what, 5 years old now?) is because I did software tweaks to keep it from holding wake locks when apps are inactive.
I think testing heavy use is probably different than a straight up stress test in a number of ways that are more applicable to my daily use and more useful. It's good to know if Safari's caching strategy saves you battery, but as a non-Safari user that often does not refresh from cache I'm not sure I would appreciate buying a laptop advertising X battery and finding it to have changed significantly or be caveated by lots of required settings.
It couldn't be that the battery is actually quite literally smaller. No, it is the tester and tests fault.
http://www.imore.com/consumer-reports-fails-earn-macbook-pro...
> Consumer Reports uses a hidden Safari setting for developing web sites which turns off the browser cache. This is not a setting used by customers and does not reflect real-world usage. Their use of this developer setting also triggered an obscure and intermittent bug reloading icons which created inconsistent results in their lab.
Additionally, this seems to be a case where CR (a venerable publication with many good qualities) used results in a home-brew single-point test to characterize a system's quality.
This is not an isolated problem with this particular review. There's a lot of snake oil in audio systems, but CR reviews of audio equipment were a joke for many years because of over-reliance on specific lab measurements. CR has had issues over the years with mistaken testing of child safety seats, pet food, and other products over the years (see their wiki page).
This caveat is apparent after subscribing for a few years.