207 comments

[ 3.7 ms ] story [ 290 ms ] thread
More comprehensive studies have ben performed: https://www.washingtonpost.com/news/where-we-live/wp/2014/06...

I just bought a home myself and looked this up, but the takeaway is that the Zestimate is within +- 5% 50% of the time. So, not that accurate. Are people really using it in negotiations though? I'm sure people do, but you'd have to be pretty uninformed to do so, and I'm also sure you're real estate agent would tell you exactly that up front.

Yeah, but how accurate are appraisals? Several years ago I was reading about an analysis that found that Zestimates were about as accurate as appraisals are 80%+ of the time.
If appraisals are not accurate, then what is the definition of 'accurate'? Appraisals are what the lender relies upon to verify the mortgage amount. I also highly doubt that number.
Sales price is really the only 'accurate' statement of value.
In a way, that's true and false. A house may be worth $500k, and you might get a dozen people to offer $500k, but if someone comes along and wants to buy back her childhood home for $1 million for the sentimental value, isn't the house still only worth $500k? The next buyer will still think the house is only worth $500k.
I suppose that's kind of my point (not the parent.) 'value' is an amorphous term. Ultimately, you have comps, a detailed appraisal, and emotion to guide the purchase. There is no hard number, but the first two get you very close.
The article uses the actual sales price as the "accurate" number.

As with anything, the 'value' is what a willing and able seller and a willing a able buyer agrees to. An appraiser attempts to determine the most likely price for something (in this case a home) within reasonable amount of marketing time. The appraisal report will usually specify the expected marketing time. I think in most areas this will be 3-6 months, some areas longer and others shorter, depending on the market.

Of course, appraisals effect prices, because most buyers need to finance, and most financiers won't do it without an appraisal that is at least as much as the purchase amount. (That is the "able" part of the willing and able requirement).

Two appraisals performed on the same property at the same time can come back with very different numbers. I have bought two houses in the last 4 years- both had 2 appraisals done, and one the values were almost almost 17% different- a difference greater than most of the "errors" that the articles notes for Zestimate vs sale price. On the other house, the appraisals were came back with nearly identical values, but the first appraisal was done wrong. Still this value (the appraised value from both appraisals) was 16.2% higher than what I paid for the property.

My experiences are just as anecdotal as the articles, though. But I have spoken with real estate professionals all over the country that have the same experiences repeatedly with appraisals.

The job of the appraiser is to write down a number as large as or larger than the amount to be financed. They know this and they know the number in advance. What they do is not rigorous.
Certainly you can influence the appraiser, but the way you state it is nonsense. Their job is also to provide feedback to the buyer, they won't just pull any number you like out of a hat.
Appraisers absolutely do pull numbers out of hats, because appraisers who regularly fail to write down a large enough number will stop getting calls.
That has not been my experience at all, and I'm pretty sure lenders would have a big issue with that if it were as rampant as you say.
What incentive does the lender have? The involved parties are the seller, the buyer, their agents, and the originator of the loan. All of these people are working to close the deal at the negotiated price. The originator of the loan will hold the loan for somewhere between 72 hours and 30 days, so they don't care.
Of course they care; any money they give you over the value of the home is unsecured! They're not going to lend you e.g. 20k @ 3.5% with no collateral. C'mon.
They won't write down a completely insane number. They're a first level filter not a fine detailed filter. "I'm willing to go on record that I found three recent sales in the neighborhood for about $whatever" where whatever is reasonably near what you're financing. One need only go a few miles to find houses a tenth the cost or ten times the cost. If your neighbors house sells for $X and you want a loan for $X they're not going to sweat it, if you are trying for a loan of $X times ten its going to be difficult.

I paid a couple hundred bucks for an ex-general contractor with a thick binder to inspect my house for hours, looking inside the furnace heat exchanger and testing every wall outlet and inspecting the plumbing and roof inside and out and insulation and foundation and everything. That is a detailed analysis of what you're likely to spend on maintenance and upgrades over the next ten years. Its probably more useful than an appraisal but it takes a lot more time and money.

Definitely not true... there are many checks in place to ensure this doesn't happen

http://www.freddiemac.com/loanadvisorsuite/loancollateraladv... http://www.freddiemac.com/loanadvisorsuite/faq/loancollatera...

This doesn't even take into account the process for appraisal management companies (third parties).

You seem to think these industry controls are effective. Maybe they prevent an appraisal from being, say, 10x higher than the recent sale prices on either side, but they don't prevent the equally bad event of a series of 10 sales on the same block each being 10% higher than the previous. Every one of those houses in Las Vegas that turned out to be worthless in 2007 was appraised for at least the sale price the year before. The appraisers in those cases didn't do a damn thing, for example they didn't come out to the house and write down "this neighborhood is in a desert with no access to water where the unemployment rate is 20% and the employed people make minimum wage therefore this cannot be a neighborhood of $850000 houses." They just said "the last house sold for $800000 and this market seems pretty hot so $850000 okey dokey."
SF Bay area, I've had 3 formal house assessments, & each time they "start with" Zillow et al to look at "reasonable values in the neighborhood". Which seriously affects your ability to refi, etc.

So Yeah Zillow has an effect. It's worth it to create a login & enter in your upgrades, so your ZESTIMATE goes up.

Who is doing these "formal assessments"? Are these broker price opinions (performed by a real estate broker)? It seems very odd they would use Zillow data instead of the MLS to find comps.
When you refinance the bank usually hire someone independent to estimate the value of a property.

I did it before Zillow, and they start with the property value as estimated by the city for tax purposed and look at recent sales that are similar in the area, and adjust.

Yeah, usually either a professional appraiser or a real estate broker- neither one should be relying on data from Zillow, both usually would use data from a local MLS.
It's worth it to create a login & enter in your upgrades, so your ZESTIMATE goes up.

Very nice, thank you, my Zillow estimate is now $5965 higher. We have no intention to sell but if we do, at least I can start the negotiations at a higher price point by pointing to Zillow.

Incredible -- I just did the same, and my house's "zestimate" went up about $20,000.

Of course, if I compare my house to actual sales in the neighborhood over the past few months, that's probably 10% high, so correcting Zillow's data about my house made the zestimate less accurate, not more.

Try changing the number of bedrooms. That used to have an effect, even for the same square footage. In one case, 2 bedrooms came out higher than 3.
You just have been 'rewarded' for your interaction with the platform. Thank you.
Ooooor, Zillow has no way of knowing that you've renovated your kitchen unless you tell them you have...

But yeah, your conspiracy theory nonsense is probably just as legitimate.

I'm pretty sure that when they say they start with Zillow, they are referring to the feature where it shows recent past sales in your neighborhood, which is an entirely legitimate input into determining a formal appraisal.
Zillow should use that data to actually make estimates. Because they clearly don't, and don't really know how to do predictive analytics.
What is it that makes it "clear" to you they aren't "using data"? What else could the estimates possibly be based on but "data"? You think the computer just randomly chooses a number?
Terribly scary that a huge financial decision would depend on giving signing up and giving data to some website.
In the past it requires that you report the data to the local permitting board.
5M home sales yearly, so this shouldn't be done by anecdote.
That was my first thought as well. Analyzing 3 houses in 3 markets... It's an interesting look, but I wouldn't consider any takeaway message to be necessarily accurate.

Algorithms have outliers, and quantitative analysis requires adequate sample size.

While we're at it, lets also figure out why my Zestimate dropped $40k overnight two months ago. If I was planning to sell, I would have a huge problem on my hands. On top of it the Zestimate chart shows no indication that they dropped it $40k. Aiming to fix the number is a fool's game : my "private home estimate" on my owner dashboard is 23% higher.

Its a worthless number, and they should stop trying to manipulate the market. Having a secret measurement formula that works for 70% of the houses but not the other 30% needs to either have a disclaimer that its all fantasy, or just outright be removed. I'm guesstimating those 70/30 numbers, but their site gives some locations 2 stars on their own scale: http://www.zillow.com/zestimate/#acc. If its not 4 stars, then don't publish it.

From time to time, Zillow rejiggers (that's a technical term) their algorithm, resulting in sudden readjustments of estimates. Given that the Zestimate itself is presented as a fairly wide range, the error margin is hardly hidden.

Also, it is important to remember that no appraiser from Zillow has actually looked at your house; the data is drawn entirely from sales data, public records, and volunteered information. So there's no way it can really account for every factor, nor does it even try. And there's a lag; the market can change faster than Zillow can. It is, at best, a rough ballpark estimate of what's going on in your neighborhood.

point taken, but two observations:

1) its ridiculous to also "fix" the zestimate history. Put the old one in the history, and explain near the chart what rejiggered.

2) The market in my immediate area, as evidence by the comps from zillow and the sales prices in the newspaper, has only increased. The conservative comps are near the old zestimate - this is a suburban area. The wild ones are higher than my private estimate.

I think what you're actually seeing is the output of a Kalman Filter temporal model. The zestimate is the mean of the filter output, but the error bounds of the filter are more important than the actual mean/reported value.

What happens is that the model uses trends to extrapolate from each "real" data point (in this case, a house sale in your neighborhood). The problem is, and what Kalman Filters help manage, is the uncertainty propagation between each house sale. When it has been a long time from when a sale has occurred, it is unclear what the real/actual price is of a home. This means that on the estimated price the error bounds are large, and zestimate still just reports the mean value of this huge uncertainty.

What then happens is that a house is sold in your area, a new data point is recorded, and the filter re-adjusts itself and collapses its uncertainty/error bounds in the time of that measurement around the measurement. And you get correction. This is why the "old zestimate" is updated.

Yes, the little chart they show you has no history--it just extends back from the current data and the current algorithm. So what it shows you now as they value five years ago is different from what it showed you five years ago. In that sense it does not function as actually tracking the market. Does it really need to? What do you really care about beyond what you originally paid and what you might sell it for if you have to?
With how slow real estate moves, thats saying a lot.
My house is recently on the market and the Zestimate and Redfin estimate dropped by 50k the day we listed. It's still higher than we listed but much closer to our number. I think we priced it fairly as we are having plenty of interest but I wonder what would be happening if it didn't change.
I paid $265,000 for my house, which at the time had a Zestimate around $500,000. After our purchase, the Zestimate slowly rose over $950,000. While I understand we got a good price on our building, I knew the Zestimate was garbage.

A couple of weeks ago, the Zestimate dropped to $435,000. This would definitely be closer to an accurate value, but there's also no record of the recalculation at all in the graph.

At this point, it's basically noise.

In my experience, Zillow does a terrible job compared to local knowledge. My zip code is weird—our ~$275k house backs up to houses worth $50-70k (the disparity is due to build quality, lot size, and the streets they're on). Not surprisingly, Zillow handles this very poorly, and has consistently valued our house at much lower than it is actually worth (based on city assessments and sales of nearby comps).

Interestingly, Zillow recently bumped up our Zestimate from $190k to $230k. What's puzzling is that they also bumped up the historical Zestimates—there isn't a big jump shown on the graph on their site (though there is a big jump shown in my Personal Capital account, which uses Zestimates as part of their net worth scores). Either they modified their algorithm in a way that affected values retroactively, or they're trying to disguise the fact that their previous Zestimates were way off.

> Either they modified their algorithm in a way that affected values retroactively, or they're trying to disguise the fact that their previous Zestimates were way off.

Probably the latter. Although, it could be that the data points are calculated on the fly or starting from some arbitrary value.

Then again, since it's shady to begin with, disguising their mistakes wouldn't surprise me.

> What's puzzling is that they also bumped up the historical Zestimates

There are two things a "historical estimate" could mean:

1) What did Zillow say the value was on day X?

2) What does Zillow now think the value was on day X?

You and others seem to be thinking of (1), but I think Zillow actually is doing (2). Since the goal is to figure out what the house is worth and how its value has changed, I think (2) is actually the right metric here.

(2), of course, has the advantage of obscuring past mistakes and producing a nice smooth chart.
Unless you think they should never update their algorithm, approach (2) is just making the chart consistent with itself.

The useful part of these charts isn't really the current Zestimate, but rather the trend line of the home values, especially at the neighborhood level.

If you don't recalculate the historical values, then you break the ability to see the trend line.

If you really want to see the results of previous models, then the way to do that is to plot a separate line for each model, but you can see why they wouldn't want to do that from a product perspective.

Every county in the US that publishes historical valuations seem to get by without overwriting history.
Yes, but those are actual valuations based on sales prices, not predicted valuations based on an algorithm. Different thing.
Interesting. While I agree the results are likely to be inaccurate, this case study targets only the three largest cities in the US. Urban centers probably aren't the best population from which to sample.

In a case like this, we'd probably want to use locations based on number of home sales; and even then, we'd need to account for the amount of time between homes changing hands i/e houses that are flipped. There's likely something to be said about the difference between the nation's average sale price and each area's average sale price; assuming that may affect Zillow's estimations.

Anyways, I like that you're doing studies like this, regardless.

I'd guess part of why those cities were chosen were to have enough data points to draw conclusions, since all the examples are from a narrow time frame.
I don't have an opinion on whether zestimates are accurate. But this blog post has 9 samples from only 3 cities. And the author seems to be making claims based on that. This is a lame "thought leadership" post as a subtle ad for his company.

At least give me 10-20 homes in each of 10-20 cities if you are trying to convince me of something systematic.

Agreed. If you're going to try to do any sort of data-driven study, citing a sample size of 3 houses per market is ineffective and very likely to be misleading.
This is a good example of how commonly and easily 'faux' statistical evidence can incite large reactions primarily composed of anecdotes.
The only really interesting thing I saw in this post was the discussion with realtors. It's significant that Zillow uses long-term averages, and therefore can't price in abrupt market shifts even once they're common knowledge. If nothing else, it implies that "Zestimates" are a bit dubious in tight urban markets where high variance is more possible. Of course, it also looks like Zillow is aware of that challenge.

The ostensibly data-driven part was by far the least convincing section - as soon as I saw the sample size I decided it was nothing more than a starting spot-check.

Perhaps Zillow is more interested in showing the longer-term market value of a home so buyers know if they're being asked for a high demand-driven price. A tight market like that can spring up quickly and fade away just as quickly. At some point most people are going to look to resell or to at least not be forever upside down because they bought during a tight market.
It's even worse than that, really. The author actually concedes that Zillow's listed median error is accurate--but then cherry picks a few cases that are outside that margin, and uses these to make sweeping claims about how its time to stop using Zestimates. But, of course, a few cases outside the median error, here or there, are virtually guaranteed to exist no matter how good the model is.

It seems to me that the real news here is actually that the Zillow median error is really good, given the data they have to work with and the scale on which they are operating. Of course, this does not mean that the Zestimate will be valuable in all cases. Individual buyers, sellers, and their agents and appraisers will often have fer better information about any given home. But everyone should have realized this already.

I bought a home last year, and each realtor I talked to brought up how terrible Zillow was before I ever mentioned it. If you look at the WaPo article all the comments are by other realtors, so I'd take those comments with a grain of salt. Oddly, my mortgage guy never looked at Zillow (that I saw) for figuring out values, so shrug.

I'm sure the data is inaccurate, but it's somewhere to start the conversation and let people feel like they have some power in the process of buying and selling homes.

And as people like to say about Uber, realtors are welcome to make a competing site that's as free and easy to use.

No one takes a Zestimate seriously in real estate (not realtors, not banks), so there's no need to create a free competitor.

Financing still lives and dies by the professional appriasal; as long as banks prefer that, Zillow has zero pull.

Unfortunately I wasn't able to afford to have a professional appraisal done for every house my realtor showed me. I'm glad you were in a different position.
Comparables are what a Zestimate is without the BS "special sauce" that Zillow makes you think exists. Comparables are free, and based on existing property market data (typically sold similar properties looking back a certain period of time).

https://www.biggerpockets.com/rei/real-estate-comps-house-va...

Yet again the tech community believing a problem exists to be solved that has already been solved.

> In most markets comparables are acquired through an MLS database, and access to MLS is usually not free.

This is true; you need to pay for access to the data source (no different than having to pay for access to equities market data). Redfin succeeds at this because they're a brokerage that happens to be a tech company, unlike Zillow who tries to just pay for feeds (that usually have stale data).

In most markets comparables are acquired through an MLS database, and access to MLS is usually not free.

You can hire or become friends with a real estate professional that pays for access to MLS (or become one yourself).

But before sites like Zillow, there wasn't anything like it easily available to the average person.

> But before sites like Zillow, there wasn't anything like it easily available to the average person.

Zillow unfortunately doesn't help though, as no one takes the value seriously (except perhaps uneducated consumers entering the real estate market). Bad data is worse than no data.

Or they can be acquired via county assessor data, for free, from the court house. From the county assessor, you'll get sold price, deed type, beds/baths, and high level sundry information.
This works for historical data analysis, but falls over flat for trying to ascertain asset prices in markets, due to how much more quickly the MLS data is updated compared to public records (although, to Redfin's credit, they provide both the MLS record and public record for property events for completeness).

Some properties are only on the market for hours/days; you'd never have a chance as a market participant if you only relied on stale public records.

Redfin is indeed awesome, but they are only available in select markets where they have a brokerage.
Not all counties in all states have or make public this data. In fact, I think sold price is not reported to the county in many (maybe most?) states.
(comment deleted)
Most of Zillow listings come direct from MLS now. They've talked about this on their earnings calls.
You usually do a professional appraisal after you have a house under contract, to verify that it is worth what you were willing to pay for it- this is usually done to protect the bank... though in many cases it may protect you as well, that (protecting you) is not the appraisers job. (At least, in most cases).
When I bought my house a year ago the bank sent someone out to appraise it.
This is correct; you don't do an appraisal before getting a house, you get an appraisal before getting a mortgage. Typically you also get reappraised every time you refinance, and often the bank pays for it.
Well, yeah. Does anyone need an appraiser to tell them how much the house is worth? Don't you look at what is available and negotiate the price accordingly?
Banks are the biggest customers of appraisers. In most cases, they require an appraisal before they will lend on a property. As far as what you say- negotiating- yes, most buyers don't get appraisals done as part of the negotiating process. Some buyers do, however, get an appraisal to give them an idea what to ask for. Especially if their home is especially unique (and many people think their home is special, even when it isn't... but that is a whole 'nother ball of wax! :)
Why would you do an appraisal for every house you looked at? If you aren't seriously interested in it, what's the point in knowing what it's worth? This might not be the best analogy, but when looking for a wedding attire, do you walk into a store and look at the sticker price of every tux/suit/gown on the rack and then decide which to buy (especially without price comparing other places)?
> each realtor I talked to brought up how terrible Zillow was before I ever mentioned it

So people who take percentage of the deal disparaged a tool that would net them less money...quelle surprise.

I found and bought a house through Zillow and the selling agents were really irritated by this, and in many cases would ignore the Zillow agent (who was great). I think a lot of this is the fear of competition and the threat to their bottom line
There are competitors that work more closely with realtors and also competitors that compete with just the zestimate, such as Smartzip.
My bro-in-law became a real estate agent a couple years ago. Recently we were discussing Zillow, and he went into a tirade about how terrible it is. I just mentioned that it's a good tool to gauge the price trends of my neighborhood, but he began countering with canned, carefully scripted responses that sounded like indoctrination from his broker/employer. I just sighed, before the topic turned to how important it is to pay the full 6% when listing your house.

The real estate industry is one of the most protectionist industries out there, and any threat of disruption, no matter how minor, is met by an army of vitriol.

What's funny is that Zillow's real business is selling advertising to realtors. The idea that they are out to get realtors is ridiculous.
What were the arguments for the 6%? There is a broker in the Bay Area who advertisers that they take half that but the complaints against them see to point to the fact that you'll only get offers from other affiliated agents, and many agents won't share the listing with clients if they can avoid it because they want their full commission.

I'm the Bay Area I feel like a seller's agent is grossly overpaid for what is often a weekend of open house showings and then fending off multiple above list price zero contingency offers.

I've sold houses using discounted brokers and he hates that. He said that buyer agents will dissuade buyers from looking at houses offering less than "standard 3%" to the listing agent (another protectionist tactic).

I countered that I don't care. In this day and age buyers are savvy enough to jump on Zillow and Redfin and find my house, and they will demand they see it. If I hired a buyer agent and they tried to talk me out of showing a house over a commission I'd fire them right there.

> Oddly, my mortgage guy never looked at Zillow (that I saw) for figuring out values, so shrug.

A few years back I called Wells Fargo to get some info on refinancing, and the first thing the loan agent did was get on Zillow to calc the existing LTV.

Better question is "how bad are the compared to those of professional appraisers who look at half a dozen comprables?" An even more important question is "how biased (based on who is paying) are those professional appraisers?"
Home appraisal in the real estate market is often not very rigorous. There's a basic set of rules the vast majority of professional appraisers use to derive a market estimate. It's weighted heavily toward "comparable" (homes with similar size, room configuration, etc.) nearby recent sales with adjustments for certain renovations, condition (e.g. deterioration; estimates start from an assumption of "good, working condition"), and possibly other factors.

Actually computing the market value as a "cost-to-build-today" is rarely done, at least for home sales the typical consumer is going to be involved in.

And those appraisers can be very biased in either direction.

The standard HUD uniform residential appraisal report requires a cost to build estimate. I have never seen one without that estimate completed. However, I bought a home last month which had 2 different appraisals done on it, and neither one had any weight given to the cost to build. (Despite the fact that the appraiser claims otherwise, I can mathematically prove that there was no weight given to that method.)
I can't imagine any scenario where cost to build would weight into appraisal.
If the appraisal is being used to determine appropriate insurance coverage or even an insurance pay out; or possibly a court ordered payout if someone else were to destroy a property; then it would likely be the primary weight.

But lenders generally don't care so much about the cost to build, so an appraisal ordered by a lender generally won't.

That being said, my recent appraisal (I was financing the purchase) said "All 3 approaches to value considered with most weight placed on sales comparison approach."

I could mathematically prove this statement false, as the cost to build had zero consideration on the value. I think it is mostly a canned statement. Maybe they have a different definition of the words "considered", and "most". :)

Yes, but the estimate provided there is, in my experience at least, rarely computed rigorously. And even when it is, I've never seen it used as part of the appraisal.
Isn't it strange how the appraisal always seems to come in just a little above where it needs to be in order for the financing to be approved?
An appraisal is telling you what the market will bear for a particular property. What better indication of the market value than the most recent price agreed to?
You're missing the point. May appraisers are under the thumb of the mortgage brokers. See http://www.nakedcapitalism.com/2014/12/bill-black-mortgage-a...

[I]n 2007the New York State attorney general sued First American: relying on internal company documents, the complaint alleged the corporation improperly let Washington Mutual’s loan production staff ‘’hand-pick appraisers who bring in appraisal values high enough to permit WaMu’s loans to close, and improperly permit WaMu to pressure …. appraisers to change appraisal values that are too low to permit loans to close” [FCIC 2011: 92].

(comment deleted)
Ohio, residential home, Zillow = $310,000, sold for $210,000
Zillow isn't accurate for Austin.

Using two identical homes in my neighborhood about 3 doors apart from each other, mine / neighbors.

Zillow - $387,546 / $384,326

Redfin - $417,808 / $420,737

TCAD - $368,876 / 372,802

(Recent Appraisal) $427,000 / (Recent Sale) $429,000

What's worse, is that when I contested my taxes Zillow used the TCAD numbers to lower the value of my home on their site. Uh... Zillow, you know it's in our best interest to contest our taxes, right? Has nothing really to do with the value of our homes... it's a mix of us wanting to pay less and the City / County wanting us to pay more.

I'm sure there are more scientific studies, but for my house I'd say Zillow is ~10% under-market. When I went in to contest my taxes this last year, I was told by a friendly TCAD employee that Redfin numbers were better and that I should use Redfin when trying to cite examples of comparable properties with lower taxable values.

Keep in mind most of the value depends on your realtor's ability to show the house, how well you stage your house, and the number of of interested buyers.

As far as I can tell, Zestimate works by taking prices of recently sold nearby houses and then estimate gain over previously sold prices or the property tax estimates. If Zillow wasn't in the picture, you would likely do the same thing. This obviously would generate inaccurate estimates if house wasn't in market since many years or if nearby houses are too different or too few. Another source of inaccuracy is that they don't seem to take in account school zoning very well. So house in good school area can inflate price of nearby house which doesn't have same school.

In my experience, Zillow has played big part in house pricing. Seller in hot market will almost always set the max of Zestimate and price in their mind. That prices decreases only slowly over time if buyers don't show up. In hot market usually some sucker will show up anyway and Zestimate becomes self-fulfilling prophecy.

This article failed to compare Zestimates to any of the numerous competing estimation services and also neglected to mention that the primary competition for these computed estimates is intuition of Realtors and brokers.
At some point my home was converted to a duplex, and converted back into a single family home before I bought it. I've tried multiple times to get Zillow to fix the listing but they've been "unable" to do even such a simple thing...

I suspect their data quality is pretty terrible if they're unable to do any curation, so it's no wonder the zestimates are all over the place.

Considering a Zestimate doesn't take into account anything about the aesthetics or actual character of the house, being off by 20% doesn't seem like it is that much. People respond to more than just the pure square footage and neighborhood. The Zestimate should be taken as nothing more than a directional starting point for someone who doesn't have the time to actually look at neighborhood comps.
Isnt that what it is marketed as? I found it useful as a starting point when shopping.
Exactly. I think the writer is taking issue with trying to use it as a more exact science, and calls it out for being within 20% of the sale value 90% of the time, which I think is pretty reasonable considering the high amount of subjectivity in home purchasing.
Exactly...looking at a neighborhood of $500k houses, a fixer-upper vs a recent remodel could easily change the price by 20%, and Zillow obviously doesn't have attributes for the condition of the house.
I don't know how accurate these things are, though my gut tells me not to trust them. That said, as a recent home-buyer, I can state without uncertainty that real estate agents hate Zillow, and Zestimates in particular.
I concur. Though I tend to find this attitude pretty ridiculous since the status-quo home valuation process is just as bullshit at the zillow strategy is.

Status Quo in my experience goes:

1. List your house (usually at slightly above whatever the realtor thinks it might sell at)

2. Someone's willing to buy at listing price +/- 5%

3. Bank sends an appraiser whose job it is to figure out what a house is worth (guess what: see #2 for relevant demand evidence), and magically the number comes remarkably close to the price sorted out in #2 massively most of the time

I've never heard a real estate agent complain that zillow's numbers are too low, mind you... only ever that they're too high, which makes sense to be frustrated by if your biggest challenge is controlling your time-costs.

Yes, sites like those - even if with "wrong" info, alter the information asymmetry that estate agents had and that used as part of their profession.

Only seemingly unrelated, just in case, the classic from Frakonomics:

https://www.wired.com/2005/05/realestate/

Thought exercise: if you were Zillow, what additional data would you need to make the estimates more accurate? And where would you get it?
Zillow's Zestimates are particularly obnoxious because they provide little detail regarding why the values change and they retroactively revise them. I bought a house this summer that had an inexplicable drop in estimated value of about 15% a month or two before the house was listed. I paid closer to the previous estimated value than the post-drop value. Now when I look at the house on Zillow, the drop is nowhere to be seen and the past year is a smooth, barely sloping line. Never was there any indication of why they had a 15% drop, which was particularly frustrating when trying to determine how much to offer for the house.
The values change because estimated sales turn into real historic data over time.
I find Zestimate over time to be useful. I've been watching my Arizona property climbing up since 2010 when it was estimated at rock bottom, or about 50% of its purchase value (sigh). Now it's estimated to be about 90% of its pre-crash value. But as with all private transactions, bottom line it's worth what someone's willing to pay for it.
Realtors who feel threatened by Zillow love to complain about Zestimates. Zillow has made it clear time and time again that they are not 100% accurate and even allow homeowners to edit them.

Zestimates are a tool that homeowners like to use. Zillow knows that. Good Realtors won't feel threatened by it but instead leverage it to provide prospects with a much more complete and accurate estimate.

Realtors need to stop blaming Zillow & Zestimates for their own weaknesses. It is getting old.

Zillow deserves a fair share of the blame because they prominently feature a single number with dollar-level precision and bury the caveats multiple clicks away on a separate page.

Some basic UI changes could avoid all of that: e.g. instead of showing $123,456, show a range like $115,000-135,000, and keep the last few digits at zeros based on their estimated error rates. Similarly, their graphs could use shading to make it clear that they're showing the mid-point of a range rather than a discrete value.

> Some basic UI changes could avoid all of that: e.g. instead of showing $123,456, show a range like $115,000-135,000

So true. When you know that you are not displaying an exact dollar amount you should not use exact dollar values. Exact dollar values afford exactness even though the branding is an estimate.

This might very well backfire on them. The buyers will want to buy at the lower price and the seller would want the higher price. Both parties end up being unhappy even if the end price was within the given band.
There is a range immediatly below the number, which seems pretty reasonable to me. For my place it shows a range from -8% to +10%
Where? On the public website I see only the $123,456 value unless I'm in the wrong A/B test group.
As silly as it is to say this, people (you?) should contact them and suggest these things.
Just over two months ago, I sold my house within $1,600 of the "Zestimate". I then turned around and purchased another house over 10% below its "Zestimate"
The Zestimate for my house is incredibly close, within $2k of a recent appraisal. There are a lot of comparables in the area, so they have a decent amount of data to work with.

I have a friend who just bought a 3,000sf house on 1/2 of an acre in Lexington, VA for $31k. Zillow has it at $400,000. He's either incredibly lucky and should buy me a beer, or Zillow is the worst. Leaning towards the latter.

"$31k. Zillow has it at $400,000."

Drop a 0 there? It seems likely that $31k would go beyond lucky into probable theft....

It was an investment property that needed some work. He had to put about $10k into it, but it's good to go now.
Thank you for clarifying. I mean, even being off by $90k would have been plenty wrong enough to justify your statement.
Like the previous reply, I'm going to assume you meant $310k, not $31k, as $31k for that property doesn't make sense based on the factors you listed unless the place is a dump.

As to the disperity, this could be easily explained by seller factors. Maybe the house is actually worth $400k, but they needed to sell it, even if less than it was worth (had a contract on another home, home was a foreclosure, etc etc).

I did not drop a zero. He paid cash.
is it a nuclear waste dumping ground? What's the mitigating factor?
In the country and needed about $10k of renovations. Sold as-is. It may end up being worth $200k, if he's lucky. Zillow is just incredibly far off.
I don't know about "zestimates", but they seems to not be able to do something as simple as email address verification properly. Admittedly, based only on the five email's I have received from realtors in North Carolina just this morning (I live in Nebraska and have no plans to move any time soon).
I am quite disappointed in the research that went into this article.

(1) Author discovers that Zillow has a specific claim about accuracy: "around 90% of the listing Zestimate is “Within 20% of Sale Price” of the home". Author then gripes that this is "a really large margin for error". Really? Because I was prepared to praise Zillow for such honest and clear reporting. It only costs a few hundred dollars per home (Zillow covers nearly every home in the US) to get a professional to produce an assessment, and these assessments are usually only a little more accurate than what Zillow is claiming.

(2) Author pulls up tables from Zillow showing a median error. Which is an excellent way to analyze this and is (I think) extraordinarily small. Author then proceeds to do other stuff.

(3) Author decides to verify the values by checking a total of NINE houses. Is the accuracy of that test expected to impress us? Furthermore, one of these 9 is rejected because "(obviously a data glitch or input error. Dismiss this result)" -- but that's the whole POINT of using median error; it isn't affected by outliers.

(4) Author finds that all of the values (all 3 of them) in Staten Island were pretty far off. Author asks professionals who say that the local market o Staten Island is behaving quite unusually at the moment. Reporting on this is the only thing I think the author did right.

Yeah people dump on Zillow quite a bit. Probably because home values can be a very emotionally stirring event.

Zillow's aggregate economic data is solid, often a better predictor than the Case Shiller Index. I expect that Zestimates will improve over time as the data trickles down and lessons are learned.

This is correct. I used to work at Redfin and estimates are a sore point for homeowners. It's one large pissing contest for their egos. Everyone has the same complaint, "Why isn't my home priced as high as that one?!?"
I'm the other guy. I bought my house for 260k a year ago, redfin now lists it at 330k, and there is just no possible way that is correct.
I'm in the same boat as you. The Zillow estimate in my neighborhood seem to be about 15% higher than the sale price on any particular property. It's kind of academic for me, since I don't plan on selling any time soon, but it would be a mistake to do financial planning based on those estimates.

The funny thing is the estimate doesn't change when a sale closes. I would think the surest indication of what you can get for a property is what it sold for last week.

> "Why isn't my home priced as high as that one?!?"

Could Redfin not explain why?

I've never bought a home in an area Redfin is in, so I havent used the site before. If Redfin knows the formulas/parameters used for the estimates couldn't they break it down and do side by side comp of $ amounts?

A long time ago I got a real estate appraiser's license. Here is the procedure for appraising real estate, as taught by my state:

1) Pick three homes that you feel are comparable.

2) Pick out the material differences between those homes and your client's

3) Make up a value for each of those differences, and adjust accordingly

(Unofficially speaking) If you don't like the number you come up with, just make different decisions in steps 1-3 until you get the number you want.

In the late 90's I worked for an online mortgage broker, and 9 times out of 10 when we ordered an appraisal, the appraiser would ask us what the selling price was and return an identical figure for their estimate.
I purchased a home couple years ago. The appraiser I used did exactly this.
Amazing that people still pay for that.
"People" don't. It's a required part of the due diligence to get a mortgage. No appraisal, no loan.

For the most part, the banks and brokers re-sell those loans immediately, so they don't stay on their books. All they need to protect them from lawsuits later is proof that they followed the process to the letter -- which involves getting a written appraisal.

No mortgage broker or Realtor ever got rich by picking a fight with the appraiser to lower the appraised value ... there is literally nobody present at the table who has an incentive to push the value down.

Aside from the buyer, who can pay for his own appraisal.
Usually a buyer has already agreed to buy a house at a certain price. So if the appraisal comes back low, the seller doesn't lower the price. Instead the buyer just doesn't get a loan so they don't get to purchase the house they had their heart set on.
A friend of mine saved $15K on a house because the appraisal came in low.

If the bank won't write a loan for the agreed price, the original contract is void, since it's contingent on loan approval. If the seller's heart is set on moving, he might well lower the price.

Really depends. In the Bay Area these days, sellers will usually only take offers with zero contingencies (because they can). Even if you do get a financing contingency in your offer and the deal falls through because of a low appraisal, there's almost always a back-up offer waiting in the wings that can cover the low appraisal with an additional downpayment (the bank doesn't really care about if the house appraises for the selling price, they just want to make sure that their Loan-to-value (LTV) ratio is OK.
That's really playing with fire. Good RE agent would tell their client so. Even if you're rich and buying in cash, at least inspection contingency is a must, but otherwise unless you have tons of money and don't really care for nasty surprises inspection could uncover - zero contingency is the riskiest bet one can make in one's life. My opinion, if you're on the market where sellers won't accept a contingent offer, it's better to get out of that market. I've bought house in BA recently, and there were plenty of sellers willing to take contingent offers (backed by some solid bank paperwork, maybe, but not just requiring to commit blindly). Of course, BA has many different places and in some market could be so hot sellers can get away with anything. But it's usually not good to get in such place as a buyer.
(comment deleted)
Usually, the seller provides an inspection report performed in advance. It's a big leap (I did this a year ago ...) and my Realtor is as old-school as they come, but if you want a single-family property in a hot area, it seems to be table stakes.
Sure, just depends on whether or not there are other parties who will pay the Delta.
Happened to me when I sold my house the value came in 10k lower (really... now that same place is 30-40k higher than what I was asking a year ago).
I understand that it's required, and that organizations often follow processes long after they're revealed to be foolish.

But it's been a decade now since the housing bubble revealed that appraisals are meaningless.

Why has no one eliminated them?

They are required by law or by long-standing policy in a government-run body (which is practically the same thing). Laws DO change to keep up with the times, but they do so rather slowly. In 50 to 100 years, if there is continuous clear evidence that home assessments are highly unreliable and not particularly profitable to anyone, then you should expect them to be eliminated.
Because they reasons they were required were still valid, so instead efforts were directed at the factors that made them ineffective at addressing the need motivating them.

Whether those efforts were successful or not is another discussion.

This. As long as the appraisal is close enough for plausible deniability, it serves the purpose. The originator just needs a number that's close enough for reality that whoever they sell the loan to (probably the government) can't come back and accuse them of fraud.
Apparently you've never actually taken out a mortgage.
When we were selling our home we didn't have a good idea of what it was worth so we called an appraiser and the first think he asked was "what are you selling it for?" Then we found out that even if we paid him a couple hundred bucks he wouldn't even come out and inspect the house before "appraising" it.

We ended up just guessing high and lowering our price a couple of times until someone bit.

That actually makes some sense. That number is supposed to indicate the maximum someone would pay for the property. If you're in contract, especially after competitive bidding, then we know that number: it's in your contract!

The appraiser just needs to make sure that the number isn't fraudulent or crazy.

As an example of above, A couple of years ago my wife and I were selling our home and the appraisal came back lower than agreed sale price, we asked appraiser to pick different comparables and the appraised price went up 10-20k
Thats interesting, why would the seller have any control over the appraisal? The mortgage broker who requests the appraisal is typically working with the buyer.
I don't think this is "control", the appraiser [sp?] did not have to agree to do that.
You are correct they didnt have to agree to change anything. But there were other houses that were closer ( one of the comparables that he pcked was in a different neighborhood and was waaay cheaper ) and houses that were more similar so we pointed that out. He agreed and redid the estimate with houses that were closer both on distance and in size/amenities. All of this was throught the realtors ours and theirs.
True, but the appraiser has zero incentive to disagree (unless it's overtly fraudulent there's no such thing as wrong appraisal) and probably some incentive (repeat business) to keep the agent they are working with happy.
This is how it USED to be done, and was a significant contributor the housing bubble that lead to the recession.

It's different now, not necessarily less subjective but the incentive for creative appraisals has been drastically reduced. Prices have to be justified and mortgage brokers can't interact with the appraisers now. Apprasing has to be entirely separate from the underwriting process and is now entirely blind. In some states, appraisers are now accountable for significant deviations. The turnaround time is also significantly longer. I've heard reports from some people of appraisals in bigger markets taking as long 9 weeks.

It's actually a pretty interesting process, ripe for disruption if anyone wants to take a crack at it.

The separation part only sounds good. However, if you get an incompetent appraiser, there's literally nothing you can do about it. It's also hard to weed out the incompetent ones, as they are now independent from their customers. We got an incompetent on our last deal. Couldn't find the house. Misidentified the street. Couldn't find the electric panel. I could go on. Took several weeks to get the appraisal done, and there was nothing we could do about it because independent.
ya, i understand. but that sounds like a very different issue.
It's kind of scary that million dollars' decisions (and in aggregate, probably billions) are made on the basis of solely "let's look what others are doing and then fudge it based on gut feeling until it looks kinda plausible" and that's supposed to be expert verdict. Not that I can offer any better idea :)
Haven't you just described 90% of the VC investments made in startups in the last few years?
Well, I guess it depends. Some investors do a lot of due diligence before investing, ask for financials, plans, market analysis, etc.
> "around 90% of the listing Zestimate is “Within 20% of Sale Price” of the home"

Yes, this sounds pretty good to me. I spent some time trying to do something sophisticated for the estimates on my site https://houseprices.io/ using repeat sales of similar properties weighted by distance, but I wasn't getting anywhere near that. I gave up and just used the Land Registry index for the county in the end. There are just so many potential local factors which you're never going to capture algorithmically, not least of which is how much the current owner might have spent doing it up. Although I do get the occasional irate message in my Contact Us form telling me exactly how much!

I feel like disappointing research is going to be a given when you have such an obvious conflict. The author has a strong motivation to make Zestimates seem bad (the less reliable the resources you have to buy/sell a home on your own are the more you'll need to contact them) and a million degrees of freedom (which neighborhoods in which cities, price floor for consideration, how many examples to pull ...) to get the answer they want.

"Research" of this type should probably never be trusted.

I'm not sure at all how you can make conclusions from just 9 houses. I've been house hunting recently. Even houses in the same neighborhood are very different, and there are a lot of factors going into valuing a house. Also may be very subjective - some people would pay a lot for things that other people don't care about. Maybe Zillow is uniquely bad at one or two out of 20 factors - e.g., it can not detect a house in a good neighborhood that has been neglected and needs a lot of repair (I've seen houses in very good places that looked great from outside and required tens of thousands dollars of work on the inside just to be livable - maybe that's what happened to that -75% house - maybe it had a recent fire or flooding). Or maybe it doesn't value having a pool correctly. Who knows. But it's impossible to figure that out from only 9 houses, 3 per huge market! You need hundreds of data points to smooth out the individual differences and cancel out noise from all different criteria. That's exactly what Zillow's advantage is supposed to be - to aggregate thousands of data points. Maybe they do a good job, maybe not, but I don't think you can evaluate it by just using mere 3 houses per market of millions!