Gosh, it's amazing how they just refuse to make this tool powerful. Still can't set a custom date range, or add technical indicators. I mean these are like 2003 requirements for a decent site let alone 2020. Just off the top of my head..historical data, SEC filings, better volume data, options pricing..
I think Yahoo! Finance is an outlier in that it was pushed by internal management at Yahoo as something they wanted personally, rather than meeting the needs of a large userbase. At least that was the rumor I heard.
Still missing transaction tracking. Hopefully it will come back some day. I migrated to https://wallmine.com/ when Google Finance announced it was neutering itself. However, I've noticed Wallmine's listings are not always complete. As a couple examples: the insurance company Root isn't listed, the label for JAMF isn't correct.
Haven't looked at what is out there in a while, any other alternatives people are using?
That's because Google aims itself at the mass market; power users will be on, well, power user tools. Google and Yahoo Finance are fine for a quick glance for me, but I spend at most a few hours a year looking at these things.
And the timeline was annotated with significant news events, which was truly fantastic. This used to be an amazing product. Still, it's free, so I'm not going to complain too much.
I thought they killed this of many years ago, but maybe that was just the portfolio piece? Hard to want to delve into a Google product these days when the norm is many years without meaningful updates and the constant possibility of the project being killed.
Google Finance used to have all these features. They were already implemented and working fine in 2009! The original chart tool was much better than the current version. You could see every dividend that was issued, splits, add moving averages, etc.
Then they "simplified" it by removing all the best features and are now adding them back?
Why would I bother to use this when they have so thoroughly torched their goodwill by senselessly killing off working (and good!) products?
The world of computers is "reinvent, but slightly different".
Remember the "Briefcase" sync feature of windows 95... Now compare that to Dropbox syncing... Pretty much the same thing 25 years on. Different UI, same principle.
I never did get what the Briefcase was supposed to do. It was so poorly explained that that was it's purpose. Perhaps because I didn't have any networked computers, and the utility of a Dropbox via floppy disk was so limited...
Completely off-topic rant incoming, but Microsoft has very often sucked at advertising itself. Here's a more recent example. I bought a Windows Phone way back when. It was amazing. My favourite feature of it, believe it or not, was the keyboard. I have a sneaky suspicion that they used MS Word's dictionary somehow, because it could correct typos not just when the wrong letter was typed (for instance, typing 'aentence' instead of 'sentence'), but also words typed in the wrong order ('snetence'). It was delightful. I could type as fast as I wanted, confident that auto-correct would take care of any errors. Then I moved to Android, and I could not believe how appalling the keyboard was. But it seems that no one else was talking about it, because android users just didn't know of the excellence that existed out there.
Then, at some point, Microsoft went ahead and ruined their keyboard when they went to Windows Phone 10 or 8.1 I can't remember, and I've been searching for perfection ever since. GBoard, with it's fucking emoji page is an affront to engineering.
> Then, at some point, Microsoft went ahead and ruined
When Microsoft does things like this, and it's not clearly a dumb product decision, it's often to avoid a patent that they don't want to license or fight. I've been using 'simple keyboard' on Android, because it's just a g-d keyboard, but it doesn't do good spell check either. I didn't particularly notice spell check getting worse when I left windows phone, but there was a lot of other stuff going on.
Yes the idea was that you'd have files that you wanted to use on different computers (e.g. at work office and at home). At the end of the work day you'd sync the files to floppy, take the floppy home, and sync it to your home machine.
Remember at the time many people did not have laptops. They often didn't have home internet either. Maybe they had AOL, or maybe nothing. They also didn't have a work VPN or any way to access files on their work computer.
You are 100% correct. There are those among us that overlook this as the reason for new releases and discontinuing products.
I know I've experienced a point where the decision is made to discontinue something as adding what is needed to move forward will require too much investment versus benefit.
As far as simple interfaces are concerned for the average layperson yahoo finance is as good as it gets since google literally gimped their finance product in such a stupid fashion since they didnt know how to monetize it. So not only did you have a competitor pick up marketshare, but also offer a premium sub feature that does not compete with their insanely good free public api. what a stupid company
MarketWatch has some of the features but Google Finance used to have almost everything. It was the easiest way to see financial ratios, previous year finances, and basic screening. There’s a few free stock screeners out there like finviz but they’re all less intuitive than Google was.
So obviously they didn't choose to remove features, they were forced to rebuild it from scratch.
And along the way, instead of it being a standalone product (which was really only justified because of competition from Yahoo Finance etc.), it made more sense for it to be simply part of Search, and so give simpler results.
Does that make sense? I don't think they torched anyone's goodwill here. It's not senseless -- to the contrary, there was a lot of sense to what they did.
If you want a serious finance tool, Google (or Apple) stock "apps" are probably not going to be the right choice in the first place.
The Flash switch could explain a few months but they had years to do a small project which much smaller companies did for more complicated projects. This seems a lot more like the usual Google problem where everyone involved moved on and nobody had an incentive to do maintenance work.
They could've just ported features to HTML5. Not sure why you're trying to justify clearly sketchy behavior of billion dollar company, but maybe Google's recruiters scour HN, so I might be the crazy one
Assuming the project had any headcount to do so. But Google has (apparently) not invested much in Google Finance which is . . . fine? Tons of alternatives. Are we now saying it's a bad thing when Google cedes part of the web ecosystem to other companies?
I have no idea what you're getting at with the comment about Google recruiters.
They are really taking their time with the process of destroying financial website tools. But I'm sure that in fifty years or so they'll get around to destroying that market. ;)
Are you a google product manager? I ask because, wow, this response is condescending.
I know the original chart tool was in Flash... I used it and described the features it had. Why after 10 years is the new version less useful than before?
Yahoo finance stayed separate and has been better for it. It doesn't make sense for stocks to be integrated into search - I completely disagree.
I never said anything about Google Finance being the only tool I used. It was a perfect place to start and once I found an interesting security I would move over into something more serious.
I didn't react as strongly as the person you're replying to but I get it. The "Does that make sense?" bit, along with the rest of the comment, makes it read like a schoolteacher trying to patiently explain why something is the way it is to a student who they are sure they have superior knowledge over. The last line of the comment you're replying to makes it clear that's how
oramit interpreted it.
A bit more charity may have been warranted but I do see where he's coming from.
The first person said that Google dumbed down Google Finance "senselessly".
So, in context, I definitely took the response's "Does that make sense?" to mean "Was it a rational product decision on Google's part?" rather than "Can you understand this?".
No, it didn’t make sense to take an existing and useful site like Google Finance and dumb it down to make it a part of Search instead.
If it made sense, then there wouldn’t be a need to reverse course today.
I think it only makes sense if you look at the incentives of Google as a corporation, and disregard the opinions of the people who used and liked Google Finance.
Yesterday I saw a post on Indian investment subreddit where people were discussing how they switched to Google sheets after Google finance since it allows fetching data for stocks in spreadsheets using some function.
Anyhow it doesn't seem like Google might have ditched finance for this result. Increase in sheets usage is just a side effect.
Could someone who knows about doing this in a large tech firm speak to the difficultly of converting standard flash --> js/html? For something as common as stock charts, it doesnt seem like a big investment, is there some complexity that comes with scale?
We have transitioned a few major silverlight based applications.
A conversion made no sense at all. What we ended up having to do (and in hindsight was absolutely the right decision), was creating a new application completely.
The best way to "transition" for Google with Flash would be to probably, similarly, just create a new application.
Google Finance wasn't a Flash app though. It was just the charts which is a much less daunting situation than what you are describing with a Silverlight conversion of a full app.
We're doing a a large silverlight app right now and the auto-conversion to blazor got us < 65% of the way there, so instead of jumping from MS's old destined-to-fail tech to their new destined-to-fail tech we're going to rewrite from scratch as a web app. I hate to do this but it's the path of least smell.
> ... is there some complexity that comes with scale?
I can't see how scaling would be an issue since the tricky/ resource intense bits happen on the client end. We're building interactive charts which are comparable to the Google Finance ones with a fairly tiny developer team.
If Google hasn't migrated functionality, it's because the Finance product hasn't met their goals and they've trimmed the team back.
It’s sooooo busy! And prioritizes displaying businesses and points of interests over street names...I really dislike it now and though I vehemently refused to use apple maps for years, this distinction made me cave.
The good old bloatification. At some point Maps was nearly perfect. However the hundreds of people behind it would be let go if they called Maps "complete" so they turned the growth inwards and started creating the bloat.
> Yes but they were implemented in Flash. So obviously they didn't choose to remove features, they were forced to rebuild it from scratch.
Who forced them? I have always been Ok with Flash and see no real reason to abandon it (besides driving sales for more powerful hardware which can do the same Flash did on Pentium 2 and with less code).
> I have always been Ok with Flash and see no real reason to abandon it
Assuming you're okay Flash being a closed product of Adobe (which was a huge problem for the web), it was also a bountiful garden of security issues, which Adobe didn't do a great job of managing.
I personally never cared about products being closed. In fact I still use Windows 7 and XP (albeit my main OS is Arch Linux). It (Flash, but Windows 7 and XP also) works (and is way more efficient than modern alternatives, both in runtime and in development) and that's Ok.
There also are open-source implementation like Ruffle, Lightspark, Gnash etc (I never bothered to try any of them however).
Even the original Adobe Flash player is not dead - it will[1] be maintained by Harman for years to come.
Nevertheless, what I meant to say was nobody really forced or could force Google - AFAIK killing Flash was Google's and Apple's own initiative.
I also hate it when apps/services move backward in terms of features. I'll still give them credit when they start to move in the correct direction again, even if they would have been better off not screwing things up in the first place.
Not a good one that is targeted at non-professional investors. I agree, it's a good opportunity. The issue is that companies have a lot of latitude in how they use XBRL once you get outside of the totally standard financial metrics like revenue, gross profit, total liabilities, etc. And those are the metrics that are already largely available in services such as Yahoo Finance for free. What would be really valuable is taking company specific metrics such as unit volumes, orders, backlog, etc. And that requires a ton of work for each company and some knowledge of what the business does. Also, a lot of that data is not yet broken out into XBRL. For example, look how much data from THO (an RV manufacturer) is useful but not detailed in XBRL fields:
You can see which parts have XBRL tags by the red lines above and below. If you search the document for "orders" or "units" you can see lots of great information that is basically stranded. I've always thought that it shouldn't be that hard to use a bunch of heuristic rules to find data that is present in the last N 10-K or 10-Q filings (particularly because they tend to reference the previous year results, so you can search for numbers that are the same that are located within K words of certain strings).
But it's hard to do something really useful as a one-person effort with some clever coding, which is probably why the services that are out there are niche and expensive.
I've looked into the XBRL data - it's closer to an XML version of each company's 10-K rather than a unified accounting/reporting system for all public companies. If one company reports their income statement using 20 lines and another uses 15 lines, the XBRL data will show both exactly as they appear in the 10-K and they would not be comparable (unless your software can understand that "Net Interest Expense/(Income)" for one is the same thing as the sum of "Interest Expense" and "Interest Income" for the other). To add to that I've found some companies are filing XBRL is a non-conforming XML standard (probably software issues) and some companies just include the last 3 years of data in every XBRL document (rather than just picking the same data that's reported in the 10-K), making the consumer figure out what is the actual new information in the document.
Bigger issue, as learnt from capitalIQ, is restatements. They are SO common. Not as in mistakes that were made but also on reclassification of business lines, discontinued operations, etc etc. So just parsing is not sufficient, there’s a probably hair pulling historical alignment exercise underneath as well.
We (https://tagnifi.com) offer standardized fundamentals based on the XBRL data. As others have stated, XBRL is complex and full of corner cases. Our standardization process softens these sharp edges.
It's not perfect – XBRL is complicated and companies file with lots of proprietary metrics that aren't included in GAAP/FASB standards – but we hope to do more with it in the coming months.
I was disappointed when Google Finance was demoted a few years ago, and many of its handy features were stripped away. I hope it's back for good, and that Google will continue to improve it.
I was just playing with building a tiny stock tool for fundamental analysis a few days ago[0]. It's in the extremely early stage and currently uses Polygon's API (made by ex-googlers), although we're looking for an alternative. Polygon's data is often flawed, buggy, absend or outdated.
I did this a few years ago with a friend. We settled on using a thing or 2 from Quandl as the best fit (hundreds of $ instead of tens of thousands) after we realized XBRL was a nightmare. We did this privately for investment purposes; if you're making a user-facing thing, perhaps their terms won't fit your case.
I have looked into Quandl, as well as other finance API's. I just had a call with Intrinio, which will offer me a demo soon. I will have another demo from S&P Global, which I fear will be unaffordable, but it's always worth a try.
May I ask what specific usecases your internal tool was built for? And were you in the end able to build it to your satisfaction? I welcome any thoughts.
At the time, there was nowhere that we knew of on the net that showed fundamental data and metrics yearly or quarterly in columns and didn't cost many thousands. Google Finance used to sort of do this for 2-3 periods, but the available fields were limited and the UI was not friendly to research.
So we made a tool where you could: select various series of data, see their historical series, and also visualize your selections. This was helpful to assist our manual screening after we ran our automated filters (think Morningstar, just as a database query).
A real manual screen setup might start with something like: "I want retained earnings, outstanding debt, and fully diluted share counts for the last 5 years for AIZ and their top3 competitors". Plotted out, perhaps you notice that property insurance segments are fairly cyclical (weather/season) which can be a useful thing to tease out for an outsider/amateur. Or maybe you note that companies don't always break their revenue sections out the way you want, so you can't even do this analysis! (Looking at you, Alphabet & Youtube.)
We started with Yahoo prices and Yahoo fundamental data, but eventually realized that we still had to verify a lot of their data (against EDGAR). Quandl was a step up in price but we worried a lot less about data integrity with that, and by this time we were seeing some confirmation of results for our strategy.
We had a good ARR sustained over a couple years. However, the scalar value of the (risky!) gains was far too small to justify either quitting our tech jobs or sustaining an extra 40+ hours a week. The lessons were great, and seeing shenanigans in real life (looking at you, TSLA) but not in headlines is always fascinating.
We did do some concepting of a subscription for the research product, but I'd be surprised if that was realistic given our lack of marketing knowledge (and $). RH is a thing now though, so perhaps we were wrong.
And you still can't do a basic slider to set a time range?
I want to see a zoom of how the stock did from 2012 through 2015. This functionality was stripped and still not returned?
This is the equivalent of having 5 inconsistently scaled preset zoom levels on Google maps. Zooming in isn't complex, and a lack of it completely weakens how you used a map product (or stock viewer).
It's amazing to me that they still haven't fixed what they tore out. Google really struggles with supporting customer products.
Oof, this is like a glorified "my first stock monitor" app. No options drill down, no relevant news or dates, not even comparisons! This is sad and significantly inferior to the old experience - certainly material UI alone wasn't THAT big of a feature?
A bit off topic, but does anyone know of a good competitor to the old Google Finance? I loved the old site where you can stack indicators and fancy box and whisker charts.
The original from years ago had a solid portfolio management tool where you could track your cost basis and positions. Still haven't found another replacement for that other than good 'ol Excel...
I don't think Symbols are technically unique between exchanges, since some exchanges are very old, although I would have trouble coming up with an example where the major US exchanges have conflicts. Given that the Google Finance has a search bar prominently on the page. I would expect/hope that the top result would be used in the event of a conflict. It's very probably in the event of a conflict that one of the potential companies is very common while the rest are niche listings on foreign exchanges anyway.
They're generally unique within a country but not between countries. Eg "Intrepid Mines trades in Canada on the TSX as IAU whereas in the US, IAU is the iShares gold trust ETF" [0]
at a point in time and within a country/region, mostly yes. It works fine for humans, but is really messy when you try to make it rigorous enough for software.
It's a horrible mess. As someone mentioned, some use numeric symbols that I'd expect not to be unique, and while I assume most are unique because everyone uses a different format, you will probably have collisions.
Even worse is that there seem to be variants (or is it the same stock simply traded on different marketplaces), which may be denominated in the same or a different currency.
And getting from one of the symbols to an ISIN and from an ISIN to the symbol is rarely easy. And I'm not sure if an exchange:ticker tuple always maps to exactly one ISIN (and one ISIN can definitely map to multiple exchange:ticket pairs).
Has the same bizarre bug as yahoo finance. Go to the search box and type "F" without the quotes. That's Ford Motor's stock symbol. Press enter. What does it bring up? The first match for "F" which was "FTSE 100". Dumb dumb dumb. If you want "F" [1] you have to click the 2nd or 3rd or whatever result. Painfully bad design.
[1] Or TIPS or IEF or SHY or basically everything I've tested.
I used to love this product. They explicitly told users to go away a couple years ago. These updates still pale to the old functionality and I am not putting my data back into Google Finance because at some point they will get tired of supporting it and tell me to go away again.
142 comments
[ 4.7 ms ] story [ 197 ms ] threadhttps://blog.google/products/search/google-finance-makes-inv...
Haven't looked at what is out there in a while, any other alternatives people are using?
The new SVG replacement is not as good.
Sadly this seems to continue down Google's trend of simplified controls, fewer power-user type interfaces.
And the timeline was annotated with significant news events, which was truly fantastic. This used to be an amazing product. Still, it's free, so I'm not going to complain too much.
It flew through every action without ever hiccuping for me.
Google trends predicts interest on the "Stocks" topic is about to go through the roof in the US. Interesting.
Google Finance used to have all these features. They were already implemented and working fine in 2009! The original chart tool was much better than the current version. You could see every dividend that was issued, splits, add moving averages, etc. Then they "simplified" it by removing all the best features and are now adding them back?
Why would I bother to use this when they have so thoroughly torched their goodwill by senselessly killing off working (and good!) products?
Remember the "Briefcase" sync feature of windows 95... Now compare that to Dropbox syncing... Pretty much the same thing 25 years on. Different UI, same principle.
Then, at some point, Microsoft went ahead and ruined their keyboard when they went to Windows Phone 10 or 8.1 I can't remember, and I've been searching for perfection ever since. GBoard, with it's fucking emoji page is an affront to engineering.
When Microsoft does things like this, and it's not clearly a dumb product decision, it's often to avoid a patent that they don't want to license or fight. I've been using 'simple keyboard' on Android, because it's just a g-d keyboard, but it doesn't do good spell check either. I didn't particularly notice spell check getting worse when I left windows phone, but there was a lot of other stuff going on.
The MS team that did Clippy would like to have a word with you...
It was an awesome piece of software, that nobody knew about.
Remember at the time many people did not have laptops. They often didn't have home internet either. Maybe they had AOL, or maybe nothing. They also didn't have a work VPN or any way to access files on their work computer.
I know I've experienced a point where the decision is made to discontinue something as adding what is needed to move forward will require too much investment versus benefit.
So obviously they didn't choose to remove features, they were forced to rebuild it from scratch.
And along the way, instead of it being a standalone product (which was really only justified because of competition from Yahoo Finance etc.), it made more sense for it to be simply part of Search, and so give simpler results.
Does that make sense? I don't think they torched anyone's goodwill here. It's not senseless -- to the contrary, there was a lot of sense to what they did.
If you want a serious finance tool, Google (or Apple) stock "apps" are probably not going to be the right choice in the first place.
I have no idea what you're getting at with the comment about Google recruiters.
I know the original chart tool was in Flash... I used it and described the features it had. Why after 10 years is the new version less useful than before?
Yahoo finance stayed separate and has been better for it. It doesn't make sense for stocks to be integrated into search - I completely disagree.
I never said anything about Google Finance being the only tool I used. It was a perfect place to start and once I found an interesting security I would move over into something more serious.
Does that make sense?
A bit more charity may have been warranted but I do see where he's coming from.
The first person said that Google dumbed down Google Finance "senselessly".
So, in context, I definitely took the response's "Does that make sense?" to mean "Was it a rational product decision on Google's part?" rather than "Can you understand this?".
You not personally finding it useful is not the same as it not "making sense".
If it made sense, then there wouldn’t be a need to reverse course today.
I think it only makes sense if you look at the incentives of Google as a corporation, and disregard the opinions of the people who used and liked Google Finance.
Anyhow it doesn't seem like Google might have ditched finance for this result. Increase in sheets usage is just a side effect.
A conversion made no sense at all. What we ended up having to do (and in hindsight was absolutely the right decision), was creating a new application completely.
The best way to "transition" for Google with Flash would be to probably, similarly, just create a new application.
I can't see how scaling would be an issue since the tricky/ resource intense bits happen on the client end. We're building interactive charts which are comparable to the Google Finance ones with a fairly tiny developer team.
If Google hasn't migrated functionality, it's because the Finance product hasn't met their goals and they've trimmed the team back.
Do you have a recommendation of any free tool?
If it's NIH Syndrome and/or a low-status in terms of Google promotion fuel, we should just say that.
It went from being a navigation tool too being the "how do I get to consumption" tool.
Who forced them? I have always been Ok with Flash and see no real reason to abandon it (besides driving sales for more powerful hardware which can do the same Flash did on Pentium 2 and with less code).
Assuming you're okay Flash being a closed product of Adobe (which was a huge problem for the web), it was also a bountiful garden of security issues, which Adobe didn't do a great job of managing.
There also are open-source implementation like Ruffle, Lightspark, Gnash etc (I never bothered to try any of them however).
Even the original Adobe Flash player is not dead - it will[1] be maintained by Harman for years to come.
Nevertheless, what I meant to say was nobody really forced or could force Google - AFAIK killing Flash was Google's and Apple's own initiative.
[1] https://services.harman.com/partners/adobe
https://www.sec.gov/ix?doc=/Archives/edgar/data/730263/00007...
You can see which parts have XBRL tags by the red lines above and below. If you search the document for "orders" or "units" you can see lots of great information that is basically stranded. I've always thought that it shouldn't be that hard to use a bunch of heuristic rules to find data that is present in the last N 10-K or 10-Q filings (particularly because they tend to reference the previous year results, so you can search for numbers that are the same that are located within K words of certain strings).
But it's hard to do something really useful as a one-person effort with some clever coding, which is probably why the services that are out there are niche and expensive.
It's not perfect – XBRL is complicated and companies file with lots of proprietary metrics that aren't included in GAAP/FASB standards – but we hope to do more with it in the coming months.
I was disappointed when Google Finance was demoted a few years ago, and many of its handy features were stripped away. I hope it's back for good, and that Google will continue to improve it.
[0] https://eftegarie.com/StockDNA.html
May I ask what specific usecases your internal tool was built for? And were you in the end able to build it to your satisfaction? I welcome any thoughts.
So we made a tool where you could: select various series of data, see their historical series, and also visualize your selections. This was helpful to assist our manual screening after we ran our automated filters (think Morningstar, just as a database query).
A real manual screen setup might start with something like: "I want retained earnings, outstanding debt, and fully diluted share counts for the last 5 years for AIZ and their top3 competitors". Plotted out, perhaps you notice that property insurance segments are fairly cyclical (weather/season) which can be a useful thing to tease out for an outsider/amateur. Or maybe you note that companies don't always break their revenue sections out the way you want, so you can't even do this analysis! (Looking at you, Alphabet & Youtube.)
We started with Yahoo prices and Yahoo fundamental data, but eventually realized that we still had to verify a lot of their data (against EDGAR). Quandl was a step up in price but we worried a lot less about data integrity with that, and by this time we were seeing some confirmation of results for our strategy.
We had a good ARR sustained over a couple years. However, the scalar value of the (risky!) gains was far too small to justify either quitting our tech jobs or sustaining an extra 40+ hours a week. The lessons were great, and seeing shenanigans in real life (looking at you, TSLA) but not in headlines is always fascinating.
We did do some concepting of a subscription for the research product, but I'd be surprised if that was realistic given our lack of marketing knowledge (and $). RH is a thing now though, so perhaps we were wrong.
I haven't used the crippled version in years, why would I ever go back?
I want to see a zoom of how the stock did from 2012 through 2015. This functionality was stripped and still not returned?
This is the equivalent of having 5 inconsistently scaled preset zoom levels on Google maps. Zooming in isn't complex, and a lack of it completely weakens how you used a map product (or stock viewer).
It's amazing to me that they still haven't fixed what they tore out. Google really struggles with supporting customer products.
Biggest gripe is that the URL format can't be user to directly open a stock's page without remembering the exchange a given stock is on.
This URL will bring up Google's stock:
> https://www.google.com/finance/quote/GOOG:NASDAQ
This URL will not:
> https://www.google.com/finance/quote/GOOG
[0] https://www.quora.com/Can-2-publicly-traded-companies-have-t...
Even worse is that there seem to be variants (or is it the same stock simply traded on different marketplaces), which may be denominated in the same or a different currency.
And getting from one of the symbols to an ISIN and from an ISIN to the symbol is rarely easy. And I'm not sure if an exchange:ticker tuple always maps to exactly one ISIN (and one ISIN can definitely map to multiple exchange:ticket pairs).
Are there any good embeddable stock widgets out there?
[1] Or TIPS or IEF or SHY or basically everything I've tested.