I wanted to build a simple calendar for the London 2012 Olympics (and actually built another one for the Sochi Olympics). Both times I ended up using a JSON file found on NBC's website.
Like the linked article suggests, it would be nice to have access to an official (and even simple) API, to dig up some interesting statistics or just have some fun playing with the data.
But there's probably a violation of the terms of use, so maybe it's not a good idea to say to everybody that you take the data on the official website.
Yeah, it seems the facts are the most important part. I mean the data should be openly available, right? and that lowering the barrier to use by programmers and data scientists should improve the quality of how people interact with and experience the olympics. We're hoping it will help devs build super interesting visualizations and/or apps (that otherwise weren't possible).
But, they prohibit the copy of the content from the website, no matter the way you did it. Point 5.2:
"Except as expressly stated on the Website, you are not entitled to copy, publish, download, modify and redistribute the information provided on the Website without the prior written consent of the Sochi 2014 Organizing Committee."
IANAL, but that sounds either like you can't do all of those things together (the "and" conjunction), or you can't even read the website. After all, if you can't download it, you can't view it.
Well, good luck to you. I think you'll find out that the IOC disagree with you, and that the Olympics is far more of a money-grab than an international coming together of the world.
Serious question - have you look very deep into the legality of what you're doing? I have no doubt in the world there is huge value in your service but generally speaking your website violates the most basic terms of use of pretty much any site that exists. That being said - you (and pretty much every other web scraper in the world) enters into this grey area of "binding contract" in the terms of use. Personally I'd be very scared about your service being able to scale for two fundamental reasons (1) the more attention you attract as you scale, the more legislation will be targeted at you and (2) people who want to protect their data will create services to block your efforts.
note - IANAL and I say this not to scare you, but hopefully to help.
There are numerous web-scraping tools out there, many are easy enough to use for non-programmers (Connotate and Mozenda for instance) so, while you have a point, precedent shows that web scraping is prominent and not going away anytime soon. I applaud innovation in this area.
Your #2 is simply not plausible unless Kimono carefully follows robots.txt and consistent user-agent, which is rarely the case for a competent web-scraping platform.
Legality is also a strong word. Web scraping is rarely illegal in the sense that you're committing a crime. A company could definitely issue a C&D and potentially go after you in a civil lawsuit but it's very infrequent (sadly not so much anymore with all the bullshit CFAA suits against "hackers" lately) that a government entity would pursue someone for scraping data.
The general consensus is if the information is available to end users with a regular web browser, no logins or agreement checkboxes, then it's fair game for web scraping.
Precedent is also a strong word. There are numerous people on the road who go over the speed limit. It's prominent and not going away anytime soon. This is not a cause for precedence.
Precedence implies that a legal case take place in order to dictate the law. AFAIK, between your unsubstantiated comment and my general knowledge of this area, the reason no one has provided a scraper service at scale isn't because the technology or execution wasn't right, but rather that it's simply illegal to do so.
> The general consensus is if the information is available to end users with a regular web browser, no logins or agreement checkboxes, then it's fair game for web scraping.
"By accessing or using the Site, you are agreeing to these Terms and concluding a legally binding contract with Yelp Inc., a Delaware corporation headquartered in San Francisco, California ("Yelp"). Do not access or use the Site if you are unwilling or unable to be bound by the Terms."
"Use any robot, spider, site search/retrieval application, or other automated device, process or means to access, retrieve, scrape, or index any portion of the Site or any Site Content;"
My overall point? Kimonolabs is slowly painting a target on their back and it's only a matter of time before they'll have to "lawyer up".
Assuming you mean that it only supports GET requests against the existing data. it's amazing though that something as simple as querying a table of olympics data wasn't possible without this (without a whole bunch of manual work). if we did support transactions what would you have in mind?
The Olympic trademarks live at a meta-IP level [1], with trademarks enforced by specific laws rather than standard trademark. Legal corruption at its finest.
I love the way this API is presented on the page. It's awesome. It exposes itself in such an obvious way that it leaves no question about its capabilities and usage. Makes me want to jump right in.
29 comments
[ 5.7 ms ] story [ 44.1 ms ] threadLike the linked article suggests, it would be nice to have access to an official (and even simple) API, to dig up some interesting statistics or just have some fun playing with the data.
There is most definitely is. It's just not free.
But there's probably a violation of the terms of use, so maybe it's not a good idea to say to everybody that you take the data on the official website.
http://www.sochi2014.com/en/terms-of-use
"Except as expressly stated on the Website, you are not entitled to copy, publish, download, modify and redistribute the information provided on the Website without the prior written consent of the Sochi 2014 Organizing Committee."
note - IANAL and I say this not to scare you, but hopefully to help.
Your #2 is simply not plausible unless Kimono carefully follows robots.txt and consistent user-agent, which is rarely the case for a competent web-scraping platform.
Legality is also a strong word. Web scraping is rarely illegal in the sense that you're committing a crime. A company could definitely issue a C&D and potentially go after you in a civil lawsuit but it's very infrequent (sadly not so much anymore with all the bullshit CFAA suits against "hackers" lately) that a government entity would pursue someone for scraping data.
The general consensus is if the information is available to end users with a regular web browser, no logins or agreement checkboxes, then it's fair game for web scraping.
Precedence implies that a legal case take place in order to dictate the law. AFAIK, between your unsubstantiated comment and my general knowledge of this area, the reason no one has provided a scraper service at scale isn't because the technology or execution wasn't right, but rather that it's simply illegal to do so.
> The general consensus is if the information is available to end users with a regular web browser, no logins or agreement checkboxes, then it's fair game for web scraping.
That's simply not true. Here's an example:
http://www.yelp.com/static?country_=US&p=tos
"By accessing or using the Site, you are agreeing to these Terms and concluding a legally binding contract with Yelp Inc., a Delaware corporation headquartered in San Francisco, California ("Yelp"). Do not access or use the Site if you are unwilling or unable to be bound by the Terms."
"Use any robot, spider, site search/retrieval application, or other automated device, process or means to access, retrieve, scrape, or index any portion of the Site or any Site Content;"
My overall point? Kimonolabs is slowly painting a target on their back and it's only a matter of time before they'll have to "lawyer up".
Interesting data though !
[1] http://www.inta.org/TrademarkBasics/FactSheets/Pages/Protect...