This could be useful for finding new places to go that are look similar to your holiday snaps.
Also if I could do this with a picture of food I want to eat.. and it guess the probable location of a restaurant serving it, that would be great.
I started working on a similar project - my approach was to train a model based on all of the geoguessr contest sets.
Geoguessr gives you both the street view _and_ the correct answer when you guess wrong.
They're all hand-curated so there are only so many points that you need to identify (the largest sets are 100k+ points) so the idea would be to run through it thousands of times with wrongs answers but use the correct answer to train the model
It's only a matter of time before the game has cheat bots and that problem, much in the same way online chess has that issue
What would be the advantage to train a model on the whole dataset, you would be running it against later? Wouldn't that overfit the network?
And for the cheating part; geoGuessr actually exposes the right answer in their API. You you could just use that to pinpoint the exact location automated.
> What would be the advantage to train a model on the whole dataset, you would be running it against later? Wouldn't that overfit the network?
The opposite, it would almost be cheating since the model would just have memorized all the correct solutions. You'd usually want to train on a subset of the data and evaluate against the remainder to protect against overfitting. So to that point I agree with your evaluation. But after that you'd use it with real data in the wild for your actual use case. With geogussr you'd know already what all the "real-world" situations are and overfitting wouldn't matter as long as you retrain wherever they add new sets.
GeoGuessr is a lot of fun. I played it a lot a few years ago, but only recently discovered that there’s a community of very serious players, and got interested in it again. For example check this video from yesterday where Tom Davies aka GeoWizard[1] gets a pretty good score on An Urban World: https://youtube.com/watch?v=TVt1GKBZMzc
Tom Davis is very good at the game. I also recommend checking out his "straight line mission" videos. He tries crossing Wales (and more recently Norway) in a single straight line. Deviating by only a few feet across the trips. He manages to make it quite interesting to "just" watch a man walking in fields.
I'm a fairly casual player too, and I guess I have my favorite aspects of the game - I've worked with foreign languages before so language identification is fun for me, and I'm a car enthusiast so I like informing guesses based on the cars I see. One thing this article raised that I'd never considered was the direction that satellite dishes face. Definitely going to keep my eye out for that!
I got into GeoGuessr during the pandemic, and am still very into it! I can really recommend it as a way to learn about more of the world while you're stuck at home.
Since I'm very interested in languages, I'm often surprised at how difficult other players sometimes find it to be to recognize specific languages (e.g. the differences between Spanish, Portuguese, Catalan, French, and Italian are quite obvious to me, but often not to many other players).
But I also notice that there are many, many world languages that I can't recognize easily; for Geogussr purposes in particular the most relevant for me right now are the Slavic languages (I can tell Polish apart from the others written in Latin script, but have trouble distinguishing the others that are written in that script; I have trouble distinguishing the Slavic languages that are written in Cyrillic; and I also have trouble distinguishing the Central Asian languages that are written in Cyrillic that are not from the Slavic family).
Wikipedia has a great chart to help you recognize languages even without learning to speak or understand them
and I'm currently writing a language recognition trainer (which, when it's done, will give you a sample of text and ask you to guess what language it is, and, if you get it wrong, provide some kind of tips that could help you to recognize that language). I plan to make it available to the public when I'm finished. I'd love for the ability to recognize and distinguish languages from one another to become more widespread. Because of the ubiquity and importance of English, we native English speakers are said to be especially bad at this... not just at understanding other languages, but even at telling them apart!
I doubt I would ever think of language recognition this way, but I bet those flowcharts do include individual tricks that people (like avid Geoguessr players) could choose to learn pretty easily.
I mean, I guess a fact like "Łł is in Polish and no other national language" feels OK to me but I personally feel better about learning other kinds of features when at all possible (individual words, morphemes, n-grams?). Although I don't plan for my own trainer software to force users to learn any particular kind of tips rather than others; I'm hoping to have it generate several kinds of recognition tips that might appeal to different people.
For example, I like knowing (and would like to teach people) that the English suffix -tion corresponds (regularly!) to
* Latin -tio [accusative form "-tionem" often cited in etymology]
* French -tion
* Spanish -ción
* Portuguese -ção
* Catalan -ció
* Italian -zione
But I could imagine that some people would say "I'm going to focus on how Portuguese and French have ç and these other languages don't, and Portuguese has ã and these other languages don't, and only Spanish and Catalan can have -ó, and Italian words almost always end with vowels and Catalan words commonly end with consonants" without learning the individual endings. And I hope I'll get my trainer into shape to help them learn that too, if they want to think about it that way.
I've only seen someone else play it but it's crazy how serious he was about it. Apart from the basic things like narrowing down based on languages in sign posts, he'd also use techniques like deriving latitude based on trees. This is gorgc on twitch, but I'm sure there are other streamers who play this game.
I wondered if this could be useful for open source journalism stuff like bellingcat does. Essentially combing through tons of photos online to figure out where / when they were taken and Put together an understanding of an incident
You know how you get off a plane and everything about a place 3000 miles away feels different? Not just smells and heat, but the light seems different. The surfaces and materials are different. The uh...(actual) atmosphere seems different.
Is for example, the atmosphere seeming different a real physical phenomenon that could be detected by ML?
Sure, why not? Chemical composition, temperature, and pressure of the atmosphere are just physical parameters (e.g. elements of a feature set) like any other (e.g. colour and intensity of pixels in an image).
Everything that can be measured can be used as an input for ML algorithms. The things that "feel" different are the result of the "post-processing" your brain applies to the inputs it receives (olfactory, temperature, light intensity and -spectral composition, etc.).
The sensation is based on physical phenomena, though it might be interesting how much of it remains if we were to take the knowledge about the change of location away, e.g. would you feel the same way if you didn't know you were 3000 miles away? In other words, does knowing about the change in location sharpen your senses such that you subconsciously look for changes in the environment?
Interesting post, but I disagree with the conclusion that with a bit of work AI could take down the best Geoguessrs. The best players already take into account all the metadata like camera quality and distinctive features of the Google car. Furthermore, they would surely have an edge in urban locations where proximate locations are visible in street-signs. I could be proven wrong though, and it would be super interesting to reverse engineer AI guesses that turn out to be surprisingly correct based off little info.
To see just how good the top players are at instantly recognising a country using these clues, then check out https://www.youtube.com/watch?v=zEmoAYpTJuA . Or maybe don't if you don't want the game spoiled!
> I disagree with the conclusion that with a bit of work AI could take down the best Geoguessrs. The best players already take into account all the metadata like camera quality and distinctive features of the Google car.
Isn’t this exactly what an AI might do even better, though? There might be statistical variations in image quality between cameras and processing that are imperceptible to a human but easy for machine learning to pick up?
True, but less than half of world countries are covered by street view and it's possible for a human to learn the distinctive metadata features of every single one, together with all the common roadside features like poles, road markings, and vegetation. So I don't know if the level of granularity where statistical variations are noticed is required to succeed at least in determining the country.
But in terms of picking the closest point within a country,
perhaps with a large enough dataset the AI would be able to distinguish based on local weather conditions when the Google car was driving through certain regions. And this would trump the player's ability to read place names and signage.
I'm a GeoGuessr fan, and like the author, experienced a bit of a GeoGuessr renaissance when I had a baby. I guess it's the sort of slow, quiet game that lends itself to being played between baby care tasks.
In any case, the criteria discussed here is the sort of thing that all too often makes me skeptical of deep learning results.
There are the things that I consider "real" discriminators when playing the game - architectural styles, building materials, road quality, driving direction, alphabets/languages on signage, flags, flora, topography, makes/types of vehicles, skin color and clothing style of humans, etc. Then there are the "artificial" things - glimpses at the street view vehicle, copyright notices, image quality, knowledge of which countries do or don't have Google Street View coverage, etc.
Obviously by taking what I'm calling the "artificial" criteria into account a deep learning approach would score better today than if only the "real" criteria were considered. But I feel like if tomorrow, GeoGuessr swapped Apple Maps Look Around or Bing Maps StreetSide in place of Google Street View (or if Google released vastly updated imagery), the deep learning approach would fall apart, but humans who have built knowledge of the "real" features to look for would continue to do just about as well.
Coming from 64 degrees north I always react when I see satellite dishes tilted up and realized a satellite dish is an exact indication - to the degree! - of latitude.
Some climates can be deceiving depending on local conditions (Gulf Stream makes Western Europe have flora like much more southern parts of North America for example) but satellite dishes work the same everywhere.
Ooh, this is great. I love GeoGuessr and have pointed several people to it. I have thought that a trained AI could do better than me on average.
I've been posting my runs publicly to Strava for years. I recently started living in a small town and decided to stop posting my location, and using an open source tool called RunnerUp to track my runs. I have still been posting running photos that don't reveal my location. I know someday people will be able to figure out my location despite avoiding taking pictures with signs and stuff. If anyone wants to try to figure out one of my recent locations, my instagram link is in my bio.
It's wild how we're anonymous on the internet, right up until we're not! I was watching an auction on bringatrailer.com (an auto auction website where people sell cars and discuss them in a comments section) the other day. One commenter realized he could distantly see his house in a stranger's auction photos! The person posting the photos probably never considered that the photo location would be precisely identifiable (and for this purpose, probably didn't care). But if you get enough eyes on something, someone is going to recognize it.
26 comments
[ 3.0 ms ] story [ 25.4 ms ] threadGeoguessr gives you both the street view _and_ the correct answer when you guess wrong.
They're all hand-curated so there are only so many points that you need to identify (the largest sets are 100k+ points) so the idea would be to run through it thousands of times with wrongs answers but use the correct answer to train the model
It's only a matter of time before the game has cheat bots and that problem, much in the same way online chess has that issue
And for the cheating part; geoGuessr actually exposes the right answer in their API. You you could just use that to pinpoint the exact location automated.
The opposite, it would almost be cheating since the model would just have memorized all the correct solutions. You'd usually want to train on a subset of the data and evaluate against the remainder to protect against overfitting. So to that point I agree with your evaluation. But after that you'd use it with real data in the wild for your actual use case. With geogussr you'd know already what all the "real-world" situations are and overfitting wouldn't matter as long as you retrain wherever they add new sets.
1: https://en.wikipedia.org/wiki/GeoWizard
https://youtu.be/M7w986ni7_g
Since I'm very interested in languages, I'm often surprised at how difficult other players sometimes find it to be to recognize specific languages (e.g. the differences between Spanish, Portuguese, Catalan, French, and Italian are quite obvious to me, but often not to many other players).
But I also notice that there are many, many world languages that I can't recognize easily; for Geogussr purposes in particular the most relevant for me right now are the Slavic languages (I can tell Polish apart from the others written in Latin script, but have trouble distinguishing the others that are written in that script; I have trouble distinguishing the Slavic languages that are written in Cyrillic; and I also have trouble distinguishing the Central Asian languages that are written in Cyrillic that are not from the Slavic family).
Wikipedia has a great chart to help you recognize languages even without learning to speak or understand them
https://en.wikipedia.org/wiki/Wikipedia:Language_recognition...
and I'm currently writing a language recognition trainer (which, when it's done, will give you a sample of text and ask you to guess what language it is, and, if you get it wrong, provide some kind of tips that could help you to recognize that language). I plan to make it available to the public when I'm finished. I'd love for the ability to recognize and distinguish languages from one another to become more widespread. Because of the ubiquity and importance of English, we native English speakers are said to be especially bad at this... not just at understanding other languages, but even at telling them apart!
That would be awesome as an interactive flowchart
I doubt I would ever think of language recognition this way, but I bet those flowcharts do include individual tricks that people (like avid Geoguessr players) could choose to learn pretty easily.
I mean, I guess a fact like "Łł is in Polish and no other national language" feels OK to me but I personally feel better about learning other kinds of features when at all possible (individual words, morphemes, n-grams?). Although I don't plan for my own trainer software to force users to learn any particular kind of tips rather than others; I'm hoping to have it generate several kinds of recognition tips that might appeal to different people.
For example, I like knowing (and would like to teach people) that the English suffix -tion corresponds (regularly!) to
* Latin -tio [accusative form "-tionem" often cited in etymology] * French -tion * Spanish -ción * Portuguese -ção * Catalan -ció * Italian -zione
like: nation, natio, nation, nación, nação, nació, nazione
But I could imagine that some people would say "I'm going to focus on how Portuguese and French have ç and these other languages don't, and Portuguese has ã and these other languages don't, and only Spanish and Catalan can have -ó, and Italian words almost always end with vowels and Catalan words commonly end with consonants" without learning the individual endings. And I hope I'll get my trainer into shape to help them learn that too, if they want to think about it that way.
Is for example, the atmosphere seeming different a real physical phenomenon that could be detected by ML?
Everything that can be measured can be used as an input for ML algorithms. The things that "feel" different are the result of the "post-processing" your brain applies to the inputs it receives (olfactory, temperature, light intensity and -spectral composition, etc.).
The sensation is based on physical phenomena, though it might be interesting how much of it remains if we were to take the knowledge about the change of location away, e.g. would you feel the same way if you didn't know you were 3000 miles away? In other words, does knowing about the change in location sharpen your senses such that you subconsciously look for changes in the environment?
To see just how good the top players are at instantly recognising a country using these clues, then check out https://www.youtube.com/watch?v=zEmoAYpTJuA . Or maybe don't if you don't want the game spoiled!
Isn’t this exactly what an AI might do even better, though? There might be statistical variations in image quality between cameras and processing that are imperceptible to a human but easy for machine learning to pick up?
But in terms of picking the closest point within a country, perhaps with a large enough dataset the AI would be able to distinguish based on local weather conditions when the Google car was driving through certain regions. And this would trump the player's ability to read place names and signage.
In any case, the criteria discussed here is the sort of thing that all too often makes me skeptical of deep learning results.
There are the things that I consider "real" discriminators when playing the game - architectural styles, building materials, road quality, driving direction, alphabets/languages on signage, flags, flora, topography, makes/types of vehicles, skin color and clothing style of humans, etc. Then there are the "artificial" things - glimpses at the street view vehicle, copyright notices, image quality, knowledge of which countries do or don't have Google Street View coverage, etc.
Obviously by taking what I'm calling the "artificial" criteria into account a deep learning approach would score better today than if only the "real" criteria were considered. But I feel like if tomorrow, GeoGuessr swapped Apple Maps Look Around or Bing Maps StreetSide in place of Google Street View (or if Google released vastly updated imagery), the deep learning approach would fall apart, but humans who have built knowledge of the "real" features to look for would continue to do just about as well.
Some climates can be deceiving depending on local conditions (Gulf Stream makes Western Europe have flora like much more southern parts of North America for example) but satellite dishes work the same everywhere.
I've been posting my runs publicly to Strava for years. I recently started living in a small town and decided to stop posting my location, and using an open source tool called RunnerUp to track my runs. I have still been posting running photos that don't reveal my location. I know someday people will be able to figure out my location despite avoiding taking pictures with signs and stuff. If anyone wants to try to figure out one of my recent locations, my instagram link is in my bio.