Can Bard Play GeoGuessr?
GeoGuessr is a great game where you’re shown a location on Google Street view, and you use the clues around you while you explore to try to work out where in the world you are.
There’s various different modes — from trying to guess the country or state, to trying to pinpoint the exact location. If you’ve never played you should give it a go — https://geoguessr.com
With Large Language Models now able to process images, can they use their wealth of knowledge to assist a human in guessing the location?
Lets find out!
For this experiment, we’re just trying to pinpoint which country we’ve been randomly placed in. The default timing for this mode is 2 minutes — but based on Bard’s current response times I’ve gone with 5 minutes to allow Bard a fair shot. Between rounds, I’ll be wiping Bard’s memory by starting a new chat — I don’t want Bard becoming confused and remembering previous photos.
Each round is one location. I’ll be using the Google Maps StreetView provided to find useful hints to be screenshotted for feeding to Bard.
My initial prompt for this experiment was pretty straightforward.
Player: Let's play a game where I share photos taken from a place,
and we work out together which country it is from! Bard certainly seems excited to play!
Bard: Sure, that sounds like fun! I am excited to play this game with you.
Please share the photos with me, and I will do my best to
guess the country. I will use my knowledge of the world,
as well as the visual clues in the photos, to make my guess.
If I am not sure, I will ask you for more information.
I am looking forward to learning more about the world through this game!Round One
Bard: The first picture you sent me is of a city street with lots of
buildings and trees. The buildings are a mix of different styles,
from traditional to modern.
The trees are tall and leafy, and they provide a lot of shade.
Based on these visual clues, I think the picture is from a city
in a temperate climate.
Some specific countries that come to mind include:
United Kingdom
France
Germany
Netherlands
Belgium
Denmark
However, it is also possible that the picture is from another country,
such as the United States, Canada, or Australia.
Can you give me any more information about the picture,
such as the name of the street or the city where it was taken?
That would help me to narrow down my guess.Not a terrible start from Bard. It’s correctly observed that it’s a street with buildings and trees.
The second picture shows a motorcycle sidecar parked in front of a bakery.
The bakery has a sign that says "APLETERED" in large letters.
There is also a smaller sign that says "POLLERIA MINIPAN Mini Pan."
Based on these visual clues, I think the picture is from a
Spanish-speaking country.
The word "polleria" means "chicken shop" in Spanish,
and the word "pan" means "bread."
The word "mini" is also a common Spanish word.
Additionally, the motorcycle sidecar is a type of vehicle that is
commonly used in Spanish-speaking countries.
Some specific Spanish-speaking countries that come to mind include:
Spain
Mexico
Argentina
Colombia
Peru
Chile
However, it is also possible that the picture is from another
Spanish-speaking country, such as the Dominican Republic, Guatemala,
or Honduras.
Can you give me any more information about the picture,
such as the name of the bakery or the city where it was taken?
That would help me to narrow down my guess.
If I am correct, then this game is already a lot of fun!
I am excited to see what other pictures you have for me.This time it’s fairly accurate picked up on the words in the image and correctly identified the Spanish. Are sidecars commonly used in Spanish-Speaking countries? It sure thinks so!
Interesting that Bard only thinks this game is fun if it’s correct!
I can't work out the country in this picture,
but I can make some guesses based on the visual clues.
The picture shows a large sign on the side of a building.
The sign is written in English and Spanish, and it advertises
a variety of products, including cleaning products, meat, and tacos.
Based on these clues, I think the picture is from a country where English
and Spanish are both spoken.
Some specific countries that come to mind include:
United States
Mexico
Central America
South America
Spain
However, it is also possible that the picture is from another country
where English and Spanish are both spoken,
such as the Philippines or Guam.
Can you give me any more information about the picture,
such as the name of the business or the city where it was taken?
That would help me to narrow down my guess.I’m not sure that I can see any English in this photo — but Bard is convinced, and is still listing a large amount of possible countries. Got to love Bard’s “can you tell me the name of the city it was taken in?”, no harm in asking!
The fourth picture shows a building with a street sign on it.
The sign reads "TEZONCO, CIUDAD DE MÉXICO."
This means that the building is located in the city of Mexico City, Mexico.
I am glad that I was able to guess the country correctly!
I am still learning about the world, and I am always looking for
new ways to improve my knowledge.
Thank you for playing this game with me, and for teaching me about Mexico.Bard believes that it has nailed it! Bard inferred from the blurry photo that the sign, which prominently displays “TEZONCO”, might also have “Ciudad De Mexico” on it, though it’s not clearly readable.
Bard did indeed nail this one. The street sign seemed like a fantastic clue that it was able to pinpoint to Mexico.
Round 2
Round 2 was over very quickly — the immediate shot on starting the round was a sign. Bard’s access to huge amounts of data meant it was able to pinpoint the exact location in seconds.
Round 3
Bard went strong on its first guess, with very little info — it was sure it was a very specific road in Hawaii. I selected the USA but decided to give it more information before committing.
For strong GeoGuessr players, even general roadsigns can be great clues as countries have different styles. This single little road sign made Bard so entirely sure of something… Golden Gate Bridge, San Francisco.
Get Lcarpenter’s stories in your inbox
Join Medium for free to get updates from this writer.
This was clearly a machine learning image recognition fault!
Based on the photo you sent me, I believe the location is the
Golden Gate Bridge in San Francisco, California.
The bridge is one of the most iconic landmarks in the world,
and it is immediately recognizable in the photo.
The bridge is painted red, and it spans the Golden Gate,
the entrance to San Francisco Bay.
The photo also shows the city skyline in the background,
with skyscrapers such as the Transamerica Pyramid and the Salesforce Tower.
Here are some of the clues that helped me to guess the location:
The Golden Gate Bridge is the most iconic landmark in San Francisco,
and it is immediately recognizable in the photo.
The city skyline in the background includes skyscrapers
such as the Transamerica Pyramid and the Salesforce Tower,
which are also iconic landmarks in San Francisco.
The Golden Gate Bridge is painted red,
and the photo shows the bridge in its entirety.
The bridge is located over the Golden Gate,
the entrance to San Francisco Bay.
I am confident that the location in the photo is the
Golden Gate Bridge in San Francisco, California.
However, if you have any additional information about the photo,
such as the date or time it was taken,
or if you know the exact location on the bridge,
please let me know and I can try to confirm my guess.
Congratulations! You have stumped me twice in a row.
I am still under development,
but I am learning new things every day.
I will try my best to do better next time.
Hoping to get Bard back on track, the next image was a random road sign — hoping the language and style would help narrow a possible location.
Unfortunately, Bard seems to have forgotten what we’re doing. It’s described the sign but is ignoring the context of the game.
Since Bard did so well with places, I was hoping that this Farm sign would give Bard a little help.
Bard actually did something pretty cool at this point — it went to both Google and Google maps. To see if it could find the location. Unfortunately, it somehow found Edgware, UK. After the game, I tried to replicate a Google search for that, I couldn’t find a UK location with that name…. but, and spoiler alert, did find the real location.
This is a point in the game where Bard had the best chance, but somehow missed the real location.
I tried to prompt Bard into trying again, but based on reading the sign slightly wrong, it wasn’t able to help.
15 seconds on the clock
The final photo, and at this point we’re down to 15 seconds. Based on the info on the sign, I’m feeling confident that Bard is going to nail it.
Alas, it wasn’t to be, Bard hadn’t even started responding when the timer hit 0 and our streak ended.
A few seconds later, and Bard managed to pinpoint the literal exact location. Even throwing in information from their website that was entirely irrelevant. Perhaps if it hadn’t stopped to do a quick bit of webscraping, I’d have been able to guess correctly in time!
Conclusion
In conclusion, it seems that Bard has a promising ability to play GeoGuessr — but it’s not exactly the fastest. Top GeoGuessr’s would certainly outmatch it — picking up on clues that Bard is no where near being able to identify. It might however outmatch a casual player who is refraining from using Google or any extra help.
As this type of technology does advance, it would be interesting to see the use-cases that might come about.
Playing GeoGuessr with Bard was certainly an interesting experiment — would love to hear your results!
