Google's AI Doesn't Understand Restaurant Menus

Google’s AI Doesn’t Understand Restaurant Menus – Terence Eden’s BlogTerence Eden’s Blog

In the glorious future, every website will be chock-full of semantic metadata. Restaurants won't have a 50MB PDF explaining the chef's vision for organic cuisine - instead, they'll have little scraps of data on the HTML page like:

"hasMenuItem":{
  "@type":"MenuItem",
   "name":"Dodo In A Bun",
   "description":"The legendary extinct bird cooked in tomato sauce, served in a gluten-free bun.",
     "offers":{
       "@type":"Offer",
       "price":"7.99",
       "priceCurrency":"GBP"
     },

But, for now, they don't. So Google uses MACHINE LEARNING and DEEP AI to scrape the menus out of PDFs and photos submitted by patrons. It works about as well as you'd expect.

Examples

Here's a bunch of screenshots from Google Maps. Would a human have made these mistakes? (No they'd make much more interesting mistakes!)

Are those realistic prices for burgers? Would a place that sells a gold-covered burger also sell second-hand chicken? Is it usual that UK menus offer items in different currencies? What about this:

Is that a likely name for a side dish? You wouldn't find that even in trendy Shoreditch. And, again with the weird pricing. Side dishes rarely cost that much. Still, it could be worse…

I guarantee you the little curry shop I was looking at didn't have anything on the menu that pricey!

Other times, it seems to just pick random words in a vague attempt to be "useful".

Perhaps my favourite is this British restaurant which offers spam, spam, and more spam:

There were dozens (dozens!) of examples I found within a few minutes of looking for somewhere nearby to grab a cheap lunch. To be fair, when the website has supplied data in an appropriate format, Google is good at displaying it. But when Google tries to be clever... ah.

There is, of course, no way for a user to contact Google and say "I dunno, man. This looks kinda off."

Whose Fault Is It Anyway?

Obviously, restaurateurs should provide machine-readable descriptions of their menus. That way search engines, smart speakers, and all sorts of funky computers will be able to parse the food available.

But they don't. Because they're busy cooking.

Google has discovered that it takes 90% of the effort to get 90% of the way there - but the last 10% takes the next 90% of the effort.

As ever, a feature has launched and it is of dubious value to business owners and customers - but I'm sure a product manager at Google got their wings.