'... apparently, when Udi was demoing this for Larry Page, he asked why the images weren't translating. Obviously, there is still work to be done, but it is quite amazing ...'
That's got to be a joke. Aside from being time consuming, not scaling well it's beyond even googles core skillset(s).
Searching text inside images seems like an obvious thing for Google to do. Translating is just the next step beyond that. It's a hard problem but not insurmountable. If someone wanted an idea for a company to sell to Google...
Add a layer of interaction to the process. The default behavior would be to leave the text in images untranslated, but when viewing the translated page there would be an easy way to select specific images for translation. There would also be a "translate all images" option.
Perhaps an even better product to create would be an interactive translation overlay that semi-intelligently created "areas" on a page (such as body text, images, menus, etc). You could simply click on the area you desired to have translated. The app would take note of which areas you clicked on and if you browsed to another page on the same site would automatically translate the corresponding area on the next page.
"... Translating is just the next step beyond that. It's a hard problem but not insurmountable. If someone wanted an idea for a company to sell to Google ..."
So parse each image, use text recognition to find the text (determine language), then re-render the text using the same font (have to recognise fonts) & render the changed image (or just re-build an approximation), the same colour (edge cases like colour fading) then store the image and display it.
A much better way would be just to substitute text extracted from the image. At least you could read it (technically it works) but the result would (without some clever hackery) look crap.
Maybe a better way would simply extract the text & rebuild a page in a standard format?
"... It's a hard problem but not insurmountable. If someone wanted an idea for a company to sell to Google... ..."
Google just open sourced their OCR the other day, it's multi-lingual, has natural language modeling, and a plug-in system for lay-out analysis and character recognition.
Even if you somehow tackle image translation, those crappy auto-translated pages will look even crappier. As an example open www.sun.com and try to imagine what would happen to their front page. "Sun" and "microsystems" on the logo will be translated too, not to mention that the whole graphical look of the page will be ruined, which won't be welcomed by the owner of the web site.
(Not that I care about Sun's web site too much, but this is an example.)
That's got to be a joke. Aside from being time consuming, not scaling well it's beyond even googles core skillset(s).