Changes Is Good

There is alot of google bashing these days from sites that track its "evilness" to pundits urging it to embrace its evilness. As an early, and modestly loyal, user of most Google applications I'm inclined to like their products (except Google Video, which sucks ass and Picasa, which still isnt as good as ACDSee8 ). So when I came across a paper called "Social- and Interactive-Television Applications Based on Real-Time Ambient-Audio Identification" I thought it would be about as interesting as reading a Microsoft patent application. But I took the plunge and came away with a startling realization; Google groks the future. While the cable networks are throwing video at every buzzword that appears in the WSJ and tripping over each other trying to replicate broadcast on the web, Google is thinking about ways to bring mass-personalization into your home.

Google researchers, Michele Covell and Shumeet Buluja along with Michael Fink, of the HUJ, have come up with an application that matches whats happening on the TV in your room with what happens on your computer screen. Using the mic built into most computers, the proposed application would capture audio snippets and convert them into an audio fingerprint, which gets sent back to the google servers. No actual audio is transmitted to Google, to preserve user privacy, but the "irreversible" audio fingerprint is matched to other audio fingerprints in a Google audio fingerprint database. This can then push a number of different information/services to the computer screen based on the match. The paper proposes four different applications: personalized information layers, ad-hoc peer communities, real-time popularity ratings and TV based bookmarks. I have a few others which are no more then wild conjecture and sophist musings.

The Applications
The first one they describe is really a suite of possible applications (layers) that would present contextual information based on the TV program being watched. For example, if your watching reruns of Sienfeld, the fashion layer might present information on the clothes different characters are wearing, the politics layer might give a profile of donations via Opensecrets.org and the health layer might tell you which street pharmaceuticals different celebrities in the program prefer. The paper also suggests that using layers might be a great way to generate revenue from contextual ads and commerce, which could be a part of the presentation or a separate layer by itself.

Ad-hoc peer networks is what you'd get if IRC and TV had a bastard child raised by a channel surfer. The idea behind this application is that it would dynamically assemble communities around whatever show is currently playing on the TV set. So channel surfers would be automatically switched to different communities with every click of their remote. Users of this application wouldnt have to program or configure anything since the communities would be automatically generated using the ambient audio from the TV. Combine this with a personal information layer that applies this comments as time-synched pop-ups and you have a rudimentary version of pop-up video.

"Having real-time, fine-grain ratings is more valuable than ratings achieved by the Nielsen system."

Do you have a Nielsen box in your home? I dont either. But, somehow they have established themselves as the defacto standard for audience measurement of TV programs. Google thinks it can do better with its real-time popularity ratings. Again, according to the researchers, using their application would provide better real-time, easy to use (ie no hardware to fiddle with) and accessible data on TV viewership. Google also sees this as a means of helping advertisers strengthen their spots through real-time feedback on the effectiveness of campaigns, though viewer rating levels.

The last proposed application is a video bookmarking product. The idea is to allow users to mark selections of video for later access and sharing with friends. Bookmarking is conceived as a way to make compilations of whole videos or video segments, putting the user in the role of aggregator.

Trouble
Lets assume that Google goes ahead and releases this application to the public and has an API for developer generated extensions. The folks in the most danger are the local broadcasters who may see ad revenue shift towards the net in a big way as advertisers, unhappy with spending, move dollars towards Googles more targeted options. Nielsen is also clearly in the path of the new application, with real-time stats coming from the server farms at Google the Danish conglom will have to figure out why its still little unit that could still matters.

A smart developer will mash-up this application with BitTorrent and a number of Torrent sites to create a virtual-DVR. Lets say your last to figure out that BattleStar Galactica was the best show on TV and you needed to get caught-up you could use this while watching the show to get every episode of BSG available online. Sounds nice to me.