Automatic content recognition

Automatic content recognition (ACR) is an identification technology to recognize content played on a media device or present in a media file. Devices containing ACR support enable users quickly obtain additional information about the content they have just experienced without any user based input or search efforts. For example, developers of the application can then provide personalized complementary content to viewers.

To start the recognition, a short audio clip is recorded by the device and sent to the identification service. Through algorithms such as Fingerprinting ACR uses, information from the audio is taken and matched to the corresponding media.The database also contains information about the content and associated information, including complementary media. If the fingerprint of the recorded audio sample is matched, the identification service returns the corresponding metadata to the client.

Audio based ACR is commonly used in the market. The two leading methodology are acoustic fingerprinting and watermarking. There are alternative approaches that involve a focus on Video fingerprinting but augment the accuracy and scalability with other content recognition solutions running in parallel and in series.

Acoustic fingerprinting generates unique fingerprints from the content itself. Fingerprinting techniques work regardless of content format, codec, bitrate and compression techniques. This makes it possible to use across networks and channels. Therefore, it is widely used for interactive TV, second screen application and content monitoring sectors. Popular apps like Shazam, YouTube, Facebook, Thetake, Wechat and Weibo are using audio fingerprinting methodology to recognize the contented played from a TV and trigger additional features like votes, lottery, topic or purchase.

In contrast to fingerprinting, digital watermarking requires inserting digital tags containing information about the content into the content itself, prior to distribution. For example, a broadcast encoder might insert a watermark every few seconds that could be used to identify to broadcast channel, program id, and time stamp. The watermark is normally inaudible or invisible to the users. Terminal devices like phones or tablets read the watermarks instead of actually recognizing the played content. Watermarking technology is utilized in media protection field to trace where the illegal copies originate.

It is expected by Next/Market Insights that 2.5 billion devices will be integrated with ACR technology to provide synchronized live and on-demand video watching experience.

...
Wikipedia