Instant Video Search

Abstract

Mobile video is quickly becoming a mass consumer phenomenon. More and more people are using their smartphones to search and browse video content while on the move. In this paper, we have developed an innovative instant mobile video search system through which users can discover videos by simply pointing their phones at a screen to capture a very few seconds of what they are watching. The system is able to index large-scale video data using a new layered audio-video indexing approach in the cloud, as well as extract light-weight joint audio-video signatures in real time and perform progressive search on mobile devices. Un-like most existing mobile video search applications that simply send the original video query to the cloud, the proposed mobile system is one of the first attempts at instant and progressive video search leveraging the light-weight computing capacity of mobile devices. The core metric of this system is based on video-indexing technology and an automatic related-term collection framework. Digital files on local disks are crawled, and images of representative scenes are extracted from video files using video indexing technology. These images are then stored in the metadata database. We describe a unique search result interface that helps users distinguish the target video. In the search results interface, the results of the video search are listed at two levels to let users search videos faster. The effectiveness of images on our interface derived from video-indexing is evaluated through a comparison with a conventional file search application.