I am quite excited that the technology is finally finding its way to a broad audience on very interesting video assets, see for example the concert of Moke on Pinkpop 2008. This is my best Sinterklaas present in years.
The program for the first International Workshop on Internet Multimedia Mining is now available. With the explosion of video and image data available on the Internet, online multimedia applications become more and more important. Moreover, mining semantics and other useful information from large-scale Internet multimedia data to facilitate online and local multimedia content analysis, search and other related applications has also gained more and more attention from both academia and industry. The program covers the breadth of internet multimedia mining, with papers focusing on auto-annotation and new retrieval models. We are proud to have a keynote by Zhongfei Zhang, who will deliver a keynote on Multimedia Data Mining Theory and Its Applications. The workshop is co-located with the IEEE International Conference on Data Mining in Miami, Florida and will be held on Sunday December 6th.
Last Friday I was given the opportunity to speak at the Society of the Query Conference, organized by the Amsterdam-based Institute of Network Cultures. The conference critically reflected on the information society and the dominant role of the (Google) search engine in our culture. Most speakers and audience had a background in media studies, as Wikipedia puts it “an academic discipline that deals with the content, history and effects of various media; in particular, the ‘mass media’.” I was invited to speak on alternative search methods, especially concept-based video search of course. The conference was quite an interesting experience, both in terms of speaker presentation conventions (some presenters only used one slide, for decoration), and a very different type of audience (heavily debating via twitter during the conference). I even learned some new words, most notably scookies and clickworkers. Although the conventions are different, there are still many similarities between media studies and the field of multimedia,which could results in interesting cross-fertilizations in the near future. All in all this was a rewaring experience, and I would like to thank the Institute of Network Cultures again for having me as a speaker.
The draft notebook paper for TRECVID 2009 by the MediaMill team, containing members of the University of Amsterdam, INESC-ID and the University of Surrey, is now available. In the paper we describe our TRECVID 2009 video retrieval experiments. The MediaMill team participated in three tasks: concept detection, automatic search, and interactive search. Starting point for the MediaMill concept detection approach is our top-performing bag-of-words system of last year, which uses multiple color descriptors, codebooks with soft-assignment, and kernel-based supervised learning. We improve upon this baseline system by exploring two novel research directions. Firstly, we study a multi-modal extension by the inclusion of 20 audio concepts and fusing using two novel multi-kernel supervised learning methods. Secondly, with the help of recently proposed algorithmic refinements of bag-of-words, a bag-of-words GPU implementation, and compute clusters, we scale-up the amount of visual information analyzed by an order of magnitude, to a total of 1,000,000 i-frames. Our experiments evaluate the merit of these new components, ultimately leading to 64 robust concept detectors for video retrieval. For retrieval, a robust but limited set of concept detectors necessitates the need to rely on as many auxiliary information channels as possible. For automatic search we therefore explore how we can learn to rank various information channels simultaneously to maximize video search results for a given topic. To improve the video retrieval results further, our interactive search experiments investigate the roles of visualizing preview results for a certain browse-dimension and relevance feedback mechanisms that learn to solve complex search topics by analysis from user browsing behavior. The 2009 edition of the TRECVID benchmark has again been a fruitful participation for the MediaMill team, resulting in the top ranking for both concept detection and interactive search.
