The Video Browser Showdown: a live evaluation of interactive video search tools

[en] The Video Browser Showdown evaluates the performance of exploratory video search tools on a common data set in a common environment and in presence of the audience. The main goal of this competition is to enable researchers in the field of interactive video search to directly compare their tools at work. In this paper, we present results from the second Video Browser Showdown (VBS2013) and describe and evaluate the tools of all participating teams in detail. The evaluation results give insights on how exploratory video search tools are used and how they perform in direct comparison. Moreover, we compare the achieved performance to results from another user study where 16 participants employed a standard video player to complete the same tasks as performed in VBS2013. This comparison shows that the sophisticated tools enable better performance in general, but for some tasks common video players provide similar performance and could even outperform the expert tools. Our results highlight the need for further improvement of professional tools for interactive search in videos.

Author, co-author :

Schoeffmann, Klaus

Ahlström, David

Bailer, Werner

Cobârzan, Claudiu

Hopfgartner, Frank

McGuinness, Kevin

Gurrin, Cathal

Frisson, Christian ; Université de Mons > Faculté Polytechnique > Information, Signal et Intelligence artificielle

Le, Duy-Dinh

Del Fabro, Manfred

Bai, Hongliang

Weiss, Wolfgang

Language :

English

Title :

The Video Browser Showdown: a live evaluation of interactive video search tools

Publication date :

01 June 2014

Journal title :

International Journal of Multimedia Information Retrieval

ISSN :

2192-6611

Publisher :

Springer, United Kingdom

Volume :

Issue :

Pages :

113-127

Peer reviewed :

Peer Reviewed verified by ORBi

Research unit :

F105 - Information, Signal et Intelligence artificielle

Research institute :

R450 - Institut NUMEDIART pour les Technologies des Arts Numériques

Available on ORBi UMONS :

since 05 January 2014

Statistics

Number of views

64 (0 by UMONS)

Number of downloads

0 (0 by UMONS)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

OpenAlex citations

Bibliography

Adams B, Greenhill S, Venkatesh S (2012) Towards a video browser for the digital native. In: Proceedings of IEEE International Conference on Multimedia and Expo Workshops, pp 127–132. doi:10.1109/ICMEW.2012.29
Ahlström D, Hudelist MA, Schoeffmann K, Schaefer G (2012) A user study on image browsing on touchscreens. In: Proceedings of the 20th ACM International Conference on Multimedia, MM ’12, pp 925–928. ACM, New York. doi:10.1145/2393347.2396348
Bailer W, Schoeffmann K, Ahlström D, Weiss W, del Fabro M (2013) Interactive evaluation of video browsing tools. In: Proceedings of the Multimedia Modeling Conference, pp 81–91
Bailer W, Weiss W, Kienast G, Thallinger G, Haas W (2010) A video browsing tool for content management in post-production. Int J Digit Multimed Broadcasting 2010:1–17. doi:10.1155/2010/856761
Barnes C, Goldman DB, Shechtman E, Finkelstein A (2010) Video tapestries with continuous temporal zoom. In: ACM SIGGRAPH 2010 Papers, SIGGRAPH ’10, pp 89:1–89:9. ACM, New York. doi:10.1145/1833349.1778826
Borgo R, Chen M, Daubney B, Grundy E, Heidemann G, H-ferlin B, H-ferlin M, Leitte H, Weiskopf D, Xie X (2012) State of the art report on video-based graphics and video visualization. Comput Graph Forum 31(8):2450–2477. doi:10.1111/j.1467-8659.2012.03158.x
Christel M, Stevens S, Kanade T, Mauldin M, Reddy R, Wactlar H (1996) Multimedia tools and applications. In: Furht B (ed) Techniques for the creation and exploration of digital video libraries., The Kluwer International series in engineering and computer scienceSpringer, US, pp 283–327. doi:10.1007/978-1-4613-1387-8_8
Christel MG, Smith MA, Taylor CR, Winkler DB (1998) Evolving video skims into useful multimedia abstractions. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’98, pp 171–178. ACM Press/Addison-Wesley Publishing Co., New York. doi:10.1145/274644.274670
del Fabro M, Münzer B, Böszörmenyi L (2013) AAU video browser with augmented navigation bars. In: Proceedings of the Multimedia Modeling Conference, pp 544–546
del Fabro M, Schoeffmann K, Böszörmenyi L (2010) Instant video browsing: a tool for fast non-sequential hierarchical video browsing. In: Proceedings of the 6th Symposium of the Workgroup Human-Computer Interaction and Usability, Engineering, pp 443–446. doi:10.1007/978-3-642-16607-5_30
Frisson C, Dupont S, Moinet A, Picard-Limpens C, Ravet T, Siebert X, Dutoit T(2013) Videocycle: user-friendly navigation by similarity in video databases. In: Proceedings of the Multimedia Modeling Conference, pp 550–553
Girgensohn A, Shipman F, Wilcox L (2011) Adaptive clustering and interactive visualizations to support the selection of video clips. In: Proceedings of the 1st ACM International Conference on Multimedia Retrieval, pp 34:1–34:8. doi:10.1145/1991996.1992030
Huber J, Steimle J, Lissermann R, Olberding S, Mühlhäuser M (2010) Wipe‘n’watch: spatial interaction techniques for interrelated video collections on mobile devices. In: Proceedings of the 24th BCS Interaction Specialist Group Conference, pp 423–427
Hürst W, Götz G, Welte M (2007) Interactive video browsing on mobile devices. In: Proceedings of the 15th international conference on Multimedia, pp 247–256. doi:10.1145/1291233.1291284
Jansen M, Heeren W, van Dijk B (2008) Videotrees: improving video surrogate presentation using hierarchy. In: Content-based multimedia indexing. International Workshop on CBMI 2008, pp 560–567. doi:10.1109/CBMI.2008.4564997
Jégou H, Douze M, Schmid C (2011) Product quantization for nearest neighbor search. IEEE Trans Pattern Anal Mach Intell 33:117–128
Jégou H, Douze M, Schmid C, Perez P (2010) Aggregating local descriptors into a compact image representation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 3304–3311
Jiang J, Zhang XP (2011) A smart video player with content-based fast-forward playback. In: Proceedings of the 19th ACM international conference on Multimedia, pp 1061–1064. doi:10.1145/2072298.2071938
Le DD, Lam V, Ngo TD, Tran VQ, Nguyen VH, Duong DA, Satoh S (2013) NII-UIT-VBS: A video browsing tool for known item search. In: Proceedings of the Multimedia Modeling Conference, pp 547–549
Lowe D (1999) Object recognition from local scale-invariant features. In: IEEE International Conference on Computer Vision, pp 1150–1157
Marchand-Maillet S, Morrison D, Szekely E, Bruno E (2010) Interactive representations of multimodal databases. In: Thiran J-P, Marques F, Bourlard H (eds) Multimodal signal processing, theory and applications for human-computer interaction. Elsevier, Amsterdam, ISBN: 9780123748256
Meixner B, Köstler J, Kosch H (2011) A mobile player for interactive non-linear video. In: Proceedings of the 19th ACM international conference on Multimedia, pp 779–780. doi:10.1145/2072298.2072453
Mueller C, Smole M, Schöffmann K (2012) A demonstration of a hierarchical multi-layout 3D video browser. In: Proceedings of the IEEE International Conference on Multimedia and Expo Workshops. doi:10.1109/ICMEW.2012.121
Over P, Awad G, Michel M, Fiscus J, Sanders G, Shaw B, Kraaij W, Smeaton AF, Quénot G (2012) Trecvid 2012: an overview of the goals, tasks, data, evaluation mechanisms and metrics. In: Proceedings of TRECVID 2012
de Rooij O, Snoek CG, Worring M (2008) Balancing thread based navigation for targeted video search. In: Proceedings of the 2008 international conference on content-based image and video retrieval, pp 485–494. doi:10.1145/1386352.1386414
Schoeffmann K, Boeszoermenyi L (2011) Image and video browsing with a cylindrical 3D storyboard. In: Proceedings of the 1st ACM International Conference on Multimedia Retrieval, ICMR ’11, pp 63:1–63:2. ACM, New York. doi:10.1145/1991996.1992059
Schoeffmann K, Cobarzan C (2013) An evaluation of interactive search with modern video players. In: IEEE International Conference on Multimedia and Expo Workshops (ICMEW), 2013, pp 1–4. doi:10.1109/ICMEW.2013.6618282
Schoeffmann K, Fabro MD (2011) Hierarchical video browsing with a 3D carousel. In: Proceedings of the 19th ACM international conference on Multimedia, pp 827–828. doi:10.1145/2072298.2072479
Schoeffmann K, Hopfgartner F, Marques O, Böszörmenyi L, Jose JM (2010) Video browsing interfaces and applications: a review. SPIE Rev 1(1):018004. doi:10.1117/6.0000005
Schoeffmann K, Taschwer M, Böszörmenyi L (2010) The video explorer: a tool for navigation and searching within a single video based on fast content analysis. In: Proceedings of the first annual ACM SIGMM conference on Multimedia systems, pp 247–258. doi:10.1145/1730836.1730867
Scott D, Guo J, Gurrin C, Hopfgartner F, McGuinness K, O’Connor N, Smeaton A, Yang Y, Zhang Z (2013) DCU at MMM 2013 Video Browser Showdown. In: Proceedings of the Multimedia Modeling Conference, pp 541–543
Scott D, Hopfgartner F, Guo J, Gurrin C (2013) Evaluating novice and expert users on handheld video retrieval systems. In: Proceedings of MMM, pp 69–78
Smeaton AF, Over P, Kraaij W (2006) Evaluation campaigns and TRECVid. In: Proceedings of the 8th ACM International Workshop on Multimedia Information Retrieval, pp 321–330. http://doi.acm.org/10.1145/1178677.1178722
Snoek C, Worring M, de Rooij O, van de Sande K, Yan R, Hauptmann A (2008) Videolympics: real-time evaluation of multimedia retrieval systems. IEEE Multimed 15(1):86–91. doi:10.1109/MMUL.2008.21
Tao K, Dong Y, Bian Y, Chang X, Bai H (2012) The France Telecom Orange Labs (Beijing) video semantic indexing systems. In: TRECVID 2012
Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 511–518 (2001)
Wilkins P, Byrne D, Jones GJF, Lee H, Keenan G, McGuinness K, O’Connor NE, O’Hare N, Smeaton AF, Adamek T, Troncy R, Amin A, Benmokhtar R, Dumont E, Huet B, Mérialdo B, Tolias G, Spyrou E, Avrithis YS, Papadopoulos G, Mezaris V, Kompatsiaris I, Mörzinger R, Schallauer P, Bailer W, Chandramouli K, Izquierdo E, Goldmann L, Haller M, Samour A, Cobet A, Sikora T, Praks P, Hannah D, Halvey M, Hopfgartner F, Villa R, Punitha P, Goyal A, Jose JM (2008) K-space at trecvid 2008. In: TRECVID
Worring M, Sajda P, Santini S, Shamma DA, Smeaton AF, Yang Q (2012) Where is the user in multimedia retrieval? IEEE MultiMed 19(4):6–10