Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Creating Segments and Effects on Comics by Clustering Gaze Data

Creating Segments and Effects on Comics by Clustering Gaze Data Traditional comics are increasingly being augmented with digital effects, such as recoloring, stereoscopy, and animation. An open question in this endeavor is identifying where in a comic panel the effects should be placed. We propose a fast, semi-automatic technique to identify effects-worthy segments in a comic panel by utilizing gaze locations as a proxy for the importance of a region. We take advantage of the fact that comic artists influence viewer gaze towards narrative important regions. By capturing gaze locations from multiple viewers, we can identify important regions and direct a computer vision segmentation algorithm to extract these segments. The challenge is that these gaze data are noisy and difficult to process. Our key contribution is to leverage a theoretical breakthrough in the computer networks community towards robust and meaningful clustering of gaze locations into semantic regions, without needing the user to specify the number of clusters. We present a method based on the concept of relative eigen quality that takes a scanned comic image and a set of gaze points and produces an image segmentation. We demonstrate a variety of effects such as defocus, recoloring, stereoscopy, and animations. We also investigate the use of artificially generated gaze locations from saliency models in place of actual gaze locations. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) Association for Computing Machinery

Creating Segments and Effects on Comics by Clustering Gaze Data

Download PDF
23 pages

Loading next page...
 
/lp/association-for-computing-machinery/creating-segments-and-effects-on-comics-by-clustering-gaze-data-0ZKdgaujjo

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
Association for Computing Machinery
Copyright
Copyright © 2017 ACM
ISSN
1551-6857
eISSN
1551-6865
DOI
10.1145/3078836
Publisher site
See Article on Publisher Site

Abstract

Traditional comics are increasingly being augmented with digital effects, such as recoloring, stereoscopy, and animation. An open question in this endeavor is identifying where in a comic panel the effects should be placed. We propose a fast, semi-automatic technique to identify effects-worthy segments in a comic panel by utilizing gaze locations as a proxy for the importance of a region. We take advantage of the fact that comic artists influence viewer gaze towards narrative important regions. By capturing gaze locations from multiple viewers, we can identify important regions and direct a computer vision segmentation algorithm to extract these segments. The challenge is that these gaze data are noisy and difficult to process. Our key contribution is to leverage a theoretical breakthrough in the computer networks community towards robust and meaningful clustering of gaze locations into semantic regions, without needing the user to specify the number of clusters. We present a method based on the concept of relative eigen quality that takes a scanned comic image and a set of gaze points and produces an image segmentation. We demonstrate a variety of effects such as defocus, recoloring, stereoscopy, and animations. We also investigate the use of artificially generated gaze locations from saliency models in place of actual gaze locations.

Journal

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) – Association for Computing Machinery

Published: May 31, 2017

Keywords: Comics

References

  • Slic Superpixels
    Achanta, Radhakrishna; Shaji, Appu; Smith, Kevin; Lucchi, Aurelien; Fua, Pascal; Süsstrunk, Sabine
  • Examining eye-tracker scan paths for elderly people using smart phones
    Al-Showarah, Suleyman; Al-Jawad, Naseer; Sellahewa, Harin
  • Interactive segmentation for manga
    Aramaki, Yuji; Matsui, Yusuke; Yamasaki, Toshihiko; Aizawa, Kiyoharu
  • Contour detection and hierarchical image segmentation
    Arbelaez, Pablo; Maire, Michael; Fowlkes, Charless; Malik, Jitendra
  • Internet Archive
    Archive,
  • Defocus magnification
    Bae, Soonmin; Durand, Frédo
  • Fixation identification: The optimum threshold for a dispersion algorithm
    Blignaut, Pieter
  • CAT2000: A large scale fixation dataset for boosting saliency research
    Borji, Ali; Itti, Laurent
  • MIT Saliency Benchmark
    Bylinskii, Zoya; Judd, Tilke; Borji, Ali; Itti, Laurent; Durand, Frédo; Oliva, Aude; Torralba, Antonio
  • Palette-based photo recoloring
    Chang, Huiwen; Fried, Ohad; Liu, Yiming; DiVerdi, Stephen; Finkelstein, Adam
  • Human attention estimation for natural images: An automatic gaze refinement approach
    Choi, Jinsoo; Oh, Tae-Hyun; Kweon, In So
  • Animating pictures with stochastic motion textures
    Chuang, Yung-Yu; Goldman, Dan B.; Zheng, Ke Colin; Curless, Brian; Salesin, David H.; Szeliski, Richard
  • Comics in public domain
    ComicBookPlus,
  • Comichron sales data
    Comichron,
  • A deep multi-level network for saliency prediction
    Cornia, Marcella; Baraldi, Lorenzo; Serra, Giuseppe; Cucchiara, Rita
  • Stylization and abstraction of photographs
    DeCarlo, Doug; Santella, Anthony
  • Learning hierarchical features for scene labeling
    Farabet, Clement; Couprie, Camille; Najman, Laurent; LeCun, Yann
  • Scanpath clustering and aggregation
    Goldberg, Joseph H.; Helfman, Jonathan I.
  • A model of saliency-based visual attention for rapid scene analysis
    Itti, Laurent; Koch, Christof; Niebur, Ernst
  • Attention-Guided Algorithms to Retarget and Augment Animations, Stills, and Videos
    Jain, Eakta
  • Inferring artistic intention in comic art through viewer gaze
    Jain, Eakta; Sheikh, Yaser; Hodgins, Jessica
  • SALICON: Saliency in context
    Jiang, Ming; Huang, Shengsheng; Duan, Juanyong; Zhao, Qi
  • Fixations on low-resolution images
    Judd, Tilke; Durand, Fredo; Torralba, Antonio
  • Learning to predict where humans look
    Judd, Tilke; Ehinger, Krista; Durand, Frédo; Torralba, Antonio
  • Depth extraction from video using non-parametric sampling
    Karsch, Kevin; Liu, Ce; Kang, Sing Bing
  • Eye tracking assisted extraction of attentionally important objects from videos
    Karthikeyan, S.; Ngo, T.; Eckstein, M.; Manjunath, B. S.
  • Human Visual Perception, Study and Applications to Understanding Images and Videos
    Katti, Harish
  • Making computers look the way we look: Exploiting visual attention for image understanding
    Katti, Harish; Subramanian, Ramanathan; Kankanhalli, Mohan; Sebe, Nicu; Chua, Tat-Seng; Ramakrishnan, Kalpathi R.
  • Draco: Bringing life to illustrations with kinetic textures
    Kazi, Rubaiat Habib; Chevalier, Fanny; Grossman, Tovi; Zhao, Shengdong; Fitzmaurice, George
  • A preliminary benchmark of four saliency algorithms on comic art
    Khetarpal, Khimya; Jain, Eakta
  • 3D object manipulation in a single photograph using stock 3D models
    Kholgade, Natasha; Simon, Tomas; Efros, Alexei; Sheikh, Yaser
  • Digital reconstruction of halftoned color comics
    Kopf, Johannes; Lischinski, Dani
  • Eye tracking for everyone
    Krafka, Kyle; Khosla, Aditya; Kellnhofer, Petr; Kannan, Harini; Bhandarkar, Suchendra; Matusik, Wojciech; Torralba, Antonio
  • Deepfix: A fully convolutional neural network for predicting human eye fixations
    Kruthiventi, Srinivas S. S.; Ayush, Kumar; Babu, R. Venkatesh
  • DeepGaze II: Reading fixations from deep features trained on object recognition
    Kümmerer, Matthias; Wallis, Thomas S. A.; Bethge, Matthias
  • Introducing context-dependent and spatially-variant viewing biases in saccadic models
    Meur, Olivier Le; Coutrot, Antoine
  • Saccadic model of eye movements for free-viewing condition
    Meur, Olivier Le; Liu, Zhi
  • Colorization using optimization
    Levin, Anat; Lischinski, Dani; Weiss, Yair
  • A closed-form solution to natural image matting
    Levin, Anat; Lischinski, Dani; Weiss, Yair
  • Stereoscopic 3D copy 8 paste
    Lo, Wan-Yen; van Baar, Jeroen; Knaus, Claude; Zwicker, Matthias; Gross, Markus
  • A stochastic model of multistable visual perception
    Merk, I.; Schnakenberg, J.
  • Active segmentation with fixation
    Mishra, Ajay; Aloimonos, Yiannis; Fah, Cheong Loong
  • Intelligent scissors for image composition
    Mortensen, Eric N.; Barrett, William A.
  • Creating fluid animation from a single image using video database
    Okabe, Makoto; Anjyo, Ken; Onai, Rikio
  • Training object class detectors from eye tracking data
    Papadopoulos, Dim P.; Clarke, Alasdair D. F.; Keller, Frank; Ferrari, Vittorio
  • Manga colorization
    Qu, Yingge; Wong, Tien-Tsin; Heng, Pheng-Ann
  • An eye fixation database for saliency detection in images
    Ramanathan, Subramanian; Katti, Harish; Sebe, Nicu; Kankanhalli, Mohan; Chua, Tat-Seng
  • LabelMe: A database and web-based tool for image annotation
    Russell, Bryan C.; Torralba, Antonio; Murphy, Kevin P.; Freeman, William T.
  • Identifying fixations and saccades in eye-tracking protocols
    Salvucci, Dario D.; Goldberg, Joseph H.
  • Gaze-based interaction for semi-automatic photo cropping
    Santella, Anthony; Agrawala, Maneesh; DeCarlo, Doug; Salesin, David; Cohen, Michael
  • Robust clustering of eye movement recordings for quantification of visual interest
    Santella, Anthony; DeCarlo, Doug
  • Automatic selection of number of clusters in networks using relative eigenvalue quality
    Shea, John M.; Macker, Joseph P.
  • Normalized cuts and image segmentation
    Shi, Jianbo; Malik, Jitendra
  • Application of clustering algorithms in eye gaze visualizations
    Špakov, Oleg; Miniotas, Darius
  • Graph-based joint clustering of fixations and visual entities
    Sugano, Yusuke; Matsushita, Yasuyuki; Sato, Yoichi
  • Video input driven animation (VIDA)
    Sun, M.; Jepson, A. D.; Fiume, E.
  • Segmentation of black and white cartoons
    Sỳkora, Daniel; Buriánek, Jan; Žára, Jiří
  • LazyBrush: Flexible painting tool for hand-drawn cartoons
    Sỳkora, Daniel; Dingliana, John; Collins, Steven
  • Bayesian online clustering of eye movement data
    Tafaj, Enkelejda; Kasneci, Gjergji; Rosenstiel, Wolfgang; Bogdan, Martin
  • Detecting eye fixations by projection clustering
    Urruty, Thierry; Lew, Stanislas; Ihadaddene, Nacim; Simovici, Dan A.
  • Modeling attention to salient proto-objects
    Walther, Dirk; Koch, Christof
  • Animating animal motion from still
    Xu, Xuemiao; Wan, Liang; Liu, Xiaopei; Wong, Tien-Tsin; Wang, Liansheng; Leung, Chi-Sing
  • Vectorizing cartoon animations
    Zhang, Song-Hai; Chen, Tao; Zhang, Yi-Fei; Hu, Shi-Min; Martin, Ralph R.