Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Recognizing Human Actions with Outlier Frames by Observation Filtering and Completion

Recognizing Human Actions with Outlier Frames by Observation Filtering and Completion This article addresses the problem of recognizing partially observed human actions. Videos of actions acquired in the real world often contain corrupt frames caused by various factors. These frames may appear irregularly, and make the actions only partially observed. They change the appearance of actions and degrade the performance of pretrained recognition systems. In this article, we propose an approach to address the corrupt-frame problem without knowing their locations and durations in advance. The proposed approach includes two key components: outlier filtering and observation completion. The former identifies and filters out unobserved frames, and the latter fills up the filtered parts by retrieving coherent alternatives from training data. Hidden Conditional Random Fields (HCRFs) are then used to recognize the filtered and completed actions. Our approach has been evaluated on three datasets, which contain both fully observed actions and partially observed actions with either real or synthetic corrupt frames. The experimental results show that our approach performs favorably against the other state-of-the-art methods, especially when corrupt frames are present. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) Association for Computing Machinery

Recognizing Human Actions with Outlier Frames by Observation Filtering and Completion

Download PDF
23 pages

Loading next page...
 
/lp/association-for-computing-machinery/recognizing-human-actions-with-outlier-frames-by-observation-filtering-AzGcCxHFbG

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
Association for Computing Machinery
Copyright
Copyright © 2017 ACM
ISSN
1551-6857
eISSN
1551-6865
DOI
10.1145/3089250
Publisher site
See Article on Publisher Site

Abstract

This article addresses the problem of recognizing partially observed human actions. Videos of actions acquired in the real world often contain corrupt frames caused by various factors. These frames may appear irregularly, and make the actions only partially observed. They change the appearance of actions and degrade the performance of pretrained recognition systems. In this article, we propose an approach to address the corrupt-frame problem without knowing their locations and durations in advance. The proposed approach includes two key components: outlier filtering and observation completion. The former identifies and filters out unobserved frames, and the latter fills up the filtered parts by retrieving coherent alternatives from training data. Hidden Conditional Random Fields (HCRFs) are then used to recognize the filtered and completed actions. Our approach has been evaluated on three datasets, which contain both fully observed actions and partially observed actions with either real or synthetic corrupt frames. The experimental results show that our approach performs favorably against the other state-of-the-art methods, especially when corrupt frames are present.

Journal

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) – Association for Computing Machinery

Published: Jun 28, 2017

Keywords: Human action recognition

References

  • Exploiting unconscious user signals in multimodal human-computer interaction
    Andre, Elisabeth
  • Sparse occlusion detection with optical flow
    Ayvaci, Alper; Raptis, Michalis; Soatto, Stefano
  • Pose filter based hidden-CRF models for activity detection
    Banerjee, Prithviraj; Nevatia, Ram
  • Recognize human activities from partially observed videos
    Cao, Yu; Barrett, Daniel; Barbu, Andrei; Narayanaswamy, Swaminathan; Yu, Haonan; Michaux, Aaron; Lin, Yuewei; Dickinson, Sven; Siskind, Jeffrey Mark; Wang, Song
  • Realtime multi-person 2D pose estimation using part affinity fields
    Cao, Zhe; Simon, Tomas; Wei, Shih-En; Sheikh, Yaser
  • Combining multiple dynamic models and deep learning architectures for tracking the left ventricle endocardium in ultrasound data
    Carneiro, Gustavo; Nascimento, Jacinto C.
  • Fusion of skeletal and silhouette-based features for human action recognition with RGB-D devices
    Chaaraoui, Alexandros Andre; Padilla-López, José Ramón; Flórez-Revuelta, Francisco
  • Multiple structured-instance learning for semantic segmentation with uncertain training data
    Chang, Feng-Ju; Lin, Yen-Yu; Hsu, Kuang-Jui
  • Learning partially-observed hidden conditional random fields for facial expression recognition
    Chang, Kai-Yueh; Liu, Tyng-Luh; Lai, Shang-Hong
  • Modeling human activities as speech
    Chen, Chia-Chih; Aggarwal, J. K.
  • Accurate and real-time human action recognition based on 3D skeleton
    Chen, Hongzhao; Wang, Guijin; He, Li
  • Adaptive human motion analysis and prediction
    Chen, Zhuo; Wang, Lu; Yung, Nelson H. C.
  • Structured feature learning for pose estimation
    Chu, Xiao; Ouyang, Wanli; Li, Hongsheng; Wang, Xiaogang
  • Minimal-latency human action recognition using reliable-inference
    Davis, James W.; Tyagi, Ambrish
  • 3-D human action recognition by shape analysis of motion trajectories on Riemannian manifold
    Devanne, Maxime; Wannous, Hazem; Berretti, Stefano; Pala, Pietro; Daoudi, Meroua; Del Bimbo, Alberto
  • Behavior recognition via sparse spatio-temporal features
    Dollár, Piotr; Rabaud, Vincent; Cottrell, Garrison; Belongie, Serge
  • Long-term recurrent convolutional networks for visual recognition and description
    Donahue, Jeffrey; Hendricks, Lisa Anne; Guadarrama, Sergio; Rohrbach, Marcus; Venugopalan, Subhashini; Saenko, Kate; Darrell, Trevor
  • A Bayesian hierarchical model for learning natural scene categories
    Fei-Fei, Li; Perona, Pietro
  • Convolutional two-stream network fusion for video action recognition
    Feichtenhofer, Christoph; Pinz, Axel; Zisserman, Andrew
  • Training structural SVMs when exact inference is intractable
    Finley, Thomas; Joachims, Thorsten
  • Instructing people for training gestural interactive systems
    Fothergill, Simon; Mentis, Helena M.; Kohli, Pushmeet; Nowozin, Sebastian
  • Devnet: A deep event network for multimedia event detection and evidence recounting
    Gan, Chuang; Wang, Naiyan; Yang, Yi; Yeung, Dit-Yan; Hauptmann, Alex G
  • Histogram of oriented displacements (HOD): Describing trajectories of human joints for action recognition
    Gowayyed, Mohammad A.; Torki, Marwan; Hussein, Mohamed E.; El-Saban, Motaz
  • Max-margin early event detectors
    Hoai, Minh; De la Torre, Fernando
  • Dynamic action classification based on iterative data selection and feedforward neural networks
    Iosifidis, Alexandros; Tefas, Anastasios; Pitas, Ioannis
  • Modeling high-dimensional humans for activity anticipation using Gaussian process latent CRFs
    Jiang, Yun; Saxena, Ashutosh
  • Imagenet classification with deep convolutional neural networks
    Krizhevsky, Alex; Sutskever, Ilya; Hinton, Geoffrey E.
  • A hierarchical representation for future action prediction
    Lan, Tian; Chen, Tsung-Chuan; Savarese, Silvio
  • On space-time interest points
    Laptev, Ivan
  • Segmentation and recognition of motion streams by similarity search
    Li, Chuanjun; Zheng, S. Q.; Prabhakaran, B.
  • Prediction of human activity by discovering temporal sequence patterns
    Li, Kang; Fu, Yun
  • Action recognition by learning deep multi-granular spatio-temporal video representation
    Li, Qing; Qiu, Zhaofan; Yao, Ting; Mei, Tao; Rui, Yong; Luo, Jiebo
  • Expandable data-driven graphical modeling of human actions based on salient postures
    Li, Wanqing; Zhang, Zhengyou; Liu, Zicheng
  • Learning coupled classifiers with RGB images for RGB-D object recognition
    Li, Xiao; Fang, Min; Zhang, Ju-Jie; Wu, Jinqiao
  • Learning and inferring human actions with temporal pyramid features based on conditional random fields
    Lin, Shih-Yao; Lin, Yen-Yu; Chen, Chu-Song; Hung, Yi-Ping
  • Depth and skeleton associated action recognition without online accessible RGB-D cameras
    Lin, Yen-Yu; Hua, Ju-Hsuan; Tang, Nick C.; Chen, Min-Hung; Liao, Hong-Yuan Mark
  • Learning spatio-temporal representations for action recognition: A genetic programming approach
    Liu, Li; Shao, Ling; Li, Xuelong; Lu, Ke
  • DAP3D-Net: Where, what and how actions occur in videos? arXiv Preprint arXiv:1602
    Liu, Li; Zhou, Yi; Shao, Ling
  • Recognition and segmentation of 3-D human action using HMM and multi-class adaboost
    Lv, Fengjun; Nevatia, Ramakant
  • Some methods for classification and analysis of multivariate observations
    MacQueen, James
  • Action recognition from a distributed representation of pose and appearance
    Maji, Subhransu; Bourdev, Lubomir; Malik, Jitendra
  • Learning recurrent neural networks with Hessian-free optimization
    Martens, James; Sutskever, Ilya
  • HON4D: Histogram of oriented 4D normals for activity recognition from depth sequences
    Oreifej, Omar; Liu, Zicheng
  • Capturing the relative distribution of features for action recognition
    Oshin, Olusegun; Gilbert, Andrew; Bowden, Richard
  • Conditional neural fields
    Peng, Jian; Bo, Liefeng; Xu, Jinbo
  • Gaussian mixture based HMM for human daily activity recognition using 3D skeleton features
    Piyathilaka, Lasitha; Kodagoda, Sarath
  • Hidden conditional random fields
    Quattoni, Ariadna; Wang, Sybor; Morency, Louis-Philippe; Collins, Michael; Darrell, Trevor
  • Poselet key-framing: A model for human activity recognition
    Raptis, Michalis; Sigal, Leonid
  • Human activity prediction: Early recognition of ongoing activities from streaming videos
    Ryoo, M. S.
  • UT-Interaction Dataset, ICPR contest on Semantic Description of Human Activities (SDHA)
    Ryoo, M. S.; Aggarwal, J. K.
  • Action snippets: How many frames does human action recognition require?
    Schindler, Konrad; Van Gool, Luc
  • Exemplar-based human action pose correction and tagging
    Shen, Wei; Deng, Ke; Bai, Xiang; Leyvand, Tommer; Guo, Baining; Tu, Zhuowen
  • Deep co-occurrence feature learning for visual object recognition
    Shih, Ya-Fang; Yeh, Yang-Ming; Lin, Yen-Yu; Weng, Ming-Feng; Lu, Yi-Chang; Chuang, Yung-Yu
  • Part-based multiple-person tracking with partial occlusion handling
    Shu, Guang; Dehghan, Afshin; Oreifej, Omar; Hand, Emily; Shah, Mubarak
  • Two-stream convolutional networks for action recognition in videos
    Simonyan, Karen; Zisserman, Andrew
  • Multi-view latent variable discriminative models for action recognition
    Song, Yale; Morency, Louis-Philippe; Davis, Randall
  • Action recognition by hierarchical sequence summarization
    Song, Yale; Morency, Louis-Philippe; Davis, Ronald W.
  • Unstructured human activity detection from RGBD images
    Sung, Jaeyong; Ponce, Colin; Selman, Bart; Saxena, Ashutosh
  • An Introduction to Conditional Random Fields for Relational Learning
    Sutton, C.; McCallum, A.
  • Robust action recognition via borrowing information across video modalities
    Tang, Nick C.; Lin, Yen-Yu; Hua, Ju-Hsuan; Wei, Shih-En; Weng, Ming-Fang; Liao, Hong-Yuan Mark
  • Learning spatiotemporal features with 3D convolutional networks
    Tran, Du; Bourdev, Lubomir; Fergus, Rob; Torresani, Lorenzo; Paluri, Manohar
  • Human action recognition by representing 3D skeletons as points in a lie group
    Vemulapalli, Raviteja; Arrate, Felipe; Chellappa, Rama
  • Mining actionlet ensemble for action recognition with depth cameras
    Wang, Jiang; Liu, Zicheng; Wu, Ying; Yuan, Junsong
  • Learning actionlet ensemble for 3D human action recognition
    Wang, Jiang; Liu, Zicheng; Wu, Ying; Yuan, Junsong
  • Recognizing human activities from silhouettes: Motion subspace and factorial discriminative graphical model
    Wang, Liang; Suter, David
  • An HOG-LBP human detector with partial occlusion handling
    Wang, Xiaoyu; Han, Tony X.; Yan, Shuicheng
  • Making action recognition robust to occlusions and viewpoint changes
    Weinland, Daniel; Özuysal, Mustafa; Fua, Pascal
  • Spatio-temporal depth cuboid similarity feature for activity recognition using depth camera
    Xia, Lu; Aggarwal, J. K.
  • Learning deep feature representations with domain guided dropout for person re-identification
    Xiao, Tong; Li, Hongsheng; Ouyang, Wanli; Wang, Xiaogang
  • Predicting human activities using spatio-temporal structure of interest points
    Yu, Gang; Yuan, Junsong; Liu, Zicheng
  • Propagative Hough voting for human activity detection and recognition
    Yu, Gang; Yuan, Junsong; Liu, Zicheng
  • Segmentation of discriminative patches in human activity video
    Zhang, Bo; Conci, Nicola; De Natale, Francesco G. B.
  • Action categorization with modified hidden conditional random field
    Zhang, Jianguo; Gong, Shaogang
  • Probabilistic image modeling with an extended chain graph for human activity recognition and image segmentation
    Zhang, Lei; Zeng, Zhi; Ji, Qiang
  • Structured streaming skeleton—A new feature for online human gesture recognition
    Zhao, Xin; Li, Xue; Pang, Chaoyi; Sheng, Quan Z.; Wang, Sen; Ye, Mao