Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

From Annotation to Computer-Aided Diagnosis

From Annotation to Computer-Aided Diagnosis Holistic medical multimedia systems covering end-to-end functionality from data collection to aided diagnosis are highly needed, but rare. In many hospitals, the potential value of multimedia data collected through routine examinations is not recognized. Moreover, the availability of the data is limited, as the health care personnel may not have direct access to stored data. However, medical specialists interact with multimedia content daily through their everyday work and have an increasing interest in finding ways to use it to facilitate their work processes. In this article, we present a novel, holistic multimedia system aiming to tackle automatic analysis of video from gastrointestinal (GI) endoscopy. The proposed system comprises the whole pipeline, including data collection, processing, analysis, and visualization. It combines filters using machine learning, image recognition, and extraction of global and local image features. The novelty is primarily in this holistic approach and its real-time performance, where we automate a complete algorithmic GI screening process. We built the system in a modular way to make it easily extendable to analyze various abnormalities, and we made it efficient in order to run in real time. The conducted experimental evaluation proves that the detection and localization accuracy are comparable or even better than existing systems, but it is by far leading in terms of real-time performance and efficient resource consumption. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) Association for Computing Machinery

From Annotation to Computer-Aided Diagnosis

Download PDF
26 pages

Loading next page...
 
/lp/association-for-computing-machinery/from-annotation-to-computer-aided-diagnosis-0qPSu9Isfy

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
Association for Computing Machinery
Copyright
Copyright © 2017 ACM
ISSN
1551-6857
eISSN
1551-6865
DOI
10.1145/3079765
Publisher site
See Article on Publisher Site

Abstract

Holistic medical multimedia systems covering end-to-end functionality from data collection to aided diagnosis are highly needed, but rare. In many hospitals, the potential value of multimedia data collected through routine examinations is not recognized. Moreover, the availability of the data is limited, as the health care personnel may not have direct access to stored data. However, medical specialists interact with multimedia content daily through their everyday work and have an increasing interest in finding ways to use it to facilitate their work processes. In this article, we present a novel, holistic multimedia system aiming to tackle automatic analysis of video from gastrointestinal (GI) endoscopy. The proposed system comprises the whole pipeline, including data collection, processing, analysis, and visualization. It combines filters using machine learning, image recognition, and extraction of global and local image features. The novelty is primarily in this holistic approach and its real-time performance, where we automate a complete algorithmic GI screening process. We built the system in a modular way to make it easily extendable to analyze various abnormalities, and we made it efficient in order to run in real time. The conducted experimental evaluation proves that the detection and localization accuracy are comparable or even better than existing systems, but it is by far leading in terms of real-time performance and efficient resource consumption.

Journal

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) – Association for Computing Machinery

Published: May 31, 2017

Keywords: Medical multimedia system

References

  • TensorFlow: A system for large-scale machine learning
    Abadi, Martın; Barham, Paul; Chen, Jianmin; Chen, Zhifeng; Davis, Andy; Dean, Jeffrey; Devin, Matthieu; Ghemawat, Sanjay; Irving, Geoffrey; Isard, Michael; Kudlur, Manjunath; Levenberg, Josh; Monga, Rajat; Moore, Sherry; Murray, Derek G.; Steiner, Benoit; Tucker, Paul; Vasudevan, Vijay; Warden, Pete; Wicke, Martin; Yu, Yuan; Zheng, Xiaoqiang
  • Expert driven semi-supervised elucidation tool for medical endoscopic videos
    Albisser, Zeno; Riegler, Michael; Halvorsen, Pål; Zhou, Jiang; Griwodz, Carsten; Balasingham, IIangko; Gurrin, Cathal
  • Polyp detection in endoscopic video using SVMs
    Alexandre, Luís A.; Casteleiro, Joao; Nobreinst, Nuno
  • Texture-based polyp detection in colonoscopy
    Ameling, Stefan; Wirth, Stephan; Paulus, Dietrich; Lacey, Gerard; Vilarino, Fernando
  • Colorectal cancer
    Brenner, Hermann; Kloor, Matthias; Pox, Christian Peter
  • Non-local means denoising
    Buades, Antoni; Coll, Bartomeu; Morel, Jean-Michel
  • Colorectal polyps detection using texture features and support vector machine
    Cheng, Da-Chuan; Ting, Wen-Chien; Chen, Yung-Fu; Pu, Qin; Jiang, Xiaoyi
  • Learning in science: A comparison of deep and surface approaches
    Chin, Christine; Brown, David E.
  • Scholten, Cornelia Schaefer-Prokop, Mathilde M
    Ciompi, Francesco; Chung, Kaman; van Riel, Sarah J.; Setio, Arnaud Arindra Adiyoso; Gerke, Paul K.; Jacobs, Colin; Th, Ernst
  • Deep sparse feature selection for computer aided endoscopy diagnosis
    Cong, Yang; Wang, Shuai; Liu, Ji; Cao, Jun; Yang, Yunsheng; Luo, Jiebo
  • Image documentation of endoscopic findings in ulcerative colitis: Photographs or video clips? Gastrointestinal Endoscopy 61, 6 (2005), 715--720
    de Lange, Thomas; Larsen, Stig; Aabakken, Lars
  • Polyp measurement based on CT colonography and colonoscopy: Variability and systematic differences
    de Vries, Ayso H.; Bipat, Shandra; Dekker, Evelien; Liedenbaum, Marjolein H.; Florie, Jasper; Fockens, Paul; van der Kraan, Roel; Mathus-Vliegen, Elizabeth M.; Reitsma, Johannes B.; Truyen, Roel; others,
  • Improvements on cross-validation: The
    Efron, Bradley; Tibshirani, Robert
  • Late fusion of heterogeneous methods for multimedia image retrieval
    Escalante, Hugo Jair; Hérnadez, Carlos A.; Sucar, Luis Enrique; Montes, Manuel
  • Cancer incidence and mortality patterns in Europe: Estimates for 40 countries in 2012
    Ferlay, J.; Steliarova-Foucher, E.; Lortet-Tieulent, J.; Rosso, S.; Coebergh, J. W.; Comber, H.; Forman, D.; Bray, F.
  • A buyer’s guide to conic fitting
    Fitzgibbon, Andrew W.; Fisher, Robert B.
  • Apache Lucene - Index File Formats
    Foundation, The Apache Software
  • Bleeding detection from capsule endoscopy videos
    Giritharan, B.; Yuan, Xiaohui; Liu, Jianguo; Buckles, B.; Oh, JungHwan; Tang, Shou Jiang
  • Flexible sigmoidoscopy versus faecal occult blood testing for colorectal cancer screening in asymptomatic individuals
    Holme, O.; Bretthauer, M.; Fretheim, A.; Odgaard-Jensen, J.; Hoff, G.
  • Polyp detection in colonoscopy video using elliptical shape feature
    Hwang, Sae; Oh, JungHwan; Tavanapong, W.; Wong, J.; de Groen, P. C.
  • The Paris endoscopic classification of superficial neoplastic lesions: Esophagus, stomach, and colon: November 30 to December 1, 2002
    Inoue, H.; Kashida, H.; Kudo, S.; Sasako, M.; Shimoda, T.; Watanabe, H.; Yoshida, S.; Guelrud, M.; Lightdale, C. J.; Wang, K.; others,
  • Computer-aided diagnosis of mammographic masses using scalable image retrieval
    Jiang, Menglin; Zhang, Shaoting; Li, Hongsheng; Metaxas, Dimitris N.
  • Quality indicators for colonoscopy and the risk of interval cancer
    Kaminski, M. F.; Regula, J.; Kraszewska, E.; Polkowski, M.; Wojciechowska, U.; Didkowska, J.; Zwierko, M.; Rupinski, M.; Nowacki, M. P.; Butruk, E.
  • Real-time image processing system for endoscopic applications
    Kang, J.; Doraiswami, R.
  • Wireless communication link for capsule endoscope at 600 MHz
    Khaleghi, A.; Balasingham, I.
  • The Art of Computer Programming: Sorting and Searching
    Knuth, Donald Ervin
  • Colorectal tumours and pit pattern
    Kudo, S.; Hirota, S.; Nakajima, T.; Hosobe, S.; Kusaka, H.; Kobayashi, T.; Himori, M.; Yagyuu, A.
  • Tumor recognition in wireless capsule endoscopy images using textural features and SVM-based feature selection
    Li, Baopu; Meng, M. Q.-H.
  • Computer-based detection of bleeding and ulcer in wireless capsule endoscopy images by chromaticity moments
    Li, Baopu; Meng, Max Q. H.
  • Unsupervised multi-feature tag relevance learning for social image retrieval
    Li, Xirong; Snoek, Cees G. M.; Worring, Marcel
  • Arthemis: Annotation software in an integrated capturing and analysis system for colonoscopy
    Liu, Danyu; Cao, Yu; Kim, Kihwan; Stanek, Sean; Doungratanaex-Chai, Bancha; Lin, Kungen; Tavanapong, Wallapak; Wong, Johnny S.; Oh, Jung-Hwan; de Groen, Piet C.
  • LIRE: Open source image retrieval in Java
    Lux, Mathias
  • Annotation of endoscopic videos on mobile devices: A bottom-up approach
    Lux, Mathias; Riegler, Michael
  • Advances in diagnostic and therapeutic endoscopy
    Mallery, Shawn; Van Dam, Jacques
  • Automated polyp detection in colon capsule endoscopy
    Mamonov, A. V.; Figueiredo, I. N.; Figueiredo, P. N.; Tsai, Y.-H. R.
  • Abnormal image detection in endoscopy videos using a filter bank and local binary patterns
    Nawarathna, Ruwan; Oh, JungHwan; Muthukudage, Jayantha; Tavanapong, Wallapak; Wong, Johnny; De Groen, Piet C.; Tang, Shou Jiang
  • Deep neural networks are easily fooled: High confidence predictions for unrecognizable images
    Nguyen, Anh; Yosinski, Jason; Clune, Jeff
  • Current status of narrow-band imaging magnifying colonoscopy for colorectal neoplasia in Japan
    Oba, S.; Tanaka, S.; Sano, Y.; Oka, S.; Chayama, K.
  • Polyp detection in colonoscopy videos using deeply-learned hierarchical features
    Park, Sungheon; Lee, Myunggi; Kwak, Nojun
  • A holistic multimedia system for gastrointestinal tract disease detection
    Pogorelov, Konstantin; Losada, Sigrun; Griwodz, Carsten; de Lange, Thomas; Randel, Kristin Ranheim; Nguyen, Duc Tien Dang; Stensland, Håkon Kvale; De Natale, Francesco; Johansen, Dag; Riegler, Michael; Halvorsen, Pål
  • You only look once: Unified, real-time object detection
    Redmon, Joseph; Divvala, Santosh; Girshick, Ross; Farhadi, Ali
  • How ‘how’ reflects what’s what: Content-based exploitation of how users frame social images
    Riegler, Michael; Larson, Martha; Lux, Mathias; Kofler, Christoph
  • VideoJot: A multifunctional video annotation tool
    Riegler, Michael; Lux, Mathias; Charvillat, Vincent; Carlier, Axel; Vliegendhart, Raynor; Larson, Martha
  • Multimedia and medicine: Teammates for better disease detection and survival
    Riegler, Michael; Lux, Mathias; Griwodz, Carsten; Spampinato, Concetto; de Lange, Thomas; Eskeland, Sigrun L.; Pogorelov, Konstantin; Tavanapong, Wallapak; Schmidt, Peter T.; Gurrin, Cathal; Johansen, Dag; Johansen, Håvard; Halvorsen, Pål
  • EIR - efficient computer aided diagnosis framework for gastrointestinal endoscopies
    Riegler, Michael; Pogorelov, Konstantin; Halvorsen, Pål; de Lange, Thomas; Griwodz, Carsten; Schmidt, Peter Thelin; Eskeland, Sigrun Losada; Johansen, Dag
  • Deep learning in neural networks: An overview
    Schmidhuber, Jürgen
  • A case--control study of screening sigmoidoscopy and mortality from colorectal cancer
    Selby, Joe V.; Friedman, Gary D.; Jr, Charles P. Quesenberry; Weiss, Noel S.
  • Multimedia indexing, search, and retrieval in large databases of social networks
    Semertzidis, Theodoros; Rafailidis, Dimitrios; Tiakas, Eleftherios; Strintzis, Michael G.; Daras, Petros
  • Early versus late fusion in semantic video analysis
    Snoek, Cees G. M.; Worring, Marcel; Smeulders, Arnold W. M.
  • Effective hashing for large-scale multimedia search
    Song, Jingkuan
  • Probabilistic neural networks
    Specht, Donald F.
  • Automated polyp detection in colonoscopy videos using shape and context information
    Tajbakhsh, Nima; Gurudu, Suryakanth; Liang, Jianming
  • Automatic polyp detection in colonoscopy videos using an ensemble of convolutional neural networks
    Tajbakhsh, Nima; Gurudu, Suryakanth R.; Liang, Jianming
  • Narrow-band imaging magnifying endoscopy in adult patients with eosinophilic esophagitis/esophageal eosinophilia and lymphocytic esophagitis
    Tanaka, Kyosuke; Rubio, Carlos A.; Dlugosz, Aldona; Truskaite, Kotryna; Befrits, Ragnar; Lindberg, Greger; Schmidt, Peter T.
  • The $2
    Times, The New York
  • European guidelines for quality assurance in colorectal cancer screening and diagnosis
    von Karsa, L.; Patnick, J.; Segnan, N.
  • Deep learning for identifying metastatic breast cancer
    Wang, Dayong; Khosla, Aditya; Gargeya, Rishab; Irshad, Humayun; Beck, Andrew H.
  • Computer-aided endoscopic diagnosis without human-specific labeling
    Wang, Shuai; Cong, Yang; Fan, Huijie; Liu, Lianqing; Li, Xiaoqiu; Yang, Yunsheng; Tang, Yandong; Zhao, Huaici; Yu, Haibin
  • Computer-aided detection of retroflexion in colonoscopy
    Wang, Yi; Tavanapong, Wallapak; Wong, Johnny; Oh, JungHwan; de Groen, Piet C.
  • Near real-time retroflexion detection in colonoscopy
    Wang, Yi; Tavanapong, Wallapak; Wong, Johnny; Oh, JungHwan; de Groen, Piet C.
  • Part-based multiderivative edge cross-sectional profiles for polyp detection in colonoscopy
    Wang, Yi; Tavanapong, Wallapak; Wong, Johnson; Oh, JungHwan; de Groen, Piet C.
  • Polyp-alert: Near real-time feedback during colonoscopy
    Wang, Yi; Tavanapong, Wallapak; Wong, Johnny; Oh, Jung Hwan; de Groen, Piet C.
  • Detection of quality visualization of appendiceal orifices using local edge cross-section profile features and near pause detection
    Wang, Yi; Tavanapong, Wallapak; Wong, Johnny S.; Oh, JungHwan; de Groen, Piet C.
  • Estimated Cancer Incidence, Mortality and Prevalence Worldwide in 2012
    Cancer, World Health Organization - International Agency for Research on
  • Polyp detection and radius measurement in small intestine using video capsule endoscopy
    Zhou, Mingda; Bao, Guanqun; Geng, Yishuang; Alkandari, B.; Li, Xiaoxi