Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

“Hands On” Visual Recognition for Visually Impaired Users

“Hands On” Visual Recognition for Visually Impaired Users Blind or visually impaired (BVI) individuals are capable of identifying an object in their hands by combining the available visual cues (if available) with manipulation. It is harder for them to associate the object with a specific brand, a model, or a type. Starting from this observation, we propose a collaborative system designed to deliver visual feedback automatically and to help the user filling this semantic gap. Our visual recognition module is implemented by means of an image retrieval procedure that provides real-time feedback, performs the computation locally on the device, and is scalable to new categories and instances. We carry out a thorough experimental analysis of the visual recognition module, which includes a comparative analysis with the state of the art. We also present two different system implementations that we test with the help of BVI users to evaluate the technical soundness, the usability, and the effectiveness of the proposed concept. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Accessible Computing (TACCESS) Association for Computing Machinery

“Hands On” Visual Recognition for Visually Impaired Users

Download PDF
30 pages

Loading next page...
 
/lp/association-for-computing-machinery/hands-on-visual-recognition-for-visually-impaired-users-LM2qLr2p9d
Publisher
Association for Computing Machinery
Copyright
Copyright © 2017 ACM
ISSN
1936-7228
eISSN
1936-7236
DOI
10.1145/3060056
Publisher site
See Article on Publisher Site

Abstract

Blind or visually impaired (BVI) individuals are capable of identifying an object in their hands by combining the available visual cues (if available) with manipulation. It is harder for them to associate the object with a specific brand, a model, or a type. Starting from this observation, we propose a collaborative system designed to deliver visual feedback automatically and to help the user filling this semantic gap. Our visual recognition module is implemented by means of an image retrieval procedure that provides real-time feedback, performs the computation locally on the device, and is scalable to new categories and instances. We carry out a thorough experimental analysis of the visual recognition module, which includes a comparative analysis with the state of the art. We also present two different system implementations that we test with the help of BVI users to evaluate the technical soundness, the usability, and the effectiveness of the proposed concept.

Journal

ACM Transactions on Accessible Computing (TACCESS) – Association for Computing Machinery

Published: Aug 11, 2017

Keywords: Single-instance object recognition

References

  • Neural codes for image retrieval
    Babenko, Artem; Slesarev, Anton; Chigorin, Alexandr; Lempitsky, Victor
  • A comparison of nearest neighbor search algorithms for generic object recognition
    Bajramovic, Ferid; Mattern, Frank; Butko, Nicholas; Denzler, Joachim
  • Speeded-up robust features (SURF)
    Bay, Herbert; Ess, Andreas; Tuytelaars, Tinne; Van Gool, Luc
  • VizWiz::LocateIt—enabling blind people to locate objects in their environment
    Bigham, Jeffrey P.; Jayan, Chandrika; Miller, Andrew; White, Brandyn; Yeh, Tom
  • Ask the locals: Multi-way local pooling for image recognition
    Boureau, Y.; Roux, N. Le; Bach, F.; Ponce, J.; LeCun, Y.
  • Spatially selective binaural hearing aids
    Brayda, Luca; Traverso, Federico; Giuliani, Luca; Diotalevi, Francesco; Repetto, Stefania; Sansalone, Sara; Trucco, Andrea; Sandini, Giulio
  • Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data?Perspectives on Psychological Science 6, 1, 3–5
    Buhrmester, Michael; Kwang, Tracy; Gosling, Samuel D.
  • Return of the devil in the details: Delving deep into convolutional nets
    Chatfield, Ken; Simonyan, Karen; Vedaldi, Andrea; Zisserman, Andrew
  • An integrated artificial vision framework for assisting visually impaired users
    Chessa, Manuela; Noceti, Nicoletta; Odone, Francesca; Solari, Fabio; Sosa-García, Joan; Zini, Luca
  • Finding objects for blind people based on SURF features
    Chincha, Ricardo; Tian, YingLi
  • Total recall: Automatic query expansion with a generative feature model for object retrieval
    Chum, Ondrej; Philbin, James; Sivic, Josef; Isard, Michael; Zisserman, Andrew
  • The MOPED framework: Object recognition and pose estimation for manipulation
    Collet, Alvaro; Martinez, Manuel; Srinivasa, Siddhartha S.
  • Functional assessment of a camera phone-based wayfinding system operated by blind and visually impaired users
    Coughlan, James; Manduchi, Roberto
  • Cell phone-based wayfinding for the visually impaired
    Coughlan, James; Manduchi, Roberto; Shen, Huiying
  • Visual categorization with bags of keypoints
    Csurka, Gabriella; Dance, Christopher; Fan, Lixin; Willamowski, Jutta; Bray, Cédric
  • Wearable obstacle avoidance electronic travel aids for blind: A survey
    Dakopoulos, Dimitrios; Bourbakis, Nikolaos G.
  • Understanding how image quality affects deep neural networks
    Dodge, Samuel F.; Karam, Lina J.
  • Eye of the beholder: Phone-based text-recognition for the visually-impaired
    Dumitraş, Tudor; Lee, Matthew; Quinones, Pablo; Smailagic, Asim; Siewiorek, Dan; Narasimhan, Priya
  • Appliance displays: Accessibility challenges and proposed solutions
    Fusco, Giovanni; Tekin, Ender; Giudice, Nicholas A.; Coughlan, James M.
  • Blind navigation and the role of technology
    Giudice, Nicholas A.; Legge, Gordon E.
  • Robust and effective component-based banknote recognition for the blind
    Hasanuzzaman, Faiz M.; Yang, Xiaodong; Tian, YingLi
  • Computer vision-based object recognition for the visually impaired using visual tags
    Jafri, Rabia; Ali, Syed Abid; Arabnia, Hamid R.
  • Computer vision-based object recognition for the visually impaired in an indoors environment: A survey
    Jafri, Rabia; Ali, Syed Abid; Arabnia, Hamid R.; Fatima, Shameem
  • Negative evidences and co-occurences in image retrieval: The benefit of PCA and whitening
    Jégou, Hervé; Chum, Ondřej
  • Improving bag-of-features for large scale image search
    Jégou, Hervé; Douze, Matthijs; Schmid, Cordelia
  • Product quantization for nearest neighbor search
    Jégou, Hervé; Douze, Matthijs; Schmid, Cordelia
  • Aggregating local image descriptors into compact codes
    Jégou, Hervé; Perronnin, Florent; Douze, Matthijs; Sanchez, Jorge; Perez, Patrick; Schmid, Cordelia
  • Triangulation embedding and democratic aggregation for image search
    Jégou, Hervé; Zisserman, Andrew
  • Caffe: Convolutional architecture for fast feature embedding
    Jia, Yangqing; Shelhamer, Evan; Donahue, Jeff; Karayev, Sergey; Long, Jonathan; Girshick, Ross; Guadarrama, Sergio; Darrell, Trevor
  • ImageNet classification with deep convolutional neural networks
    Krizhevsky, Alex; Sutskever, Ilya; Hinton, Geoffrey E.
  • A novel method for visually impaired using object recognition
    Kumar, Rahul; Meher, Sukadev
  • Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories
    Lazebnik, Svetlana; Schmid, Cordelia; Ponce, Jean
  • Designing search engine user interfaces for the visually impaired
    Leporini, Barbara; Andronico, Patrizia; Buzzi, Marina
  • Object recognition based on the region of interest and optimal bag of words model
    Li, Weisheng; Dong, Peng; Xiao, Bin; Zhou, Lifang
  • Improving Web access for visually impaired users
    Liu, Simon; Ma, Wei; Schalow, Dale; Spruill, Kevin
  • A camera phone based currency reader for the visually impaired
    Liu, Xu
  • (Computer) vision without sight
    Manduchi, Roberto; Coughlan, James
  • A compressive sensing approach to describe indoor scenes for blind people
    Mekhalfi, Mohamed Lamine; Melgani, Farid; Bazi, Yakoub; Alajlan, Naif
  • Toward an assisted indoor scene perception for blind people with image multilabeling strategies
    Mekhalfi, Mohamed Lamine; Melgani, Farid; Bazi, Yakoub; Alajlan, Naif
  • Recovering the sight to blind people in indoor environments with smart technologies
    Mekhalfi, Mohamed Lamine; Melgani, Farid; Zeggada, Abdallah; De Natale, Francesco G. B.; Salem, Mohammed A.-M.; Khamis, Alaa
  • EyeRing: A finger-worn input device for seamless interactions with our surroundings
    Nanayakkara, Suranga; Shilkrot, Roy; Yeo, Kian Peen; Maes, Pattie
  • Indoor and outdoor mobile navigation by using a combination of floor plans and street maps
    Nikander, Jussi; Järvi, Juha; Usman, Muhammad; Virrantaus, Kirsi
  • Scalable recognition with a vocabulary tree
    Nister, David; Stewenius, Henrik
  • A system helping blind people to get character information in their surrounding environment
    Ohnishi, Noboru; Matsumoto, Tetsuya; Kudo, Hiroaki; Takeuchi, Yoshinori
  • Global estimates of visual impairment: 2010
    Pascolini, D.; Mariotti, S. P.
  • Large-scale image retrieval with compressed Fisher vectors
    Perronnin, Florent; Liu, Yan; Sánchez, Jorge; Poirier, Hervé
  • Improving the Fisher kernel for large-scale image classification
    Perronnin, Florent; Sánchez, Jorge; Mensink, Thomas
  • Object retrieval with large vocabularies and fast spatial matching
    Philbin, James; Chum, Ondrej; Isard, Michael; Sivic, Josef; Zisserman, Andrew
  • Lost in quantization: Improving particular object retrieval in large scale image databases
    Philbin, James; Chum, Ondrej; Isard, Michael; Sivic, Josef; Zisserman, Andrew
  • ImageNet large scale visual recognition challenge
    Russakovsky, Olga; Deng, Jia; Su, Hao; Krause, Jonathan; Satheesh, Sanjeev; Ma, Sean; Huang, Zhiheng; Karpathy, Andrej; Khosla, Aditya; Bernstein, Michael; Berg, Alexander C.; Fei-Fei, Li
  • An Assistive Vision System for the Blind That Helps Find Lost Things
    Schauerte, Boris; Martinez, Manel; Constantinescu, Angela; Stiefelhagen, Rainer
  • FingerReader: A wearable device to support text reading on the go
    Shilkrot, Roy; Huber, Jochen; Liu, Connie; Maes, Pattie; Nanayakkara, Suranga Chandima
  • Video Google: A text retrieval approach to object matching in videos
    Sivic, Josef; Zisserman, Andrew
  • Banknote recognition for visually impaired
    Solymár, Zóra; Stubendek, Attila; Radványi, Mihály; Karacs, Kristóf
  • Mean BoF per quadrant—simple and effective way to embed spatial information in bag of features
    Sosa-García, Joan; Odone, Francesca
  • Looktel—a comprehensive platform for computer-aided visual assistance
    Sudol, Jeremi; Dialameh, Orang; Blanchard, Chuck; Dorcey, Tim
  • Going deeper with convolutions
    Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A.
  • Thai banknote recognition using neural network and continues learning by DSP unit
    Takeda, Fumiaki; Sakoobunthu, Lalita; Satou, Hironobu
  • Text-tracking wearable camera system for visually-impaired people
    Tanaka, Makoto; Goto, Hideaki
  • Toward a computer vision-based wayfinding aid for blind persons to access unfamiliar indoor environments
    Tian, YingLi; Yang, Xiaodong; Yi, Chucai; Arditi, Aries
  • 3-D route-planning support for navigation in a complex indoor environment
    van Schaik, Paul; Mayouf, Mohammad; Aranyi, Gabor
  • VLFeat: An Open and Portable Library of Computer Vision Algorithms
    Vedaldi, A.; Fulkerson, B.
  • Toward real-time grocery detection for the visually impaired
    Winlock, Tess; Christiansen, Eric; Belongie, Serge
  • Context-based indoor object detection as an aid to blind persons accessing unfamiliar environments
    Yang, Xiaodong; Tian, Yingli; Yi, Chucai; Arditi, Aries
  • Current and future mobile and wearable device use by people with visual impairments
    Ye, Hanlu; Malu, Meethu; Oh, Uran; Findlater, Leah
  • Portable camera-based assistive text and product label reading from hand-held objects for blind persons
    Yi, Chucai; Tian, Yingli; Arditi, Aries
  • Indoor vision-based auditory assistance for blind people in semi controlled environments
    Zeb, Aurang; Ullah, Sehat; Rabbi, Ihsan
  • Navigation systems for individuals with visual impairment: A survey
    Zhang, J.; Ong, S. K.; Nee, A. Y. C.