Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

DeepSearch

DeepSearch Content-based image retrieval (CBIR) is one of the most important applications of computer vision. In recent years, there have been many important advances in the development of CBIR systems, especially Convolutional Neural Networks (CNNs) and other deep-learning techniques. On the other hand, current CNN-based CBIR systems suffer from high computational complexity of CNNs. This problem becomes more severe as mobile applications become more and more popular. The current practice is to deploy the entire CBIR systems on the server side while the client side only serves as an image provider. This architecture can increase the computational burden on the server side, which needs to process thousands of requests per second. Moreover, sending images have the potential of personal information leakage. As the need of mobile search expands, concerns about privacy are growing. In this article, we propose a fast image search framework, named DeepSearch, which makes complex image search based on CNNs feasible on mobile phones. To implement the huge computation of CNN models, we present a tensor Block Term Decomposition (BTD) approach as well as a nonlinear response reconstruction method to accelerate the CNNs involving in object detection and feature extraction. The extensive experiments on the ImageNet dataset and Alibaba Large-scale Image Search Challenge dataset show that the proposed accelerating approach BTD can significantly speed up the CNN models and further makes CNN-based image search practical on common smart phones. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) Association for Computing Machinery

DeepSearch

Download PDF
22 pages

Loading next page...
 
/lp/association-for-computing-machinery/deepsearch-CnIT5fymno
Publisher
Association for Computing Machinery
Copyright
Copyright © 2017 ACM
ISSN
1551-6857
eISSN
1551-6865
DOI
10.1145/3152127
Publisher site
See Article on Publisher Site

Abstract

Content-based image retrieval (CBIR) is one of the most important applications of computer vision. In recent years, there have been many important advances in the development of CBIR systems, especially Convolutional Neural Networks (CNNs) and other deep-learning techniques. On the other hand, current CNN-based CBIR systems suffer from high computational complexity of CNNs. This problem becomes more severe as mobile applications become more and more popular. The current practice is to deploy the entire CBIR systems on the server side while the client side only serves as an image provider. This architecture can increase the computational burden on the server side, which needs to process thousands of requests per second. Moreover, sending images have the potential of personal information leakage. As the need of mobile search expands, concerns about privacy are growing. In this article, we propose a fast image search framework, named DeepSearch, which makes complex image search based on CNNs feasible on mobile phones. To implement the huge computation of CNN models, we present a tensor Block Term Decomposition (BTD) approach as well as a nonlinear response reconstruction method to accelerate the CNNs involving in object detection and feature extraction. The extensive experiments on the ImageNet dataset and Alibaba Large-scale Image Search Challenge dataset show that the proposed accelerating approach BTD can significantly speed up the CNN models and further makes CNN-based image search practical on common smart phones.

Journal

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) – Association for Computing Machinery

Published: Dec 13, 2017

Keywords: Convolutional neural networks

References

  • Neural codes for image retrieval
    Babenko, Artem; Slesarev, Anton; Chigorin, Alexandr; Lempitsky, Victor
  • Retrieval by shape similarity with perceptual distance and effective indexing
    Berretti, Stefano; Del Bimbo, Alberto; Pala, Pietro
  • Return of the devil in the details: Delving deep into convolutional nets
    Chatfield, Ken; Simonyan, Karen; Vedaldi, Andrea; Zisserman, Andrew
  • Semi-supervised multi-graph hashing for scalable similarity search
    Cheng, Jian; Leng, Cong; Li, Peng; Wang, Meng; Lu, Hanqing
  • Fast and accurate image matching with cascade hashing for 3d reconstruction
    Cheng, Jian; Leng, Cong; Wu, Jiaxiang; Cui, Hainan; Lu, Hanqing
  • Active learning for image retrieval with Co-SVM
    Cheng, Jian; Wang, Kongqiao
  • Training binary multilayer neural networks for image classification using expectation backpropagation
    Cheng, Zhiyong; Soudry, Daniel; Mao, Zexi; Lan, Zhenzhong
  • Binarynet: Training deep neural networks with weights and activations constrained to+ 1 or-1
    Courbariaux, Matthieu; Bengio, Yoshua
  • Binaryconnect: Training deep neural networks with binary weights during propagations
    Courbariaux, Matthieu; Bengio, Yoshua; David, Jean-Pierre
  • Decompositions of a higher-order tensor in block terms-Part I: Lemmas for partitioned matrices
    De Lathauwer, Lieven
  • Decompositions of a higher-order tensor in block terms-part II: definitions and uniqueness
    De Lathauwer, Lieven
  • On the best rank-1 and rank-(r 1, r 2,
    De Lathauwer, Lieven; De Moor, Bart; Vandewalle, Joos
  • Decompositions of a higher-order tensor in block terms-part III: Alternating least squares algorithms
    De Lathauwer, Lieven; Nion, Dimitri
  • Predicting parameters in deep learning
    Denil, Misha; Shakibi, Babak; Dinh, Laurent; de Freitas, Nando; others,
  • Exploiting linear structure within convolutional networks for efficient evaluation
    Denton, Emily L.; Zaremba, Wojciech; Bruna, Joan; LeCun, Yann; Fergus, Rob
  • Object-aware deep network for commodity image retrieval
    Fang, Zhiwei; Liu, Jing; Wang, Yuhang; Li, Yong; Hang, Song; Tang, Jinhui; Lu, Hanqing
  • Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval
    Gong, Yunchao; Lazebnik, Svetlana; Gordo, Albert; Perronnin, Florent
  • Procrustes Problems
    Gower, John C.; Dijksterhuis, Garmt B.
  • Deep learning with limited numerical precision
    Gupta, Suyog; Agrawal, Ankur; Gopalakrishnan, Kailash; Narayanan, Pritish
  • Learning both weights and connections for efficient neural network
    Han, Song; Pool, Jeff; Tran, John; Dally, William
  • Deep residual learning for image recognition
    He, Kaiming; Zhang, Xiangyu; Ren, Shaoqing; Sun, Jian
  • Deep networks for image retrieval on large-scale databases
    Hörster, Eva; Lienhart, Rainer
  • Mobilenets: Efficient convolutional neural networks for mobile vision applications
    Howard, Andrew G.; Zhu, Menglong; Chen, Bo; Kalenichenko, Dmitry; Wang, Weijun; Weyand, Tobias; Andreetto, Marco; Adam, Hartwig
  • Binarized neural networks
    Hubara, Itay; Courbariaux, Matthieu; Soudry, Daniel; El-Yaniv, Ran; Bengio, Yoshua
  • Fixed-point feedforward deep neural network design using weights+ 1, 0, and- 1
    Hwang, Kyuyeon; Sung, Wonyong
  • Approximate nearest neighbors: Towards removing the curse of dimensionality
    Indyk, Piotr; Motwani, Rajeev
  • Speeding up convolutional neural networks with low rank expansions
    Jaderberg, Max; Vedaldi, Andrea; Zisserman, Andrew
  • Image retrieval using color and shape
    Jain, Anil K.; Vailaya, Aditya
  • Caffe: Convolutional architecture for fast feature embedding
    Jia, Yangqing; Shelhamer, Evan; Donahue, Jeff; Karayev, Sergey; Long, Jonathan; Girshick, Ross; Guadarrama, Sergio; Darrell, Trevor
  • Bitwise neural networks
    Kim, Minje; Smaragdis, Paris
  • Compression of deep convolutional neural networks for fast and low power mobile applications
    Kim, Yong-Deok; Park, Eunhyeok; Yoo, Sungjoo; Choi, Taelim; Yang, Lu; Shin, Dongjun
  • Tensor decompositions and applications
    Kolda, Tamara G.; Bader, Brett W.
  • Using very deep autoencoders for content-based image retrieval
    Krizhevsky, Alex; Hinton, Geoffrey E.
  • Imagenet classification with deep convolutional neural networks
    Krizhevsky, Alex; Sutskever, Ilya; Hinton, Geoffrey E.
  • Simultaneous feature learning and hash coding with deep neural networks
    Lai, Hanjiang; Pan, Yan; Liu, Ye; Yan, Shuicheng
  • Speeding-up convolutional neural networks using fine-tuned cp-decomposition
    Lebedev, Vadim; Ganin, Yaroslav; Rakhuba, Maksim; Oseledets, Ivan; Lempitsky, Victor
  • Fast convnets using group-wise brain damage
    Lebedev, Vadim; Lempitsky, Victor
  • Deep learning
    LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey
  • Fixed point quantization of deep convolutional networks
    Lin, Darryl D.; Talathi, Sachin S.; Annapureddy, V. Sreekanth
  • Sparse convolutional neural networks
    Liu, Baoyuan; Wang, Min; Foroosh, Hassan; Tappen, Marshall; Pensky, Marianna
  • Fully convolutional networks for semantic segmentation
    Long, Jonathan; Shelhamer, Evan; Darrell, Trevor
  • Texture features for browsing and retrieval of image data
    Manjunath, Bangalore S.; Ma, Wei-Ying
  • Fast training of convolutional networks through FFTs
    Mathieu, Michael; Henaff, Mikael; LeCun, Yann
  • Tensorizing neural networks
    Novikov, Alexander; Podoprikhin, Dmitrii; Osokin, Anton; Vetrov, Dmitry P.
  • Going deeper with embedded fpga platform for convolutional neural network
    Qiu, Jiantao; Wang, Jie; Yao, Song; Guo, Kaiyuan; Li, Boxun; Zhou, Erjin; Yu, Jincheng; Tang, Tianqi; Xu, Ningyi; Song, Sen; others,
  • XNOR-Net: ImageNet classification using binary convolutional neural networks
    Rastegari, Mohammad; Ordonez, Vicente; Redmon, Joseph; Farhadi, Ali
  • Faster R-CNN: Towards real-time object detection with region proposal networks
    Ren, Shaoqing; He, Kaiming; Girshick, Ross; Sun, Jian
  • Fitnets: Hints for thin deep nets
    Romero, Adriana; Ballas, Nicolas; Kahou, Samira Ebrahimi; Chassang, Antoine; Gatta, Carlo; Bengio, Yoshua
  • Imagenet large scale visual recognition challenge
    Russakovsky, Olga; Deng, Jia; Su, Hao; Krause, Jonathan; Satheesh, Sanjeev; Ma, Sean; Huang, Zhiheng; Karpathy, Andrej; Khosla, Aditya; Bernstein, Michael; others,
  • Very deep convolutional networks for large-scale image recognition
    Simonyan, Karen; Zisserman, Andrew
  • Convolutional neural networks with low-rank regularization
    Tai, Cheng; Xiao, Tong; Wang, Xiaogang; others,
  • Generalized constrained redundancy analysis
    Takane, Yoshio; Jung, Sunho
  • Deep learning for content-based image retrieval: A comprehensive study
    Wan, Ji; Wang, Dayong; Hoi, Steven Chu Hong; Wu, Pengcheng; Zhu, Jianke; Zhang, Yongdong; Li, Jintao
  • Accelerating convolutional neural networks for mobile applications
    Wang, Peisong; Cheng, Jian
  • Fixed-point factorized networks
    Wang, Peisong; Cheng, Jian
  • Spectral hashing
    Weiss, Yair; Torralba, Antonio; Fergus, Rob
  • Quantized convolutional neural networks for mobile devices
    Wu, Jiaxiang; Leng, Cong; Wang, Yuhang; Hu, Qinghao; Cheng, Jian
  • Exploiting local features from deep networks for image retrieval
    Ng, Joe Yue-Hei; Yang, Fan; Davis, Larry S.
  • ShuffleNet: An extremely efficient convolutional neural network for mobile devices
    Zhang, Xiangyu; Zhou, Xinyu; Lin, Mengxiao; Sun, Jian
  • Accelerating very deep convolutional networks for classification and detection
    Zhang, Xiangyu; Zou, Jianhua; He, Kaiming; Sun, Jian
  • Incremental network quantization: Towards lossless cnns with low-precision weights
    Zhou, Aojun; Yao, Anbang; Guo, Yiwen; Xu, Lin; Chen, Yurong