Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Deep Bidirectional Cross-Triplet Embedding for Online Clothing Shopping

Deep Bidirectional Cross-Triplet Embedding for Online Clothing Shopping In this article, we address the cross-domain (i.e., street and shop) clothing retrieval problem and investigate its real-world applications for online clothing shopping. It is a challenging problem due to the large discrepancy between street and shop domain images. We focus on learning an effective feature-embedding model to generate robust and discriminative feature representation across domains. Existing triplet embedding models achieve promising results by finding an embedding metric in which the distance between negative pairs is larger than the distance between positive pairs plus a margin. However, existing methods do not address the challenges in the cross-domain clothing retrieval scenario sufficiently. First, the intradomain and cross-domain data relationships need to be considered simultaneously. Second, the number of matched and nonmatched cross-domain pairs are unbalanced. To address these challenges, we propose a deep cross-triplet embedding algorithm together with a cross-triplet sampling strategy. The extensive experimental evaluations demonstrate the effectiveness of the proposed algorithms well. Furthermore, we investigate two novel online shopping applications, clothing trying on and accessories recommendation, based on a unified cross-domain clothing retrieval framework. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) Association for Computing Machinery

Deep Bidirectional Cross-Triplet Embedding for Online Clothing Shopping

Download PDF
22 pages

Loading next page...
 
/lp/association-for-computing-machinery/deep-bidirectional-cross-triplet-embedding-for-online-clothing-MLNfYnqoVA
Publisher
Association for Computing Machinery
Copyright
Copyright © 2018 ACM
ISSN
1551-6857
eISSN
1551-6865
DOI
10.1145/3152114
Publisher site
See Article on Publisher Site

Abstract

In this article, we address the cross-domain (i.e., street and shop) clothing retrieval problem and investigate its real-world applications for online clothing shopping. It is a challenging problem due to the large discrepancy between street and shop domain images. We focus on learning an effective feature-embedding model to generate robust and discriminative feature representation across domains. Existing triplet embedding models achieve promising results by finding an embedding metric in which the distance between negative pairs is larger than the distance between positive pairs plus a margin. However, existing methods do not address the challenges in the cross-domain clothing retrieval scenario sufficiently. First, the intradomain and cross-domain data relationships need to be considered simultaneously. Second, the number of matched and nonmatched cross-domain pairs are unbalanced. To address these challenges, we propose a deep cross-triplet embedding algorithm together with a cross-triplet sampling strategy. The extensive experimental evaluations demonstrate the effectiveness of the proposed algorithms well. Furthermore, we investigate two novel online shopping applications, clothing trying on and accessories recommendation, based on a unified cross-domain clothing retrieval framework.

Journal

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) – Association for Computing Machinery

Published: Jan 4, 2018

Keywords: Clothing retrieval

References

  • Learning visual similarity for product design with convolutional neural networks
    Bell, Sean; Bala, Kavita
  • Signature verification using a Siamese time delay neural network
    Bromley, Jane; Bentz, James W.; Bottou, Léon; Guyon, Isabelle; LeCun, Yann; Moore, Cliff; Säckinger, Eduard; Shah, Roopak
  • Deep domain adaptation for describing people based on fine-grained clothing attributes
    Chen, Qiang; Huang, Junshi; Feris, Rogerio; Brown, Lisa M.; Dong, Jian; Yan, Shuicheng
  • Chromirror: A real-time interactive mirror for chromatic and color-harmonic dressing
    Cheng, Chueh-Min; Chung, Meng-Fang; Yu, Ming-Yang; Ouhyoung, Ming; Chu, Hao-Hua; Chuang, Yung-Yu
  • Learning a similarity metric discriminatively, with application to face verification
    Chopra, Sumit; Hadsell, Raia; LeCun, Yann
  • Color harmonization
    Cohen-Or, Daniel; Sorkine, Olga; Gal, Ran; Leyvand, Tommer; Xu, Ying-Qing
  • Style finder: Fine-grained clothing style detection and retrieval
    Di, Wei; Wah, Catherine; Bhardwaj, Anurag; Piramuthu, Robinson; Sundaresan, Neel
  • Low-rank common subspace for multi-view learning
    Ding, Zhengming; Fu, Yun
  • Missing modality transfer learning via latent low-rank constraint
    Ding, Zhengming; Shao, Ming; Fu, Yun
  • Fashion style generator
    Fu, Yun; Jiang, Shuhui
  • Efficient clothing retrieval with semantic-preserving visual phrases
    Fu, Jianlong; Wang, Jinqiao; Li, Zechao; Xu, Min; Lu, Hanqing
  • Hipster wars: Discovering elements of fashion styles
    Kiapour, M. Hadi; Yamaguchi, Kota; Berg, Alexander C.; Berg, Tamara L.
  • Where to buy it: Matching street clothing photos in online shops
    Kiapour, M. Hadi; Han, Xufeng; Lazebnik, Svetlana; Berg, Alexander C.; Berg, Tamara L.
  • Dimensionality reduction by learning an invariant mapping
    Hadsell, Raia; Chopra, Sumit; LeCun, Yann
  • Cross-domain image retrieval with a dual attribute-aware ranking network
    Huang, Junshi; Feris, Rogerio S.; Chen, Qiang; Yan, Shuicheng
  • Fashion coordinates recommender system using photographs from fashion magazines
    Iwata, Tomoharu; Wanatabe, Shinji; Sawada, Hiroshi
  • Large scale visual recommendations from street fashion images
    Jagadeesh, Vignesh; Piramuthu, Robinson; Bhardwaj, Anurag; Di, Wei; Sundaresan, Neel
  • Caffe: Convolutional architecture for fast feature embedding
    Jia, Yangqing; Shelhamer, Evan; Donahue, Jeff; Karayev, Sergey; Long, Jonathan; Girshick, Ross; Guadarrama, Sergio; Darrell, Trevor
  • Mobile multimedia travelogue generation by exploring geo-locations and image tags
    Jiang, Shuhui; Qian, Xueming; Lan, Ke; Zhang, Lei; Mei, Tao
  • Personalized travel sequence recommendation on multi-source big social media
    Jiang, Shuhui; Qian, Xueming; Mei, Tao; Fu, Yun
  • Author topic model-based collaborative filtering for personalized POI recommendations
    Jiang, Shuhui; Qian, Xueming; Shen, Jialie; Fu, Yun; Mei, Tao
  • Generating representative images for landmark by discovering high frequency shooting locations from community-contributed photos
    Jiang, Shuhui; Qian, Xueming; Xue, Yao; Li, Fan; Hou, Xingsong
  • Consensus style centralizing auto-encoder for weak style classification
    Jiang, Shuhui; Shao, Ming; Jia, Chengcheng; Fu, Yun
  • Deep bi-directional cross-triplet embedding for cross-domain clothing retrieval
    Jiang, Shuhui; Wu, Yue; Fu, Yun
  • Getting the look: Clothing recognition and segmentation for automatic product suggestions in everyday photos
    Kalantidis, Yannis; Kennedy, Lyndon; Li, Li-Jia
  • Generating diverse and representative image search results for landmarks
    Kennedy, Lyndon S.; Naaman, Mor
  • Imagenet classification with deep convolutional neural networks
    Krizhevsky, Alex; Sutskever, Ilya; Hinton, Geoffrey E.
  • Sparse deep stacking network for image classification
    Li, J.; Chang, H. Y.; Yang, J.
  • Sparseness analysis in the pretraining of deep neural networks
    Li, Jun; Zhang, Tong; Luo, Wei; Yang, Jian; Yuan, Xiaotong; Zhang, Jian
  • Hi, magic closet, tell me what to wear!
    Liu, Si; Feng, Jiashi; Song, Zheng; Zhang, Tianzhu; Lu, Hanqing; Xu, Changsheng; Yan, Shuicheng
  • Matching-CNN meets KNN: Quasi-parametric human parsing
    Liu, Si; Liang, Xiaodan; Liu, Luoqi; Shen, Xiaohui; Yang, Jianchao; Xu, Changsheng; Lin, Liang; Cao, Xiaochun; Yan, Shuicheng
  • Street-to-shop: Cross-scenario clothing retrieval via parts alignment and auxiliary set
    Liu, Si; Song, Zheng; Liu, Guangcan; Xu, Changsheng; Lu, Hanqing; Yan, Shuicheng
  • Deepfashion: Powering robust clothes recognition and retrieval with rich annotations
    Liu, Ziwei; Luo, Ping; Qiu, Shi; Wang, Xiaogang; Tang, Xiaoou
  • Convolutional sparse autoencoders for image classification
    Luo, Wei; Li, Jun; Yang, Jian; Xu, Wei; Zhang, Jian
  • Image-based recommendations on styles and substitutes
    McAuley, Julian; Targett, Christopher; Shi, Qinfeng; van den Hengel, Anton
  • Facenet: A unified embedding for face recognition and clustering
    Schroff, Florian; Kalenichenko, Dmitry; Philbin, James
  • What am I gonna wear?: Scenario-oriented recommendation
    Shen, Edward; Lieberman, Henry; Lam, Francis
  • Neuroaesthetics in fashion: Modeling the perception of fashionability
    Simo-Serra, Edgar; Fidler, Sanja; Moreno-Noguer, Francesc; Urtasun, Raquel
  • Deep metric learning via lifted structured feature embedding
    Song, Hyun Oh; Xiang, Yu; Jegelka, Stefanie; Savarese, Silvio
  • From ensemble clustering to multi-view clustering
    Tao, Zhiqiang; Liu, Hongfu; Li, Sheng; Ding, Zhengming; Fu, Yun
  • Learning visual clothing style with heterogeneous dyadic co-occurrences
    Veit, Andreas; Kovacs, Balazs; Bell, Sean; McAuley, Julian; Bala, Kavita; Belongie, Serge
  • Runway to realway: Visual analysis of fashion
    Vittayakorn, Sirion; Yamaguchi, Kota; Berg, Alexander C.; Berg, Tamara L.
  • Learning fine-grained image similarity with deep ranking
    Wang, Jiang; Song, Yang; Leung, Thomas; Rosenberg, Chuck; Wang, Jingbin; Philbin, James; Chen, Bo; Wu, Ying
  • Matching user photos to online products with robust deep features
    Wang, Xi; Sun, Zhenfeng; Zhang, Wenqiang; Zhou, Yu; Jiang, Yu-Gang
  • Distance metric learning for large margin nearest neighbor classification
    Weinberger, Kilian Q.; Saul, Lawrence K.
  • Paper doll parsing: Retrieving similar styles to parse clothing items
    Yamaguchi, Kota; Kiapour, M.; Berg, Tamara
  • Parsing clothing in fashion photographs
    Yamaguchi, Kota; Kiapour, M. Hadi; Ortiz, Luis E.; Berg, Tamara L.
  • Multi-view clustering via deep matrix factorization
    Zhao, Handong; Ding, Zhengming; Fu, Yun