Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Cross-Modality Transfer Learning for Image-Text Information Management

Cross-Modality Transfer Learning for Image-Text Information Management In the past decades, information from all kinds of data has been on a rapid increase. With state-of-the-art performance, machine learning algorithms have been beneficial for information management. However, insufficient supervised training data is still an adversity in many real-world applications. Therefore, transfer learning (TF) was proposed to address this issue. This article studies a not well investigated but important TL problem termed cross-modality transfer learning (CMTL). This topic is closely related to distant domain transfer learning (DDTL) and negative transfer. In general, conventional TL disciplines assume that the source domain and the target domain are in the same modality. DDTL aims to make efficient transfers even when the domains or the tasks are entirely different. As an extension of DDTL, CMTL aims to make efficient transfers between two different data modalities, such as from image to text. As the main focus of this study, we aim to improve the performance of image classification by transferring knowledge from text data. Previously, a few CMTL algorithms were proposed to deal with image classification problems. However, most existing algorithms are very task specific, and they are unstable on convergence. There are four main contributions in this study. First, we propose a novel heterogeneous CMTL algorithm, which requires only a tiny set of unlabeled target data and labeled source data with associate text tags. Second, we introduce a latent semantic information extraction method to connect the information learned from the image data and the text data. Third, the proposed method can effectively handle the information transfer across different modalities (text-image). Fourth, we examined our algorithm on a public dataset, Office-31. It has achieved up to 5% higher classification accuracy than “non-transfer” algorithms and up to 9% higher than existing CMTL algorithms. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Management Information Systems (TMIS) Association for Computing Machinery

Cross-Modality Transfer Learning for Image-Text Information Management

Download PDF
14 pages

Loading next page...
 
/lp/association-for-computing-machinery/cross-modality-transfer-learning-for-image-text-information-management-5OdyMTAKQn

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
Association for Computing Machinery
Copyright
Copyright © 2021 Association for Computing Machinery.
ISSN
2158-656X
eISSN
2158-6578
DOI
10.1145/3464324
Publisher site
See Article on Publisher Site

Abstract

In the past decades, information from all kinds of data has been on a rapid increase. With state-of-the-art performance, machine learning algorithms have been beneficial for information management. However, insufficient supervised training data is still an adversity in many real-world applications. Therefore, transfer learning (TF) was proposed to address this issue. This article studies a not well investigated but important TL problem termed cross-modality transfer learning (CMTL). This topic is closely related to distant domain transfer learning (DDTL) and negative transfer. In general, conventional TL disciplines assume that the source domain and the target domain are in the same modality. DDTL aims to make efficient transfers even when the domains or the tasks are entirely different. As an extension of DDTL, CMTL aims to make efficient transfers between two different data modalities, such as from image to text. As the main focus of this study, we aim to improve the performance of image classification by transferring knowledge from text data. Previously, a few CMTL algorithms were proposed to deal with image classification problems. However, most existing algorithms are very task specific, and they are unstable on convergence. There are four main contributions in this study. First, we propose a novel heterogeneous CMTL algorithm, which requires only a tiny set of unlabeled target data and labeled source data with associate text tags. Second, we introduce a latent semantic information extraction method to connect the information learned from the image data and the text data. Third, the proposed method can effectively handle the information transfer across different modalities (text-image). Fourth, we examined our algorithm on a public dataset, Office-31. It has achieved up to 5% higher classification accuracy than “non-transfer” algorithms and up to 9% higher than existing CMTL algorithms.

Journal

ACM Transactions on Management Information Systems (TMIS) – Association for Computing Machinery

Published: Oct 5, 2021

Keywords: Machine learning

References

  • Modified machine learning technique for curve fitting on regression models for COVID-19 projections
    [,
  • The Netflix Prize
    [,
  • P2L: Predicting transfer learning for images and semantic relations
    [,
  • Integrating structured biological data by kernel maximum mean discrepancy
    [,
  • Not-so-supervised: A survey of semi-supervised, multi-instance, and transfer learning in medical image analysis
    [,
  • Translated learning: Transfer learning across different feature spaces
    [,
  • Big Data Analytics for Cyber-Physical Systems: Machine Learning for the Internet of Things
    [,
  • Exploring decomposition for solving pattern mining problems
    [,
  • On handling negative transfer and imbalanced distributions in multiple source transfer learning
    [,
  • Big data analytics for event detection in the IoT-multicriteria approach
    [,
  • Deep residual learning for image recognition
    [,
  • Universal language model fine-tuning for text classification
    [,
  • Industrial Internet of Things: Cybermanufacturing Systems
    [,
  • ImageNet classification with deep convolutional neural networks
    [,
  • ASRNN: A recurrent neural network with an attention model for sequence labeling
    [,
  • Enhanced sequence labeling based on latent variable conditional random fields
    [,
  • A Bi-LSTM mention hypergraph model with encoding schema for mention extraction
    [,
  • Big data orchestration as a service network
    [,
  • Exploring data validity in transportation systems for smart cities
    [,
  • Deep transfer learning with joint adaptation networks
    [,
  • Next-generation big data analytics: State of the art, challenges, and future research topics
    [,
  • Feature-based distant domain transfer learning
    [,
  • Distant domain transfer learning for medical imaging
    [,
  • A decade survey of transfer learning (2010–2020)
    [,
  • Transfer learning based data-efficient machine learning enabled classification
    [,
  • A survey on transfer learning
    [,
  • Transfer learning to predict missing ratings via heterogeneous user feedbacks
    [,
  • Transfer learning in collaborative filtering for sparsity reduction
    [,
  • A hyper-box approach using relational databases for large scale machine learning
    [,
  • Multi-adversarial domain adaptation
    [,
  • Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning
    [,
  • Security and Privacy in Cyber-Physical Systems
    [,
  • Cyber-Physical Systems: Foundations, Principles and Applications
    [,
  • Smart Cities: Foundations, Principles, and Applications
    [,
  • Deep CORAL: Correlation alignment for deep domain adaptation
    [,
  • Internet of Things and big data analytics for smart and connected communities
    [,
  • Transitive transfer learning
    [,
  • Distant domain transfer learning
    [,
  • MultiPLIER: A transfer learning framework for transcriptomics reveals systemic features of rare disease
    [,
  • A deep convolutional auto-encoder with pooling-unpooling layers in Caffe
    [,
  • Transfer learning with dynamic distribution adaptation
    [,
  • Transfer learning from deep features for remote sensing and poverty mapping
    [,
  • How transferable are features in deep neural networks? In Advances in Neural Information Processing Systems
    [,
  • Deep model based transfer and multi-task learning for biological image analysis
    [,
  • Stretching domain adaptation: How far is too far? arXiv:1712
    [,
  • Heterogeneous transfer learning for image classification
    [,