Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

CWI: A multimodal deep learning approach for named entity recognition from social media using character, word and image features

CWI: A multimodal deep learning approach for named entity recognition from social media using... Named entity recognition (NER) from social media posts is a challenging task. User-generated content that forms the nature of social media is noisy and contains grammatical and linguistic errors. This noisy content makes tasks such as NER much harder. We propose two novel deep learning approaches utilizing multimodal deep learning and transformers. Both of our approaches use image features from short social media posts to provide better results on the NER task. On the first approach, we extract image features using InceptionV3 and use fusion to combine textual and image features. This approach presents more reliable name entity recognition when the images related to the entities are provided by the user. On the second approach, we use image features combined with text and feed it into a BERT-like transformer. The experimental results using precision, recall, and F1 score metrics show the superiority of our work compared to other state-of-the-art NER solutions. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Neural Computing and Applications Springer Journals

CWI: A multimodal deep learning approach for named entity recognition from social media using character, word and image features

Loading next page...
 
/lp/springer-journals/cwi-a-multimodal-deep-learning-approach-for-named-entity-recognition-Go0goEvbKu

References (87)

Publisher
Springer Journals
Copyright
Copyright © The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature 2021
ISSN
0941-0643
eISSN
1433-3058
DOI
10.1007/s00521-021-06488-4
Publisher site
See Article on Publisher Site

Abstract

Named entity recognition (NER) from social media posts is a challenging task. User-generated content that forms the nature of social media is noisy and contains grammatical and linguistic errors. This noisy content makes tasks such as NER much harder. We propose two novel deep learning approaches utilizing multimodal deep learning and transformers. Both of our approaches use image features from short social media posts to provide better results on the NER task. On the first approach, we extract image features using InceptionV3 and use fusion to combine textual and image features. This approach presents more reliable name entity recognition when the images related to the entities are provided by the user. On the second approach, we use image features combined with text and feed it into a BERT-like transformer. The experimental results using precision, recall, and F1 score metrics show the superiority of our work compared to other state-of-the-art NER solutions.

Journal

Neural Computing and ApplicationsSpringer Journals

Published: Feb 1, 2022

Keywords: Deep learning; Named entity recognition; Multimodal learning; Transformer

There are no references for this article.