Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Learning Pixel Affinity Pyramid for Arbitrary-Shaped Text Detection

Learning Pixel Affinity Pyramid for Arbitrary-Shaped Text Detection Arbitrary-shaped text detection in natural images is a challenging task due to the complexity of the background and the diversity of text properties. The difficulty lies in two aspects: accurate separation of adjacent texts and sufficient text feature representation. To handle these problems, we consider text detection as instance segmentation and propose a novel text detection framework, which jointly learns semantic segmentation and a pixel affinity pyramid in a unified fully convolutional network. Specifically, the pixel affinity pyramid is proposed to encode multi-scale instance affiliation relationships of pixels, which is not only robust to varying shapes of text but also provides an accurate boundary description for separating closely located texts. In the inference phase, a simple but effective post-processing is presented to reconstruct text instances from the semantic segmentation results under the guidance of the learned pixel affinity pyramid, achieving good accuracy and efficiency. Furthermore, to enhance the representation of text features in the neural network, two modules — the Region Enhancement Module (REM) and Attentional Fusion Module (AFM) — are proposed. The REM models the semantic correlations of regional features to enhance the features from the text area, which effectively suppresses false-positive detection. The AFM adaptively fuses multi-scale textual information through an attention mechanism to obtain abundant text semantic features, which benefits multi-sized text detection. Extensive ablation experiments are conducted demonstrating the effectiveness of the REM and AFM. Evaluation results on standard benchmarks, including Total-Text, ICDAR2015, SCUT-CTW1500, and MSRA-TD500, show that our method surpasses most existing text detectors and achieves state-of-the-art performance, denoting its superior capability in detecting arbitrary-shaped texts. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Multimedia Computing Communications and Applications (TOMCCAP) Association for Computing Machinery

Learning Pixel Affinity Pyramid for Arbitrary-Shaped Text Detection

Download PDF
24 pages

Loading next page...
 
/lp/association-for-computing-machinery/learning-pixel-affinity-pyramid-for-arbitrary-shaped-text-detection-IuxrKMjwwr
Publisher
Association for Computing Machinery
Copyright
Copyright © 2023 Association for Computing Machinery.
ISSN
1551-6857
eISSN
1551-6865
DOI
10.1145/3524617
Publisher site
See Article on Publisher Site

Abstract

Arbitrary-shaped text detection in natural images is a challenging task due to the complexity of the background and the diversity of text properties. The difficulty lies in two aspects: accurate separation of adjacent texts and sufficient text feature representation. To handle these problems, we consider text detection as instance segmentation and propose a novel text detection framework, which jointly learns semantic segmentation and a pixel affinity pyramid in a unified fully convolutional network. Specifically, the pixel affinity pyramid is proposed to encode multi-scale instance affiliation relationships of pixels, which is not only robust to varying shapes of text but also provides an accurate boundary description for separating closely located texts. In the inference phase, a simple but effective post-processing is presented to reconstruct text instances from the semantic segmentation results under the guidance of the learned pixel affinity pyramid, achieving good accuracy and efficiency. Furthermore, to enhance the representation of text features in the neural network, two modules — the Region Enhancement Module (REM) and Attentional Fusion Module (AFM) — are proposed. The REM models the semantic correlations of regional features to enhance the features from the text area, which effectively suppresses false-positive detection. The AFM adaptively fuses multi-scale textual information through an attention mechanism to obtain abundant text semantic features, which benefits multi-sized text detection. Extensive ablation experiments are conducted demonstrating the effectiveness of the REM and AFM. Evaluation results on standard benchmarks, including Total-Text, ICDAR2015, SCUT-CTW1500, and MSRA-TD500, show that our method surpasses most existing text detectors and achieves state-of-the-art performance, denoting its superior capability in detecting arbitrary-shaped texts.

Journal

ACM Transactions on Multimedia Computing Communications and Applications (TOMCCAP) – Association for Computing Machinery

Published: Feb 3, 2023

Keywords: Scene text detection

References

  • Graph-based global reasoning networks
    [,
  • Total-text: A comprehensive dataset for scene text detection and recognition
    [,
  • Pixellink: Detecting scene text via instance segmentation
    [,
  • Detecting text in natural scenes with stroke width transform
    [,
  • Read like humans: Autonomous, bidirectional and iterative language modeling for scene text recognition
    [,
  • TextDragon: An end-to-end framework for arbitrary shaped text spotting
    [,
  • Dual attention network for scene segmentation
    [,
  • SSAP: Single-shot instance segmentation with affinity pyramid
    [,
  • Synthetic data for text localisation in natural images
    [,
  • Mask R-CNN
    [,
  • Deep residual learning for image recognition
    [,
  • Very deep convolutional networks for large-scale image recognition
    [,
  • Single shot text detector with regional attention
    [,
  • Deep direct regression for multi-oriented scene text detection
    [,
  • Squeeze-and-excitation networks
    [,
  • Text localization in natural images using stroke feature transform and text covariance descriptors
    [,
  • Ccnet: Criss-cross attention for semantic segmentation
    [,
  • Mask R-CNN with pyramid attention network for scene text detection
    [,
  • Spatial transformer networks
    [,
  • Detection and recognition of text embedded in online images via neural context models
    [,
  • ICDAR 2015 competition on robust reading
    [,
  • Semi-supervised classification with graph convolutional networks
    [,
  • Spatial preserved graph convolution networks for person re-identification
    [,
  • Textboxes++: A single-shot oriented scene text detector
    [,
  • Textboxes: A fast text detector with a single deep neural network
    [,
  • Rotation-sensitive regression for oriented scene text detection
    [,
  • Affinity derivation and graph merge for instance segmentation
    [,
  • Learning Markov clustering networks for scene text detection
    [,
  • Towards robust curve text detection with conditional spatial expansion
    [,
  • AB-LSTM: Attention-based bidirectional LSTM model for scene text detection
    [,
  • Textsnake: A flexible representation for detecting text of arbitrary shapes
    [,
  • Mask textspotter: An end-to-end trainable neural network for spotting text with arbitrary shapes
    [,
  • Arbitrary-oriented scene text detection via rotation proposals
    [,
  • Robust wide-baseline stereo from maximally stable extremal regions
    [,
  • ImageSense: Towards contextual image advertising
    [,
  • V-Net: Fully convolutional neural networks for volumetric medical image segmentation
    [,
  • Real-time scene text localization and recognition
    [,
  • Faster R-CNN: Towards real-time object detection with region proposal networks
    [,
  • Detecting oriented text in natural images by linking segments
    [,
  • Training region-based object detectors with online hard example mining
    [,
  • Learning shape-aware embedding for scene text detection
    [,
  • Geometry-aware scene text detection with instance transformation network
    [,
  • Word spotting in the wild
    [,
  • A single-shot arbitrarily-shaped text detector based on context attended multi-task learning
    [,
  • Shape robust text detection with progressive scale expansion network
    [,
  • Efficient and accurate arbitrary-shaped text detection with pixel aggregation network
    [,
  • Non-local neural networks
    [,
  • Arbitrary shape scene text detection with adaptive text region representation
    [,
  • From two to one: A new scene text recognizer with visual language modeling network
    [,
  • R-Net: A relationship network for efficient and accurate scene text detection
    [,
  • Scene text detection with supervised pyramid context network
    [,
  • Convolutional attention networks for scene text recognition
    [,
  • Structured attention guided convolutional neural fields for monocular depth estimation
    [,
  • TextField: Learning a deep direction field for irregular scene text detection
    [,
  • MSR: Multi-scale shape regression for scene text detection
    [,
  • A unified framework for multioriented text detection and recognition
    [,
  • Detecting texts of arbitrary orientations in natural images
    [,
  • Scene text detection via holistic, multi-channel prediction
    [,
  • TextFuseNet: Scene text detection with richer fused features
    [,
  • Detecting curve text in the wild: New dataset and new solution
    [,
  • Look more than once: An accurate detector for text of arbitrary shapes
    [,
  • Dynamic graph message passing networks
    [,
  • LatentGNN: Learning efficient non-local relations for visual recognition
    [,
  • Scale-transferrable object detection
    [,
  • EAST: An efficient and accurate scene text detector
    [,
  • Oriented response networks
    [,