Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

MFGAN: Multi-modal Feature-fusion for CT Metal Artifact Reduction Using GANs

MFGAN: Multi-modal Feature-fusion for CT Metal Artifact Reduction Using GANs Due to the existence of metallic implants in certain patients, the Computed Tomography (CT) images from these patients are often corrupted by undesirable metal artifacts, which causes severe problem of metal artifact. Although many methods have been proposed to reduce metal artifact, reduction is still challenging and inadequate. Some reduced results are suffering from symptom variance, second artifact, and poor subjective evaluation. To address these, we propose a novel method based on generative adversarial nets (GANs) to reduce metal artifacts. Specifically, we firstly encode interactive information (text) and imaging CT (image) to yield multi-modal feature-fusion representation, which overcomes representative ability limitation of single-modal CT images. The incorporation of interaction information constrains feature generation, which ensures symptom consistency between corrected and target CT. Then, we design an enhancement network to avoid second artifact and enhance edge as well as suppress noise. Besides, three radiology physicians are invited to evaluate the corrected CT image. Experiments show that our method gains significant improvement over other methods. Objectively, ours achieves an average increment of 7.44% PSNR and 6.12% SSIM on two medical image datasets. Subjectively, ours outperforms others in comparison in term of sharpness, resolution, invariance, and acceptability. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Multimedia Computing Communications and Applications (TOMCCAP) Association for Computing Machinery

MFGAN: Multi-modal Feature-fusion for CT Metal Artifact Reduction Using GANs

Download PDF
17 pages

Loading next page...
 
/lp/association-for-computing-machinery/mfgan-multi-modal-feature-fusion-for-ct-metal-artifact-reduction-using-L0VC3mtdec
Publisher
Association for Computing Machinery
Copyright
Copyright © 2023 Association for Computing Machinery.
ISSN
1551-6857
eISSN
1551-6865
DOI
10.1145/3528172
Publisher site
See Article on Publisher Site

Abstract

Due to the existence of metallic implants in certain patients, the Computed Tomography (CT) images from these patients are often corrupted by undesirable metal artifacts, which causes severe problem of metal artifact. Although many methods have been proposed to reduce metal artifact, reduction is still challenging and inadequate. Some reduced results are suffering from symptom variance, second artifact, and poor subjective evaluation. To address these, we propose a novel method based on generative adversarial nets (GANs) to reduce metal artifacts. Specifically, we firstly encode interactive information (text) and imaging CT (image) to yield multi-modal feature-fusion representation, which overcomes representative ability limitation of single-modal CT images. The incorporation of interaction information constrains feature generation, which ensures symptom consistency between corrected and target CT. Then, we design an enhancement network to avoid second artifact and enhance edge as well as suppress noise. Besides, three radiology physicians are invited to evaluate the corrected CT image. Experiments show that our method gains significant improvement over other methods. Objectively, ours achieves an average increment of 7.44% PSNR and 6.12% SSIM on two medical image datasets. Subjectively, ours outperforms others in comparison in term of sharpness, resolution, invariance, and acceptability.

Journal

ACM Transactions on Multimedia Computing Communications and Applications (TOMCCAP) – Association for Computing Machinery

Published: Jan 23, 2023

Keywords: Feature fusion

References

  • Reduction of CT artifacts caused by metallic implants
    [,
  • Wasserstein GAN
    [,
  • A radial adaptive filter for metal artifact reduction
    [,
  • Multiple-gradient descent algorithm (MGDA) for multiobjective optimization
    [,
  • Deep learning methods to guide CT image reconstruction and reduce metal artifacts
    [,
  • Generative adversarial nets
    [,
  • GANs trained by a two time-scale update rule converge to a Nash equilibrium
    [,
  • Train longer, generalize better: Closing the generalization gap in large batch training of neural networks
    [,
  • Edge-enhanced GAN for remote sensing image superresolution
    [,
  • Generalized multi-dimensional adaptive filtering for conventional and spiral single-slice, multi-slice, and cone-beam CT
    [,
  • Progressive growing of GANs for improved quality, stability, and variation
    [,
  • Photo-realistic single image super-resolution using a generative adversarial network
    [,
  • Suppression of metal artifacts in CT using a reconstruction procedure that combines MAP and projection completion
    [,
  • A closed-form solution to natural image matting
    [,
  • ADN: Artifact disentanglement network for unsupervised metal artifact reduction
    [,
  • DuDoNet: Dual domain network for CT metal artifact reduction
    [,
  • Normalized metal artifact reduction (NMAR) in computed tomography
    [,
  • Metal artifact reduction for polychromatic X-ray CT based on a beam-hardening corrector
    [,
  • A novel forward projection-based metal artifact reduction method for flat-detector computed tomography
    [,
  • Improved techniques for training GANs
    [,
  • An information fidelity criterion for image quality assessment using natural scene statistics
    [,
  • How good is my GAN? In IEEE Conference on Computer Vision and Pattern Recognition
    [,
  • Conditional generative adversarial networks for metal artifact reduction in CT images of the ear
    [,
  • Laplacian operator-based edge detectors
    [,
  • Multiscale structural similarity for image quality assessment
    [,
  • Fusing texture, shape and deep model-learned information at decision level for automated classification of lung nodules on chest CT
    [,
  • BPGAN: Bidirectional CT-to-MRI prediction using multi-generative multi-adversarial nets with spectral normalization and localization
    [,
  • Deep residual learning enabled metal artifact reduction in CT
    [,
  • A reweighted joint spatial-radon domain CT image reconstruction model for metal artifact reduction
    [,
  • FSIM: A feature similarity index for image quality assessment
    [,
  • Convolutional neural network based metal artifact reduction in X-ray computed tomography
    [,