Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Research on automatic cleaning algorithm of multi-dimensional network redundant data based on big data

Research on automatic cleaning algorithm of multi-dimensional network redundant data based on big... In order to realize the research on network redundant data cleaning based on big data, this paper designs a set of redundant data cleaning framework according to the data processing flow before data analysis. According to the spatial correlation of redundant data, a method of data cleaning is designed. In the data cleaning method, appropriate cleaning algorithms are designed for abnormal data and missing data respectively, in which mathematical probability design is applied to abnormal data to delete the data with obvious deviation from the normal data value. The spatial model and algorithm are designed by applying spatial correlation to the missing data to fill the missing data value after the redundant data is cleaned by other steps in the method. The accuracy of the model is compared with that of the common data prediction algorithm, and the accuracy between the algorithm and the redundant data set is verified. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Evolutionary Intelligence Springer Journals

Research on automatic cleaning algorithm of multi-dimensional network redundant data based on big data

Evolutionary Intelligence , Volume 15 (4): 9 – Dec 1, 2022

Loading next page...
 
/lp/springer-journals/research-on-automatic-cleaning-algorithm-of-multi-dimensional-network-r9Qx0OXi4T

References (44)

Publisher
Springer Journals
Copyright
Copyright © The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2021
ISSN
1864-5909
eISSN
1864-5917
DOI
10.1007/s12065-021-00620-y
Publisher site
See Article on Publisher Site

Abstract

In order to realize the research on network redundant data cleaning based on big data, this paper designs a set of redundant data cleaning framework according to the data processing flow before data analysis. According to the spatial correlation of redundant data, a method of data cleaning is designed. In the data cleaning method, appropriate cleaning algorithms are designed for abnormal data and missing data respectively, in which mathematical probability design is applied to abnormal data to delete the data with obvious deviation from the normal data value. The spatial model and algorithm are designed by applying spatial correlation to the missing data to fill the missing data value after the redundant data is cleaned by other steps in the method. The accuracy of the model is compared with that of the common data prediction algorithm, and the accuracy between the algorithm and the redundant data set is verified.

Journal

Evolutionary IntelligenceSpringer Journals

Published: Dec 1, 2022

Keywords: Network redundant data; Big data; Multi-dimensional; Cleaning algorithm

There are no references for this article.