Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Evaluating the Predictive Performance of Positive- Unlabelled Classifiers

Evaluating the Predictive Performance of Positive- Unlabelled Classifiers Positive-Unlabelled (PU) learning is a growing area of machine learning that aims to learn classifiers from data consisting of labelled positive and unlabelled instances. Whilst much work has been done proposing methods for PU learning, little has been written on the subject of evaluating these methods. Many popular standard classification metrics cannot be precisely calculated due to the absence of fully labelled data, so alternative approaches must be taken. This short commentary paper critically reviews the main PU learning evaluation approaches and the choice of predictive accuracy measures in 51 articles proposing PU classifiers and provides practical recommendations for improvements in this area. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM SIGKDD Explorations Newsletter Association for Computing Machinery

Evaluating the Predictive Performance of Positive- Unlabelled Classifiers

Download PDF
7 pages

Loading next page...
 
/lp/association-for-computing-machinery/evaluating-the-predictive-performance-of-positive-unlabelled-HermoA5QjL
Publisher
Association for Computing Machinery
Copyright
Copyright © 2022 Copyright is held by the owner/author(s)
ISSN
1931-0145
eISSN
1931-0153
DOI
10.1145/3575637.3575642
Publisher site
See Article on Publisher Site

Abstract

Positive-Unlabelled (PU) learning is a growing area of machine learning that aims to learn classifiers from data consisting of labelled positive and unlabelled instances. Whilst much work has been done proposing methods for PU learning, little has been written on the subject of evaluating these methods. Many popular standard classification metrics cannot be precisely calculated due to the absence of fully labelled data, so alternative approaches must be taken. This short commentary paper critically reviews the main PU learning evaluation approaches and the choice of predictive accuracy measures in 51 articles proposing PU classifiers and provides practical recommendations for improvements in this area.

Journal

ACM SIGKDD Explorations Newsletter – Association for Computing Machinery

Published: Dec 5, 2022

Keywords: classification

References

  • Learning from positive and unlabeled data: A survey
    Bekker, ; J., ; Davis, ; J.,,
  • Learning classifiers from only positive and unlabeled data
    Elkan, ; C., ; Noto, ; K.,,
  • Disease genes prediction by HMM based PU-learning using gene expression profiles
    Nikdelfaz, ; O., ; Jalili, ; S.,,
  • C-PUGP: A cluster-based positive unlabelled learning method for disease gene prediction and prioritisation
    Vasighizaker, ; A., ; Jalili, ; S.,
  • Positive-unlabelled learning for disease gene identification
    Yang, ; P., ; Li, ; X., ; Mei, ; K.,
  • Clustering-based Method for Positive and Unlabelled Text Categorization Enhanced by Improved TFIDF
    Liu, ; L., ; Peng, ; T.,,
  • Building highperformance classifiers using positive and unlabelled examples for text
    Ke, ; T., ; Yang, ; B., ; Zhen, ; L.,
  • Partially supervised classification of text documents
    Liu, ; B., ; Yu, ; P., ; Li, ; X.,
  • Poster: A PU learning based system for potential malicious URL detection
    Zhang, ; Y., ; Li, ; L., ; Zhou, ; J.,
  • PU learning in payload-based web anomaly detection
    Luo, ; Y., ; Cheng, ; S., ; Liu, ; C.,
  • A survey on semisupervised learning
    Engelen, Van; J.E., ; Hoos, ; H.H.,,
  • Positive and unlabeled learning algorithms and applications: A survey
    Jaskie, ; K., ; Spanias, ; A.,,
  • A survey on positive and unlabelled learning
    Li, ; G.,,
  • Evaluating Learning Algorithms: a classification perspective
    Japkowicz, ; N., ; Shah, ; M.,,
  • Estimating the class prior in positive and unlabeled data through decision tree induction
    Bekker, ; J., ; Davis, ; J.,,
  • Class prior estimation from positive and unlabeled data
    Plessis, Du; M.C., ; Sugiyama, ; M.,,
  • Positive unlabeled learning for time series classification
    Nguyen, ; M.N., ; Li, ; X.L., ; Ng, ; S.K.,,
  • Learning with positive and unlabelled examples using topic-sensitive PLSA
    Zhou, ; K., ; Gui-Rong, ; X., ; Yang, ; Q.,
  • Density estimators for positive-unlabelled learning
    Basile, ; T., ; Mauro, Di; N., ; Esposito, ; F.,
  • Positive and unlabelled relational classification through label frequency estimation
    Bekker, ; J., ; Davis, ; J.,,
  • Learning Bayesian classifiers from positive and unlabelled examples
    Calvo, ; B., ; Larrañaga, ; P., ; Lozano, ; J.,,
  • Learning from positive and unlabelled examples using maximum margin clustering
    Chaudhari, ; S., ; Shevade, ; S.,,
  • Learning with a generative adversarial network from a positive unlabeled dataset for image classification
    Chiaroni, ; F., ; Rahal, ; M., ; Hueber, ; N.,
  • A robust ensemble approach to learn from positive and unlabeled data using SVM base models
    Claesen, ; M., ; Smet, De; F., ; Suykens, ; J.A., ; Moor, De; B.,,
  • Learning from positive and unlabeled examples
    Denis, ; F., ; Gilleron, ; R., ; Letouzey, ; F.,,
  • Text classification without negative examples revisit
    Fung, ; C., ; Yu, ; J., ; Lu, ; H.,
  • Bayesian belief network for positive unlabeled learning with uncertainty
    Gan, ; H., ; Zhang, ; Y., ; Song, ; Q.,,
  • Instancedependent PU learning by Bayesian optimal relabeling
    He, ; F., ; Liu, ; T., ; Webb, ; G.I., ; Tao, ; D.,,
  • J
    He,
  • Generative adversarial positive-unlabeled learning
    Hou, ; M., ; Chaib-draa, ; B., ; Li, ; C.,
  • Positive and unlabeled learning in categorical data
    Ienco, ; D., ; Pensa, ; R.,,
  • Learning from positive and unlabeled data with a selection bias
    Kato, ; M., ; Teshima, ; T., ; Honda, ; J.,,
  • A biased least squares support vector machine based on Mahalanobis distance for PU learning
    Ke, ; T., ; Lv, ; H., ; Sun, ; M.,
  • Global and local learning from positive and unlabeled examples
    Ke, ; T., ; Jing, ; L., ; Lv, ; H.,
  • Predicting drug-target interaction using positive-unlabeled learning
    Lan, ; W., ; Wang, ; J., ; Li, ; M.,
  • Text classification and co-training from positive and unlabeled examples
    Denis, ; F., ; Laurent, ; A., ; Gilleron, ; R.,,
  • Learning with positive and unlabeled examples using weighted logistic regression
    Lee, ; W.S., ; Liu, ; B.,,
  • A positive and unlabeled learning algorithm for one-class classification of remote-sensing data
    Li, ; W., ; Guo, ; Q., ; Elkan, ; C.,,
  • Learning to classify texts using positive and unlabeled data
    Li, ; X., ; Liu, ; B.,,
  • Learning from positive and unlabeled examples with different data distributions
    Li, ; X.L., ; Liu, ; B.,,
  • Learning to Identify Unexpected Instances in the Test Set
    Li, ; X., ; Liu, ; B., ; Ng, ; S.K.,,
  • Positive unlabeled learning for data stream classification
    Li, ; X.L., ; Yu, ; P.S., ; Liu, ; B., ; Ng, ; S.K.,,
  • Positive-unlabelled learning of glycosylation sites in the human proteome
    Li, ; F., ; Zhang, ; Y., ; Purcell, ; A.W., ; Webb, ; G.I., ; Chou, ; K.C., ; Lithgow, ; T., ; Li, ; C., ; Song, ; J.,,
  • Learning very fast decision tree from uncertain data streams with positive and unlabeled samples
    Liang, ; C., ; Zhang, ; Y., ; Shi, ; P., ; Hu, ; Z.,,
  • Building text classifiers using positive and unlabeled examples
    Liu, ; B., ; Dai, ; Y., ; Li, ; X., ; Lee, ; W.S., ; Yu, ; P.S.,,
  • A bagging SVM to learn from positive and unlabeled examples
    Mordelet, ; F., ; Vert, ; J.P.,,
  • SVM based adaptive learning method for text classification from positive and unlabeled documents
    Peng, ; T., ; Zuo, ; W., ; He, ; F.,,
  • Learning from data streams with only positive and unlabeled data
    Qin, ; X., ; Zhang, ; Y., ; Li, ; C., ; Li, ; X.,,
  • Learning with positive and unlabeled examples using biased twin support vector machine
    Xu, ; Z., ; Qi, ; Z., ; Zhang, ; J.,,
  • AdaSampling for positive-unlabeled and label noise learning with bioinformatics applications
    Yang, ; P., ; Ormerod, ; J.T., ; Liu, ; W., ; Ma, ; C., ; Zomaya, ; A.Y., ; Yang, ; J.Y.,,
  • Single-class classification with mapping convergence
    Yu, ; H.,,
  • Predicting disease-associated circular RNAs using deep forests combined with positive-unlabeled learning methods
    Zeng, ; X., ; Zhong, ; Y., ; Lin, ; W., ; Zou, ; Q.,,
  • Nonparallel hyperplane support vector machine for pu learning
    Zhang, ; Y., ; Ju, ; X., ; Tian, ; Y.,,
  • A simple probabilistic approach to learning from positive and unlabeled examples
    Zhang, ; D., ; Lee, ; W.S.,,
  • Reliable Negative Extracting Based on kNN for Learning from Positive and Unlabeled Examples
    Zhang, ; B., ; Zuo, ; W.,,
  • DDI-PULearn: a positive-unlabeled learning method for large-scale prediction of drug-drug interactions
    Zheng, ; Y., ; Peng, ; H., ; Zhang, ; X., ; Zhao, ; Z., ; Gao, ; X., ; Li, ; J.,,
  • Multiview positive and unlabeled learning
    Zhou, ; J.T., ; Pan, ; S.J., ; Mao, ; Q., ; Tsang, ; I.W.,,
  • Precision-recall versus accuracy and the role of large data sets
    Juba, ; B., ; Le, ; H.S.,,