Large amounts of labeled data are usually needed for training deep neural networks in medical image studies,particularly in medical image classification.However,in the field of semi-supervised medical image analysis,l...Large amounts of labeled data are usually needed for training deep neural networks in medical image studies,particularly in medical image classification.However,in the field of semi-supervised medical image analysis,labeled data is very scarce due to patient privacy concerns.For researchers,obtaining high-quality labeled images is exceedingly challenging because it involves manual annotation and clinical understanding.In addition,skin datasets are highly suitable for medical image classification studies due to the inter-class relationships and the inter-class similarities of skin lesions.In this paper,we propose a model called Coalition Sample Relation Consistency(CSRC),a consistency-based method that leverages Canonical Correlation Analysis(CCA)to capture the intrinsic relationships between samples.Considering that traditional consistency-based models only focus on the consistency of prediction,we additionally explore the similarity between features by using CCA.We enforce feature relation consistency based on traditional models,encouraging the model to learn more meaningful information from unlabeled data.Finally,considering that cross-entropy loss is not as suitable as the supervised loss when studying with imbalanced datasets(i.e.,ISIC 2017 and ISIC 2018),we improve the supervised loss to achieve better classification accuracy.Our study shows that this model performs better than many semi-supervised methods.展开更多
With the successive application of deep learning(DL)in classification tasks,the DL-based modulation classification method has become the preference for its state-of-the-art performance.Nevertheless,once the DL recogni...With the successive application of deep learning(DL)in classification tasks,the DL-based modulation classification method has become the preference for its state-of-the-art performance.Nevertheless,once the DL recognition model is pre-trained with fixed classes,the pre-trained model tends to predict incorrect results when identifying incremental classes.Moreover,the incremental classes are usually emergent without label information or only a few labeled samples of incremental classes can be obtained.In this context,we propose a graphbased semi-supervised approach to address the fewshot classes-incremental(FSCI)modulation classification problem.Our proposed method is a twostage learning method,specifically,a warm-up model is trained for classifying old classes and incremental classes,where the unlabeled samples of incremental classes are uniformly labeled with the same label to alleviate the damage of the class imbalance problem.Then the warm-up model is regarded as a feature extractor for constructing a similar graph to connect labeled samples and unlabeled samples,and the label propagation algorithm is adopted to propagate the label information from labeled nodes to unlabeled nodes in the graph to achieve the purpose of incremental classes recognition.Simulation results prove that the proposed method is superior to other finetuning methods and retrain methods.展开更多
Deep Learning(DL)is such a powerful tool that we have seen tremendous success in areas such as Computer Vision,Speech Recognition,and Natural Language Processing.Since Automated Modulation Classification(AMC)is an imp...Deep Learning(DL)is such a powerful tool that we have seen tremendous success in areas such as Computer Vision,Speech Recognition,and Natural Language Processing.Since Automated Modulation Classification(AMC)is an important part in Cognitive Radio Networks,we try to explore its potential in solving signal modulation recognition problem.It cannot be overlooked that DL model is a complex model,thus making them prone to over-fitting.DL model requires many training data to combat with over-fitting,but adding high quality labels to training data manually is not always cheap and accessible,especially in real-time system,which may counter unprecedented data in dataset.Semi-supervised Learning is a way to exploit unlabeled data effectively to reduce over-fitting in DL.In this paper,we extend Generative Adversarial Networks(GANs)to the semi-supervised learning will show it is a method can be used to create a more dataefficient classifier.展开更多
Generative adversarial networks(GANs) have become a competitive method among computer vision tasks. There have been many studies devoted to utilizing generative network to do generative tasks, such as images synthesis...Generative adversarial networks(GANs) have become a competitive method among computer vision tasks. There have been many studies devoted to utilizing generative network to do generative tasks, such as images synthesis. In this paper, a semi-supervised learning scheme is incorporated with generative adversarial network on image classification tasks to improve the image classification accuracy. Two applications of GANs are mainly focused on: semi-supervised learning and generation of images which can be as real as possible. The whole process is divided into two sections. First, only a small part of the dataset is utilized as labeled training data. And then a huge amount of samples generated from the generator is added into the training samples to improve the generalization of the discriminator. Through the semi-supervised learning scheme, full use of the unlabeled data is made which may contain potential information. Thus, the classification accuracy of the discriminator can be improved. Experimental results demonstrate the improvement of the classification accuracy of discriminator among different datasets, such as MNIST, CIFAR-10.展开更多
A semi-supervised vector machine is a relatively new learning method using both labeled and unlabeled data in classifi- cation. Since the objective function of the model for an unstrained semi-supervised vector machin...A semi-supervised vector machine is a relatively new learning method using both labeled and unlabeled data in classifi- cation. Since the objective function of the model for an unstrained semi-supervised vector machine is not smooth, many fast opti- mization algorithms cannot be applied to solve the model. In order to overcome the difficulty of dealing with non-smooth objective functions, new methods that can solve the semi-supervised vector machine with desired classification accuracy are in great demand. A quintic spline function with three-times differentiability at the ori- gin is constructed by a general three-moment method, which can be used to approximate the symmetric hinge loss function. The approximate accuracy of the quintic spiine function is estimated. Moreover, a quintic spline smooth semi-support vector machine is obtained and the convergence accuracy of the smooth model to the non-smooth one is analyzed. Three experiments are performed to test the efficiency of the model. The experimental results show that the new model outperforms other smooth models, in terms of classification performance. Furthermore, the new model is not sensitive to the increasing number of the labeled samples, which means that the new model is more efficient.展开更多
Non-collaborative radio transmitter recognition is a significant but challenging issue, since it is hard or costly to obtain labeled training data samples. In order to make effective use of the unlabeled samples which...Non-collaborative radio transmitter recognition is a significant but challenging issue, since it is hard or costly to obtain labeled training data samples. In order to make effective use of the unlabeled samples which can be obtained much easier, a novel semi-supervised classification method named Elastic Sparsity Regularized Support Vector Machine (ESRSVM) is proposed for radio transmitter classification. ESRSVM first constructs an elastic-net graph over data samples to capture the robust and natural discriminating information and then incorporate the information into the manifold learning framework by an elastic sparsity regularization term. Experimental results on 10 GMSK modulated Automatic Identification System radios and 15 FM walkie-talkie radios show that ESRSVM achieves obviously better performance than KNN and SVM, which use only labeled samples for classification, and also outperforms semi-supervised classifier LapSVM based on manifold regularization.展开更多
Objective To explore the semi-supervised learning(SSL) algorithm for long-tail endoscopic image classification with limited annotations.Method We explored semi-supervised long-tail endoscopic image classification in H...Objective To explore the semi-supervised learning(SSL) algorithm for long-tail endoscopic image classification with limited annotations.Method We explored semi-supervised long-tail endoscopic image classification in HyperKvasir,the largest gastrointestinal public dataset with 23 diverse classes.Semi-supervised learning algorithm FixMatch was applied based on consistency regularization and pseudo-labeling.After splitting the training dataset and the test dataset at a ratio of 4:1,we sampled 20%,50%,and 100% labeled training data to test the classification with limited annotations.Results The classification performance was evaluated by micro-average and macro-average evaluation metrics,with the Mathews correlation coefficient(MCC) as the overall evaluation.SSL algorithm improved the classification performance,with MCC increasing from 0.8761 to 0.8850,from 0.8983 to 0.8994,and from 0.9075 to 0.9095 with 20%,50%,and 100% ratio of labeled training data,respectively.With a 20% ratio of labeled training data,SSL improved both the micro-average and macro-average classification performance;while for the ratio of 50% and 100%,SSL improved the micro-average performance but hurt macro-average performance.Through analyzing the confusion matrix and labeling bias in each class,we found that the pseudo-based SSL algorithm exacerbated the classifier’ s preference for the head class,resulting in improved performance in the head class and degenerated performance in the tail class.Conclusion SSL can improve the classification performance for semi-supervised long-tail endoscopic image classification,especially when the labeled data is extremely limited,which may benefit the building of assisted diagnosis systems for low-volume hospitals.However,the pseudo-labeling strategy may amplify the effect of class imbalance,which hurts the classification performance for the tail class.展开更多
Sentiment classification is a useful tool to classify reviews about sentiments and attitudes towards a product or service.Existing studies heavily rely on sentiment classification methods that require fully annotated ...Sentiment classification is a useful tool to classify reviews about sentiments and attitudes towards a product or service.Existing studies heavily rely on sentiment classification methods that require fully annotated inputs.However,there is limited labelled text available,making the acquirement process of the fully annotated input costly and labour-intensive.Lately,semi-supervised methods emerge as they require only partially labelled input but perform comparably to supervised methods.Nevertheless,some works reported that the performance of the semi-supervised model degraded after adding unlabelled instances into training.Literature also shows that not all unlabelled instances are equally useful;thus identifying the informative unlabelled instances is beneficial in training a semi-supervised model.To achieve this,an informative score is proposed and incorporated into semisupervised sentiment classification.The evaluation is performed on a semisupervised method without an informative score and with an informative score.By using the informative score in the instance selection strategy to identify informative unlabelled instances,semi-supervised models perform better compared to models that do not incorporate informative scores into their training.Although the performance of semi-supervised models incorporated with an informative score is not able to surpass the supervised models,the results are still found promising as the differences in performance are subtle with a small difference of 2%to 5%,but the number of labelled instances used is greatly reduced from100%to 40%.The best finding of the proposed instance selection strategy is achieved when incorporating an informative score with a baseline confidence score at a 0.5:0.5 ratio using only 40%labelled data.展开更多
These problems of nonlinearity, fuzziness and few labeled data were rarely considered in traditional remote sensing image classification. A semi-supervised kernel fuzzy C-means （SSKFCM） algorithm is proposed to over...These problems of nonlinearity, fuzziness and few labeled data were rarely considered in traditional remote sensing image classification. A semi-supervised kernel fuzzy C-means （SSKFCM） algorithm is proposed to overcome these disadvantages of remote sensing image classification in this paper. The SSKFCM algorithm is achieved by introducing a kernel method and semi-supervised learning technique into the standard fuzzy C-means （FCM） algorithm. A set of Beijing-1 micro-satellite＇s multispectral images are adopted to be classified by several algorithms, such as FCM, kernel FCM （KFCM）, semi-supervised FCM （SSFCM） and SSKFCM. The classification results are estimated by corresponding indexes. The results indicate that the SSKFCM algorithm significantly improves the classification accuracy of remote sensing images compared with the others.展开更多
Accurate classification of encrypted traffic plays an important role in network management.However,current methods confronts several problems:inability to characterize traffic that exhibits great dispersion,inability ...Accurate classification of encrypted traffic plays an important role in network management.However,current methods confronts several problems:inability to characterize traffic that exhibits great dispersion,inability to classify traffic with multi-level features,and degradation due to limited training traffic size.To address these problems,this paper proposes a traffic granularity-based cryptographic traffic classification method,called Granular Classifier(GC).In this paper,a novel Cardinality-based Constrained Fuzzy C-Means(CCFCM)clustering algorithm is proposed to address the problem caused by limited training traffic,considering the ratio of cardinality that must be linked between flows to achieve good traffic partitioning.Then,an original representation format of traffic is presented based on granular computing,named Traffic Granules(TG),to accurately describe traffic structure by catching the dispersion of different traffic features.Each granule is a compact set of similar data with a refined boundary by excluding outliers.Based on TG,GC is constructed to perform traffic classification based on multi-level features.The performance of the GC is evaluated based on real-world encrypted network traffic data.Experimental results show that the GC achieves outstanding performance for encrypted traffic classification with limited size of training traffic and keeps accurate classification in dynamic network conditions.展开更多
Generative adversarial network(GAN)has achieved great success in many fields such as computer vision,speech processing,and natural language processing,because of its powerful capabilities for generating realistic samp...Generative adversarial network(GAN)has achieved great success in many fields such as computer vision,speech processing,and natural language processing,because of its powerful capabilities for generating realistic samples.In this paper,we introduce GAN into the field of electromagnetic signal classification(ESC).ESC plays an important role in both military and civilian domains.However,in many specific scenarios,we can’t obtain enough labeled data,which cause failure of deep learning methods because they are easy to fall into over-fitting.Fortunately,semi-supervised learning(SSL)can leverage the large amount of unlabeled data to enhance the classification performance of classifiers,especially in scenarios with limited amount of labeled data.We present an SSL framework by incorporating GAN,which can directly process the raw in-phase and quadrature(IQ)signal data.According to the characteristics of the electromagnetic signal,we propose a weighted loss function,leading to an effective classifier to realize the end-to-end classification of the electromagnetic signal.We validate the proposed method on both public RML2016.04c dataset and real-world Aircraft Communications Addressing and Reporting System(ACARS)signal dataset.Extensive experimental results show that the proposed framework obtains a significant increase in classification accuracy compared with the state-of-the-art studies.展开更多
A Machine Learning (ML)-based Intrusion Detection and Prevention System (IDPS)requires a large amount of labeled up-to-date training data to effectively detect intrusions and generalize well to novel attacks.However,t...A Machine Learning (ML)-based Intrusion Detection and Prevention System (IDPS)requires a large amount of labeled up-to-date training data to effectively detect intrusions and generalize well to novel attacks.However,the labeling of data is costly and becomes infeasible when dealing with big data,such as those generated by Intemet of Things applications.To this effect,building an ML model that learns from non-labeled or partially labeled data is of critical importance.This paper proposes a Semi-supervised Mniti-Layered Clustering ((SMLC))model for the detection and prevention of network intrusion.SMLC has the capability to learn from partially labeled data while achieving a detection performance comparable to that of supervised ML-based IDPS.The performance of SMLC is compared with that of a well-known semi-supervised model (tri-training)and of supervised ensemble ML models, namely Random.Forest,Bagging,and AdaboostM1on two benchmark network-intrusion datasets,NSL and Kyoto 2006+.Experimental resnits show that SMLC is superior to tri-training,providing a comparable detection accuracy with 20%less labeled instances of training data.Furthermore,our results demonstrate that our scheme has a detection accuracy comparable to that of the supervised ensemble models.展开更多
Automatic classification of blog entries is generally treated as a semi-supervised machine learning task, in which the blog entries are automatically assigned to one of a set of pre-defined classes based on the featur...Automatic classification of blog entries is generally treated as a semi-supervised machine learning task, in which the blog entries are automatically assigned to one of a set of pre-defined classes based on the features extracted from their textual content. This paper attempts automatic classification of unstructured blog entries by following pre-processing steps like tokenization, stop-word elimination and stemming;statistical techniques for feature set extraction, and feature set enhancement using semantic resources followed by modeling using two alternative machine learning models—the na?ve Bayesian model and the artificial neural network model. Empirical evaluations indicate that this multi-step classification approach has resulted in good overall classification accuracy over unstructured blog text datasets with both machine learning model alternatives. However, the na?ve Bayesian classification model clearly out-performs the ANN based classification model when a smaller feature-set is available which is usually the case when a blog topic is recent and the number of training datasets available is restricted.展开更多
Deep learning significantly improves the accuracy of remote sensing image scene classification,benefiting from the large-scale datasets.However,annotating the remote sensing images is time-consuming and even tough for...Deep learning significantly improves the accuracy of remote sensing image scene classification,benefiting from the large-scale datasets.However,annotating the remote sensing images is time-consuming and even tough for experts.Deep neural networks trained using a few labeled samples usually generalize less to new unseen images.In this paper,we propose a semi-supervised approach for remote sensing image scene classification based on the prototype-based consistency,by exploring massive unlabeled images.To this end,we,first,propose a feature enhancement module to extract discriminative features.This is achieved by focusing the model on the foreground areas.Then,the prototype-based classifier is introduced to the framework,which is used to acquire consistent feature representations.We conduct a series of experiments on NWPU-RESISC45 and Aerial Image Dataset(AID).Our method improves the State-Of-The-Art(SOTA)method on NWPU-RESISC45 from 92.03%to 93.08%and on AID from 94.25%to 95.24%in terms of accuracy.展开更多
In many machine learning problems, a large amount of data is available but only a few of them can be labeled easily. This provides a research branch to effectively combine unlabeled and labeled data to infer the label...In many machine learning problems, a large amount of data is available but only a few of them can be labeled easily. This provides a research branch to effectively combine unlabeled and labeled data to infer the labels of unlabeled ones, that is, to develop transductive learning. In this article, based on Pattern classification via single sphere （SSPC）, which seeks a hypersphere to separate data with the maximum separation ratio, a progressive transductive pattern classification method via single sphere （PTSSPC） is proposed to construct the classifier using both the labeled and unlabeled data. PTSSPC utilize the additional information of the unlabeled samples and obtain better classification performance than SSPC when insufficient labeled data information is available. Experiment results show the algorithm can yields better performance.展开更多
Multi-label data with high dimensionality often occurs,which will produce large time and energy overheads when directly used in classification tasks.To solve this problem,a novel algorithm called multi-label dimension...Multi-label data with high dimensionality often occurs,which will produce large time and energy overheads when directly used in classification tasks.To solve this problem,a novel algorithm called multi-label dimensionality reduction via semi-supervised discriminant analysis(MSDA) was proposed.It was expected to derive an objective discriminant function as smooth as possible on the data manifold by multi-label learning and semi-supervised learning.By virtue of the latent imformation,which was provided by the graph weighted matrix of sample attributes and the similarity correlation matrix of partial sample labels,MSDA readily made the separability between different classes achieve maximization and estimated the intrinsic geometric structure in the lower manifold space by employing unlabeled data.Extensive experimental results on several real multi-label datasets show that after dimensionality reduction using MSDA,the average classification accuracy is about 9.71% higher than that of other algorithms,and several evaluation metrices like Hamming-loss are also superior to those of other dimensionality reduction methods.展开更多
Semi-supervised Support Vector Machines is an appealing method for using unlabeled data in classification. Smoothing homotopy method is one of feasible method for solving semi-supervised support vector machines. In th...Semi-supervised Support Vector Machines is an appealing method for using unlabeled data in classification. Smoothing homotopy method is one of feasible method for solving semi-supervised support vector machines. In this paper, an inexact implementation of the smoothing homotopy method is considered. The numerical implementation is based on a truncated smoothing technique. By the new technique, many “non-active” data can be filtered during the computation of every iteration so that the computation cost is reduced greatly. Besides this, the global convergence can make better local minima and then result in lower test errors. Final numerical results verify the efficiency of the method.展开更多
In this paper, we proposed a new semi-supervised multi-manifold learning method, called semi- supervised sparse multi-manifold embedding (S3MME), for dimensionality reduction of hyperspectral image data. S3MME exploit...In this paper, we proposed a new semi-supervised multi-manifold learning method, called semi- supervised sparse multi-manifold embedding (S3MME), for dimensionality reduction of hyperspectral image data. S3MME exploits both the labeled and unlabeled data to adaptively find neighbors of each sample from the same manifold by using an optimization program based on sparse representation, and naturally gives relative importance to the labeled ones through a graph-based methodology. Then it tries to extract discriminative features on each manifold such that the data points in the same manifold become closer. The effectiveness of the proposed multi-manifold learning algorithm is demonstrated and compared through experiments on a real hyperspectral images.展开更多
Image classification is an important application of deep learning. In a typical classification task, the classification accuracy is strongly related to the features that are extracted via deep learning methods. An aut...Image classification is an important application of deep learning. In a typical classification task, the classification accuracy is strongly related to the features that are extracted via deep learning methods. An autoencoder is a special type of neural network, often used for dimensionality reduction and feature extraction. The proposed method is based on the traditional autoencoder, incorporating the "distance" information between samples from different categories. The model is called a semisupervised distance autoencoder. Each layer is first pre-trained in an unsupervised manner. In the subsequent supervised training, the optimized parameters are set as the initial values. To obtain more suitable features, we use a stacked model to replace the basic autoencoder structure with a single hidden layer. A series of experiments are carried out to test the performance of different models on several datasets, including the MNIST dataset, street view house numbers(SVHN) dataset, German traffic sign recognition benchmark(GTSRB), and CIFAR-10 dataset. The proposed semi-supervised distance autoencoder method is compared with the traditional autoencoder, sparse autoencoder, and supervised autoencoder. Experimental results verify the effectiveness of the proposed model.展开更多
Semi-Supervised Classification (SSC),which makes use of both labeled and unlabeled data to determine classification borders in feature space,has great advantages in extracting classification information from mass data...Semi-Supervised Classification (SSC),which makes use of both labeled and unlabeled data to determine classification borders in feature space,has great advantages in extracting classification information from mass data.In this paper,a novel SSC method based on Gaussian Mixture Model (GMM) is proposed,in which each class’s feature space is described by one GMM.Experiments show the proposed method can achieve high classification accuracy with small amount of labeled data.However,for the same accuracy,supervised classification methods such as Support Vector Machine,Object Oriented Classification,etc.should be provided with much more labeled data.展开更多
基金sponsored by the National Natural Science Foundation of China Grant No.62271302the Shanghai Municipal Natural Science Foundation Grant 20ZR1423500.
文摘Large amounts of labeled data are usually needed for training deep neural networks in medical image studies,particularly in medical image classification.However,in the field of semi-supervised medical image analysis,labeled data is very scarce due to patient privacy concerns.For researchers,obtaining high-quality labeled images is exceedingly challenging because it involves manual annotation and clinical understanding.In addition,skin datasets are highly suitable for medical image classification studies due to the inter-class relationships and the inter-class similarities of skin lesions.In this paper,we propose a model called Coalition Sample Relation Consistency(CSRC),a consistency-based method that leverages Canonical Correlation Analysis(CCA)to capture the intrinsic relationships between samples.Considering that traditional consistency-based models only focus on the consistency of prediction,we additionally explore the similarity between features by using CCA.We enforce feature relation consistency based on traditional models,encouraging the model to learn more meaningful information from unlabeled data.Finally,considering that cross-entropy loss is not as suitable as the supervised loss when studying with imbalanced datasets(i.e.,ISIC 2017 and ISIC 2018),we improve the supervised loss to achieve better classification accuracy.Our study shows that this model performs better than many semi-supervised methods.
基金supported in part by the National Natural Science Foundation of China under Grant No.62171334,No.11973077 and No.12003061。
文摘With the successive application of deep learning(DL)in classification tasks,the DL-based modulation classification method has become the preference for its state-of-the-art performance.Nevertheless,once the DL recognition model is pre-trained with fixed classes,the pre-trained model tends to predict incorrect results when identifying incremental classes.Moreover,the incremental classes are usually emergent without label information or only a few labeled samples of incremental classes can be obtained.In this context,we propose a graphbased semi-supervised approach to address the fewshot classes-incremental(FSCI)modulation classification problem.Our proposed method is a twostage learning method,specifically,a warm-up model is trained for classifying old classes and incremental classes,where the unlabeled samples of incremental classes are uniformly labeled with the same label to alleviate the damage of the class imbalance problem.Then the warm-up model is regarded as a feature extractor for constructing a similar graph to connect labeled samples and unlabeled samples,and the label propagation algorithm is adopted to propagate the label information from labeled nodes to unlabeled nodes in the graph to achieve the purpose of incremental classes recognition.Simulation results prove that the proposed method is superior to other finetuning methods and retrain methods.
基金This work is supported by the National Natural Science Foundation of China(Nos.61771154,61603239,61772454,6171101570).
文摘Deep Learning(DL)is such a powerful tool that we have seen tremendous success in areas such as Computer Vision,Speech Recognition,and Natural Language Processing.Since Automated Modulation Classification(AMC)is an important part in Cognitive Radio Networks,we try to explore its potential in solving signal modulation recognition problem.It cannot be overlooked that DL model is a complex model,thus making them prone to over-fitting.DL model requires many training data to combat with over-fitting,but adding high quality labels to training data manually is not always cheap and accessible,especially in real-time system,which may counter unprecedented data in dataset.Semi-supervised Learning is a way to exploit unlabeled data effectively to reduce over-fitting in DL.In this paper,we extend Generative Adversarial Networks(GANs)to the semi-supervised learning will show it is a method can be used to create a more dataefficient classifier.
基金Supported by the National Natural Science Foundation of China(No.61501457)National Key Technology R&D Program(No.2015BAK21B00)
文摘Generative adversarial networks(GANs) have become a competitive method among computer vision tasks. There have been many studies devoted to utilizing generative network to do generative tasks, such as images synthesis. In this paper, a semi-supervised learning scheme is incorporated with generative adversarial network on image classification tasks to improve the image classification accuracy. Two applications of GANs are mainly focused on: semi-supervised learning and generation of images which can be as real as possible. The whole process is divided into two sections. First, only a small part of the dataset is utilized as labeled training data. And then a huge amount of samples generated from the generator is added into the training samples to improve the generalization of the discriminator. Through the semi-supervised learning scheme, full use of the unlabeled data is made which may contain potential information. Thus, the classification accuracy of the discriminator can be improved. Experimental results demonstrate the improvement of the classification accuracy of discriminator among different datasets, such as MNIST, CIFAR-10.
基金supported by the Fundamental Research Funds for University of Science and Technology Beijing(FRF-BR-12-021)
文摘A semi-supervised vector machine is a relatively new learning method using both labeled and unlabeled data in classifi- cation. Since the objective function of the model for an unstrained semi-supervised vector machine is not smooth, many fast opti- mization algorithms cannot be applied to solve the model. In order to overcome the difficulty of dealing with non-smooth objective functions, new methods that can solve the semi-supervised vector machine with desired classification accuracy are in great demand. A quintic spline function with three-times differentiability at the ori- gin is constructed by a general three-moment method, which can be used to approximate the symmetric hinge loss function. The approximate accuracy of the quintic spiine function is estimated. Moreover, a quintic spline smooth semi-support vector machine is obtained and the convergence accuracy of the smooth model to the non-smooth one is analyzed. Three experiments are performed to test the efficiency of the model. The experimental results show that the new model outperforms other smooth models, in terms of classification performance. Furthermore, the new model is not sensitive to the increasing number of the labeled samples, which means that the new model is more efficient.
基金Supported by the Hi-Tech Research and Development Program of China (No. 2009AAJ130)
文摘Non-collaborative radio transmitter recognition is a significant but challenging issue, since it is hard or costly to obtain labeled training data samples. In order to make effective use of the unlabeled samples which can be obtained much easier, a novel semi-supervised classification method named Elastic Sparsity Regularized Support Vector Machine (ESRSVM) is proposed for radio transmitter classification. ESRSVM first constructs an elastic-net graph over data samples to capture the robust and natural discriminating information and then incorporate the information into the manifold learning framework by an elastic sparsity regularization term. Experimental results on 10 GMSK modulated Automatic Identification System radios and 15 FM walkie-talkie radios show that ESRSVM achieves obviously better performance than KNN and SVM, which use only labeled samples for classification, and also outperforms semi-supervised classifier LapSVM based on manifold regularization.
文摘Objective To explore the semi-supervised learning(SSL) algorithm for long-tail endoscopic image classification with limited annotations.Method We explored semi-supervised long-tail endoscopic image classification in HyperKvasir,the largest gastrointestinal public dataset with 23 diverse classes.Semi-supervised learning algorithm FixMatch was applied based on consistency regularization and pseudo-labeling.After splitting the training dataset and the test dataset at a ratio of 4:1,we sampled 20%,50%,and 100% labeled training data to test the classification with limited annotations.Results The classification performance was evaluated by micro-average and macro-average evaluation metrics,with the Mathews correlation coefficient(MCC) as the overall evaluation.SSL algorithm improved the classification performance,with MCC increasing from 0.8761 to 0.8850,from 0.8983 to 0.8994,and from 0.9075 to 0.9095 with 20%,50%,and 100% ratio of labeled training data,respectively.With a 20% ratio of labeled training data,SSL improved both the micro-average and macro-average classification performance;while for the ratio of 50% and 100%,SSL improved the micro-average performance but hurt macro-average performance.Through analyzing the confusion matrix and labeling bias in each class,we found that the pseudo-based SSL algorithm exacerbated the classifier’ s preference for the head class,resulting in improved performance in the head class and degenerated performance in the tail class.Conclusion SSL can improve the classification performance for semi-supervised long-tail endoscopic image classification,especially when the labeled data is extremely limited,which may benefit the building of assisted diagnosis systems for low-volume hospitals.However,the pseudo-labeling strategy may amplify the effect of class imbalance,which hurts the classification performance for the tail class.
基金This research is supported by Fundamental Research Grant Scheme(FRGS),Ministry of Education Malaysia(MOE)under the project code,FRGS/1/2018/ICT02/USM/02/9 titled,Automated Big Data Annotation for Training Semi-Supervised Deep Learning Model in Sentiment Classification.
文摘Sentiment classification is a useful tool to classify reviews about sentiments and attitudes towards a product or service.Existing studies heavily rely on sentiment classification methods that require fully annotated inputs.However,there is limited labelled text available,making the acquirement process of the fully annotated input costly and labour-intensive.Lately,semi-supervised methods emerge as they require only partially labelled input but perform comparably to supervised methods.Nevertheless,some works reported that the performance of the semi-supervised model degraded after adding unlabelled instances into training.Literature also shows that not all unlabelled instances are equally useful;thus identifying the informative unlabelled instances is beneficial in training a semi-supervised model.To achieve this,an informative score is proposed and incorporated into semisupervised sentiment classification.The evaluation is performed on a semisupervised method without an informative score and with an informative score.By using the informative score in the instance selection strategy to identify informative unlabelled instances,semi-supervised models perform better compared to models that do not incorporate informative scores into their training.Although the performance of semi-supervised models incorporated with an informative score is not able to surpass the supervised models,the results are still found promising as the differences in performance are subtle with a small difference of 2%to 5%,but the number of labelled instances used is greatly reduced from100%to 40%.The best finding of the proposed instance selection strategy is achieved when incorporating an informative score with a baseline confidence score at a 0.5:0.5 ratio using only 40%labelled data.
基金Supported by the National High Technology Research and Development Programme （No.2007AA12Z227） and the National Natural Science Foundation of China （No.40701146）.
文摘These problems of nonlinearity, fuzziness and few labeled data were rarely considered in traditional remote sensing image classification. A semi-supervised kernel fuzzy C-means （SSKFCM） algorithm is proposed to overcome these disadvantages of remote sensing image classification in this paper. The SSKFCM algorithm is achieved by introducing a kernel method and semi-supervised learning technique into the standard fuzzy C-means （FCM） algorithm. A set of Beijing-1 micro-satellite＇s multispectral images are adopted to be classified by several algorithms, such as FCM, kernel FCM （KFCM）, semi-supervised FCM （SSFCM） and SSKFCM. The classification results are estimated by corresponding indexes. The results indicate that the SSKFCM algorithm significantly improves the classification accuracy of remote sensing images compared with the others.
基金supported in part by the Shandong Provincial Natural Science Foundation under Grant ZR2021QF008the National Natural Science Foundation of China under Grant 62072351+1 种基金in part by the open research project of ZheJiang Lab under grant 2021PD0AB01in part by the 111 Project under Grant B16037。
文摘Accurate classification of encrypted traffic plays an important role in network management.However,current methods confronts several problems:inability to characterize traffic that exhibits great dispersion,inability to classify traffic with multi-level features,and degradation due to limited training traffic size.To address these problems,this paper proposes a traffic granularity-based cryptographic traffic classification method,called Granular Classifier(GC).In this paper,a novel Cardinality-based Constrained Fuzzy C-Means(CCFCM)clustering algorithm is proposed to address the problem caused by limited training traffic,considering the ratio of cardinality that must be linked between flows to achieve good traffic partitioning.Then,an original representation format of traffic is presented based on granular computing,named Traffic Granules(TG),to accurately describe traffic structure by catching the dispersion of different traffic features.Each granule is a compact set of similar data with a refined boundary by excluding outliers.Based on TG,GC is constructed to perform traffic classification based on multi-level features.The performance of the GC is evaluated based on real-world encrypted network traffic data.Experimental results show that the GC achieves outstanding performance for encrypted traffic classification with limited size of training traffic and keeps accurate classification in dynamic network conditions.
基金the National Natural Science Foundation of China(Nos.61771380,U19B2015,U1730109).
文摘Generative adversarial network(GAN)has achieved great success in many fields such as computer vision,speech processing,and natural language processing,because of its powerful capabilities for generating realistic samples.In this paper,we introduce GAN into the field of electromagnetic signal classification(ESC).ESC plays an important role in both military and civilian domains.However,in many specific scenarios,we can’t obtain enough labeled data,which cause failure of deep learning methods because they are easy to fall into over-fitting.Fortunately,semi-supervised learning(SSL)can leverage the large amount of unlabeled data to enhance the classification performance of classifiers,especially in scenarios with limited amount of labeled data.We present an SSL framework by incorporating GAN,which can directly process the raw in-phase and quadrature(IQ)signal data.According to the characteristics of the electromagnetic signal,we propose a weighted loss function,leading to an effective classifier to realize the end-to-end classification of the electromagnetic signal.We validate the proposed method on both public RML2016.04c dataset and real-world Aircraft Communications Addressing and Reporting System(ACARS)signal dataset.Extensive experimental results show that the proposed framework obtains a significant increase in classification accuracy compared with the state-of-the-art studies.
文摘A Machine Learning (ML)-based Intrusion Detection and Prevention System (IDPS)requires a large amount of labeled up-to-date training data to effectively detect intrusions and generalize well to novel attacks.However,the labeling of data is costly and becomes infeasible when dealing with big data,such as those generated by Intemet of Things applications.To this effect,building an ML model that learns from non-labeled or partially labeled data is of critical importance.This paper proposes a Semi-supervised Mniti-Layered Clustering ((SMLC))model for the detection and prevention of network intrusion.SMLC has the capability to learn from partially labeled data while achieving a detection performance comparable to that of supervised ML-based IDPS.The performance of SMLC is compared with that of a well-known semi-supervised model (tri-training)and of supervised ensemble ML models, namely Random.Forest,Bagging,and AdaboostM1on two benchmark network-intrusion datasets,NSL and Kyoto 2006+.Experimental resnits show that SMLC is superior to tri-training,providing a comparable detection accuracy with 20%less labeled instances of training data.Furthermore,our results demonstrate that our scheme has a detection accuracy comparable to that of the supervised ensemble models.
文摘Automatic classification of blog entries is generally treated as a semi-supervised machine learning task, in which the blog entries are automatically assigned to one of a set of pre-defined classes based on the features extracted from their textual content. This paper attempts automatic classification of unstructured blog entries by following pre-processing steps like tokenization, stop-word elimination and stemming;statistical techniques for feature set extraction, and feature set enhancement using semantic resources followed by modeling using two alternative machine learning models—the na?ve Bayesian model and the artificial neural network model. Empirical evaluations indicate that this multi-step classification approach has resulted in good overall classification accuracy over unstructured blog text datasets with both machine learning model alternatives. However, the na?ve Bayesian classification model clearly out-performs the ANN based classification model when a smaller feature-set is available which is usually the case when a blog topic is recent and the number of training datasets available is restricted.
基金supported in part by the National Natural Science Foundation of China(No.12302252)。
文摘Deep learning significantly improves the accuracy of remote sensing image scene classification,benefiting from the large-scale datasets.However,annotating the remote sensing images is time-consuming and even tough for experts.Deep neural networks trained using a few labeled samples usually generalize less to new unseen images.In this paper,we propose a semi-supervised approach for remote sensing image scene classification based on the prototype-based consistency,by exploring massive unlabeled images.To this end,we,first,propose a feature enhancement module to extract discriminative features.This is achieved by focusing the model on the foreground areas.Then,the prototype-based classifier is introduced to the framework,which is used to acquire consistent feature representations.We conduct a series of experiments on NWPU-RESISC45 and Aerial Image Dataset(AID).Our method improves the State-Of-The-Art(SOTA)method on NWPU-RESISC45 from 92.03%to 93.08%and on AID from 94.25%to 95.24%in terms of accuracy.
基金supported by the National Natural Science of China(6057407560705004).
文摘In many machine learning problems, a large amount of data is available but only a few of them can be labeled easily. This provides a research branch to effectively combine unlabeled and labeled data to infer the labels of unlabeled ones, that is, to develop transductive learning. In this article, based on Pattern classification via single sphere （SSPC）, which seeks a hypersphere to separate data with the maximum separation ratio, a progressive transductive pattern classification method via single sphere （PTSSPC） is proposed to construct the classifier using both the labeled and unlabeled data. PTSSPC utilize the additional information of the unlabeled samples and obtain better classification performance than SSPC when insufficient labeled data information is available. Experiment results show the algorithm can yields better performance.
基金Project(60425310) supported by the National Science Fund for Distinguished Young ScholarsProject(10JJ6094) supported by the Hunan Provincial Natural Foundation of China
文摘Multi-label data with high dimensionality often occurs,which will produce large time and energy overheads when directly used in classification tasks.To solve this problem,a novel algorithm called multi-label dimensionality reduction via semi-supervised discriminant analysis(MSDA) was proposed.It was expected to derive an objective discriminant function as smooth as possible on the data manifold by multi-label learning and semi-supervised learning.By virtue of the latent imformation,which was provided by the graph weighted matrix of sample attributes and the similarity correlation matrix of partial sample labels,MSDA readily made the separability between different classes achieve maximization and estimated the intrinsic geometric structure in the lower manifold space by employing unlabeled data.Extensive experimental results on several real multi-label datasets show that after dimensionality reduction using MSDA,the average classification accuracy is about 9.71% higher than that of other algorithms,and several evaluation metrices like Hamming-loss are also superior to those of other dimensionality reduction methods.
文摘Semi-supervised Support Vector Machines is an appealing method for using unlabeled data in classification. Smoothing homotopy method is one of feasible method for solving semi-supervised support vector machines. In this paper, an inexact implementation of the smoothing homotopy method is considered. The numerical implementation is based on a truncated smoothing technique. By the new technique, many “non-active” data can be filtered during the computation of every iteration so that the computation cost is reduced greatly. Besides this, the global convergence can make better local minima and then result in lower test errors. Final numerical results verify the efficiency of the method.
文摘In this paper, we proposed a new semi-supervised multi-manifold learning method, called semi- supervised sparse multi-manifold embedding (S3MME), for dimensionality reduction of hyperspectral image data. S3MME exploits both the labeled and unlabeled data to adaptively find neighbors of each sample from the same manifold by using an optimization program based on sparse representation, and naturally gives relative importance to the labeled ones through a graph-based methodology. Then it tries to extract discriminative features on each manifold such that the data points in the same manifold become closer. The effectiveness of the proposed multi-manifold learning algorithm is demonstrated and compared through experiments on a real hyperspectral images.
基金Project supported by the National Natural Science Foundation of China (Nos. U1664264 and U1509203)。
文摘Image classification is an important application of deep learning. In a typical classification task, the classification accuracy is strongly related to the features that are extracted via deep learning methods. An autoencoder is a special type of neural network, often used for dimensionality reduction and feature extraction. The proposed method is based on the traditional autoencoder, incorporating the "distance" information between samples from different categories. The model is called a semisupervised distance autoencoder. Each layer is first pre-trained in an unsupervised manner. In the subsequent supervised training, the optimized parameters are set as the initial values. To obtain more suitable features, we use a stacked model to replace the basic autoencoder structure with a single hidden layer. A series of experiments are carried out to test the performance of different models on several datasets, including the MNIST dataset, street view house numbers(SVHN) dataset, German traffic sign recognition benchmark(GTSRB), and CIFAR-10 dataset. The proposed semi-supervised distance autoencoder method is compared with the traditional autoencoder, sparse autoencoder, and supervised autoencoder. Experimental results verify the effectiveness of the proposed model.
基金supported by the State Key Laboratory of Remote Sensing Science and Chinese Academy of Surveying & Mapping (Grant No.20903)
文摘Semi-Supervised Classification (SSC),which makes use of both labeled and unlabeled data to determine classification borders in feature space,has great advantages in extracting classification information from mass data.In this paper,a novel SSC method based on Gaussian Mixture Model (GMM) is proposed,in which each class’s feature space is described by one GMM.Experiments show the proposed method can achieve high classification accuracy with small amount of labeled data.However,for the same accuracy,supervised classification methods such as Support Vector Machine,Object Oriented Classification,etc.should be provided with much more labeled data.
