The field of biometric identification has seen significant advancements over the years,with research focusing on enhancing the accuracy and security of these systems.One of the key developments is the integration of d...The field of biometric identification has seen significant advancements over the years,with research focusing on enhancing the accuracy and security of these systems.One of the key developments is the integration of deep learning techniques in biometric systems.However,despite these advancements,certain challenges persist.One of the most significant challenges is scalability over growing complexity.Traditional methods either require maintaining and securing a growing database,introducing serious security challenges,or relying on retraining the entiremodelwhen new data is introduced-a process that can be computationally expensive and complex.This challenge underscores the need for more efficient methods to scale securely.To this end,we introduce a novel approach that addresses these challenges by integrating multimodal biometrics,cancelable biometrics,and incremental learning techniques.This work is among the first attempts to seamlessly incorporate deep cancelable biometrics with dynamic architectural updates,applied incrementally to the deep learning model as new users are enrolled,achieving high performance with minimal catastrophic forgetting.By leveraging a One-Dimensional Convolutional Neural Network(1D-CNN)architecture combined with a hybrid incremental learning approach,our system achieves high recognition accuracy,averaging 98.98% over incrementing datasets,while ensuring user privacy through cancelable templates generated via a pre-trained CNN model and random projection.The approach demonstrates remarkable adaptability,utilizing the least intrusive biometric traits like facial features and fingerprints,ensuring not only robust performance but also long-term serviceability.展开更多
The visions of Industry 4.0 and 5.0 have reinforced the industrial environment.They have also made artificial intelligence incorporated as a major facilitator.Diagnosing machine faults has become a solid foundation fo...The visions of Industry 4.0 and 5.0 have reinforced the industrial environment.They have also made artificial intelligence incorporated as a major facilitator.Diagnosing machine faults has become a solid foundation for automatically recognizing machine failure,and thus timely maintenance can ensure safe operations.Transfer learning is a promising solution that can enhance the machine fault diagnosis model by borrowing pre-trained knowledge from the source model and applying it to the target model,which typically involves two datasets.In response to the availability of multiple datasets,this paper proposes using selective and adaptive incremental transfer learning(SA-ITL),which fuses three algorithms,namely,the hybrid selective algorithm,the transferability enhancement algorithm,and the incremental transfer learning algorithm.It is a selective algorithm that enables selecting and ordering appropriate datasets for transfer learning and selecting useful knowledge to avoid negative transfer.The algorithm also adaptively adjusts the portion of training data to balance the learning rate and training time.The proposed algorithm is evaluated and analyzed using ten benchmark datasets.Compared with other algorithms from existing works,SA-ITL improves the accuracy of all datasets.Ablation studies present the accuracy enhancements of the SA-ITL,including the hybrid selective algorithm(1.22%-3.82%),transferability enhancement algorithm(1.91%-4.15%),and incremental transfer learning algorithm(0.605%-2.68%).These also show the benefits of enhancing the target model with heterogeneous image datasets that widen the range of domain selection between source and target domains.展开更多
A new incremental support vector machine （SVM） algorithm is proposed which is based on multiple kernel learning. Through introducing multiple kernel learning into the SVM incremental learning, large scale data set l...A new incremental support vector machine （SVM） algorithm is proposed which is based on multiple kernel learning. Through introducing multiple kernel learning into the SVM incremental learning, large scale data set learning problem can be solved effectively. Furthermore, different punishments are adopted in allusion to the training subset and the acquired support vectors, which may help to improve the performance of SVM. Simulation results indicate that the proposed algorithm can not only solve the model selection problem in SVM incremental learning, but also improve the classification or prediction precision.展开更多
The maintenance of an aero-engine usually includes three levels,and the maintenance cost and period greatly differ depending on the different maintenance levels.To plan a reasonable maintenance budget program, airline...The maintenance of an aero-engine usually includes three levels,and the maintenance cost and period greatly differ depending on the different maintenance levels.To plan a reasonable maintenance budget program, airlines would like to predict the maintenance level of aero-engine before repairing in terms of performance parameters,which can provide more economic benefits.The maintenance level decision rules are mined using the historical maintenance data of a civil aero-engine based on the rough set theory,and a variety of possible models of updating rules produced by newly increased maintenance cases added to the historical maintenance case database are investigated by the means of incremental machine learning.The continuously updated rules can provide reasonable guidance suggestions for engineers and decision support for planning a maintenance budget program before repairing. The results of an example show that the decision rules become more typical and robust,and they are more accurate to predict the maintenance level of an aero-engine module as the maintenance data increase,which illustrates the feasibility of the represented method.展开更多
Intelligent seismic facies identification based on deep learning can alleviate the time-consuming and labor-intensive problem of manual interpretation,which has been widely applied.Supervised learning can realize faci...Intelligent seismic facies identification based on deep learning can alleviate the time-consuming and labor-intensive problem of manual interpretation,which has been widely applied.Supervised learning can realize facies identification with high efficiency and accuracy;however,it depends on the usage of a large amount of well-labeled data.To solve this issue,we propose herein an incremental semi-supervised method for intelligent facies identification.Our method considers the continuity of the lateral variation of strata and uses cosine similarity to quantify the similarity of the seismic data feature domain.The maximum-diff erence sample in the neighborhood of the currently used training data is then found to reasonably expand the training sets.This process continuously increases the amount of training data and learns its distribution.We integrate old knowledge while absorbing new ones to realize incremental semi-supervised learning and achieve the purpose of evolving the network models.In this work,accuracy and confusion matrix are employed to jointly control the predicted results of the model from both overall and partial aspects.The obtained values are then applied to a three-dimensional(3D)real dataset and used to quantitatively evaluate the results.Using unlabeled data,our proposed method acquires more accurate and stable testing results compared to conventional supervised learning algorithms that only use well-labeled data.A considerable improvement for small-sample categories is also observed.Using less than 1%of the training data,the proposed method can achieve an average accuracy of over 95%on the 3D dataset.In contrast,the conventional supervised learning algorithm achieved only approximately 85%.展开更多
In order to improve the efficiency of learning the triangular membership functions( TMFs) for mining fuzzy association rule( FAR) in dynamic database,a single-pass fuzzy c means( SPFCM)algorithm is combined with the r...In order to improve the efficiency of learning the triangular membership functions( TMFs) for mining fuzzy association rule( FAR) in dynamic database,a single-pass fuzzy c means( SPFCM)algorithm is combined with the real-coded CHC genetic model to incrementally learn the TMFs. The cluster centers resulting from SPFCM are regarded as the midpoint of TMFs. The population of CHC is generated randomly according to the cluster center and constraint conditions among TMFs. Then a new population for incremental learning is composed of the excellent chromosomes stored in the first genetic process and the chromosomes generated based on the cluster center adjusted by SPFCM. The experiments on real datasets show that the number of generations converging to the solution of the proposed approach is less than that of the existing batch learning approach. The quality of TMFs generated by the approach is comparable to that of the batch learning approach. Compared with the existing incremental learning strategy,the proposed approach is superior in terms of the quality of TMFs and time cost.展开更多
In this study, we propose an incremental learning approach based on a machine-machine interaction via relative attribute feedbacks that exploit comparative relationships among top level image categories. One machine a...In this study, we propose an incremental learning approach based on a machine-machine interaction via relative attribute feedbacks that exploit comparative relationships among top level image categories. One machine acts as ＇Student （S）＇ with initially limited information and it endeavors to capture the task domain gradually by questioning its mentor on a pool of unlabeled data. The other machine is ＇Teacher （T）＇ with the implicit knowledge for helping S on learning the class models. T initiates relative attributes as a communication channel by randomly sorting the classes on attribute space in an unsupervised manner. S starts modeling the categories in this intermediate level by using only a limited number of labeled data. Thereafter, it first selects an entropy-based sample from the pool of unlabeled data and triggers the conversation by propagating the selected image with its belief class in a query. Since T already knows the ground truth labels, it not only decides whether the belief is true or false, but it also provides an attribute-based feedback to S in each case without revealing the true label of the query sample if the belief is false. So the number of training data is increased virtually by dropping the falsely predicted sample back into the unlabeled pool. Next, S updates the attribute space which, in fact, has an impact on T＇s future responses, and then the category models are updated concurrently for the next run. We experience the weakly supervised algorithm on the real world datasets of faces and natural scenes in comparison with direct attribute prediction and semi-supervised learning approaches, and a noteworthy performance increase is achieved.展开更多
Deep Convolution Neural Networks(DCNNs)can capture discriminative features from large datasets.However,how to incrementally learn new samples without forgetting old ones and recognize novel classes that arise in the d...Deep Convolution Neural Networks(DCNNs)can capture discriminative features from large datasets.However,how to incrementally learn new samples without forgetting old ones and recognize novel classes that arise in the dynamically changing world,e.g.,classifying newly discovered fish species,remains an open problem.We address an even more challenging and realistic setting of this problem where new class samples are insufficient,i.e.,Few-Shot Class-Incremental Learning(FSCIL).Current FSCIL methods augment the training data to alleviate the overfitting of novel classes.By contrast,we propose Filter Bank Networks(FBNs)that augment the learnable filters to capture fine-detailed features for adapting to future new classes.In the forward pass,FBNs augment each convolutional filter to a virtual filter bank containing the canonical one,i.e.,itself,and multiple transformed versions.During back-propagation,FBNs explicitly stimulate fine-detailed features to emerge and collectively align all gradients of each filter bank to learn the canonical one.FBNs capture pattern variants that do not yet exist in the pretraining session,thus making it easy to incorporate new classes in the incremental learning phase.Moreover,FBNs introduce model-level prior knowledge to efficiently utilize the limited few-shot data.Extensive experiments on MNIST,CIFAR100,CUB200,andMini-ImageNet datasets show that FBNs consistently outperformthe baseline by a significantmargin,reporting new state-of-the-art FSCIL results.In addition,we contribute a challenging FSCIL benchmark,Fishshot1K,which contains 8261 underwater images covering 1000 ocean fish species.The code is included in the supplementary materials.展开更多
This article presents hourly load forecasting by using an incremental learning model called Online Sequential Extreme Learning Machine(OSELM),which can learn and adapt automatically according to new arrival input.Howe...This article presents hourly load forecasting by using an incremental learning model called Online Sequential Extreme Learning Machine(OSELM),which can learn and adapt automatically according to new arrival input.However,the use of OS-ELM requires a sufficient amount of initial training sample data,which makes OS-ELM inoperable if sufficiently accurate sample data cannot be obtained.To solve this problem,a synthesis of the initial training sample is proposed.The synthesis of the initial sample is achieved by taking the first data received at the start of working and adding random noises to that data to create new and sufficient samples.Then the synthesis samples are used to initial train the OS-ELM.This proposed method is compared with Fully Online Extreme Learning Machine(FOS-ELM),which is an incremental learning model that also does not require the initial training samples.Both the proposed method and FOS-ELM are used for hourly load forecasting from the Hourly Energy Consumption dataset.Experiments have shown that the proposed method with a wide range of noise levels,can forecast hourly load more accurately than the FOS-ELM.展开更多
Background With the development of information technology,there is a significant increase in the number of network traffic logs mixed with various types of cyberattacks.Traditional intrusion detection systems(IDSs)are...Background With the development of information technology,there is a significant increase in the number of network traffic logs mixed with various types of cyberattacks.Traditional intrusion detection systems(IDSs)are limited in detecting new inconstant patterns and identifying malicious traffic traces in real time.Therefore,there is an urgent need to implement more effective intrusion detection technologies to protect computer security.Methods In this study,we designed a hybrid IDS by combining our incremental learning model(KANSOINN)and active learning to learn new log patterns and detect various network anomalies in real time.Conclusions Experimental results on the NSLKDD dataset showed that KAN-SOINN can be continuously improved and effectively detect malicious logs.Meanwhile,comparative experiments proved that using a hybrid query strategy in active learning can improve the model learning efficiency.展开更多
In this paper, an incremental learning model called Resource Allocating Network with Long-Term Memory (RAN-LTM) is extended such that the learning is conducted with some autonomy for the following functions: 1) data c...In this paper, an incremental learning model called Resource Allocating Network with Long-Term Memory (RAN-LTM) is extended such that the learning is conducted with some autonomy for the following functions: 1) data collection for initial learning, 2) data normalization, 3) addition of radial basis functions (RBFs), and 4) determination of RBF cen-ters and widths. The proposed learning algorithm called Autonomous Learning algorithm for Resource Allocating Network (AL-RAN) is divided into the two learning phases: initial learning phase and incremental learning phase. And the former is further divided into the autonomous data collection and the initial network learning. In the initial learning phase, training data are first collected until the class separability is converged or has a significant dif-ference between normalized and unnormalized data. Then, an initial structure of AL-RAN is autonomously determined by selecting a moderate number of RBF centers from the collected data and by defining as large RBF widths as possible within a proper range. After the initial learning, the incremental learning of AL-RAN is conducted in a sequential way whenever a new training data is given. In the experiments, we evaluate AL-RAN using five benchmark data sets. From the experimental results, we confirm that the above autonomous functions work well and the efficiency in terms of network structure and learning time is improved without sacrificing the recognition accuracy as compared with the previous version of AL-RAN.展开更多
At this current time,data stream classification plays a key role in big data analytics due to its enormous growth.Most of the existing classification methods used ensemble learning,which is trustworthy but these metho...At this current time,data stream classification plays a key role in big data analytics due to its enormous growth.Most of the existing classification methods used ensemble learning,which is trustworthy but these methods are not effective to face the issues of learning from imbalanced big data,it also supposes that all data are pre-classified.Another weakness of current methods is that it takes a long evaluation time when the target data stream contains a high number of features.The main objective of this research is to develop a new method for incremental learning based on the proposed ant lion fuzzy-generative adversarial network model.The proposed model is implemented in spark architecture.For each data stream,the class output is computed at slave nodes by training a generative adversarial network with the back propagation error based on fuzzy bound computation.This method overcomes the limitations of existing methods as it can classify data streams that are slightly or completely unlabeled data and providing high scalability and efficiency.The results show that the proposed model outperforms stateof-the-art performance in terms of accuracy(0.861)precision(0.9328)and minimal MSE(0.0416).展开更多
Recently, deep convolutional neural networks (DCNNs) have achieved remarkable results in image classification tasks. Despite convolutional networks’ great successes, their training process relies on a large amount of...Recently, deep convolutional neural networks (DCNNs) have achieved remarkable results in image classification tasks. Despite convolutional networks’ great successes, their training process relies on a large amount of data prepared in advance, which is often challenging in real-world applications, such as streaming data and concept drift. For this reason, incremental learning (continual learning) has attracted increasing attention from scholars. However, incremental learning is associated with the challenge of catastrophic forgetting: the performance on previous tasks drastically degrades after learning a new task. In this paper, we propose a new strategy to alleviate catastrophic forgetting when neural networks are trained in continual domains. Specifically, two components are applied: data translation based on transfer learning and knowledge distillation. The former translates a portion of new data to reconstruct the partial data distribution of the old domain. The latter uses an old model as a teacher to guide a new model. The experimental results on three datasets have shown that our work can effectively alleviate catastrophic forgetting by a combination of the two methods aforementioned.展开更多
Intrusion detection is a hot field in the direction of network security.Classical intrusion detection systems are usually based on supervised machine learning models.These offline-trained models usually have better pe...Intrusion detection is a hot field in the direction of network security.Classical intrusion detection systems are usually based on supervised machine learning models.These offline-trained models usually have better performance in the initial stages of system construction.However,due to the diversity and rapid development of intrusion techniques,the trained models are often difficult to detect new attacks.In addition,very little noisy data in the training process often has a considerable impact on the performance of the intrusion detection system.This paper proposes an intrusion detection system based on active incremental learning with the adaptive capability to solve these problems.IDS consists of two modules,namely the improved incremental stacking ensemble learning detection method called Multi-Stacking model and the active learning query module.The stacking model can cope well with concept drift due to the diversity and generalization selection of its base classifiers,but the accuracy does not meet the requirements.The Multi-Stacking model improves the accuracy of the model by adding a voting layer on the basis of the original stacking.The active learning query module improves the detection of known attacks through the committee algorithm,and the improved KNN algorithm can better help detect unknown attacks.We have tested the latest industrial IoT dataset with satisfactory results.展开更多
In recently proposed partial oblique projection (POP) learning, a function space is decomposed into two complementary subspaces, so that functions belonging to one of which can be optimally estimated. This paper shows...In recently proposed partial oblique projection (POP) learning, a function space is decomposed into two complementary subspaces, so that functions belonging to one of which can be optimally estimated. This paper shows that when the decomposition is specially performed so that the above subspace becomes the largest, a special learning called SPOP learning is obtained and correspondingly an incremental learning is implemented, result of which equals exactly to that of batch learning including novel data. The effectiveness of the method is illustrated by experimental results.展开更多
Humans are experiencing the inclusion of artificial agents in their lives,such as unmanned vehicles,service robots,voice assistants,and intelligent medical care.If the artificial agents cannot align with social values...Humans are experiencing the inclusion of artificial agents in their lives,such as unmanned vehicles,service robots,voice assistants,and intelligent medical care.If the artificial agents cannot align with social values or make ethical decisions,they may not meet the expectations of humans.Traditionally,an ethical decision-making framework is constructed by rule-based or statistical approaches.In this paper,we propose an ethical decision-making framework based on incremental ILP(Inductive Logic Programming),which can overcome the brittleness of rule-based approaches and little interpretability of statistical approaches.As the current incremental ILP makes it difficult to solve conflicts,we propose a novel ethical decision-making framework considering conflicts in this paper,which adopts our proposed incremental ILP system.The framework consists of two processes:the learning process and the deduction process.The first process records bottom clauses with their score functions and learns rules guided by the entailment and the score function.The second process obtains an ethical decision based on the rules.In an ethical scenario about chatbots for teenagers’mental health,we verify that our framework can learn ethical rules and make ethical decisions.Besides,we extract incremental ILP from the framework and compare it with the state-of-the-art ILP systems based on ASP(Answer Set Programming)focusing on conflict resolution.The results of comparisons show that our proposed system can generate better-quality rules than most other systems.展开更多
In scenarios where a large amount of data needs to be learned,incremental learning can make full use of old knowledge,signif-icantly reduce the computational cost of the overall learning process,and maintain high perf...In scenarios where a large amount of data needs to be learned,incremental learning can make full use of old knowledge,signif-icantly reduce the computational cost of the overall learning process,and maintain high performance.In this paper,taking the MaxCut problem as our example,we introduce the idea of incremental learning into quantum computing,and propose a Quantum Proactive Incremental Learning algorithm(QPIL).Instead of a one-off training of quantum circuit,QPIL contains a multi-phase training on gradually-increased subgraphs of all vertices,proactively reducing large-scale problems to smaller ones to solve in steps,providing an efficient solution for MaxCut.Specifically,some vertices and corresponding edges are randomly selected for training to obtain optimized parameters of the quantum circuit at first.Then,in each incremental phase,the remaining vertices and corresponding edges are gradually added and the parameters obtained from the previous phase are reused in the parameter initialization of the current phase.We perform experiments on 120 different small-scale graphs,and it shows that QPIL performs superior to prevalent quantum and classical baselines in terms of approximation ratio(AR),time cost,anti-forgetting,and solv-ing stability.In particular,QPIL’s AR surpasses 20%of mainstream quantum baselines,and the time cost is less than 1/5 of them.The idea of QPIL is expected to inspire efficient and high-quality solutions in large-scale MaxCut and other combinatorial optimization problems.展开更多
Fault diagnosis is an important measure to ensure the safety of production, and all kinds of fault diagnosis methods are of importance in actual production process. However, the complexity and uncertainty of productio...Fault diagnosis is an important measure to ensure the safety of production, and all kinds of fault diagnosis methods are of importance in actual production process. However, the complexity and uncertainty of production process often lead to the changes of data distribution and the emergence of new fault classes, and the number of the new fault classes is unpredictable. The reconstruction of the fault diagnosis model and the identification of new fault classes have become core issues under the circumstances. This paper presents a fault diagnosis method based on model transfer learning and the main contributions of the paper are as follows: 1) An incremental model transfer fault diagnosis method is proposed to reconstruct the new process diagnosis model. 2) Breaking the limit of existing method that the new process can only have one more class of faults than the old process, this method can identify M faults more in the new process with the thought of incremental learning. 3) The method offers a solution to a series of problems caused by the increase of fault classes. Experiments based on Tennessee-Eastman process and ore grinding classification process demonstrate the effectiveness and the feasibility of the method.展开更多
The scheme for tracking maneuvering target based on neural fuzzy network with incremental neural learning is proposed. When tracked target maneuver occurs, the scheme can detect maneuver immediately and estimate the m...The scheme for tracking maneuvering target based on neural fuzzy network with incremental neural learning is proposed. When tracked target maneuver occurs, the scheme can detect maneuver immediately and estimate the maneuver value accurately , then the tracking filter can be compensated correctly and duly by the estimated maneuver value. When environment changes, neural fuzzy network with incremental neural learning （INL-SONFIN） can find its optimal structure and parameters automatically to adopt to changed environment. So, it always produce estimated output very close to the true maneuver value that leads to good tracking performance and avoids misstracking. Simulation results show that the performance is superior to the traditional schemes and the scheme can fit changed dynamic environment to track maneuvering target accurately and duly.展开更多
基金the Deputyship for Research&Innovation,Ministry of Education in Saudi Arabia for funding this research work through project number RI-44-0833.
文摘The field of biometric identification has seen significant advancements over the years,with research focusing on enhancing the accuracy and security of these systems.One of the key developments is the integration of deep learning techniques in biometric systems.However,despite these advancements,certain challenges persist.One of the most significant challenges is scalability over growing complexity.Traditional methods either require maintaining and securing a growing database,introducing serious security challenges,or relying on retraining the entiremodelwhen new data is introduced-a process that can be computationally expensive and complex.This challenge underscores the need for more efficient methods to scale securely.To this end,we introduce a novel approach that addresses these challenges by integrating multimodal biometrics,cancelable biometrics,and incremental learning techniques.This work is among the first attempts to seamlessly incorporate deep cancelable biometrics with dynamic architectural updates,applied incrementally to the deep learning model as new users are enrolled,achieving high performance with minimal catastrophic forgetting.By leveraging a One-Dimensional Convolutional Neural Network(1D-CNN)architecture combined with a hybrid incremental learning approach,our system achieves high recognition accuracy,averaging 98.98% over incrementing datasets,while ensuring user privacy through cancelable templates generated via a pre-trained CNN model and random projection.The approach demonstrates remarkable adaptability,utilizing the least intrusive biometric traits like facial features and fingerprints,ensuring not only robust performance but also long-term serviceability.
文摘The visions of Industry 4.0 and 5.0 have reinforced the industrial environment.They have also made artificial intelligence incorporated as a major facilitator.Diagnosing machine faults has become a solid foundation for automatically recognizing machine failure,and thus timely maintenance can ensure safe operations.Transfer learning is a promising solution that can enhance the machine fault diagnosis model by borrowing pre-trained knowledge from the source model and applying it to the target model,which typically involves two datasets.In response to the availability of multiple datasets,this paper proposes using selective and adaptive incremental transfer learning(SA-ITL),which fuses three algorithms,namely,the hybrid selective algorithm,the transferability enhancement algorithm,and the incremental transfer learning algorithm.It is a selective algorithm that enables selecting and ordering appropriate datasets for transfer learning and selecting useful knowledge to avoid negative transfer.The algorithm also adaptively adjusts the portion of training data to balance the learning rate and training time.The proposed algorithm is evaluated and analyzed using ten benchmark datasets.Compared with other algorithms from existing works,SA-ITL improves the accuracy of all datasets.Ablation studies present the accuracy enhancements of the SA-ITL,including the hybrid selective algorithm(1.22%-3.82%),transferability enhancement algorithm(1.91%-4.15%),and incremental transfer learning algorithm(0.605%-2.68%).These also show the benefits of enhancing the target model with heterogeneous image datasets that widen the range of domain selection between source and target domains.
基金supported by the National Natural Science Key Foundation of China(69974021)
文摘A new incremental support vector machine （SVM） algorithm is proposed which is based on multiple kernel learning. Through introducing multiple kernel learning into the SVM incremental learning, large scale data set learning problem can be solved effectively. Furthermore, different punishments are adopted in allusion to the training subset and the acquired support vectors, which may help to improve the performance of SVM. Simulation results indicate that the proposed algorithm can not only solve the model selection problem in SVM incremental learning, but also improve the classification or prediction precision.
基金Supported by the National Natural Science Foundation of China(60939003)
文摘The maintenance of an aero-engine usually includes three levels,and the maintenance cost and period greatly differ depending on the different maintenance levels.To plan a reasonable maintenance budget program, airlines would like to predict the maintenance level of aero-engine before repairing in terms of performance parameters,which can provide more economic benefits.The maintenance level decision rules are mined using the historical maintenance data of a civil aero-engine based on the rough set theory,and a variety of possible models of updating rules produced by newly increased maintenance cases added to the historical maintenance case database are investigated by the means of incremental machine learning.The continuously updated rules can provide reasonable guidance suggestions for engineers and decision support for planning a maintenance budget program before repairing. The results of an example show that the decision rules become more typical and robust,and they are more accurate to predict the maintenance level of an aero-engine module as the maintenance data increase,which illustrates the feasibility of the represented method.
基金financially supported by the National Key R&D Program of China(No.2018YFA0702504)the National Natural Science Foundation of China(No.42174152 and No.41974140)+1 种基金the Science Foundation of China University of Petroleum,Beijing(No.2462020YXZZ008 and No.2462020QZDX003)the Strategic Cooperation Technology Projects of CNPC and CUPB(No.ZLZX2020-03).
文摘Intelligent seismic facies identification based on deep learning can alleviate the time-consuming and labor-intensive problem of manual interpretation,which has been widely applied.Supervised learning can realize facies identification with high efficiency and accuracy;however,it depends on the usage of a large amount of well-labeled data.To solve this issue,we propose herein an incremental semi-supervised method for intelligent facies identification.Our method considers the continuity of the lateral variation of strata and uses cosine similarity to quantify the similarity of the seismic data feature domain.The maximum-diff erence sample in the neighborhood of the currently used training data is then found to reasonably expand the training sets.This process continuously increases the amount of training data and learns its distribution.We integrate old knowledge while absorbing new ones to realize incremental semi-supervised learning and achieve the purpose of evolving the network models.In this work,accuracy and confusion matrix are employed to jointly control the predicted results of the model from both overall and partial aspects.The obtained values are then applied to a three-dimensional(3D)real dataset and used to quantitatively evaluate the results.Using unlabeled data,our proposed method acquires more accurate and stable testing results compared to conventional supervised learning algorithms that only use well-labeled data.A considerable improvement for small-sample categories is also observed.Using less than 1%of the training data,the proposed method can achieve an average accuracy of over 95%on the 3D dataset.In contrast,the conventional supervised learning algorithm achieved only approximately 85%.
基金Supported by the National Natural Science Foundation of China(No.61301245,U1533104)
文摘In order to improve the efficiency of learning the triangular membership functions( TMFs) for mining fuzzy association rule( FAR) in dynamic database,a single-pass fuzzy c means( SPFCM)algorithm is combined with the real-coded CHC genetic model to incrementally learn the TMFs. The cluster centers resulting from SPFCM are regarded as the midpoint of TMFs. The population of CHC is generated randomly according to the cluster center and constraint conditions among TMFs. Then a new population for incremental learning is composed of the excellent chromosomes stored in the first genetic process and the chromosomes generated based on the cluster center adjusted by SPFCM. The experiments on real datasets show that the number of generations converging to the solution of the proposed approach is less than that of the existing batch learning approach. The quality of TMFs generated by the approach is comparable to that of the batch learning approach. Compared with the existing incremental learning strategy,the proposed approach is superior in terms of the quality of TMFs and time cost.
文摘In this study, we propose an incremental learning approach based on a machine-machine interaction via relative attribute feedbacks that exploit comparative relationships among top level image categories. One machine acts as ＇Student （S）＇ with initially limited information and it endeavors to capture the task domain gradually by questioning its mentor on a pool of unlabeled data. The other machine is ＇Teacher （T）＇ with the implicit knowledge for helping S on learning the class models. T initiates relative attributes as a communication channel by randomly sorting the classes on attribute space in an unsupervised manner. S starts modeling the categories in this intermediate level by using only a limited number of labeled data. Thereafter, it first selects an entropy-based sample from the pool of unlabeled data and triggers the conversation by propagating the selected image with its belief class in a query. Since T already knows the ground truth labels, it not only decides whether the belief is true or false, but it also provides an attribute-based feedback to S in each case without revealing the true label of the query sample if the belief is false. So the number of training data is increased virtually by dropping the falsely predicted sample back into the unlabeled pool. Next, S updates the attribute space which, in fact, has an impact on T＇s future responses, and then the category models are updated concurrently for the next run. We experience the weakly supervised algorithm on the real world datasets of faces and natural scenes in comparison with direct attribute prediction and semi-supervised learning approaches, and a noteworthy performance increase is achieved.
基金support from the Strategic Priority Research Program of the Chinese Academy of Sciences under Grant No.XDA27000000.
文摘Deep Convolution Neural Networks(DCNNs)can capture discriminative features from large datasets.However,how to incrementally learn new samples without forgetting old ones and recognize novel classes that arise in the dynamically changing world,e.g.,classifying newly discovered fish species,remains an open problem.We address an even more challenging and realistic setting of this problem where new class samples are insufficient,i.e.,Few-Shot Class-Incremental Learning(FSCIL).Current FSCIL methods augment the training data to alleviate the overfitting of novel classes.By contrast,we propose Filter Bank Networks(FBNs)that augment the learnable filters to capture fine-detailed features for adapting to future new classes.In the forward pass,FBNs augment each convolutional filter to a virtual filter bank containing the canonical one,i.e.,itself,and multiple transformed versions.During back-propagation,FBNs explicitly stimulate fine-detailed features to emerge and collectively align all gradients of each filter bank to learn the canonical one.FBNs capture pattern variants that do not yet exist in the pretraining session,thus making it easy to incorporate new classes in the incremental learning phase.Moreover,FBNs introduce model-level prior knowledge to efficiently utilize the limited few-shot data.Extensive experiments on MNIST,CIFAR100,CUB200,andMini-ImageNet datasets show that FBNs consistently outperformthe baseline by a significantmargin,reporting new state-of-the-art FSCIL results.In addition,we contribute a challenging FSCIL benchmark,Fishshot1K,which contains 8261 underwater images covering 1000 ocean fish species.The code is included in the supplementary materials.
文摘This article presents hourly load forecasting by using an incremental learning model called Online Sequential Extreme Learning Machine(OSELM),which can learn and adapt automatically according to new arrival input.However,the use of OS-ELM requires a sufficient amount of initial training sample data,which makes OS-ELM inoperable if sufficiently accurate sample data cannot be obtained.To solve this problem,a synthesis of the initial training sample is proposed.The synthesis of the initial sample is achieved by taking the first data received at the start of working and adding random noises to that data to create new and sufficient samples.Then the synthesis samples are used to initial train the OS-ELM.This proposed method is compared with Fully Online Extreme Learning Machine(FOS-ELM),which is an incremental learning model that also does not require the initial training samples.Both the proposed method and FOS-ELM are used for hourly load forecasting from the Hourly Energy Consumption dataset.Experiments have shown that the proposed method with a wide range of noise levels,can forecast hourly load more accurately than the FOS-ELM.
基金Supported by SJTU-HUAWEI TECH Cybersecurity Innovation Lab。
文摘Background With the development of information technology,there is a significant increase in the number of network traffic logs mixed with various types of cyberattacks.Traditional intrusion detection systems(IDSs)are limited in detecting new inconstant patterns and identifying malicious traffic traces in real time.Therefore,there is an urgent need to implement more effective intrusion detection technologies to protect computer security.Methods In this study,we designed a hybrid IDS by combining our incremental learning model(KANSOINN)and active learning to learn new log patterns and detect various network anomalies in real time.Conclusions Experimental results on the NSLKDD dataset showed that KAN-SOINN can be continuously improved and effectively detect malicious logs.Meanwhile,comparative experiments proved that using a hybrid query strategy in active learning can improve the model learning efficiency.
文摘In this paper, an incremental learning model called Resource Allocating Network with Long-Term Memory (RAN-LTM) is extended such that the learning is conducted with some autonomy for the following functions: 1) data collection for initial learning, 2) data normalization, 3) addition of radial basis functions (RBFs), and 4) determination of RBF cen-ters and widths. The proposed learning algorithm called Autonomous Learning algorithm for Resource Allocating Network (AL-RAN) is divided into the two learning phases: initial learning phase and incremental learning phase. And the former is further divided into the autonomous data collection and the initial network learning. In the initial learning phase, training data are first collected until the class separability is converged or has a significant dif-ference between normalized and unnormalized data. Then, an initial structure of AL-RAN is autonomously determined by selecting a moderate number of RBF centers from the collected data and by defining as large RBF widths as possible within a proper range. After the initial learning, the incremental learning of AL-RAN is conducted in a sequential way whenever a new training data is given. In the experiments, we evaluate AL-RAN using five benchmark data sets. From the experimental results, we confirm that the above autonomous functions work well and the efficiency in terms of network structure and learning time is improved without sacrificing the recognition accuracy as compared with the previous version of AL-RAN.
基金Taif University Researchers Supporting Project Number(TURSP-2020/126),Taif University,Taif,Saudi Arabia.
文摘At this current time,data stream classification plays a key role in big data analytics due to its enormous growth.Most of the existing classification methods used ensemble learning,which is trustworthy but these methods are not effective to face the issues of learning from imbalanced big data,it also supposes that all data are pre-classified.Another weakness of current methods is that it takes a long evaluation time when the target data stream contains a high number of features.The main objective of this research is to develop a new method for incremental learning based on the proposed ant lion fuzzy-generative adversarial network model.The proposed model is implemented in spark architecture.For each data stream,the class output is computed at slave nodes by training a generative adversarial network with the back propagation error based on fuzzy bound computation.This method overcomes the limitations of existing methods as it can classify data streams that are slightly or completely unlabeled data and providing high scalability and efficiency.The results show that the proposed model outperforms stateof-the-art performance in terms of accuracy(0.861)precision(0.9328)and minimal MSE(0.0416).
文摘Recently, deep convolutional neural networks (DCNNs) have achieved remarkable results in image classification tasks. Despite convolutional networks’ great successes, their training process relies on a large amount of data prepared in advance, which is often challenging in real-world applications, such as streaming data and concept drift. For this reason, incremental learning (continual learning) has attracted increasing attention from scholars. However, incremental learning is associated with the challenge of catastrophic forgetting: the performance on previous tasks drastically degrades after learning a new task. In this paper, we propose a new strategy to alleviate catastrophic forgetting when neural networks are trained in continual domains. Specifically, two components are applied: data translation based on transfer learning and knowledge distillation. The former translates a portion of new data to reconstruct the partial data distribution of the old domain. The latter uses an old model as a teacher to guide a new model. The experimental results on three datasets have shown that our work can effectively alleviate catastrophic forgetting by a combination of the two methods aforementioned.
基金sponsored by the National Natural Science Foundation of China under Grants 62271264,61972207,and 42175194the Project through the Priority Academic Program Development(PAPD)of Jiangsu Higher Education Institution.
文摘Intrusion detection is a hot field in the direction of network security.Classical intrusion detection systems are usually based on supervised machine learning models.These offline-trained models usually have better performance in the initial stages of system construction.However,due to the diversity and rapid development of intrusion techniques,the trained models are often difficult to detect new attacks.In addition,very little noisy data in the training process often has a considerable impact on the performance of the intrusion detection system.This paper proposes an intrusion detection system based on active incremental learning with the adaptive capability to solve these problems.IDS consists of two modules,namely the improved incremental stacking ensemble learning detection method called Multi-Stacking model and the active learning query module.The stacking model can cope well with concept drift due to the diversity and generalization selection of its base classifiers,but the accuracy does not meet the requirements.The Multi-Stacking model improves the accuracy of the model by adding a voting layer on the basis of the original stacking.The active learning query module improves the detection of known attacks through the committee algorithm,and the improved KNN algorithm can better help detect unknown attacks.We have tested the latest industrial IoT dataset with satisfactory results.
文摘In recently proposed partial oblique projection (POP) learning, a function space is decomposed into two complementary subspaces, so that functions belonging to one of which can be optimally estimated. This paper shows that when the decomposition is specially performed so that the above subspace becomes the largest, a special learning called SPOP learning is obtained and correspondingly an incremental learning is implemented, result of which equals exactly to that of batch learning including novel data. The effectiveness of the method is illustrated by experimental results.
基金This work was funded by the National Natural Science Foundation of China Nos.U22A2099,61966009,62006057the Graduate Innovation Program No.YCSW2022286.
文摘Humans are experiencing the inclusion of artificial agents in their lives,such as unmanned vehicles,service robots,voice assistants,and intelligent medical care.If the artificial agents cannot align with social values or make ethical decisions,they may not meet the expectations of humans.Traditionally,an ethical decision-making framework is constructed by rule-based or statistical approaches.In this paper,we propose an ethical decision-making framework based on incremental ILP(Inductive Logic Programming),which can overcome the brittleness of rule-based approaches and little interpretability of statistical approaches.As the current incremental ILP makes it difficult to solve conflicts,we propose a novel ethical decision-making framework considering conflicts in this paper,which adopts our proposed incremental ILP system.The framework consists of two processes:the learning process and the deduction process.The first process records bottom clauses with their score functions and learns rules guided by the entailment and the score function.The second process obtains an ethical decision based on the rules.In an ethical scenario about chatbots for teenagers’mental health,we verify that our framework can learn ethical rules and make ethical decisions.Besides,we extract incremental ILP from the framework and compare it with the state-of-the-art ILP systems based on ASP(Answer Set Programming)focusing on conflict resolution.The results of comparisons show that our proposed system can generate better-quality rules than most other systems.
基金supported by the National Natural Science Foundation of China(Grant Nos.62272048,61976056,and 62371069)BUPT Excellent Ph.D.Students Foundation(Grant No.CX20241055)。
文摘In scenarios where a large amount of data needs to be learned,incremental learning can make full use of old knowledge,signif-icantly reduce the computational cost of the overall learning process,and maintain high performance.In this paper,taking the MaxCut problem as our example,we introduce the idea of incremental learning into quantum computing,and propose a Quantum Proactive Incremental Learning algorithm(QPIL).Instead of a one-off training of quantum circuit,QPIL contains a multi-phase training on gradually-increased subgraphs of all vertices,proactively reducing large-scale problems to smaller ones to solve in steps,providing an efficient solution for MaxCut.Specifically,some vertices and corresponding edges are randomly selected for training to obtain optimized parameters of the quantum circuit at first.Then,in each incremental phase,the remaining vertices and corresponding edges are gradually added and the parameters obtained from the previous phase are reused in the parameter initialization of the current phase.We perform experiments on 120 different small-scale graphs,and it shows that QPIL performs superior to prevalent quantum and classical baselines in terms of approximation ratio(AR),time cost,anti-forgetting,and solv-ing stability.In particular,QPIL’s AR surpasses 20%of mainstream quantum baselines,and the time cost is less than 1/5 of them.The idea of QPIL is expected to inspire efficient and high-quality solutions in large-scale MaxCut and other combinatorial optimization problems.
文摘Fault diagnosis is an important measure to ensure the safety of production, and all kinds of fault diagnosis methods are of importance in actual production process. However, the complexity and uncertainty of production process often lead to the changes of data distribution and the emergence of new fault classes, and the number of the new fault classes is unpredictable. The reconstruction of the fault diagnosis model and the identification of new fault classes have become core issues under the circumstances. This paper presents a fault diagnosis method based on model transfer learning and the main contributions of the paper are as follows: 1) An incremental model transfer fault diagnosis method is proposed to reconstruct the new process diagnosis model. 2) Breaking the limit of existing method that the new process can only have one more class of faults than the old process, this method can identify M faults more in the new process with the thought of incremental learning. 3) The method offers a solution to a series of problems caused by the increase of fault classes. Experiments based on Tennessee-Eastman process and ore grinding classification process demonstrate the effectiveness and the feasibility of the method.
基金This project was supported by Spaceflight Support Fund ( HIT01) and the Spaceflight Science Project Group
文摘The scheme for tracking maneuvering target based on neural fuzzy network with incremental neural learning is proposed. When tracked target maneuver occurs, the scheme can detect maneuver immediately and estimate the maneuver value accurately , then the tracking filter can be compensated correctly and duly by the estimated maneuver value. When environment changes, neural fuzzy network with incremental neural learning （INL-SONFIN） can find its optimal structure and parameters automatically to adopt to changed environment. So, it always produce estimated output very close to the true maneuver value that leads to good tracking performance and avoids misstracking. Simulation results show that the performance is superior to the traditional schemes and the scheme can fit changed dynamic environment to track maneuvering target accurately and duly.
