This paper presents a two-phase genetic algorithm (TPGA) based on the multi- parent genetic algorithm (MPGA). Through analysis we find MPGA will lead the population' s evol vement to diversity or convergence accor...This paper presents a two-phase genetic algorithm (TPGA) based on the multi- parent genetic algorithm (MPGA). Through analysis we find MPGA will lead the population' s evol vement to diversity or convergence according to the population size and the crossover size, so we make it run in different forms during the global and local optimization phases and then forms TPGA. The experiment results show that TPGA is very efficient for the optimization of low-dimension multi-modal functions, usually we can obtain all the global optimal solutions.展开更多
In this paper, a new algorithm for solving multi-modal function optimization problems-two-level subspace evolutionary algorithm is proposed. In the first level, the improved GT algorithm is used to do global recombina...In this paper, a new algorithm for solving multi-modal function optimization problems-two-level subspace evolutionary algorithm is proposed. In the first level, the improved GT algorithm is used to do global recombination search so that the whole population can be separated into several niches according to the position of solutions; then, in the second level, the niche evolutionary strategy is used for local search in the subspaces gotten in the first level till solutions of the problem are found. The new algorithm has been tested on some hard problems and some good results are obtained.展开更多
A novel immune genetic algorithm with the elitist selection and elitist crossover was proposed, which is called the immune genetic algorithm with the elitism （IGAE）. In IGAE, the new methods for computing antibody s...A novel immune genetic algorithm with the elitist selection and elitist crossover was proposed, which is called the immune genetic algorithm with the elitism （IGAE）. In IGAE, the new methods for computing antibody similarity, expected reproduction probability, and clonal selection probability were given. IGAE has three features. The first is that the similarities of two antibodies in structure and quality are all defined in the form of percentage, which helps to describe the similarity of two antibodies more accurately and to reduce the computational burden effectively. The second is that with the elitist selection and elitist crossover strategy IGAE is able to find the globally optimal solution of a given problem. The third is that the formula of expected reproduction probability of antibody can be adjusted through a parameter r, which helps to balance the population diversity and the convergence speed of IGAE so that IGAE can find the globally optimal solution of a given problem more rapidly. Two different complex multi-modal functions were selected to test the validity of IGAE. The experimental results show that IGAE can find the globally maximum/minimum values of the two functions rapidly. The experimental results also confirm that IGAE is of better performance in convergence speed, solution variation behavior, and computational efficiency compared with the canonical genetic algorithm with the elitism and the immune genetic algorithm with the information entropy and elitism.展开更多
Multi-modal knowledge graph completion(MMKGC)aims to complete missing entities or relations in multi-modal knowledge graphs,thereby discovering more previously unknown triples.Due to the continuous growth of data and ...Multi-modal knowledge graph completion(MMKGC)aims to complete missing entities or relations in multi-modal knowledge graphs,thereby discovering more previously unknown triples.Due to the continuous growth of data and knowledge and the limitations of data sources,the visual knowledge within the knowledge graphs is generally of low quality,and some entities suffer from the issue of missing visual modality.Nevertheless,previous studies of MMKGC have primarily focused on how to facilitate modality interaction and fusion while neglecting the problems of low modality quality and modality missing.In this case,mainstream MMKGC models only use pre-trained visual encoders to extract features and transfer the semantic information to the joint embeddings through modal fusion,which inevitably suffers from problems such as error propagation and increased uncertainty.To address these problems,we propose a Multi-modal knowledge graph Completion model based on Super-resolution and Detailed Description Generation(MMCSD).Specifically,we leverage a pre-trained residual network to enhance the resolution and improve the quality of the visual modality.Moreover,we design multi-level visual semantic extraction and entity description generation,thereby further extracting entity semantics from structural triples and visual images.Meanwhile,we train a variational multi-modal auto-encoder and utilize a pre-trained multi-modal language model to complement the missing visual features.We conducted experiments on FB15K-237 and DB13K,and the results showed that MMCSD can effectively perform MMKGC and achieve state-of-the-art performance.展开更多
BACKGROUND Stress ulcers are common complications in critically ill patients,with a higher incidence observed in older patients following gastrointestinal surgery.This study aimed to develop and evaluate the effective...BACKGROUND Stress ulcers are common complications in critically ill patients,with a higher incidence observed in older patients following gastrointestinal surgery.This study aimed to develop and evaluate the effectiveness of a multi-modal intervention protocol to prevent stress ulcers in this high-risk population.AIM To assess the impact of a multi-modal intervention on preventing stress ulcers in older intensive care unit(ICU)patients postoperatively.METHODS A randomized controlled trial involving critically ill patients(aged≥65 years)admitted to the ICU after gastrointestinal surgery was conducted.Patients were randomly assigned to either the intervention group,which received a multimodal stress ulcer prevention protocol,or the control group,which received standard care.The primary outcome measure was the incidence of stress ulcers.The secondary outcomes included ulcer healing time,complication rates,and length of hospital stay.RESULTS A total of 200 patients(100 in each group)were included in this study.The intervention group exhibited a significantly lower incidence of stress ulcers than the control group(15%vs 30%,P<0.01).Additionally,the intervention group demonstrated shorter ulcer healing times(mean 5.2 vs 7.8 days,P<0.05),lower complication rates(10%vs 22%,P<0.05),and reduced length of hospital stay(mean 12.3 vs 15.7 days,P<0.05).CONCLUSION This multi-modal intervention protocol significantly reduced the incidence of stress ulcers and improved clinical outcomes in critically ill older patients after gastrointestinal surgery.This comprehensive approach may provide a valuable strategy for managing high-risk populations in intensive care settings.展开更多
Objective:To explore the effectiveness of multi-modal teaching based on an online case library in the education of gene methylation combined with spiral computed tomography(CT)screening for pulmonary ground-glass opac...Objective:To explore the effectiveness of multi-modal teaching based on an online case library in the education of gene methylation combined with spiral computed tomography(CT)screening for pulmonary ground-glass opacity(GGO)nodules.Methods:From October 2023 to April 2024,66 medical imaging students were selected and randomly divided into a control group and an observation group,each with 33 students.The control group received traditional lecture-based teaching,while the observation group was taught using a multi-modal teaching approach based on an online case library.Performance on assessments and teaching quality were analyzed between the two groups.Results:The observation group achieved higher scores in theoretical and practical knowledge compared to the control group(P<0.05).Additionally,the teaching quality scores were significantly higher in the observation group(P<0.05).Conclusion:Implementing multi-modal teaching based on an online case library for pulmonary GGO nodule screening with gene methylation combined with spiral CT can enhance students’knowledge acquisition,improve teaching quality,and have significant clinical application value.展开更多
[Objective]Accurate prediction of tomato growth height is crucial for optimizing production environments in smart farming.However,current prediction methods predominantly rely on empirical,mechanistic,or learning-base...[Objective]Accurate prediction of tomato growth height is crucial for optimizing production environments in smart farming.However,current prediction methods predominantly rely on empirical,mechanistic,or learning-based models that utilize either images data or environmental data.These methods fail to fully leverage multi-modal data to capture the diverse aspects of plant growth comprehensively.[Methods]To address this limitation,a two-stage phenotypic feature extraction(PFE)model based on deep learning algorithm of recurrent neural network(RNN)and long short-term memory(LSTM)was developed.The model integrated environment and plant information to provide a holistic understanding of the growth process,emploied phenotypic and temporal feature extractors to comprehensively capture both types of features,enabled a deeper understanding of the interaction between tomato plants and their environment,ultimately leading to highly accurate predictions of growth height.[Results and Discussions]The experimental results showed the model's ef‐fectiveness:When predicting the next two days based on the past five days,the PFE-based RNN and LSTM models achieved mean absolute percentage error(MAPE)of 0.81%and 0.40%,respectively,which were significantly lower than the 8.00%MAPE of the large language model(LLM)and 6.72%MAPE of the Transformer-based model.In longer-term predictions,the 10-day prediction for 4 days ahead and the 30-day prediction for 12 days ahead,the PFE-RNN model continued to outperform the other two baseline models,with MAPE of 2.66%and 14.05%,respectively.[Conclusions]The proposed method,which leverages phenotypic-temporal collaboration,shows great potential for intelligent,data-driven management of tomato cultivation,making it a promising approach for enhancing the efficiency and precision of smart tomato planting management.展开更多
Multi-modal fusion technology gradually become a fundamental task in many fields,such as autonomous driving,smart healthcare,sentiment analysis,and human-computer interaction.It is rapidly becoming the dominant resear...Multi-modal fusion technology gradually become a fundamental task in many fields,such as autonomous driving,smart healthcare,sentiment analysis,and human-computer interaction.It is rapidly becoming the dominant research due to its powerful perception and judgment capabilities.Under complex scenes,multi-modal fusion technology utilizes the complementary characteristics of multiple data streams to fuse different data types and achieve more accurate predictions.However,achieving outstanding performance is challenging because of equipment performance limitations,missing information,and data noise.This paper comprehensively reviews existing methods based onmulti-modal fusion techniques and completes a detailed and in-depth analysis.According to the data fusion stage,multi-modal fusion has four primary methods:early fusion,deep fusion,late fusion,and hybrid fusion.The paper surveys the three majormulti-modal fusion technologies that can significantly enhance the effect of data fusion and further explore the applications of multi-modal fusion technology in various fields.Finally,it discusses the challenges and explores potential research opportunities.Multi-modal tasks still need intensive study because of data heterogeneity and quality.Preserving complementary information and eliminating redundant information between modalities is critical in multi-modal technology.Invalid data fusion methods may introduce extra noise and lead to worse results.This paper provides a comprehensive and detailed summary in response to these challenges.展开更多
Fusing hand-based features in multi-modal biometric recognition enhances anti-spoofing capabilities.Additionally,it leverages inter-modal correlation to enhance recognition performance.Concurrently,the robustness and ...Fusing hand-based features in multi-modal biometric recognition enhances anti-spoofing capabilities.Additionally,it leverages inter-modal correlation to enhance recognition performance.Concurrently,the robustness and recognition performance of the system can be enhanced through judiciously leveraging the correlation among multimodal features.Nevertheless,two issues persist in multi-modal feature fusion recognition:Firstly,the enhancement of recognition performance in fusion recognition has not comprehensively considered the inter-modality correlations among distinct modalities.Secondly,during modal fusion,improper weight selection diminishes the salience of crucial modal features,thereby diminishing the overall recognition performance.To address these two issues,we introduce an enhanced DenseNet multimodal recognition network founded on feature-level fusion.The information from the three modalities is fused akin to RGB,and the input network augments the correlation between modes through channel correlation.Within the enhanced DenseNet network,the Efficient Channel Attention Network(ECA-Net)dynamically adjusts the weight of each channel to amplify the salience of crucial information in each modal feature.Depthwise separable convolution markedly reduces the training parameters and further enhances the feature correlation.Experimental evaluations were conducted on four multimodal databases,comprising six unimodal databases,including multispectral palmprint and palm vein databases from the Chinese Academy of Sciences.The Equal Error Rates(EER)values were 0.0149%,0.0150%,0.0099%,and 0.0050%,correspondingly.In comparison to other network methods for palmprint,palm vein,and finger vein fusion recognition,this approach substantially enhances recognition performance,rendering it suitable for high-security environments with practical applicability.The experiments in this article utilized amodest sample database comprising 200 individuals.The subsequent phase involves preparing for the extension of the method to larger databases.展开更多
Predicting the motion of other road agents enables autonomous vehicles to perform safe and efficient path planning.This task is very complex,as the behaviour of road agents depends on many factors and the number of po...Predicting the motion of other road agents enables autonomous vehicles to perform safe and efficient path planning.This task is very complex,as the behaviour of road agents depends on many factors and the number of possible future trajectories can be consid-erable(multi-modal).Most prior approaches proposed to address multi-modal motion prediction are based on complex machine learning systems that have limited interpret-ability.Moreover,the metrics used in current benchmarks do not evaluate all aspects of the problem,such as the diversity and admissibility of the output.The authors aim to advance towards the design of trustworthy motion prediction systems,based on some of the re-quirements for the design of Trustworthy Artificial Intelligence.The focus is on evaluation criteria,robustness,and interpretability of outputs.First,the evaluation metrics are comprehensively analysed,the main gaps of current benchmarks are identified,and a new holistic evaluation framework is proposed.Then,a method for the assessment of spatial and temporal robustness is introduced by simulating noise in the perception system.To enhance the interpretability of the outputs and generate more balanced results in the proposed evaluation framework,an intent prediction layer that can be attached to multi-modal motion prediction models is proposed.The effectiveness of this approach is assessed through a survey that explores different elements in the visualisation of the multi-modal trajectories and intentions.The proposed approach and findings make a significant contribution to the development of trustworthy motion prediction systems for autono-mous vehicles,advancing the field towards greater safety and reliability.展开更多
Mill vibration is a common problem in rolling production,which directly affects the thickness accuracy of the strip and may even lead to strip fracture accidents in serious cases.The existing vibration prediction mode...Mill vibration is a common problem in rolling production,which directly affects the thickness accuracy of the strip and may even lead to strip fracture accidents in serious cases.The existing vibration prediction models do not consider the features contained in the data,resulting in limited improvement of model accuracy.To address these challenges,this paper proposes a multi-dimensional multi-modal cold rolling vibration time series prediction model(MDMMVPM)based on the deep fusion of multi-level networks.In the model,the long-term and short-term modal features of multi-dimensional data are considered,and the appropriate prediction algorithms are selected for different data features.Based on the established prediction model,the effects of tension and rolling force on mill vibration are analyzed.Taking the 5th stand of a cold mill in a steel mill as the research object,the innovative model is applied to predict the mill vibration for the first time.The experimental results show that the correlation coefficient(R^(2))of the model proposed in this paper is 92.5%,and the root-mean-square error(RMSE)is 0.0011,which significantly improves the modeling accuracy compared with the existing models.The proposed model is also suitable for the hot rolling process,which provides a new method for the prediction of strip rolling vibration.展开更多
Intelligent personal assistants play a pivotal role in in-vehicle systems,significantly enhancing life efficiency,driving safety,and decision-making support.In this study,the multi-modal design elements of intelligent...Intelligent personal assistants play a pivotal role in in-vehicle systems,significantly enhancing life efficiency,driving safety,and decision-making support.In this study,the multi-modal design elements of intelligent personal assistants within the context of visual,auditory,and somatosensory interactions with drivers were discussed.Their impact on the driver’s psychological state through various modes such as visual imagery,voice interaction,and gesture interaction were explored.The study also introduced innovative designs for in-vehicle intelligent personal assistants,incorporating design principles such as driver-centricity,prioritizing passenger safety,and utilizing timely feedback as a criterion.Additionally,the study employed design methods like driver behavior research and driving situation analysis to enhance the emotional connection between drivers and their vehicles,ultimately improving driver satisfaction and trust.展开更多
Media convergence works by processing information from different modalities and applying them to different domains.It is difficult for the conventional knowledge graph to utilise multi-media features because the intro...Media convergence works by processing information from different modalities and applying them to different domains.It is difficult for the conventional knowledge graph to utilise multi-media features because the introduction of a large amount of information from other modalities reduces the effectiveness of representation learning and makes knowledge graph inference less effective.To address the issue,an inference method based on Media Convergence and Rule-guided Joint Inference model(MCRJI)has been pro-posed.The authors not only converge multi-media features of entities but also introduce logic rules to improve the accuracy and interpretability of link prediction.First,a multi-headed self-attention approach is used to obtain the attention of different media features of entities during semantic synthesis.Second,logic rules of different lengths are mined from knowledge graph to learn new entity representations.Finally,knowledge graph inference is performed based on representing entities that converge multi-media features.Numerous experimental results show that MCRJI outperforms other advanced baselines in using multi-media features and knowledge graph inference,demonstrating that MCRJI provides an excellent approach for knowledge graph inference with converged multi-media features.展开更多
Multi-modal 3D object detection has achieved remarkable progress,but it is often limited in practical industrial production because of its high cost and low efficiency.The multi-view camera-based method provides a fea...Multi-modal 3D object detection has achieved remarkable progress,but it is often limited in practical industrial production because of its high cost and low efficiency.The multi-view camera-based method provides a feasible solution due to its low cost.However,camera data lacks geometric depth,and only using camera data to obtain high accuracy is challenging.This paper proposes a multi-modal Bird-Eye-View(BEV)distillation framework(MMDistill)to make a trade-off between them.MMDistill is a carefully crafted two-stage distillation framework based on teacher and student models for learning cross-modal knowledge and generating multi-modal features.It can improve the performance of unimodal detectors without introducing additional costs during inference.Specifically,our method can effectively solve the cross-gap caused by the heterogeneity between data.Furthermore,we further propose a Light Detection and Ranging(LiDAR)-guided geometric compensation module,which can assist the student model in obtaining effective geometric features and reduce the gap between different modalities.Our proposed method generally requires fewer computational resources and faster inference speed than traditional multi-modal models.This advancement enables multi-modal technology to be applied more widely in practical scenarios.Through experiments,we validate the effectiveness and superiority of MMDistill on the nuScenes dataset,achieving an improvement of 4.1%mean Average Precision(mAP)and 4.6%NuScenes Detection Score(NDS)over the baseline detector.In addition,we also present detailed ablation studies to validate our method.展开更多
Recently,there have been significant advancements in the study of semantic communication in single-modal scenarios.However,the ability to process information in multi-modal environments remains limited.Inspired by the...Recently,there have been significant advancements in the study of semantic communication in single-modal scenarios.However,the ability to process information in multi-modal environments remains limited.Inspired by the research and applications of natural language processing across different modalities,our goal is to accurately extract frame-level semantic information from videos and ultimately transmit high-quality videos.Specifically,we propose a deep learning-basedMulti-ModalMutual Enhancement Video Semantic Communication system,called M3E-VSC.Built upon a VectorQuantized Generative AdversarialNetwork(VQGAN),our systemaims to leverage mutual enhancement among different modalities by using text as the main carrier of transmission.With it,the semantic information can be extracted fromkey-frame images and audio of the video and performdifferential value to ensure that the extracted text conveys accurate semantic information with fewer bits,thus improving the capacity of the system.Furthermore,a multi-frame semantic detection module is designed to facilitate semantic transitions during video generation.Simulation results demonstrate that our proposed model maintains high robustness in complex noise environments,particularly in low signal-to-noise ratio conditions,significantly improving the accuracy and speed of semantic transmission in video communication by approximately 50 percent.展开更多
Traumatic brain injury involves complex pathophysiological mechanisms,among which oxidative stress significantly contributes to the occurrence of secondary injury.In this study,we evaluated hypidone hydrochloride(YL-0...Traumatic brain injury involves complex pathophysiological mechanisms,among which oxidative stress significantly contributes to the occurrence of secondary injury.In this study,we evaluated hypidone hydrochloride(YL-0919),a self-developed antidepressant with selective sigma-1 receptor agonist properties,and its associated mechanisms and targets in traumatic brain injury.Behavioral experiments to assess functional deficits were followed by assessment of neuronal damage through histological analyses and examination of blood-brain barrier permeability and brain edema.Next,we investigated the antioxidative effects of YL-0919 by assessing the levels of traditional markers of oxidative stress in vivo in mice and in vitro in HT22 cells.Finally,the targeted action of YL-0919 was verified by employing a sigma-1 receptor antagonist(BD-1047).Our findings demonstrated that YL-0919 markedly improved deficits in motor function and spatial cognition on day 3 post traumatic brain injury,while also decreasing neuronal mortality and reversing blood-brain barrier disruption and brain edema.Furthermore,YL-0919 effectively combated oxidative stress both in vivo and in vitro.The protective effects of YL-0919 were partially inhibited by BD-1047.These results indicated that YL-0919 relieved impairments in motor and spatial cognition by restraining oxidative stress,a neuroprotective effect that was partially reversed by the sigma-1 receptor antagonist BD-1047.YL-0919 may have potential as a new treatment for traumatic brain injury.展开更多
BACKGROUND At present,the influencing factors of social function in patients with residual depressive symptoms are still unclear.Residual depressive symptoms are highly harmful,leading to low mood in patients,affectin...BACKGROUND At present,the influencing factors of social function in patients with residual depressive symptoms are still unclear.Residual depressive symptoms are highly harmful,leading to low mood in patients,affecting work and interpersonal communication,increasing the risk of recurrence,and adding to the burden on families.Studying the influencing factors of their social function is of great significance.AIM To explore the social function score and its influencing factors in patients with residual depressive symptoms.METHODS This observational study surveyed patients with residual depressive symptoms(case group)and healthy patients undergoing physical examinations(control group).Participants were admitted between January 2022 and December 2023.Social functioning was assessed using the Sheehan Disability Scale(SDS),and scores were compared between groups.Factors influencing SDS scores in patients with residual depressive symptoms were analyzed by applying multiple linear regression while using the receiver operating characteristic curve,and these RESULTS The SDS scores of the 158 patients with depressive symptoms were 11.48±3.26.Compared with the control group,the SDS scores and all items in the case group were higher.SDS scores were higher in patients with relapse,discon-tinuous medication,drug therapy alone,severe somatic symptoms,obvious residual symptoms,and anxiety scores≥8.Disease history,medication compliance,therapy method,and residual symptoms correlated positively with SDS scores(r=0.354,0.414,0.602,and 0.456,respectively).Independent influencing factors included disease history,medication compliance,therapy method,somatic symptoms,residual symptoms,and anxiety scores(P<0.05).The areas under the curve for predicting social functional impairment using these factors were 0.713,0.559,0.684,0.729,0.668,and 0.628,respectively,with sensitivities of 79.2%,61.8%,76.8%,81.7%,63.6%,and 65.5%and specificities of 83.3%,87.5%,82.6%,83.3%,86.7%,and 92.1%,respectively.CONCLUSION The social function scores of patients with residual symptoms of depression are high.They are affected by disease history,medication compliance,therapy method,degree of somatic symptoms,residual symptoms,and anxiety.展开更多
The unsupervised multi-modal image translation is an emerging domain of computer vision whose goal is to transform an image from the source domain into many diverse styles in the target domain.However,the multi-genera...The unsupervised multi-modal image translation is an emerging domain of computer vision whose goal is to transform an image from the source domain into many diverse styles in the target domain.However,the multi-generator mechanism is employed among the advanced approaches available to model different domain mappings,which results in inefficient training of neural networks and pattern collapse,leading to inefficient generation of image diversity.To address this issue,this paper introduces a multi-modal unsupervised image translation framework that uses a generator to perform multi-modal image translation.Specifically,firstly,the domain code is introduced in this paper to explicitly control the different generation tasks.Secondly,this paper brings in the squeeze-and-excitation(SE)mechanism and feature attention(FA)module.Finally,the model integrates multiple optimization objectives to ensure efficient multi-modal translation.This paper performs qualitative and quantitative experiments on multiple non-paired benchmark image translation datasets while demonstrating the benefits of the proposed method over existing technologies.Overall,experimental results have shown that the proposed method is versatile and scalable.展开更多
Very long chain-saturated and-polyunsaturated fatty acids(VLC-SFA and VLC-PUFA, respectively) are a functionally important class of fatty acids containing 28 carbons or more in their acyl chain. They are synthesized b...Very long chain-saturated and-polyunsaturated fatty acids(VLC-SFA and VLC-PUFA, respectively) are a functionally important class of fatty acids containing 28 carbons or more in their acyl chain. They are synthesized by the elongation of very long fatty acids-4(ELOVL4) enzyme, expressed mainly in the brain, retina, skin, testes, and meibomian gland, where these fatty acids are found(Agbaga et al., 2008). Further, these organs exhibit tissuespecific VLC-PUFA and VLC-SFA biosynthesis and incorporation into complex lipids for specific functions. In the brain, skin, and Meibomian glands, the ELOVL4 mainly makes VLC-SFA, which are incorporated into complex sphingolipids. In the retina, the ELOVL4 makes VLC-PUFA that are incorporated into phosphatidylcholine, that are critical for visual function, while in testes and sperm, the VLC-PUFA are incorporated into sphingolipids that are critical for fertility(Yeboah et al., 2021).展开更多
Mutations in the protocadherin-19(PCDH19)gene(Xq22.1)cause the X-linked syndrome known as developmental and epileptic encephalopathy 9(DEE9,OMIM#300088)(Dibbens et al.,2008).DEE9 is characterized by early-onset cluste...Mutations in the protocadherin-19(PCDH19)gene(Xq22.1)cause the X-linked syndrome known as developmental and epileptic encephalopathy 9(DEE9,OMIM#300088)(Dibbens et al.,2008).DEE9 is characterized by early-onset clustering epilepsy associated with intellectual disability ranging from mild to profound,autism spectrum disorder,and other neuropsychiatric features including schizophrenia,anxiety,attentiondeficit/hyperactivity,and obsessive or aggressive behaviors.While seizures may become less frequent in adolescence,psychiatric comorbidities persist and often worsen with age(Dibbens et al.,2008;Kolc et al.,2020).展开更多
基金Supported by the National Natural Science Foundation of China (70071042,60073043,60133010)
文摘This paper presents a two-phase genetic algorithm (TPGA) based on the multi- parent genetic algorithm (MPGA). Through analysis we find MPGA will lead the population' s evol vement to diversity or convergence according to the population size and the crossover size, so we make it run in different forms during the global and local optimization phases and then forms TPGA. The experiment results show that TPGA is very efficient for the optimization of low-dimension multi-modal functions, usually we can obtain all the global optimal solutions.
基金Supported by the National Natural Science Foundation of China (70071042,60073043,60133010)
文摘In this paper, a new algorithm for solving multi-modal function optimization problems-two-level subspace evolutionary algorithm is proposed. In the first level, the improved GT algorithm is used to do global recombination search so that the whole population can be separated into several niches according to the position of solutions; then, in the second level, the niche evolutionary strategy is used for local search in the subspaces gotten in the first level till solutions of the problem are found. The new algorithm has been tested on some hard problems and some good results are obtained.
基金Project(50275150) supported by the National Natural Science Foundation of ChinaProjects(20040533035, 20070533131) supported by the National Research Foundation for the Doctoral Program of Higher Education of China
文摘A novel immune genetic algorithm with the elitist selection and elitist crossover was proposed, which is called the immune genetic algorithm with the elitism （IGAE）. In IGAE, the new methods for computing antibody similarity, expected reproduction probability, and clonal selection probability were given. IGAE has three features. The first is that the similarities of two antibodies in structure and quality are all defined in the form of percentage, which helps to describe the similarity of two antibodies more accurately and to reduce the computational burden effectively. The second is that with the elitist selection and elitist crossover strategy IGAE is able to find the globally optimal solution of a given problem. The third is that the formula of expected reproduction probability of antibody can be adjusted through a parameter r, which helps to balance the population diversity and the convergence speed of IGAE so that IGAE can find the globally optimal solution of a given problem more rapidly. Two different complex multi-modal functions were selected to test the validity of IGAE. The experimental results show that IGAE can find the globally maximum/minimum values of the two functions rapidly. The experimental results also confirm that IGAE is of better performance in convergence speed, solution variation behavior, and computational efficiency compared with the canonical genetic algorithm with the elitism and the immune genetic algorithm with the information entropy and elitism.
基金funded by Research Project,grant number BHQ090003000X03。
文摘Multi-modal knowledge graph completion(MMKGC)aims to complete missing entities or relations in multi-modal knowledge graphs,thereby discovering more previously unknown triples.Due to the continuous growth of data and knowledge and the limitations of data sources,the visual knowledge within the knowledge graphs is generally of low quality,and some entities suffer from the issue of missing visual modality.Nevertheless,previous studies of MMKGC have primarily focused on how to facilitate modality interaction and fusion while neglecting the problems of low modality quality and modality missing.In this case,mainstream MMKGC models only use pre-trained visual encoders to extract features and transfer the semantic information to the joint embeddings through modal fusion,which inevitably suffers from problems such as error propagation and increased uncertainty.To address these problems,we propose a Multi-modal knowledge graph Completion model based on Super-resolution and Detailed Description Generation(MMCSD).Specifically,we leverage a pre-trained residual network to enhance the resolution and improve the quality of the visual modality.Moreover,we design multi-level visual semantic extraction and entity description generation,thereby further extracting entity semantics from structural triples and visual images.Meanwhile,we train a variational multi-modal auto-encoder and utilize a pre-trained multi-modal language model to complement the missing visual features.We conducted experiments on FB15K-237 and DB13K,and the results showed that MMCSD can effectively perform MMKGC and achieve state-of-the-art performance.
文摘BACKGROUND Stress ulcers are common complications in critically ill patients,with a higher incidence observed in older patients following gastrointestinal surgery.This study aimed to develop and evaluate the effectiveness of a multi-modal intervention protocol to prevent stress ulcers in this high-risk population.AIM To assess the impact of a multi-modal intervention on preventing stress ulcers in older intensive care unit(ICU)patients postoperatively.METHODS A randomized controlled trial involving critically ill patients(aged≥65 years)admitted to the ICU after gastrointestinal surgery was conducted.Patients were randomly assigned to either the intervention group,which received a multimodal stress ulcer prevention protocol,or the control group,which received standard care.The primary outcome measure was the incidence of stress ulcers.The secondary outcomes included ulcer healing time,complication rates,and length of hospital stay.RESULTS A total of 200 patients(100 in each group)were included in this study.The intervention group exhibited a significantly lower incidence of stress ulcers than the control group(15%vs 30%,P<0.01).Additionally,the intervention group demonstrated shorter ulcer healing times(mean 5.2 vs 7.8 days,P<0.05),lower complication rates(10%vs 22%,P<0.05),and reduced length of hospital stay(mean 12.3 vs 15.7 days,P<0.05).CONCLUSION This multi-modal intervention protocol significantly reduced the incidence of stress ulcers and improved clinical outcomes in critically ill older patients after gastrointestinal surgery.This comprehensive approach may provide a valuable strategy for managing high-risk populations in intensive care settings.
基金supported by the Autonomous Region Industry-Education Integration Project“Application of DNA Methylation Combined with Spiral CT in the Screening of Pulmonary Ground-Glass Nodules and AI Recognition Systems in Teaching Practice”(Project No.2023210016)the“Open Project of the State Key Laboratory of High Incidence Diseases in Central Asia”(Project No.SKL-HIDCA-2021-28).
文摘Objective:To explore the effectiveness of multi-modal teaching based on an online case library in the education of gene methylation combined with spiral computed tomography(CT)screening for pulmonary ground-glass opacity(GGO)nodules.Methods:From October 2023 to April 2024,66 medical imaging students were selected and randomly divided into a control group and an observation group,each with 33 students.The control group received traditional lecture-based teaching,while the observation group was taught using a multi-modal teaching approach based on an online case library.Performance on assessments and teaching quality were analyzed between the two groups.Results:The observation group achieved higher scores in theoretical and practical knowledge compared to the control group(P<0.05).Additionally,the teaching quality scores were significantly higher in the observation group(P<0.05).Conclusion:Implementing multi-modal teaching based on an online case library for pulmonary GGO nodule screening with gene methylation combined with spiral CT can enhance students’knowledge acquisition,improve teaching quality,and have significant clinical application value.
文摘[Objective]Accurate prediction of tomato growth height is crucial for optimizing production environments in smart farming.However,current prediction methods predominantly rely on empirical,mechanistic,or learning-based models that utilize either images data or environmental data.These methods fail to fully leverage multi-modal data to capture the diverse aspects of plant growth comprehensively.[Methods]To address this limitation,a two-stage phenotypic feature extraction(PFE)model based on deep learning algorithm of recurrent neural network(RNN)and long short-term memory(LSTM)was developed.The model integrated environment and plant information to provide a holistic understanding of the growth process,emploied phenotypic and temporal feature extractors to comprehensively capture both types of features,enabled a deeper understanding of the interaction between tomato plants and their environment,ultimately leading to highly accurate predictions of growth height.[Results and Discussions]The experimental results showed the model's ef‐fectiveness:When predicting the next two days based on the past five days,the PFE-based RNN and LSTM models achieved mean absolute percentage error(MAPE)of 0.81%and 0.40%,respectively,which were significantly lower than the 8.00%MAPE of the large language model(LLM)and 6.72%MAPE of the Transformer-based model.In longer-term predictions,the 10-day prediction for 4 days ahead and the 30-day prediction for 12 days ahead,the PFE-RNN model continued to outperform the other two baseline models,with MAPE of 2.66%and 14.05%,respectively.[Conclusions]The proposed method,which leverages phenotypic-temporal collaboration,shows great potential for intelligent,data-driven management of tomato cultivation,making it a promising approach for enhancing the efficiency and precision of smart tomato planting management.
基金supported by the Natural Science Foundation of Liaoning Province(Grant No.2023-MSBA-070)the National Natural Science Foundation of China(Grant No.62302086).
文摘Multi-modal fusion technology gradually become a fundamental task in many fields,such as autonomous driving,smart healthcare,sentiment analysis,and human-computer interaction.It is rapidly becoming the dominant research due to its powerful perception and judgment capabilities.Under complex scenes,multi-modal fusion technology utilizes the complementary characteristics of multiple data streams to fuse different data types and achieve more accurate predictions.However,achieving outstanding performance is challenging because of equipment performance limitations,missing information,and data noise.This paper comprehensively reviews existing methods based onmulti-modal fusion techniques and completes a detailed and in-depth analysis.According to the data fusion stage,multi-modal fusion has four primary methods:early fusion,deep fusion,late fusion,and hybrid fusion.The paper surveys the three majormulti-modal fusion technologies that can significantly enhance the effect of data fusion and further explore the applications of multi-modal fusion technology in various fields.Finally,it discusses the challenges and explores potential research opportunities.Multi-modal tasks still need intensive study because of data heterogeneity and quality.Preserving complementary information and eliminating redundant information between modalities is critical in multi-modal technology.Invalid data fusion methods may introduce extra noise and lead to worse results.This paper provides a comprehensive and detailed summary in response to these challenges.
基金funded by the National Natural Science Foundation of China(61991413)the China Postdoctoral Science Foundation(2019M651142)+1 种基金the Natural Science Foundation of Liaoning Province(2021-KF-12-07)the Natural Science Foundations of Liaoning Province(2023-MS-322).
文摘Fusing hand-based features in multi-modal biometric recognition enhances anti-spoofing capabilities.Additionally,it leverages inter-modal correlation to enhance recognition performance.Concurrently,the robustness and recognition performance of the system can be enhanced through judiciously leveraging the correlation among multimodal features.Nevertheless,two issues persist in multi-modal feature fusion recognition:Firstly,the enhancement of recognition performance in fusion recognition has not comprehensively considered the inter-modality correlations among distinct modalities.Secondly,during modal fusion,improper weight selection diminishes the salience of crucial modal features,thereby diminishing the overall recognition performance.To address these two issues,we introduce an enhanced DenseNet multimodal recognition network founded on feature-level fusion.The information from the three modalities is fused akin to RGB,and the input network augments the correlation between modes through channel correlation.Within the enhanced DenseNet network,the Efficient Channel Attention Network(ECA-Net)dynamically adjusts the weight of each channel to amplify the salience of crucial information in each modal feature.Depthwise separable convolution markedly reduces the training parameters and further enhances the feature correlation.Experimental evaluations were conducted on four multimodal databases,comprising six unimodal databases,including multispectral palmprint and palm vein databases from the Chinese Academy of Sciences.The Equal Error Rates(EER)values were 0.0149%,0.0150%,0.0099%,and 0.0050%,correspondingly.In comparison to other network methods for palmprint,palm vein,and finger vein fusion recognition,this approach substantially enhances recognition performance,rendering it suitable for high-security environments with practical applicability.The experiments in this article utilized amodest sample database comprising 200 individuals.The subsequent phase involves preparing for the extension of the method to larger databases.
基金European Commission,Joint Research Center,Grant/Award Number:HUMAINTMinisterio de Ciencia e Innovación,Grant/Award Number:PID2020‐114924RB‐I00Comunidad de Madrid,Grant/Award Number:S2018/EMT‐4362 SEGVAUTO 4.0‐CM。
文摘Predicting the motion of other road agents enables autonomous vehicles to perform safe and efficient path planning.This task is very complex,as the behaviour of road agents depends on many factors and the number of possible future trajectories can be consid-erable(multi-modal).Most prior approaches proposed to address multi-modal motion prediction are based on complex machine learning systems that have limited interpret-ability.Moreover,the metrics used in current benchmarks do not evaluate all aspects of the problem,such as the diversity and admissibility of the output.The authors aim to advance towards the design of trustworthy motion prediction systems,based on some of the re-quirements for the design of Trustworthy Artificial Intelligence.The focus is on evaluation criteria,robustness,and interpretability of outputs.First,the evaluation metrics are comprehensively analysed,the main gaps of current benchmarks are identified,and a new holistic evaluation framework is proposed.Then,a method for the assessment of spatial and temporal robustness is introduced by simulating noise in the perception system.To enhance the interpretability of the outputs and generate more balanced results in the proposed evaluation framework,an intent prediction layer that can be attached to multi-modal motion prediction models is proposed.The effectiveness of this approach is assessed through a survey that explores different elements in the visualisation of the multi-modal trajectories and intentions.The proposed approach and findings make a significant contribution to the development of trustworthy motion prediction systems for autono-mous vehicles,advancing the field towards greater safety and reliability.
基金Project(2023JH26-10100002)supported by the Liaoning Science and Technology Major Project,ChinaProjects(U21A20117,52074085)supported by the National Natural Science Foundation of China+1 种基金Project(2022JH2/101300008)supported by the Liaoning Applied Basic Research Program Project,ChinaProject(22567612H)supported by the Hebei Provincial Key Laboratory Performance Subsidy Project,China。
文摘Mill vibration is a common problem in rolling production,which directly affects the thickness accuracy of the strip and may even lead to strip fracture accidents in serious cases.The existing vibration prediction models do not consider the features contained in the data,resulting in limited improvement of model accuracy.To address these challenges,this paper proposes a multi-dimensional multi-modal cold rolling vibration time series prediction model(MDMMVPM)based on the deep fusion of multi-level networks.In the model,the long-term and short-term modal features of multi-dimensional data are considered,and the appropriate prediction algorithms are selected for different data features.Based on the established prediction model,the effects of tension and rolling force on mill vibration are analyzed.Taking the 5th stand of a cold mill in a steel mill as the research object,the innovative model is applied to predict the mill vibration for the first time.The experimental results show that the correlation coefficient(R^(2))of the model proposed in this paper is 92.5%,and the root-mean-square error(RMSE)is 0.0011,which significantly improves the modeling accuracy compared with the existing models.The proposed model is also suitable for the hot rolling process,which provides a new method for the prediction of strip rolling vibration.
文摘Intelligent personal assistants play a pivotal role in in-vehicle systems,significantly enhancing life efficiency,driving safety,and decision-making support.In this study,the multi-modal design elements of intelligent personal assistants within the context of visual,auditory,and somatosensory interactions with drivers were discussed.Their impact on the driver’s psychological state through various modes such as visual imagery,voice interaction,and gesture interaction were explored.The study also introduced innovative designs for in-vehicle intelligent personal assistants,incorporating design principles such as driver-centricity,prioritizing passenger safety,and utilizing timely feedback as a criterion.Additionally,the study employed design methods like driver behavior research and driving situation analysis to enhance the emotional connection between drivers and their vehicles,ultimately improving driver satisfaction and trust.
基金National College Students’Training Programs of Innovation and Entrepreneurship,Grant/Award Number:S202210022060the CACMS Innovation Fund,Grant/Award Number:CI2021A00512the National Nature Science Foundation of China under Grant,Grant/Award Number:62206021。
文摘Media convergence works by processing information from different modalities and applying them to different domains.It is difficult for the conventional knowledge graph to utilise multi-media features because the introduction of a large amount of information from other modalities reduces the effectiveness of representation learning and makes knowledge graph inference less effective.To address the issue,an inference method based on Media Convergence and Rule-guided Joint Inference model(MCRJI)has been pro-posed.The authors not only converge multi-media features of entities but also introduce logic rules to improve the accuracy and interpretability of link prediction.First,a multi-headed self-attention approach is used to obtain the attention of different media features of entities during semantic synthesis.Second,logic rules of different lengths are mined from knowledge graph to learn new entity representations.Finally,knowledge graph inference is performed based on representing entities that converge multi-media features.Numerous experimental results show that MCRJI outperforms other advanced baselines in using multi-media features and knowledge graph inference,demonstrating that MCRJI provides an excellent approach for knowledge graph inference with converged multi-media features.
基金supported by the National Natural Science Foundation of China(GrantNo.62302086)the Natural Science Foundation of Liaoning Province(Grant No.2023-MSBA-070)the Fundamental Research Funds for the Central Universities(Grant No.N2317005).
文摘Multi-modal 3D object detection has achieved remarkable progress,but it is often limited in practical industrial production because of its high cost and low efficiency.The multi-view camera-based method provides a feasible solution due to its low cost.However,camera data lacks geometric depth,and only using camera data to obtain high accuracy is challenging.This paper proposes a multi-modal Bird-Eye-View(BEV)distillation framework(MMDistill)to make a trade-off between them.MMDistill is a carefully crafted two-stage distillation framework based on teacher and student models for learning cross-modal knowledge and generating multi-modal features.It can improve the performance of unimodal detectors without introducing additional costs during inference.Specifically,our method can effectively solve the cross-gap caused by the heterogeneity between data.Furthermore,we further propose a Light Detection and Ranging(LiDAR)-guided geometric compensation module,which can assist the student model in obtaining effective geometric features and reduce the gap between different modalities.Our proposed method generally requires fewer computational resources and faster inference speed than traditional multi-modal models.This advancement enables multi-modal technology to be applied more widely in practical scenarios.Through experiments,we validate the effectiveness and superiority of MMDistill on the nuScenes dataset,achieving an improvement of 4.1%mean Average Precision(mAP)and 4.6%NuScenes Detection Score(NDS)over the baseline detector.In addition,we also present detailed ablation studies to validate our method.
基金supported by the National Key Research and Development Project under Grant 2020YFB1807602Key Program of Marine Economy Development Special Foundation of Department of Natural Resources of Guangdong Province(GDNRC[2023]24)the National Natural Science Foundation of China under Grant 62271267.
文摘Recently,there have been significant advancements in the study of semantic communication in single-modal scenarios.However,the ability to process information in multi-modal environments remains limited.Inspired by the research and applications of natural language processing across different modalities,our goal is to accurately extract frame-level semantic information from videos and ultimately transmit high-quality videos.Specifically,we propose a deep learning-basedMulti-ModalMutual Enhancement Video Semantic Communication system,called M3E-VSC.Built upon a VectorQuantized Generative AdversarialNetwork(VQGAN),our systemaims to leverage mutual enhancement among different modalities by using text as the main carrier of transmission.With it,the semantic information can be extracted fromkey-frame images and audio of the video and performdifferential value to ensure that the extracted text conveys accurate semantic information with fewer bits,thus improving the capacity of the system.Furthermore,a multi-frame semantic detection module is designed to facilitate semantic transitions during video generation.Simulation results demonstrate that our proposed model maintains high robustness in complex noise environments,particularly in low signal-to-noise ratio conditions,significantly improving the accuracy and speed of semantic transmission in video communication by approximately 50 percent.
基金supported by the National Natural Science Foundation of China,Nos.82204360(to HM)and 82270411(to GW)National Science and Technology Innovation 2030 Major Program,No.2021ZD0200900(to YL)。
文摘Traumatic brain injury involves complex pathophysiological mechanisms,among which oxidative stress significantly contributes to the occurrence of secondary injury.In this study,we evaluated hypidone hydrochloride(YL-0919),a self-developed antidepressant with selective sigma-1 receptor agonist properties,and its associated mechanisms and targets in traumatic brain injury.Behavioral experiments to assess functional deficits were followed by assessment of neuronal damage through histological analyses and examination of blood-brain barrier permeability and brain edema.Next,we investigated the antioxidative effects of YL-0919 by assessing the levels of traditional markers of oxidative stress in vivo in mice and in vitro in HT22 cells.Finally,the targeted action of YL-0919 was verified by employing a sigma-1 receptor antagonist(BD-1047).Our findings demonstrated that YL-0919 markedly improved deficits in motor function and spatial cognition on day 3 post traumatic brain injury,while also decreasing neuronal mortality and reversing blood-brain barrier disruption and brain edema.Furthermore,YL-0919 effectively combated oxidative stress both in vivo and in vitro.The protective effects of YL-0919 were partially inhibited by BD-1047.These results indicated that YL-0919 relieved impairments in motor and spatial cognition by restraining oxidative stress,a neuroprotective effect that was partially reversed by the sigma-1 receptor antagonist BD-1047.YL-0919 may have potential as a new treatment for traumatic brain injury.
文摘BACKGROUND At present,the influencing factors of social function in patients with residual depressive symptoms are still unclear.Residual depressive symptoms are highly harmful,leading to low mood in patients,affecting work and interpersonal communication,increasing the risk of recurrence,and adding to the burden on families.Studying the influencing factors of their social function is of great significance.AIM To explore the social function score and its influencing factors in patients with residual depressive symptoms.METHODS This observational study surveyed patients with residual depressive symptoms(case group)and healthy patients undergoing physical examinations(control group).Participants were admitted between January 2022 and December 2023.Social functioning was assessed using the Sheehan Disability Scale(SDS),and scores were compared between groups.Factors influencing SDS scores in patients with residual depressive symptoms were analyzed by applying multiple linear regression while using the receiver operating characteristic curve,and these RESULTS The SDS scores of the 158 patients with depressive symptoms were 11.48±3.26.Compared with the control group,the SDS scores and all items in the case group were higher.SDS scores were higher in patients with relapse,discon-tinuous medication,drug therapy alone,severe somatic symptoms,obvious residual symptoms,and anxiety scores≥8.Disease history,medication compliance,therapy method,and residual symptoms correlated positively with SDS scores(r=0.354,0.414,0.602,and 0.456,respectively).Independent influencing factors included disease history,medication compliance,therapy method,somatic symptoms,residual symptoms,and anxiety scores(P<0.05).The areas under the curve for predicting social functional impairment using these factors were 0.713,0.559,0.684,0.729,0.668,and 0.628,respectively,with sensitivities of 79.2%,61.8%,76.8%,81.7%,63.6%,and 65.5%and specificities of 83.3%,87.5%,82.6%,83.3%,86.7%,and 92.1%,respectively.CONCLUSION The social function scores of patients with residual symptoms of depression are high.They are affected by disease history,medication compliance,therapy method,degree of somatic symptoms,residual symptoms,and anxiety.
基金the National Natural Science Foundation of China(No.61976080)the Academic Degrees&Graduate Education Reform Project of Henan Province(No.2021SJGLX195Y)+1 种基金the Teaching Reform Research and Practice Project of Henan Undergraduate Universities(No.2022SYJXLX008)the Key Project on Research and Practice of Henan University Graduate Education and Teaching Reform(No.YJSJG2023XJ006)。
文摘The unsupervised multi-modal image translation is an emerging domain of computer vision whose goal is to transform an image from the source domain into many diverse styles in the target domain.However,the multi-generator mechanism is employed among the advanced approaches available to model different domain mappings,which results in inefficient training of neural networks and pattern collapse,leading to inefficient generation of image diversity.To address this issue,this paper introduces a multi-modal unsupervised image translation framework that uses a generator to perform multi-modal image translation.Specifically,firstly,the domain code is introduced in this paper to explicitly control the different generation tasks.Secondly,this paper brings in the squeeze-and-excitation(SE)mechanism and feature attention(FA)module.Finally,the model integrates multiple optimization objectives to ensure efficient multi-modal translation.This paper performs qualitative and quantitative experiments on multiple non-paired benchmark image translation datasets while demonstrating the benefits of the proposed method over existing technologies.Overall,experimental results have shown that the proposed method is versatile and scalable.
基金supported by NEI/NIH R01 EY030513NIAMS/NIH R21-AR076035Multi-PI Team Science grant from Presbyterian Health Foundation。
文摘Very long chain-saturated and-polyunsaturated fatty acids(VLC-SFA and VLC-PUFA, respectively) are a functionally important class of fatty acids containing 28 carbons or more in their acyl chain. They are synthesized by the elongation of very long fatty acids-4(ELOVL4) enzyme, expressed mainly in the brain, retina, skin, testes, and meibomian gland, where these fatty acids are found(Agbaga et al., 2008). Further, these organs exhibit tissuespecific VLC-PUFA and VLC-SFA biosynthesis and incorporation into complex lipids for specific functions. In the brain, skin, and Meibomian glands, the ELOVL4 mainly makes VLC-SFA, which are incorporated into complex sphingolipids. In the retina, the ELOVL4 makes VLC-PUFA that are incorporated into phosphatidylcholine, that are critical for visual function, while in testes and sperm, the VLC-PUFA are incorporated into sphingolipids that are critical for fertility(Yeboah et al., 2021).
基金supported by a grant from Telethon Foundation(grant No.GGP20056 to SB)The generation of Pcdh19 floxed mouse model was funded by Cariplo Foundation(grant No.2014-0972 to SB)。
文摘Mutations in the protocadherin-19(PCDH19)gene(Xq22.1)cause the X-linked syndrome known as developmental and epileptic encephalopathy 9(DEE9,OMIM#300088)(Dibbens et al.,2008).DEE9 is characterized by early-onset clustering epilepsy associated with intellectual disability ranging from mild to profound,autism spectrum disorder,and other neuropsychiatric features including schizophrenia,anxiety,attentiondeficit/hyperactivity,and obsessive or aggressive behaviors.While seizures may become less frequent in adolescence,psychiatric comorbidities persist and often worsen with age(Dibbens et al.,2008;Kolc et al.,2020).
