The rapid growth of digital data necessitates advanced natural language processing(NLP)models like BERT(Bidi-rectional Encoder Representations from Transformers),known for its superior performance in text classificati...The rapid growth of digital data necessitates advanced natural language processing(NLP)models like BERT(Bidi-rectional Encoder Representations from Transformers),known for its superior performance in text classification.However,BERT’s size and computational demands limit its practicality,especially in resource-constrained settings.This research compresses the BERT base model for Bengali emotion classification through knowledge distillation(KD),pruning,and quantization techniques.Despite Bengali being the sixth most spoken language globally,NLP research in this area is limited.Our approach addresses this gap by creating an efficient BERT-based model for Bengali text.We have explored 20 combinations for KD,quantization,and pruning,resulting in improved speedup,fewer parameters,and reduced memory size.Our best results demonstrate significant improvements in both speed and efficiency.For instance,in the case of mBERT,we achieved a 3.87×speedup and 4×compression ratio with a combination of Distil+Prune+Quant that reduced parameters from 178 to 46 M,while the memory size decreased from 711 to 178 MB.These results offer scalable solutions for NLP tasks in various languages and advance the field of model compression,making these models suitable for real-world applications in resource-limited environments.展开更多
The demand for adopting neural networks in resource-constrained embedded devices is continuously increasing.Quantization is one of the most promising solutions to reduce computational cost and memory storage on embedd...The demand for adopting neural networks in resource-constrained embedded devices is continuously increasing.Quantization is one of the most promising solutions to reduce computational cost and memory storage on embedded devices.In order to reduce the complexity and overhead of deploying neural networks on Integeronly hardware,most current quantization methods use a symmetric quantization mapping strategy to quantize a floating-point neural network into an integer network.However,although symmetric quantization has the advantage of easier implementation,it is sub-optimal for cases where the range could be skewed and not symmetric.This often comes at the cost of lower accuracy.This paper proposed an activation redistribution-based hybrid asymmetric quantizationmethod for neural networks.The proposedmethod takes data distribution into consideration and can resolve the contradiction between the quantization accuracy and the ease of implementation,balance the trade-off between clipping range and quantization resolution,and thus improve the accuracy of the quantized neural network.The experimental results indicate that the accuracy of the proposed method is 2.02%and 5.52%higher than the traditional symmetric quantization method for classification and detection tasks,respectively.The proposed method paves the way for computationally intensive neural network models to be deployed on devices with limited computing resources.Codes will be available on https://github.com/ycjcy/Hybrid-Asymmetric-Quantization.展开更多
The quantization algorithm compresses the original network by reducing the numerical bit width of the model,which improves the computation speed. Because different layers have different redundancy and sensitivity to d...The quantization algorithm compresses the original network by reducing the numerical bit width of the model,which improves the computation speed. Because different layers have different redundancy and sensitivity to databit width. Reducing the data bit width will result in a loss of accuracy. Therefore, it is difficult to determinethe optimal bit width for different parts of the network with guaranteed accuracy. Mixed precision quantizationcan effectively reduce the amount of computation while keeping the model accuracy basically unchanged. In thispaper, a hardware-aware mixed precision quantization strategy optimal assignment algorithm adapted to low bitwidth is proposed, and reinforcement learning is used to automatically predict the mixed precision that meets theconstraints of hardware resources. In the state-space design, the standard deviation of weights is used to measurethe distribution difference of data, the execution speed feedback of simulated neural network accelerator inferenceis used as the environment to limit the action space of the agent, and the accuracy of the quantization model afterretraining is used as the reward function to guide the agent to carry out deep reinforcement learning training. Theexperimental results show that the proposed method obtains a suitable model layer-by-layer quantization strategyunder the condition that the computational resources are satisfied, and themodel accuracy is effectively improved.The proposed method has strong intelligence and certain universality and has strong application potential in thefield of mixed precision quantization and embedded neural network model deployment.展开更多
The recently developed magic-intensity trapping technique of neutral atoms efficiently mitigates the detrimental effect of light shifts on atomic qubits and substantially enhances the coherence time. This technique re...The recently developed magic-intensity trapping technique of neutral atoms efficiently mitigates the detrimental effect of light shifts on atomic qubits and substantially enhances the coherence time. This technique relies on applying a bias magnetic field precisely parallel to the wave vector of a circularly polarized trapping laser field. However, due to the presence of the vector light shift experienced by the trapped atoms, it is challenging to precisely define a parallel magnetic field, especially at a low bias magnetic field strength, for the magic-intensity trapping of85Rb qubits. In this work, we present a method to calibrate the angle between the bias magnetic field and the trapping laser field with the compensating magnetic fields in the other two directions orthogonal to the bias magnetic field direction. Experimentally, with a constantdepth trap and a fixed bias magnetic field, we measure the respective resonant frequencies of the atomic qubits in a linearly polarized trap and a circularly polarized one via the conventional microwave Rabi spectra with different compensating magnetic fields and obtain the corresponding total magnetic fields via the respective resonant frequencies using the Breit–Rabi formula. With known total magnetic fields, the angle is a function of the other two compensating magnetic fields.Finally, the projection value of the angle on either of the directions orthogonal to the bias magnetic field direction can be reduced to 0(4)° by applying specific compensating magnetic fields. The measurement error is mainly attributed to the fluctuation of atomic temperature. Moreover, it also demonstrates that, even for a small angle, the effect is strong enough to cause large decoherence of Rabi oscillation in a magic-intensity trap. Although the compensation method demonstrated here is explored for the magic-intensity trapping technique, it can be applied to a variety of similar precision measurements with trapped neutral atoms.展开更多
In this paper,we investigate networkassisted full-duplex(NAFD)cell-free millimeter-wave(mmWave)massive multiple-input multiple-output(MIMO)systems with digital-to-analog converter(DAC)quantization and fronthaul compre...In this paper,we investigate networkassisted full-duplex(NAFD)cell-free millimeter-wave(mmWave)massive multiple-input multiple-output(MIMO)systems with digital-to-analog converter(DAC)quantization and fronthaul compression.We propose to maximize the weighted uplink and downlink sum rate by jointly optimizing the power allocation of both the transmitting remote antenna units(T-RAUs)and uplink users and the variances of the downlink and uplink fronthaul compression noises.To deal with this challenging problem,we further apply a successive convex approximation(SCA)method to handle the non-convex bidirectional limited-capacity fronthaul constraints.The simulation results verify the convergence of the proposed SCA-based algorithm and analyze the impact of fronthaul capacity and DAC quantization on the spectral efficiency of the NAFD cell-free mmWave massive MIMO systems.Moreover,some insightful conclusions are obtained through the comparisons of spectral efficiency,which shows that NAFD achieves better performance gains than cotime co-frequency full-duplex cloud radio access network(CCFD C-RAN)in the cases of practical limited-resolution DACs.Specifically,their performance gaps with 8-bit DAC quantization are larger than that with1-bit DAC quantization,which attains a 5.5-fold improvement.展开更多
Massive computational complexity and memory requirement of artificial intelligence models impede their deploy-ability on edge computing devices of the Internet of Things(IoT).While Power-of-Two(PoT)quantization is pro...Massive computational complexity and memory requirement of artificial intelligence models impede their deploy-ability on edge computing devices of the Internet of Things(IoT).While Power-of-Two(PoT)quantization is pro-posed to improve the efficiency for edge inference of Deep Neural Networks(DNNs),existing PoT schemes require a huge amount of bit-wise manipulation and have large memory overhead,and their efficiency is bounded by the bottleneck of computation latency and memory footprint.To tackle this challenge,we present an efficient inference approach on the basis of PoT quantization and model compression.An integer-only scalar PoT quantization(IOS-PoT)is designed jointly with a distribution loss regularizer,wherein the regularizer minimizes quantization errors and training disturbances.Additionally,two-stage model compression is developed to effectively reduce memory requirement,and alleviate bandwidth usage in communications of networked heterogenous learning systems.The product look-up table(P-LUT)inference scheme is leveraged to replace bit-shifting with only indexing and addition operations for achieving low-latency computation and implementing efficient edge accelerators.Finally,comprehensive experiments on Residual Networks(ResNets)and efficient architectures with Canadian Institute for Advanced Research(CIFAR),ImageNet,and Real-world Affective Faces Database(RAF-DB)datasets,indicate that our approach achieves 2×∼10×improvement in the reduction of both weight size and computation cost in comparison to state-of-the-art methods.A P-LUT accelerator prototype is implemented on the Xilinx KV260 Field Programmable Gate Array(FPGA)platform for accelerating convolution operations,with performance results showing that P-LUT reduces memory footprint by 1.45×,achieves more than 3×power efficiency and 2×resource efficiency,compared to the conventional bit-shifting scheme.展开更多
A new steganographic method by pixel-value differencing(PVD)using general quantization ranges of pixel pairs’difference values is proposed.The objective of this method is to provide a data embedding technique with a ...A new steganographic method by pixel-value differencing(PVD)using general quantization ranges of pixel pairs’difference values is proposed.The objective of this method is to provide a data embedding technique with a range table with range widths not limited to powers of 2,extending PVD-based methods to enhance their flexibility and data-embedding rates without changing their capabilities to resist security attacks.Specifically,the conventional PVD technique partitions a grayscale image into 1×2 non-overlapping blocks.The entire range[0,255]of all possible absolute values of the pixel pairs’grayscale differences in the blocks is divided into multiple quantization ranges.The width of each quantization range is a power of two to facilitate the direct embedding of the bit information with high embedding rates.Without using power-of-two range widths,the embedding rates can drop using conventional embedding techniques.In contrast,the proposed method uses general quantization range widths,and a multiple-based number conversion mechanism is employed skillfully to implement the use of nonpower-of-two range widths,with each pixel pair being employed to embed a digit in the multiple-based number.All the message bits are converted into a big multiple-based number whose digits can be embedded into the pixel pairs with a higher embedding rate.Good experimental results showed the feasibility of the proposed method and its resistance to security attacks.In addition,implementation examples are provided,where the proposed method adopts non-power-of-two range widths and employsmultiple-based number conversion to expand the data-hiding and steganalysis-resisting capabilities of other PVD methods.展开更多
Imbalanced datasets are common in practical applications,and oversampling methods using fuzzy rules have been shown to enhance the classification performance of imbalanced data by taking into account the relationship ...Imbalanced datasets are common in practical applications,and oversampling methods using fuzzy rules have been shown to enhance the classification performance of imbalanced data by taking into account the relationship between data attributes.However,the creation of fuzzy rules typically depends on expert knowledge,which may not fully leverage the label information in training data and may be subjective.To address this issue,a novel fuzzy rule oversampling approach is developed based on the learning vector quantization(LVQ)algorithm.In this method,the label information of the training data is utilized to determine the antecedent part of If-Then fuzzy rules by dynamically dividing attribute intervals using LVQ.Subsequently,fuzzy rules are generated and adjusted to calculate rule weights.The number of new samples to be synthesized for each rule is then computed,and samples from the minority class are synthesized based on the newly generated fuzzy rules.This results in the establishment of a fuzzy rule oversampling method based on LVQ.To evaluate the effectiveness of this method,comparative experiments are conducted on 12 publicly available imbalance datasets with five other sampling techniques in combination with the support function machine.The experimental results demonstrate that the proposed method can significantly enhance the classification algorithm across seven performance indicators,including a boost of 2.15%to 12.34%in Accuracy,6.11%to 27.06%in G-mean,and 4.69%to 18.78%in AUC.These show that the proposed method is capable of more efficiently improving the classification performance of imbalanced data.展开更多
Linear temporal logic(LTL)is an intuitive and expressive language to specify complex control tasks,and how to design an efficient control strategy for LTL specification is still a challenge.In this paper,we implement ...Linear temporal logic(LTL)is an intuitive and expressive language to specify complex control tasks,and how to design an efficient control strategy for LTL specification is still a challenge.In this paper,we implement the dynamic quantization technique to propose a novel hierarchical control strategy for nonlinear control systems under LTL specifications.Based on the regions of interest involved in the LTL formula,an accepting path is derived first to provide a high-level solution for the controller synthesis problem.Second,we develop a dynamic quantization based approach to verify the realization of the accepting path.The realization verification results in the necessity of the controller design and a sequence of quantization regions for the controller design.Third,the techniques of dynamic quantization and abstraction-based control are combined together to establish the local-to-global control strategy.Both abstraction construction and controller design are local and dynamic,thereby resulting in the potential reduction of the computational complexity.Since each quantization region can be considered locally and individually,the proposed hierarchical mechanism is more efficient and can solve much larger problems than many existing methods.Finally,the proposed control strategy is illustrated via two examples from the path planning and tracking problems of mobile robots.展开更多
What does it mean to study PDE (Partial Differential Equation)? How and what to do “to claim proudly that I’m studying a certain PDE”? Newton mechanic uses mainly ODE (Ordinary Differential Equation) and describes ...What does it mean to study PDE (Partial Differential Equation)? How and what to do “to claim proudly that I’m studying a certain PDE”? Newton mechanic uses mainly ODE (Ordinary Differential Equation) and describes nicely movements of Sun, Moon and Earth etc. Now, so-called quantum phenomenum is described by, say Schrödinger equation, PDE which explains both wave and particle characters after quantization of ODE. The coupled Maxwell-Dirac equation is also “quantized” and QED (Quantum Electro-Dynamics) theory is invented by physicists. Though it is said this QED gives very good coincidence between theoretical1 and experimental observed quantities, but what is the equation corresponding to QED? Or, is it possible to describe QED by “equation” in naive sense?展开更多
The uncertainty principle is a fundamental principle of quantum mechanics, but its exact mathematical expression cannot obtain correct results when used to solve theoretical problems such as the energy levels of hydro...The uncertainty principle is a fundamental principle of quantum mechanics, but its exact mathematical expression cannot obtain correct results when used to solve theoretical problems such as the energy levels of hydrogen atoms, one-dimensional deep potential wells, one-dimensional harmonic oscillators, and double-slit experiments. Even after approximate treatment, the results obtained are not completely consistent with those obtained by solving Schrödinger’s equation. This indicates that further research on the uncertainty principle is necessary. Therefore, using the de Broglie matter wave hypothesis, we quantize the action of an elementary particle in natural coordinates and obtain the quantization condition and a new deterministic relation. Using this quantization condition, we obtain the energy level formulas of an elementary particle in different conditions in a classical way that is completely consistent with the results obtained by solving Schrödinger’s equation. A new physical interpretation is given for the particle eigenfunction independence of probability for an elementary particle: an elementary particle is in a particle state at the space-time point where the action is quantized, and in a wave state in the rest of the space-time region. The space-time points of particle nature and the wave regions of particle motion constitute the continuous trajectory of particle motion. When an elementary particle is in a particle state, it is localized, whereas in the wave state region, it is nonlocalized.展开更多
Large-scale Language Models(LLMs)have achieved significant breakthroughs in Natural Language Processing(NLP),driven by the pre-training and fine-tuning paradigm.While this approach allows models to specialize in speci...Large-scale Language Models(LLMs)have achieved significant breakthroughs in Natural Language Processing(NLP),driven by the pre-training and fine-tuning paradigm.While this approach allows models to specialize in specific tasks with reduced training costs,the substantial memory requirements during fine-tuning present a barrier to broader deployment.Parameter-Efficient Fine-Tuning(PEFT)techniques,such as Low-Rank Adaptation(LoRA),and parameter quantization methods have emerged as solutions to address these challenges by optimizing memory usage and computational efficiency.Among these,QLoRA,which combines PEFT and quantization,has demonstrated notable success in reducing memory footprints during fine-tuning,prompting the development of various QLoRA variants.Despite these advancements,the quantitative impact of key variables on the fine-tuning performance of quantized LLMs remains underexplored.This study presents a comprehensive analysis of these key variables,focusing on their influence across different layer types and depths within LLM architectures.Our investigation uncovers several critical findings:(1)Larger layers,such as MLP layers,can maintain performance despite reductions in adapter rank,while smaller layers,like self-attention layers,aremore sensitive to such changes;(2)The effectiveness of balancing factors depends more on specific values rather than layer type or depth;(3)In quantization-aware fine-tuning,larger layers can effectively utilize smaller adapters,whereas smaller layers struggle to do so.These insights suggest that layer type is a more significant determinant of fine-tuning success than layer depth when optimizing quantized LLMs.Moreover,for the same discount of trainable parameters,reducing the trainable parameters in a larger layer is more effective in preserving fine-tuning accuracy than in a smaller one.This study provides valuable guidance for more efficient fine-tuning strategies and opens avenues for further research into optimizing LLM fine-tuning in resource-constrained environments.展开更多
Growth of high-quality Nb_(3)Sn thin films for superconducting radiofrequency(SRF)applications using the vapor diffusion method requires a uniform distribution of tin nuclei on the niobium(Nb)surface.This study examin...Growth of high-quality Nb_(3)Sn thin films for superconducting radiofrequency(SRF)applications using the vapor diffusion method requires a uniform distribution of tin nuclei on the niobium(Nb)surface.This study examines the mechanism underlying the observed non-uniform distribution of tin nuclei with tin chloride SnCl_(2).Electron backscatter diffraction(EBSD)analysis was used to examine the correlation between the nucleation behavior and orientation of niobium grains in the substrate.The findings of the density functional theory(DFT)simulation are in good agreement with the experimental results,showing that the non-uniform distribution of tin nuclei is the result of the adsorption energy of SnCl_(2)molecules by varied niobium grain orientations.Further analysis indicated that the surface roughness and grain size of niobium also played significant roles in the nucleation behavior.This study provides valuable insights into enhancing the surface pretreatment of niobium substrates during the growth of Nb_(3)Sn thin films using the vapor diffusion method.展开更多
In low-light image enhancement,prevailing Retinex-based methods often struggle with precise illumina-tion estimation and brightness modulation.This can result in issues such as halo artifacts,blurred edges,and diminis...In low-light image enhancement,prevailing Retinex-based methods often struggle with precise illumina-tion estimation and brightness modulation.This can result in issues such as halo artifacts,blurred edges,and diminished details in bright regions,particularly under non-uniform illumination conditions.We propose an innovative approach that refines low-light images by leveraging an in-depth awareness of local content within the image.By introducing multi-scale effective guided filtering,our method surpasses the limitations of traditional isotropic filters,such as Gaussian filters,in handling non-uniform illumination.It dynamically adjusts regularization parameters in response to local image characteristics and significantly integrates edge perception across different scales.This balanced approach achieves a harmonious blend of smoothing and detail preservation,enabling more accurate illumination estimation.Additionally,we have designed an adaptive gamma correction function that dynamically adjusts the brightness value based on local pixel intensity,further balancing enhancement effects across different brightness levels in the image.Experimental results demonstrate the effectiveness of our proposed method for non-uniform illumination images across various scenarios.It exhibits superior quality and objective evaluation scores compared to existing methods.Our method effectively addresses potential issues that existing methods encounter when processing non-uniform illumination images,producing enhanced images with precise details and natural,vivid colors.展开更多
The intrusion of obstacles onto railway tracks presents a significant threat to train safety,characterized by sudden and unpredictable occurrences.With China leading the world in high-speed rail mileage,ensuring railw...The intrusion of obstacles onto railway tracks presents a significant threat to train safety,characterized by sudden and unpredictable occurrences.With China leading the world in high-speed rail mileage,ensuring railway security is paramount.The current laser monitoring technologies suffer from high false alarm rates and unreliable intrusion identification.This study addresses these issues by investigating high-resolution laser monitoring technology for railway obstacles,focusing on key parameters such as monitoring range and resolution.We propose an enhanced non-uniform laser scanning method,developing a laser monitoring system that reduces the obstacle false alarm rate to 2.00%,significantly lower than the 20%standard(TJ/GW135-2015).This rate is the best record for laser monitoring systems on China Railway.Our system operates seamlessly in all weather conditions,providing superior accuracy,resolution,and identification efficiency.It is the only 3D LiDAR system certified by the China State Railway Group Co.,Ltd.(Certificate No.[2023]008).Over three years,our system has been deployed at numerous points along various lines managed by the China State Railway Group,accumulating a dataset of 300,000 observations.This extensive deployment has significantly enhanced railway safety.The development and implementation of our railway laser monitoring system represent a substantial advancement in railway safety technology.Its low false alarm rate(2.00%),high accuracy(20 cm×20 cm×20 cm),and robust performance in diverse conditions underscore its potential for widespread adoption,promising to enhance railway safety in China and internationally.展开更多
Non-uniform quantization for messages in Low-Density Parity-Check(LDPC)decoding canreduce implementation complexity and mitigate performance loss.But the distribution of messagesvaries in the iterative decoding.This l...Non-uniform quantization for messages in Low-Density Parity-Check(LDPC)decoding canreduce implementation complexity and mitigate performance loss.But the distribution of messagesvaries in the iterative decoding.This letter proposes a variable non-uniform quantized Belief Propaga-tion(BP)algorithm.The BP decoding is analyzed by density evolution with Gaussian approximation.Since the probability density of messages can be well approximated by Gaussian distribution,by theunbiased estimation of variance,the distribution of messages can be tracked during the iteration.Thusthe non-uniform quantization scheme can be optimized to minimize the distortion.Simulation resultsshow that the variable non-uniform quantization scheme can achieve better error rate performance andfaster decoding convergence than the conventional non-uniform quantization and uniform quantizationschemes.展开更多
Many classical encoding algorithms of vector quantization （VQ） of image compression that can obtain global optimal solution have computational complexity O（N）. A pure quantum VQ encoding algorithm with probability...Many classical encoding algorithms of vector quantization （VQ） of image compression that can obtain global optimal solution have computational complexity O（N）. A pure quantum VQ encoding algorithm with probability of success near 100% has been proposed, that performs operations 45√N times approximately. In this paper, a hybrid quantum VQ encoding algorithm between the classical method and the quantum algorithm is presented. The number of its operations is less than √N for most images, and it is more efficient than the pure quantum algorithm.展开更多
The quantum object is in general considered as displaying both wave and particle nature. By particle is understood an item localized in a very small volume of the space, and which cannot be simultaneously in two disjo...The quantum object is in general considered as displaying both wave and particle nature. By particle is understood an item localized in a very small volume of the space, and which cannot be simultaneously in two disjoint regions of the space. By wave, to the contrary, is understood a distributed item, occupying in some cases two or more disjoint regions of the space. The quantum formalism did not explain until today the so-called “collapse” of the wave-function, i.e. the shrinking of the wave-function to one small region of the space, when a macroscopic object is encountered. This seems to happen in “which-way” experiments. A very appealing explanation for this behavior is the idea of a particle, localized in some limited part of the wave-function. The present article challenges the concept of particle. It proves in the base of a variant of the Tan, Walls and Collett experiment, that this concept leads to a situation in which the particle has to be simultaneously in two places distant from one another—situation that contradicts the very definition of a particle. Another argument is based on a modified version of the Afshar experiment, showing that the concept of particle is problematic. The concept of particle makes additional difficulties when the wave-function passes through fields. An unexpected possibility to solve these difficulties seems to arise from the cavity quantum electrodynamics studies done recently by S. Savasta and his collaborators. It involves virtual particles. One of these studies is briefly described here. Though, experimental results are needed, so that it is too soon to conclude whether it speaks in favor, or against the concept of particle.展开更多
This work studies the stabilization of a class of control systems that use communication networks as signal transmission medium. The lateral motion of independently actuated four-wheel vehicle is modeled as an uncerta...This work studies the stabilization of a class of control systems that use communication networks as signal transmission medium. The lateral motion of independently actuated four-wheel vehicle is modeled as an uncertain-linear system. Time delay and quantization density are modeled as Markov chains.The networked control systems(NCSs) with plants being lateral motion are first transformed to switched linear systems with uncertain parameters. Sufficient and necessary conditions for the stochastic stability of closed-loop networked control systems are then established. By solving the matrix inequalities, this work presents an output-feedback controller that depends on the modes of time delay and quantization density. The controller performance is illustrated via a vehicular lateral motion system.展开更多
We study quantum classical correspondence in terms of the coherent wave functions of a charged particle in two-dimensional central-scalar potentials as well as the gauge field of a magnetic flux in the sense that the ...We study quantum classical correspondence in terms of the coherent wave functions of a charged particle in two-dimensional central-scalar potentials as well as the gauge field of a magnetic flux in the sense that the probability clouds of wave functions are well localized on classical orbits. For both closed and open classical orbits, the non-integer angular-momentum quantization with the level space of angular momentum being greater or less than h is determined uniquely by the same rotational symmetry of classical orbits and probability clouds of coherent wave functions, which is not necessarily 27r-periodic. The gauge potential of a magnetic flux impenetrable to the particle cannot change the quantization rule but is able to shift the spectrum of canonical angular momentum by a flux-dependent value, which results in a common topological phase for all wave functions in the given model. The well-known quantum mechanical anyon model becomes a special case of the arbitrary quantization, where the classical orbits are 2π-periodic.展开更多
文摘The rapid growth of digital data necessitates advanced natural language processing(NLP)models like BERT(Bidi-rectional Encoder Representations from Transformers),known for its superior performance in text classification.However,BERT’s size and computational demands limit its practicality,especially in resource-constrained settings.This research compresses the BERT base model for Bengali emotion classification through knowledge distillation(KD),pruning,and quantization techniques.Despite Bengali being the sixth most spoken language globally,NLP research in this area is limited.Our approach addresses this gap by creating an efficient BERT-based model for Bengali text.We have explored 20 combinations for KD,quantization,and pruning,resulting in improved speedup,fewer parameters,and reduced memory size.Our best results demonstrate significant improvements in both speed and efficiency.For instance,in the case of mBERT,we achieved a 3.87×speedup and 4×compression ratio with a combination of Distil+Prune+Quant that reduced parameters from 178 to 46 M,while the memory size decreased from 711 to 178 MB.These results offer scalable solutions for NLP tasks in various languages and advance the field of model compression,making these models suitable for real-world applications in resource-limited environments.
基金The Qian Xuesen Youth Innovation Foundation from China Aerospace Science and Technology Corporation(Grant Number 2022JY51).
文摘The demand for adopting neural networks in resource-constrained embedded devices is continuously increasing.Quantization is one of the most promising solutions to reduce computational cost and memory storage on embedded devices.In order to reduce the complexity and overhead of deploying neural networks on Integeronly hardware,most current quantization methods use a symmetric quantization mapping strategy to quantize a floating-point neural network into an integer network.However,although symmetric quantization has the advantage of easier implementation,it is sub-optimal for cases where the range could be skewed and not symmetric.This often comes at the cost of lower accuracy.This paper proposed an activation redistribution-based hybrid asymmetric quantizationmethod for neural networks.The proposedmethod takes data distribution into consideration and can resolve the contradiction between the quantization accuracy and the ease of implementation,balance the trade-off between clipping range and quantization resolution,and thus improve the accuracy of the quantized neural network.The experimental results indicate that the accuracy of the proposed method is 2.02%and 5.52%higher than the traditional symmetric quantization method for classification and detection tasks,respectively.The proposed method paves the way for computationally intensive neural network models to be deployed on devices with limited computing resources.Codes will be available on https://github.com/ycjcy/Hybrid-Asymmetric-Quantization.
文摘The quantization algorithm compresses the original network by reducing the numerical bit width of the model,which improves the computation speed. Because different layers have different redundancy and sensitivity to databit width. Reducing the data bit width will result in a loss of accuracy. Therefore, it is difficult to determinethe optimal bit width for different parts of the network with guaranteed accuracy. Mixed precision quantizationcan effectively reduce the amount of computation while keeping the model accuracy basically unchanged. In thispaper, a hardware-aware mixed precision quantization strategy optimal assignment algorithm adapted to low bitwidth is proposed, and reinforcement learning is used to automatically predict the mixed precision that meets theconstraints of hardware resources. In the state-space design, the standard deviation of weights is used to measurethe distribution difference of data, the execution speed feedback of simulated neural network accelerator inferenceis used as the environment to limit the action space of the agent, and the accuracy of the quantization model afterretraining is used as the reward function to guide the agent to carry out deep reinforcement learning training. Theexperimental results show that the proposed method obtains a suitable model layer-by-layer quantization strategyunder the condition that the computational resources are satisfied, and themodel accuracy is effectively improved.The proposed method has strong intelligence and certain universality and has strong application potential in thefield of mixed precision quantization and embedded neural network model deployment.
基金Project supported by the National Natural Science Foundation of China(Grant Nos.12104414,12122412,12104464,and 12104413)the China Postdoctoral Science Foundation(Grant No.2021M702955).
文摘The recently developed magic-intensity trapping technique of neutral atoms efficiently mitigates the detrimental effect of light shifts on atomic qubits and substantially enhances the coherence time. This technique relies on applying a bias magnetic field precisely parallel to the wave vector of a circularly polarized trapping laser field. However, due to the presence of the vector light shift experienced by the trapped atoms, it is challenging to precisely define a parallel magnetic field, especially at a low bias magnetic field strength, for the magic-intensity trapping of85Rb qubits. In this work, we present a method to calibrate the angle between the bias magnetic field and the trapping laser field with the compensating magnetic fields in the other two directions orthogonal to the bias magnetic field direction. Experimentally, with a constantdepth trap and a fixed bias magnetic field, we measure the respective resonant frequencies of the atomic qubits in a linearly polarized trap and a circularly polarized one via the conventional microwave Rabi spectra with different compensating magnetic fields and obtain the corresponding total magnetic fields via the respective resonant frequencies using the Breit–Rabi formula. With known total magnetic fields, the angle is a function of the other two compensating magnetic fields.Finally, the projection value of the angle on either of the directions orthogonal to the bias magnetic field direction can be reduced to 0(4)° by applying specific compensating magnetic fields. The measurement error is mainly attributed to the fluctuation of atomic temperature. Moreover, it also demonstrates that, even for a small angle, the effect is strong enough to cause large decoherence of Rabi oscillation in a magic-intensity trap. Although the compensation method demonstrated here is explored for the magic-intensity trapping technique, it can be applied to a variety of similar precision measurements with trapped neutral atoms.
基金supported in part by the National Natural Science Foundation of China(NSFC)under Grants 61971127,61871465,61871122in part by the National Key Research and Development Program under Grant 2020YFB1806600in part by the open research fund of National Mobile Communications Research Laboratory,Southeast University under Grant 2022D11。
文摘In this paper,we investigate networkassisted full-duplex(NAFD)cell-free millimeter-wave(mmWave)massive multiple-input multiple-output(MIMO)systems with digital-to-analog converter(DAC)quantization and fronthaul compression.We propose to maximize the weighted uplink and downlink sum rate by jointly optimizing the power allocation of both the transmitting remote antenna units(T-RAUs)and uplink users and the variances of the downlink and uplink fronthaul compression noises.To deal with this challenging problem,we further apply a successive convex approximation(SCA)method to handle the non-convex bidirectional limited-capacity fronthaul constraints.The simulation results verify the convergence of the proposed SCA-based algorithm and analyze the impact of fronthaul capacity and DAC quantization on the spectral efficiency of the NAFD cell-free mmWave massive MIMO systems.Moreover,some insightful conclusions are obtained through the comparisons of spectral efficiency,which shows that NAFD achieves better performance gains than cotime co-frequency full-duplex cloud radio access network(CCFD C-RAN)in the cases of practical limited-resolution DACs.Specifically,their performance gaps with 8-bit DAC quantization are larger than that with1-bit DAC quantization,which attains a 5.5-fold improvement.
基金This work was supported by Open Fund Project of State Key Laboratory of Intelligent Vehicle Safety Technology by Grant with No.IVSTSKL-202311Key Projects of Science and Technology Research Programme of Chongqing Municipal Education Commission by Grant with No.KJZD-K202301505+1 种基金Cooperation Project between Chongqing Municipal Undergraduate Universities and Institutes Affiliated to the Chinese Academy of Sciences in 2021 by Grant with No.HZ2021015Chongqing Graduate Student Research Innovation Program by Grant with No.CYS240801.
文摘Massive computational complexity and memory requirement of artificial intelligence models impede their deploy-ability on edge computing devices of the Internet of Things(IoT).While Power-of-Two(PoT)quantization is pro-posed to improve the efficiency for edge inference of Deep Neural Networks(DNNs),existing PoT schemes require a huge amount of bit-wise manipulation and have large memory overhead,and their efficiency is bounded by the bottleneck of computation latency and memory footprint.To tackle this challenge,we present an efficient inference approach on the basis of PoT quantization and model compression.An integer-only scalar PoT quantization(IOS-PoT)is designed jointly with a distribution loss regularizer,wherein the regularizer minimizes quantization errors and training disturbances.Additionally,two-stage model compression is developed to effectively reduce memory requirement,and alleviate bandwidth usage in communications of networked heterogenous learning systems.The product look-up table(P-LUT)inference scheme is leveraged to replace bit-shifting with only indexing and addition operations for achieving low-latency computation and implementing efficient edge accelerators.Finally,comprehensive experiments on Residual Networks(ResNets)and efficient architectures with Canadian Institute for Advanced Research(CIFAR),ImageNet,and Real-world Affective Faces Database(RAF-DB)datasets,indicate that our approach achieves 2×∼10×improvement in the reduction of both weight size and computation cost in comparison to state-of-the-art methods.A P-LUT accelerator prototype is implemented on the Xilinx KV260 Field Programmable Gate Array(FPGA)platform for accelerating convolution operations,with performance results showing that P-LUT reduces memory footprint by 1.45×,achieves more than 3×power efficiency and 2×resource efficiency,compared to the conventional bit-shifting scheme.
文摘A new steganographic method by pixel-value differencing(PVD)using general quantization ranges of pixel pairs’difference values is proposed.The objective of this method is to provide a data embedding technique with a range table with range widths not limited to powers of 2,extending PVD-based methods to enhance their flexibility and data-embedding rates without changing their capabilities to resist security attacks.Specifically,the conventional PVD technique partitions a grayscale image into 1×2 non-overlapping blocks.The entire range[0,255]of all possible absolute values of the pixel pairs’grayscale differences in the blocks is divided into multiple quantization ranges.The width of each quantization range is a power of two to facilitate the direct embedding of the bit information with high embedding rates.Without using power-of-two range widths,the embedding rates can drop using conventional embedding techniques.In contrast,the proposed method uses general quantization range widths,and a multiple-based number conversion mechanism is employed skillfully to implement the use of nonpower-of-two range widths,with each pixel pair being employed to embed a digit in the multiple-based number.All the message bits are converted into a big multiple-based number whose digits can be embedded into the pixel pairs with a higher embedding rate.Good experimental results showed the feasibility of the proposed method and its resistance to security attacks.In addition,implementation examples are provided,where the proposed method adopts non-power-of-two range widths and employsmultiple-based number conversion to expand the data-hiding and steganalysis-resisting capabilities of other PVD methods.
基金funded by the National Science Foundation of China(62006068)Hebei Natural Science Foundation(A2021402008),Natural Science Foundation of Scientific Research Project of Higher Education in Hebei Province(ZD2020185,QN2020188)333 Talent Supported Project of Hebei Province(C20221026).
文摘Imbalanced datasets are common in practical applications,and oversampling methods using fuzzy rules have been shown to enhance the classification performance of imbalanced data by taking into account the relationship between data attributes.However,the creation of fuzzy rules typically depends on expert knowledge,which may not fully leverage the label information in training data and may be subjective.To address this issue,a novel fuzzy rule oversampling approach is developed based on the learning vector quantization(LVQ)algorithm.In this method,the label information of the training data is utilized to determine the antecedent part of If-Then fuzzy rules by dynamically dividing attribute intervals using LVQ.Subsequently,fuzzy rules are generated and adjusted to calculate rule weights.The number of new samples to be synthesized for each rule is then computed,and samples from the minority class are synthesized based on the newly generated fuzzy rules.This results in the establishment of a fuzzy rule oversampling method based on LVQ.To evaluate the effectiveness of this method,comparative experiments are conducted on 12 publicly available imbalance datasets with five other sampling techniques in combination with the support function machine.The experimental results demonstrate that the proposed method can significantly enhance the classification algorithm across seven performance indicators,including a boost of 2.15%to 12.34%in Accuracy,6.11%to 27.06%in G-mean,and 4.69%to 18.78%in AUC.These show that the proposed method is capable of more efficiently improving the classification performance of imbalanced data.
基金supported by the Fundamental Research Funds for the Central Universities(DUT22RT(3)090)the National Natural Science Foundation of China(61890920,61890921,62122016,08120003)Liaoning Science and Technology Program(2023JH2/101700361).
文摘Linear temporal logic(LTL)is an intuitive and expressive language to specify complex control tasks,and how to design an efficient control strategy for LTL specification is still a challenge.In this paper,we implement the dynamic quantization technique to propose a novel hierarchical control strategy for nonlinear control systems under LTL specifications.Based on the regions of interest involved in the LTL formula,an accepting path is derived first to provide a high-level solution for the controller synthesis problem.Second,we develop a dynamic quantization based approach to verify the realization of the accepting path.The realization verification results in the necessity of the controller design and a sequence of quantization regions for the controller design.Third,the techniques of dynamic quantization and abstraction-based control are combined together to establish the local-to-global control strategy.Both abstraction construction and controller design are local and dynamic,thereby resulting in the potential reduction of the computational complexity.Since each quantization region can be considered locally and individually,the proposed hierarchical mechanism is more efficient and can solve much larger problems than many existing methods.Finally,the proposed control strategy is illustrated via two examples from the path planning and tracking problems of mobile robots.
文摘What does it mean to study PDE (Partial Differential Equation)? How and what to do “to claim proudly that I’m studying a certain PDE”? Newton mechanic uses mainly ODE (Ordinary Differential Equation) and describes nicely movements of Sun, Moon and Earth etc. Now, so-called quantum phenomenum is described by, say Schrödinger equation, PDE which explains both wave and particle characters after quantization of ODE. The coupled Maxwell-Dirac equation is also “quantized” and QED (Quantum Electro-Dynamics) theory is invented by physicists. Though it is said this QED gives very good coincidence between theoretical1 and experimental observed quantities, but what is the equation corresponding to QED? Or, is it possible to describe QED by “equation” in naive sense?
文摘The uncertainty principle is a fundamental principle of quantum mechanics, but its exact mathematical expression cannot obtain correct results when used to solve theoretical problems such as the energy levels of hydrogen atoms, one-dimensional deep potential wells, one-dimensional harmonic oscillators, and double-slit experiments. Even after approximate treatment, the results obtained are not completely consistent with those obtained by solving Schrödinger’s equation. This indicates that further research on the uncertainty principle is necessary. Therefore, using the de Broglie matter wave hypothesis, we quantize the action of an elementary particle in natural coordinates and obtain the quantization condition and a new deterministic relation. Using this quantization condition, we obtain the energy level formulas of an elementary particle in different conditions in a classical way that is completely consistent with the results obtained by solving Schrödinger’s equation. A new physical interpretation is given for the particle eigenfunction independence of probability for an elementary particle: an elementary particle is in a particle state at the space-time point where the action is quantized, and in a wave state in the rest of the space-time region. The space-time points of particle nature and the wave regions of particle motion constitute the continuous trajectory of particle motion. When an elementary particle is in a particle state, it is localized, whereas in the wave state region, it is nonlocalized.
基金supported by the National Key R&D Program of China(No.2021YFB0301200)National Natural Science Foundation of China(No.62025208).
文摘Large-scale Language Models(LLMs)have achieved significant breakthroughs in Natural Language Processing(NLP),driven by the pre-training and fine-tuning paradigm.While this approach allows models to specialize in specific tasks with reduced training costs,the substantial memory requirements during fine-tuning present a barrier to broader deployment.Parameter-Efficient Fine-Tuning(PEFT)techniques,such as Low-Rank Adaptation(LoRA),and parameter quantization methods have emerged as solutions to address these challenges by optimizing memory usage and computational efficiency.Among these,QLoRA,which combines PEFT and quantization,has demonstrated notable success in reducing memory footprints during fine-tuning,prompting the development of various QLoRA variants.Despite these advancements,the quantitative impact of key variables on the fine-tuning performance of quantized LLMs remains underexplored.This study presents a comprehensive analysis of these key variables,focusing on their influence across different layer types and depths within LLM architectures.Our investigation uncovers several critical findings:(1)Larger layers,such as MLP layers,can maintain performance despite reductions in adapter rank,while smaller layers,like self-attention layers,aremore sensitive to such changes;(2)The effectiveness of balancing factors depends more on specific values rather than layer type or depth;(3)In quantization-aware fine-tuning,larger layers can effectively utilize smaller adapters,whereas smaller layers struggle to do so.These insights suggest that layer type is a more significant determinant of fine-tuning success than layer depth when optimizing quantized LLMs.Moreover,for the same discount of trainable parameters,reducing the trainable parameters in a larger layer is more effective in preserving fine-tuning accuracy than in a smaller one.This study provides valuable guidance for more efficient fine-tuning strategies and opens avenues for further research into optimizing LLM fine-tuning in resource-constrained environments.
基金supported by the National Natural Science Foundation of China(No.12175283)Youth Innovation Promotion Association of Chinese Academy of Sciences(2020410)Advanced Energy Science and Technology Guangdong Laboratory(HND20TDSPCD,HND22PTDZD).
文摘Growth of high-quality Nb_(3)Sn thin films for superconducting radiofrequency(SRF)applications using the vapor diffusion method requires a uniform distribution of tin nuclei on the niobium(Nb)surface.This study examines the mechanism underlying the observed non-uniform distribution of tin nuclei with tin chloride SnCl_(2).Electron backscatter diffraction(EBSD)analysis was used to examine the correlation between the nucleation behavior and orientation of niobium grains in the substrate.The findings of the density functional theory(DFT)simulation are in good agreement with the experimental results,showing that the non-uniform distribution of tin nuclei is the result of the adsorption energy of SnCl_(2)molecules by varied niobium grain orientations.Further analysis indicated that the surface roughness and grain size of niobium also played significant roles in the nucleation behavior.This study provides valuable insights into enhancing the surface pretreatment of niobium substrates during the growth of Nb_(3)Sn thin films using the vapor diffusion method.
文摘In low-light image enhancement,prevailing Retinex-based methods often struggle with precise illumina-tion estimation and brightness modulation.This can result in issues such as halo artifacts,blurred edges,and diminished details in bright regions,particularly under non-uniform illumination conditions.We propose an innovative approach that refines low-light images by leveraging an in-depth awareness of local content within the image.By introducing multi-scale effective guided filtering,our method surpasses the limitations of traditional isotropic filters,such as Gaussian filters,in handling non-uniform illumination.It dynamically adjusts regularization parameters in response to local image characteristics and significantly integrates edge perception across different scales.This balanced approach achieves a harmonious blend of smoothing and detail preservation,enabling more accurate illumination estimation.Additionally,we have designed an adaptive gamma correction function that dynamically adjusts the brightness value based on local pixel intensity,further balancing enhancement effects across different brightness levels in the image.Experimental results demonstrate the effectiveness of our proposed method for non-uniform illumination images across various scenarios.It exhibits superior quality and objective evaluation scores compared to existing methods.Our method effectively addresses potential issues that existing methods encounter when processing non-uniform illumination images,producing enhanced images with precise details and natural,vivid colors.
基金financially supported by the National Natural Science Foundation of China(Nos.62275244,62375258,62225507,U2033211,62175230,and 62175232)the CAS Project for Young Scientists in Basic Research(No.YSBR-065)+2 种基金Scientific Instrument Developing Project of the Chinese Academy of Sciences(No.YJKYYQ20200001)National Key R&D Program of China(No.2022YFB3607800,No.2022YFB3605800,and No.2022YFB4601501)Key Program of the Chinese Academy of Sciences(ZDBS-ZRKJZ-TLC018)。
文摘The intrusion of obstacles onto railway tracks presents a significant threat to train safety,characterized by sudden and unpredictable occurrences.With China leading the world in high-speed rail mileage,ensuring railway security is paramount.The current laser monitoring technologies suffer from high false alarm rates and unreliable intrusion identification.This study addresses these issues by investigating high-resolution laser monitoring technology for railway obstacles,focusing on key parameters such as monitoring range and resolution.We propose an enhanced non-uniform laser scanning method,developing a laser monitoring system that reduces the obstacle false alarm rate to 2.00%,significantly lower than the 20%standard(TJ/GW135-2015).This rate is the best record for laser monitoring systems on China Railway.Our system operates seamlessly in all weather conditions,providing superior accuracy,resolution,and identification efficiency.It is the only 3D LiDAR system certified by the China State Railway Group Co.,Ltd.(Certificate No.[2023]008).Over three years,our system has been deployed at numerous points along various lines managed by the China State Railway Group,accumulating a dataset of 300,000 observations.This extensive deployment has significantly enhanced railway safety.The development and implementation of our railway laser monitoring system represent a substantial advancement in railway safety technology.Its low false alarm rate(2.00%),high accuracy(20 cm×20 cm×20 cm),and robust performance in diverse conditions underscore its potential for widespread adoption,promising to enhance railway safety in China and internationally.
基金the Aerospace Technology Support Foun-dation of China(No.J04-2005040).
文摘Non-uniform quantization for messages in Low-Density Parity-Check(LDPC)decoding canreduce implementation complexity and mitigate performance loss.But the distribution of messagesvaries in the iterative decoding.This letter proposes a variable non-uniform quantized Belief Propaga-tion(BP)algorithm.The BP decoding is analyzed by density evolution with Gaussian approximation.Since the probability density of messages can be well approximated by Gaussian distribution,by theunbiased estimation of variance,the distribution of messages can be tracked during the iteration.Thusthe non-uniform quantization scheme can be optimized to minimize the distortion.Simulation resultsshow that the variable non-uniform quantization scheme can achieve better error rate performance andfaster decoding convergence than the conventional non-uniform quantization and uniform quantizationschemes.
文摘Many classical encoding algorithms of vector quantization （VQ） of image compression that can obtain global optimal solution have computational complexity O（N）. A pure quantum VQ encoding algorithm with probability of success near 100% has been proposed, that performs operations 45√N times approximately. In this paper, a hybrid quantum VQ encoding algorithm between the classical method and the quantum algorithm is presented. The number of its operations is less than √N for most images, and it is more efficient than the pure quantum algorithm.
文摘The quantum object is in general considered as displaying both wave and particle nature. By particle is understood an item localized in a very small volume of the space, and which cannot be simultaneously in two disjoint regions of the space. By wave, to the contrary, is understood a distributed item, occupying in some cases two or more disjoint regions of the space. The quantum formalism did not explain until today the so-called “collapse” of the wave-function, i.e. the shrinking of the wave-function to one small region of the space, when a macroscopic object is encountered. This seems to happen in “which-way” experiments. A very appealing explanation for this behavior is the idea of a particle, localized in some limited part of the wave-function. The present article challenges the concept of particle. It proves in the base of a variant of the Tan, Walls and Collett experiment, that this concept leads to a situation in which the particle has to be simultaneously in two places distant from one another—situation that contradicts the very definition of a particle. Another argument is based on a modified version of the Afshar experiment, showing that the concept of particle is problematic. The concept of particle makes additional difficulties when the wave-function passes through fields. An unexpected possibility to solve these difficulties seems to arise from the cavity quantum electrodynamics studies done recently by S. Savasta and his collaborators. It involves virtual particles. One of these studies is briefly described here. Though, experimental results are needed, so that it is too soon to conclude whether it speaks in favor, or against the concept of particle.
基金supported by Humanities and Social Sciences Youth Foundation of the Ministry of Education(17YJC630093)
文摘This work studies the stabilization of a class of control systems that use communication networks as signal transmission medium. The lateral motion of independently actuated four-wheel vehicle is modeled as an uncertain-linear system. Time delay and quantization density are modeled as Markov chains.The networked control systems(NCSs) with plants being lateral motion are first transformed to switched linear systems with uncertain parameters. Sufficient and necessary conditions for the stochastic stability of closed-loop networked control systems are then established. By solving the matrix inequalities, this work presents an output-feedback controller that depends on the modes of time delay and quantization density. The controller performance is illustrated via a vehicular lateral motion system.
基金supported by the National Natural Science Foundation of China (Grant No. 11075099)
文摘We study quantum classical correspondence in terms of the coherent wave functions of a charged particle in two-dimensional central-scalar potentials as well as the gauge field of a magnetic flux in the sense that the probability clouds of wave functions are well localized on classical orbits. For both closed and open classical orbits, the non-integer angular-momentum quantization with the level space of angular momentum being greater or less than h is determined uniquely by the same rotational symmetry of classical orbits and probability clouds of coherent wave functions, which is not necessarily 27r-periodic. The gauge potential of a magnetic flux impenetrable to the particle cannot change the quantization rule but is able to shift the spectrum of canonical angular momentum by a flux-dependent value, which results in a common topological phase for all wave functions in the given model. The well-known quantum mechanical anyon model becomes a special case of the arbitrary quantization, where the classical orbits are 2π-periodic.
