In this paper we discuss policy iteration methods for approximate solution of a finite-state discounted Markov decision problem, with a focus on feature-based aggregation methods and their connection with deep reinfor...In this paper we discuss policy iteration methods for approximate solution of a finite-state discounted Markov decision problem, with a focus on feature-based aggregation methods and their connection with deep reinforcement learning schemes. We introduce features of the states of the original problem, and we formulate a smaller "aggregate" Markov decision problem, whose states relate to the features. We discuss properties and possible implementations of this type of aggregation, including a new approach to approximate policy iteration. In this approach the policy improvement operation combines feature-based aggregation with feature construction using deep neural networks or other calculations. We argue that the cost function of a policy may be approximated much more accurately by the nonlinear function of the features provided by aggregation, than by the linear function of the features provided by neural networkbased reinforcement learning, thereby potentially leading to more effective policy improvement.展开更多
Tourism route planning is widely applied in the smart tourism field.The Pareto-optimal front obtained by the traditional multi-objective evolutionary algorithm exhibits long tails,sharp peaks and disconnected regions ...Tourism route planning is widely applied in the smart tourism field.The Pareto-optimal front obtained by the traditional multi-objective evolutionary algorithm exhibits long tails,sharp peaks and disconnected regions problems,which leads to uneven distribution and weak diversity of optimization solutions of tourism routes.Inspired by these limitations,we propose a multi-objective evolutionary algorithm for tourism route recommendation(MOTRR)with two-stage and Pareto layering based on decomposition.The method decomposes the multiobjective problem into several subproblems,and improves the distribution of solutions through a two-stage method.The crowding degree mechanism between extreme and intermediate populations is used in the two-stage method.The neighborhood is determined according to the weight of the subproblem for crossover mutation.Finally,Pareto layering is used to improve the updating efficiency and population diversity of the solution.The two-stage method is combined with the Pareto layering structure,which not only maintains the distribution and diversity of the algorithm,but also avoids the same solutions.Compared with several classical benchmark algorithms,the experimental results demonstrate competitive advantages on five test functions,hypervolume(HV)and inverted generational distance(IGD)metrics.Using the experimental results of real scenic spot datasets from two famous tourism social networking sites with vast amounts of users and large-scale online comments in Beijing,our proposed algorithm shows better distribution.It proves that the tourism routes recommended by our proposed algorithm have better distribution and diversity,so that the recommended routes can better meet the personalized needs of tourists.展开更多
Demand response, the reactive power output of distributed generation(DG), and network reconfiguration have significant impacts on a DG allocation strategy. In this context, a novel real-time price-based demand respons...Demand response, the reactive power output of distributed generation(DG), and network reconfiguration have significant impacts on a DG allocation strategy. In this context, a novel real-time price-based demand response formulation is integrated into the allocation model of DG. The tariff is regulated by the difference between the load and active power of renewable energy. Meanwhile, network reconfiguration and the capacity curve describing the active and reactive power limits of DG are included in the optimization model for promoting the allocation of DG.With these measures, the optimal allocation model of DG is established with the goal of maximizing the net annual profit while guaranteeing the efficient utilization of renewable energy. In addition, the uncertainties of renewable energy are considered on the basis of a two-stage robust optimization method. Finally, the entire optimization model is solved by the column and constraint generation algorithm in the IEEE 33-bus distribution system and a practical 99-bus distribution system. Numerical simulations show that the proposed model is effective in terms of improving both the usage of renewable energy and net annual profit.展开更多
Controlled islanding plays an essential role in preventing the blackout of power systems.Although there are several studies on this topic in the past,no enough attention is paid to the uncertainty brought by renewable...Controlled islanding plays an essential role in preventing the blackout of power systems.Although there are several studies on this topic in the past,no enough attention is paid to the uncertainty brought by renewable energy sources(RESs)that may cause unpredictable unbalanced power and the observabilit>T of power systems after islanding that is essential for back-up black-start measures.Therefore,a novel controlled islanding model based on mixed-integer second-order cone and chance-constrained programming(MISOCCP)is proposed to address these issues.First,the uncertainty of RESs is characterized by their possibility distribution models with chance constraints,and the requirements,e.g.,system observability,for rapid back-up black-start measures are also considered.Then,a law of large numbers(LLN)based method is em-ployed for converting the chance constraints into deterministic ones and reformulating the non-convex model into convex one.Finally,case studies on the revised IEEE 39-bus and 118-bus power systems as well as the comparisons among different models are given to demonstrate the effectiveness of the proposed model.The results show that the proposed model can result in less unbalanced power and better observability after islanding compared with other models.展开更多
In this paper, controller parameters of static var compensators(SVCs) at planned locations are optimized to mitigate fault-induced delayed voltage recovery issues and improve angular stability of a multi-machine power...In this paper, controller parameters of static var compensators(SVCs) at planned locations are optimized to mitigate fault-induced delayed voltage recovery issues and improve angular stability of a multi-machine power system. The problem is formulated as a nonlinear optimization problem involving constraints on post-fault trajectories of voltages and frequencies. This paper proposes a mesh adaptive direct search based algorithm interfaced with a power system simulator for the optimization of SVC controllerparameters. The proposed method is tested on an NPCC140-bus system to optimize controller parameters for three SVCs. Simulations on critical contingencies verify that post-fault transient voltages and generator speeds can both quickly recover and transient stability of the system is improved.展开更多
Demand response(DR)has received much attention for its ability to balance the changing power supply and demand with flexibility.DR aggregators play an important role in aggregating flexible loads that are too small to...Demand response(DR)has received much attention for its ability to balance the changing power supply and demand with flexibility.DR aggregators play an important role in aggregating flexible loads that are too small to participate in electricity markets.In this work,a DR operation framework is presented to enable local management of customers to participate in electricity market.A novel optimization model is proposed for the DR aggregator with multiple objectives.On one hand,it attempts to obtain the optimal design of different DR contracts as well as the portfolio management so that the DR aggregator can maximize its profit.On the other hand,the customers’welfare should be maximized to incentivize users to enroll in DR programs which ensure the effective and flexible load control.The consumer psychology is introduced to model the consumers’behavior during contract signing.Several simulation studies are performed to demonstrate the feasibility of the proposed model.The results illustrate that the proposed model can ensure the profit of the DR aggregator whereas the customers’welfare is considered.展开更多
For a hierarchical cognitive radio network(CRN),the secondary users(SUs) may access the licensed spectrum opportunistically,whenever it is not occupied by the primary users(PUs).An important issue for this kind of CRN...For a hierarchical cognitive radio network(CRN),the secondary users(SUs) may access the licensed spectrum opportunistically,whenever it is not occupied by the primary users(PUs).An important issue for this kind of CRN is the achievable qualityof-service(QoS) performance,such as traffic transmission delay,which is critical to the SUs' traffic experience.In this paper,we focus on the delay performance analysis of the SU system and the design of the corresponding optimal access strategy for the case of SUs sharing multiple licensed channels.In our analysis,the transmission of PU and SU traffic is modeled as M/G/1 queues.By merging the PU and SU traffic,we propose the model of a priority virtual queue on the licensed channels.Based on this model,we obtain the expected system delay expression for SU traffic through M/G/1 preemptive repeat priority queuing analysis.For the case of multiple licensed channel access,the access strategy is further investigated with respect to the expected system delay for SU traffic.By minimizing the expected transmission delay,the optimal access strategy is modeled as a nonlinear programming problem,which can be resolved by means of the classic Genetic Algorithm(GA).Numerical results validate our analysis and design of an optimal access strategy.Meanwhile,by considering the time taken by the GA approach,we can also adopt the inverse proportional access strategy to obtain near-optimal results in practice.展开更多
Firewalls are crucial elements that enhance network security by examining the field values of every packet and deciding whether to accept or discard a packet according to the firewall policies. With the development of...Firewalls are crucial elements that enhance network security by examining the field values of every packet and deciding whether to accept or discard a packet according to the firewall policies. With the development of networks, the number of rules in firewalls has rapidly increased, consequently degrading network performance.In addition, because most real-life firewalls have been plagued with policy conflicts, malicious traffics can be allowed or legitimate traffics can be blocked. Moreover, because of the complexity of the firewall policies, it is very important to reduce the number of rules in a firewall while keeping the rule semantics unchanged and the target firewall rules conflict-free. In this study, we make three major contributions. First, we present a new approach in which a geometric model, multidimensional rectilinear polygon, is constructed for the firewall rules compression problem.Second, we propose a new scheme, Firewall Policies Compression(FPC), to compress the multidimensional firewall rules based on this geometric model. Third, we conducted extensive experiments to evaluate the performance of the proposed method. The experimental results demonstrate that the FPC method outperforms the existing approaches, in terms of compression ratio and efficiency while maintaining conflict-free firewall rules.展开更多
A voltage security region(VSR)is a powerful tool for monitoring the voltage security in bulk power grids with high penetration of renewables.It can prevent cascading failures in wind power integration areas caused by ...A voltage security region(VSR)is a powerful tool for monitoring the voltage security in bulk power grids with high penetration of renewables.It can prevent cascading failures in wind power integration areas caused by serious over or low voltage problems.The bottlenecks of a VSR for practical applications are computational efficiency and accuracy.To bridge these gaps,a general optimization model for tracking a voltage security region boundary(VSRB)in bulk power grids is developed in this paper in accordance with the topological characteristics of the VSRB.First,the initial VSRB point on the VSRB is examined with the traditional OPF by using the base case parameters as initial values.Then,the rest of the VSRB points on the VSRB are tracked one after another,with the proposed optimization model,by using the parameters of the tracked VSRB point as the initial value to explore its adjacent VSRB point.The proposed approach can significantly improve the computational efficiency of the VSRB tracking over the existing algorithms,and case studies,in the WECC 9-bus and the Polish 2736-bus test systems,demonstrate the high accuracy and efficiency of the proposed approach on exploring the VSRB.展开更多
文摘In this paper we discuss policy iteration methods for approximate solution of a finite-state discounted Markov decision problem, with a focus on feature-based aggregation methods and their connection with deep reinforcement learning schemes. We introduce features of the states of the original problem, and we formulate a smaller "aggregate" Markov decision problem, whose states relate to the features. We discuss properties and possible implementations of this type of aggregation, including a new approach to approximate policy iteration. In this approach the policy improvement operation combines feature-based aggregation with feature construction using deep neural networks or other calculations. We argue that the cost function of a policy may be approximated much more accurately by the nonlinear function of the features provided by aggregation, than by the linear function of the features provided by neural networkbased reinforcement learning, thereby potentially leading to more effective policy improvement.
基金partially supported by the National Natural Science Foundation of China(41930644,61972439)the Collaborative Innovation Project of Anhui Province(GXXT-2022-093)the Key Program in the Youth Elite Support Plan in Universities of Anhui Province(gxyqZD2019010)。
文摘Tourism route planning is widely applied in the smart tourism field.The Pareto-optimal front obtained by the traditional multi-objective evolutionary algorithm exhibits long tails,sharp peaks and disconnected regions problems,which leads to uneven distribution and weak diversity of optimization solutions of tourism routes.Inspired by these limitations,we propose a multi-objective evolutionary algorithm for tourism route recommendation(MOTRR)with two-stage and Pareto layering based on decomposition.The method decomposes the multiobjective problem into several subproblems,and improves the distribution of solutions through a two-stage method.The crowding degree mechanism between extreme and intermediate populations is used in the two-stage method.The neighborhood is determined according to the weight of the subproblem for crossover mutation.Finally,Pareto layering is used to improve the updating efficiency and population diversity of the solution.The two-stage method is combined with the Pareto layering structure,which not only maintains the distribution and diversity of the algorithm,but also avoids the same solutions.Compared with several classical benchmark algorithms,the experimental results demonstrate competitive advantages on five test functions,hypervolume(HV)and inverted generational distance(IGD)metrics.Using the experimental results of real scenic spot datasets from two famous tourism social networking sites with vast amounts of users and large-scale online comments in Beijing,our proposed algorithm shows better distribution.It proves that the tourism routes recommended by our proposed algorithm have better distribution and diversity,so that the recommended routes can better meet the personalized needs of tourists.
基金supported by the National Key R&D Program of China (No.2019YFE0111500)the National Natural Science Foundation of China(No. 51807125)Sichuan Science and Technology Program (No.2020YFH0040)。
文摘Demand response, the reactive power output of distributed generation(DG), and network reconfiguration have significant impacts on a DG allocation strategy. In this context, a novel real-time price-based demand response formulation is integrated into the allocation model of DG. The tariff is regulated by the difference between the load and active power of renewable energy. Meanwhile, network reconfiguration and the capacity curve describing the active and reactive power limits of DG are included in the optimization model for promoting the allocation of DG.With these measures, the optimal allocation model of DG is established with the goal of maximizing the net annual profit while guaranteeing the efficient utilization of renewable energy. In addition, the uncertainties of renewable energy are considered on the basis of a two-stage robust optimization method. Finally, the entire optimization model is solved by the column and constraint generation algorithm in the IEEE 33-bus distribution system and a practical 99-bus distribution system. Numerical simulations show that the proposed model is effective in terms of improving both the usage of renewable energy and net annual profit.
基金the National Natural Science Foundation of China(No.51777185)National Key R&D Program of China(No.2016YFB0900100)Zhejiang University Academic Award for Outstanding Doctoral Candidates.
文摘Controlled islanding plays an essential role in preventing the blackout of power systems.Although there are several studies on this topic in the past,no enough attention is paid to the uncertainty brought by renewable energy sources(RESs)that may cause unpredictable unbalanced power and the observabilit>T of power systems after islanding that is essential for back-up black-start measures.Therefore,a novel controlled islanding model based on mixed-integer second-order cone and chance-constrained programming(MISOCCP)is proposed to address these issues.First,the uncertainty of RESs is characterized by their possibility distribution models with chance constraints,and the requirements,e.g.,system observability,for rapid back-up black-start measures are also considered.Then,a law of large numbers(LLN)based method is em-ployed for converting the chance constraints into deterministic ones and reformulating the non-convex model into convex one.Finally,case studies on the revised IEEE 39-bus and 118-bus power systems as well as the comparisons among different models are given to demonstrate the effectiveness of the proposed model.The results show that the proposed model can result in less unbalanced power and better observability after islanding compared with other models.
基金supported in part by the ERC Program of the NSF and DOE under NSF (No. EEC-1041877)
文摘In this paper, controller parameters of static var compensators(SVCs) at planned locations are optimized to mitigate fault-induced delayed voltage recovery issues and improve angular stability of a multi-machine power system. The problem is formulated as a nonlinear optimization problem involving constraints on post-fault trajectories of voltages and frequencies. This paper proposes a mesh adaptive direct search based algorithm interfaced with a power system simulator for the optimization of SVC controllerparameters. The proposed method is tested on an NPCC140-bus system to optimize controller parameters for three SVCs. Simulations on critical contingencies verify that post-fault transient voltages and generator speeds can both quickly recover and transient stability of the system is improved.
基金supported in part by the National Natural Science Foundation of China(No.51777030)in part by CURENT,a U.S.NSF/DOE Engineering Research Center+1 种基金through NSF under Award EEC-1081477the China Scholarship Council(No.201706090150)。
文摘Demand response(DR)has received much attention for its ability to balance the changing power supply and demand with flexibility.DR aggregators play an important role in aggregating flexible loads that are too small to participate in electricity markets.In this work,a DR operation framework is presented to enable local management of customers to participate in electricity market.A novel optimization model is proposed for the DR aggregator with multiple objectives.On one hand,it attempts to obtain the optimal design of different DR contracts as well as the portfolio management so that the DR aggregator can maximize its profit.On the other hand,the customers’welfare should be maximized to incentivize users to enroll in DR programs which ensure the effective and flexible load control.The consumer psychology is introduced to model the consumers’behavior during contract signing.Several simulation studies are performed to demonstrate the feasibility of the proposed model.The results illustrate that the proposed model can ensure the profit of the DR aggregator whereas the customers’welfare is considered.
基金supported by the National Basic Research Program of China (2009CB320405)National Natural Science Foundation of China(61071102)+1 种基金National Science and Technology Major Project of China(2010ZX03006-002-02 and 2010ZX03005-003)the Foundation Project of National Key Laboratory of Science and Technology on Communications (9140C0202061004)
文摘For a hierarchical cognitive radio network(CRN),the secondary users(SUs) may access the licensed spectrum opportunistically,whenever it is not occupied by the primary users(PUs).An important issue for this kind of CRN is the achievable qualityof-service(QoS) performance,such as traffic transmission delay,which is critical to the SUs' traffic experience.In this paper,we focus on the delay performance analysis of the SU system and the design of the corresponding optimal access strategy for the case of SUs sharing multiple licensed channels.In our analysis,the transmission of PU and SU traffic is modeled as M/G/1 queues.By merging the PU and SU traffic,we propose the model of a priority virtual queue on the licensed channels.Based on this model,we obtain the expected system delay expression for SU traffic through M/G/1 preemptive repeat priority queuing analysis.For the case of multiple licensed channel access,the access strategy is further investigated with respect to the expected system delay for SU traffic.By minimizing the expected transmission delay,the optimal access strategy is modeled as a nonlinear programming problem,which can be resolved by means of the classic Genetic Algorithm(GA).Numerical results validate our analysis and design of an optimal access strategy.Meanwhile,by considering the time taken by the GA approach,we can also adopt the inverse proportional access strategy to obtain near-optimal results in practice.
基金supported by the National Natural Science Foundation of China(Nos.61672543 and 61402542)Research Foundation of the Education Department of Hunan Province(No.17B022)Hunan Provincial Innovation Foundation for Postgraduate(No.CX2014B081)
文摘Firewalls are crucial elements that enhance network security by examining the field values of every packet and deciding whether to accept or discard a packet according to the firewall policies. With the development of networks, the number of rules in firewalls has rapidly increased, consequently degrading network performance.In addition, because most real-life firewalls have been plagued with policy conflicts, malicious traffics can be allowed or legitimate traffics can be blocked. Moreover, because of the complexity of the firewall policies, it is very important to reduce the number of rules in a firewall while keeping the rule semantics unchanged and the target firewall rules conflict-free. In this study, we make three major contributions. First, we present a new approach in which a geometric model, multidimensional rectilinear polygon, is constructed for the firewall rules compression problem.Second, we propose a new scheme, Firewall Policies Compression(FPC), to compress the multidimensional firewall rules based on this geometric model. Third, we conducted extensive experiments to evaluate the performance of the proposed method. The experimental results demonstrate that the FPC method outperforms the existing approaches, in terms of compression ratio and efficiency while maintaining conflict-free firewall rules.
基金This work was supported in part by the National Natural Science Foundation of China(No.52077029 and U2066208)National Key Research and Development Program of China(2016YFB0900903)International Clear Energy Talent Programme(iCET)of China Scholarship Council.
文摘A voltage security region(VSR)is a powerful tool for monitoring the voltage security in bulk power grids with high penetration of renewables.It can prevent cascading failures in wind power integration areas caused by serious over or low voltage problems.The bottlenecks of a VSR for practical applications are computational efficiency and accuracy.To bridge these gaps,a general optimization model for tracking a voltage security region boundary(VSRB)in bulk power grids is developed in this paper in accordance with the topological characteristics of the VSRB.First,the initial VSRB point on the VSRB is examined with the traditional OPF by using the base case parameters as initial values.Then,the rest of the VSRB points on the VSRB are tracked one after another,with the proposed optimization model,by using the parameters of the tracked VSRB point as the initial value to explore its adjacent VSRB point.The proposed approach can significantly improve the computational efficiency of the VSRB tracking over the existing algorithms,and case studies,in the WECC 9-bus and the Polish 2736-bus test systems,demonstrate the high accuracy and efficiency of the proposed approach on exploring the VSRB.
