摘要
分析神经网络学习过程中各参数的变化动态,对理解网络的动力学行为,改进网络的结构和性能等具有积极意义.本文讨论了用梯度算法优化误差平方和损失函数时RBF网隐节点参数的变化动态,即算法收敛后各隐节点参数的可能取值.主要结论包括:如果算法收敛后损失函数不为零,则各隐节点将位于样本输入的加权聚类中心;如果损失函数为零,则网络中的冗余隐节点将出现萎缩、衰减、外移或重合现象.进一步的试验发现,对结构过大的RBF网,冗余隐节点的萎缩、外移、衰减和重合是频繁出现的现象.
To understand the dynamic behavior and improve the structure and performance of neural networks, it is very important to investigate their parameter changing dynamics during the learning. For radial basis function （RBF） networks using gradient descent method to minimize the least squares error cost function, this paper discusses the learning dynamics bf the hidden unit parameters, i.e., their possible values after learning. It is proved that if the cost function is not zero after the algorithm converges, then all hidden units will move to the weighted cluster centers of sample inputs. If cost function is zero, then hidden units will have shrinking, eliminating, out-moving and overlapping happened to those redundant units. Further simulation shows that such phenomena occur frequently in oversized RBF networks.
出处
《控制理论与应用》
EI
CAS
CSCD
北大核心
2007年第3期356-360,365,共6页
Control Theory & Applications
关键词
梯度算法
RBF网
学习动态
神经网络
泛化能力
gradient method
RBF network
learning dynamics
neural networks
generalization ability
