Variations on the cascade-correlation learning architecture for fast convergence in robot c
Variations on the Cascade-Correlation Learning Architecture for Fast Convergence in Robot Control Natalio Simon Henk Corporaal Eugene Kerckhoffs
Delft University of Technology
Electrical Engineering Department
Mekelweg4
2628CD Delft,The Netherlands
phone:+31-15-785021
email:natalio@liberator.et.tudelft.nl
Abstract
Most applications of Neural Networks in Control Systems use a version of the Back-Propagation algorithm for training.Learning in these networks is generally a slow and very time consuming process.Cascade-Correlation is a supervised learning algorithm that automatically determines the size and topology of the network and is quicker than back-propagation in learning for several benchmarks.We present modi?ed versions of the Cascade-Correlation learning algorithm.This version is used to implement the inverse kinematic transformations of a robot arm controller with two and three degrees of freedom.This new version shows faster convergence than the original and scales better to bigger training sets and lower tolerances.
Keywords:supervised learning,robot control,cascade-correlation.
1Introduction
Control Systems is one area for which neural networks are attractive because of their ability to learn,to approximate functions,to classify patterns and because of their potential for massive parallel hardware implementation[3].
One of the main obstacles to the use of neural networks in Control Systems is the slow speed at which the algorithms learn.Many of todays neural network applications use supervised learning algorithms which are variations of the backpropagation algorithm [1]-[9].The learning in these networks is generally a slow and time consuming process.
Another main problem to implementing control systems with neural networks is choosing the architecture of the network.This is usually done based on empirical criteria or iterating over a number of networks to determine the best one for a given tolerance.The learning speed is then dependent on the learning parameters and also on the architecture chosen.
Cascade-Correlation is a supervised learning algorithm that automatically determines the size and topology of the network.The algorithm has been shown to converge faster than back-propagation in the XOR/Parity,Encoder/Decoder and two-spirals problems[13].
This paper discusses a control system application in which a neural network is used as a robot arm controller[4]-[11].This problem represents a real-world application and requires a network with multiple continuous outputs,making it an attractive benchmark to test a learning algorithm.
We present a modi?ed version of the Cascade-Correlation learning algorithm and use the robot arm controller problem with two and three degrees of freedom to study the new algorithm’s learning speed and generalization capabilities as a function of the training set size and the required tolerance.
The next section reviews the Cascade-Correlation learning algorithm.Section3presents the new version of Cascade-Correlation.In section4two inverse kinematic transforma-tions used for a robot arm controller are given.Section5compares performances of the original and new learning algorithms in learning the inverse kinematic transformations of section4.A discussion of the results in section6concludes this paper.
2Review of Cascade-Correlation
Cascade-Correlation is a supervised learning algorithm for arti?cial neural networks[13]. The cascade architecture is illustrated in Fig.1.It begins as a one layer perceptron,with each input unit connected to each output unit.The learning algorithm is divided in two parts:training of output units and new unit creation.
In the?rst part the direct input-output connections are trained as well as possible over the entire training set using the quickprop learning algorithm[12].Here no backpropagation is needed since only one layer is trained.When no signi?cant error reduction has occurred after a certain number of training cycles,the entire training set is run through the network one last time to measure the error.If the error is less than the speci?ed tolerance the algorithm stops.If it is necessary to reduce this error,a new hidden unit is added to the network.This new unit receives trainable input connections from all of the network’s external inputs and from all pre-existing hidden units.The output of this new unit will be
I n p u t s
Outputs
I n p u t s
Outputs
(a)(b)
Figure 1:Cascade-Correlation Architecture:(a)initial,(b)after 2hidden units are added.Boxed connections are frozen,x connections are trained repeatedly.connected to all output units of the network.
Before the output of this new unit is however connected,a number of passes are run over the training set,adjusting the new unit’s input weights after each pass.The goal of this adjustment is to maximize ,the sum over all output units of the magnitude of the correlation between ,the new unit’s output value,and ,the residual output error observed at output unit .is de?ned as
where is the network output unit at which the error is measured and is the training pattern.The quantities are the values of and averaged over all patterns.
T o maximize ,the partial derivative of with respect to each of the new unit’s incoming
weights,,is computed.This gives
I n p u t s
Outputs
I n p u t s
Outputs Figure 2:New Architecture after 2groups of 2hidden units have been added:(a)Mode 1,(b)Mode 2.
avoids propagating error signals backwards through the network connections.Since the weights of the hidden units don’t change,hidden units’output values can be cached.By correlating the value of the hidden unit to the remaining error at the output units,the output units are able to reduce the error in the next training round since for the input patterns that cause high errors in the outputs,the hidden unit will produce a high value.
3Modi?cations on Cascade-Correlation
The new version of the Cascade-Correlation algorithm changes the way new units are added to the network.Here a number of new units equal to the number of outputs is added every time the training of output units cannot decrease the error.This should accelerated the convergence of the network,since each hidden unit helps correct the remaining error at one output and not the sum at all outputs,as in the original algorithm.
New units are denoted ,for 12,where equals the number of output units,and is the layer where the new unit is added.Each new unit is independent of the other new units in the same layer so they can be trained in parallel.Each new unit receives trainable input connections from all of the network’s external inputs and from previously added units with the same index (see Fig.2).
Before the outputs of the new units are connected,a number of passes are run over the training set,adjusting each new unit’s input weights after each pass.The goal of this adjustment is to maximize ,the sum of the magnitude of the correlation between ,the output value of ,and ,the residual output error observed at output unit .is de?ned for as
where is the network output unit at which the error is measured and also the index of the respective new unit;is the training pattern.The quantities are the values
of and averaged over all patterns.
T o maximize,the partial derivative of with respect to each incoming weight to new unit is computed.This gives
212
1
122
where the angles are limited to the?rst quadrant.
Fig.3(b)presents the three-dimensional case where the robot arm consists of three segments.In this case one solution is given by
1
223
2
1
22
33
Y
X
(a)
Figure 3:Robot Arm:(a)two degrees of freedom,(b)three degrees of freedom.
2
2
Figure 4:Controller with Corrective Neural Network.
where again the angles are limited to the ?rst quadrant.
In both cases the robot arm controller can be implemented by a neural network which
is trained to perform the inverse transformation functions.The training set consist of coordinates as inputs and their respectives joint angles as outputs.In this work,where we intend to test the modi?ed Cascade-Correlation as proposed in section 3,the training set is determined on the basis of the given theoretical formulas.In real world practice they are determined from real robot experiments.
A third problem presented in [6]is the use of a neural network as a corrective network for the controller of a damaged robot arm.Here the damage is modeled by a bend at a certain point along the robot arm.In this case the neural network learns the difference in angles 12needed to correct the error between the theoretically calculated joint angles of the kinematic transformation and the actual angles required to place the robot arm at the desired position (Fig.4).
In this case the input to the neural network consists of the desired end coordinates and
the uncorrected joint angles calculated by the robot controller.The outputs are the joint angle corrections.
Algorithm Avg.Units Avg.Epochs Success C-C
10/10 10/10 10/10Mode 1 Mode 2 10/10 10/10 10/10C-C Mode 1 Mode 2Mode 1 Mode 2 10/10 10/10 10/10C-C
49
2512291103106586122316504529402175314191123Mode 1 Mode 2 10/10 10/10C-C 81
303343682042 9/1027
1833
(a)
Mode 1 Mode 2
10/10 10/10
C-C 10/10314031
620025342159
121
SpeedUp
11110.951.251.24
1.56
2.142.382.452.87
10.860.88 10/10 10/10 10/10Mode 1 Mode 2Mode 1 Mode 2 10/10 10/10
Mode 1 Mode 2
10/10
82764
422337243649
C-C
10/1011431631341584133255452688
364Mode 1 10/108318Mode 2 10/1062498/10C-C 10/10172386C-C
C-C 652806 9/10Algorithm Avg.Units
Avg.Epochs Success (b)
SpeedUp
10.881.0711.141.4611.511.7911.982.06
Training Set
Training
Set
T able 1:Average training epochs:(a)two degrees of freedom,(b)three degrees of freedom
5Simulation Results
We compare the learning speed of the algorithms as a function of training set size and as a function of the output tolerance,given in radians.The training set consists of uniformly spread points in the ?rst quadrant of the X-Y plane for the robot with two degrees of freedom and for the corrective network,and for the three degrees of freedom robot in the positive X-Y-Z space.All robot links,,are 12length units.The training of the output units uses a linear function and the training of the hidden units the symmetric sigmoid function.This combination gave the best results.The same training parameters are used in all simulations.The number of candidate units is eight.All candidate units of all new units are trained in parallel.All simulations are run on a Sun4workstation.
Robot Arm Controller
T able 1compares the average in training epochs,de?ned as the number of passes through the training set,and number of hidden units for different sizes of the robot controller’s training set.The tolerance for the output angles was 0.025rad.The notation indicates Cascade-Correlation,1and 2indicate the two modes of the new version and the success rate indicates that p out of q trials were successful,where each trial starts from the initial state with different random weights.The speedup is given for the average epochs.
The results show that the new version learns faster and scales better when the training set increases.The speedup in average epochs increases with the training set size.In the two degrees of freedom case,when the training set size is more than 81,Mode 2is not only faster but requires the same number or less hidden units than the original algorithm.Mode 2architecture,where each output receives connections from all hidden units (Fig
(a)
(b)
T able 2:Position Errors:(a)two degrees of freedom,(b)three degrees of freedom 2),performs better than Mode 1.This could be explained by the existence of a correlation between the outputs,so hidden units that correlate to one output help other outputs.We tested the network with 100previously unseen points forming a circle.T able 2gives the average,maximum and standard deviation of the error for different sizes of the training set.The error is measured as the euclidean distance between the actual end point of the robot arm and the desired end points in length units.The tolerance during the learning was 0,025rad.The results are given for the original Cascade-Correlation and for the new algorithm with Mode 2.The original algorithm could not converge with a training set size of 289patterns.The results show a decrease in average error and standard deviation as the set size increases.Both algorithms exhibit comparable generalization https://www.wendangku.net/doc/fd10707869.html,parison with Back-propagation
For the two degrees of freedom case,we can compare the accuracy of generalization of the new algorithm (Mode 2)with that obtained by Josin [7],who used back-propagation.In [7],the architecture used consisted of one hidden layer of 32neurons,2input and 2output units.Josin reports that the test set,100points on a circle of radius 0.2length units,was within an euclidean distance of 0.025length units from the desired end points when the training set,a rectangular grid covering the test circle,was of 8to 25points.Less training points could not achieve that error https://www.wendangku.net/doc/fd10707869.html,ing the same training and test sets,we obtained the same error tolerance of 0.025length units with a training set of 8points and the network built by the algorithm had 4hidden neurons.With a test set of 9points an error tolerance of 0.015length units was achieved and the network required 9hidden units.Finally with a test set of 25points,an error tolerance of 0.012length units was achieved with 9hidden units.
Corrective Network
T able 3presents the average in training epochs and number of hidden units for different tolerances in the corrective network (Fig.4).
C-C
10/10 10/10 10/10Mode 1 Mode 2 10/10 10/10 10/10C-C Mode 1 Mode 2Mode 1 Mode 2 10/10 10/10C-C Mode 1 Mode 2
10/10 10/10
C-C Tolerance
Algorithm Avg.Units Avg.Epochs
Success 410003286 2238000666101117898410
1201
153439 7/101718925x105x105x105x10
-2
-3
-4
-5
60*0/10
* Max. no. Units
13270
(a)
T able 3:Average training
epochs,(a)9train cases.(b)16train cases.
(a)
(b)
T able 4:Position Errors:(a)9train cases,(b)16train cases
The results show that the new version also scales better as low tolerance is required.For
a tolerance of 5x103or less the new algorithm requires less hidden units and converges faster than the original.For a tolerance of 5x105the original algorithm is not able to converge after adding 60hidden units.Again Mode 2architecture performs better than Mode 1.
T able 4gives the average,maximum and standard deviation of the error for different tolerances for the test circle.The results are again for the new algorithm with Mode 2.The results show the well known overtraining problem.When the training set is too accurately learned,the network generalizes less accurately.
6Discussion and Future Work
Most neural network implementations of robot control systems use a version of back-propagation and require a way of determining the architecture of the network.
The modi?ed version introduced in this paper has the advantages of the Cascade-Correlation architecture:it determines the size and topology of the network and also automatically increases its size when lower tolerance is required or more training cases are used.This?exibility makes the proposed algorithm attractive for this area of applica-tions and the algorithm solves the problem of determining the architecture of the network. The algorithm and architecture allow incremental learning.
The proposed algorithm was shown to learn quicker,scale better to more training cases and to lower tolerance,and to have a higher success rate than the original Cascade-Correlation for the robot control problems presented here.The robot arm control with two degrees of freedom allows us a comparison with previous results using back-propagation. In this case the accuracy of generalization achieved by the new algorithm is equal or better and with a much smaller network than achieved by Josin using back-propagation.Also no trial and error stage is needed to determine the amount and size of the hidden layers. The results show that only an increase in the training set size improves the accuracy of the controller.Learning the training set with very high accuracy produces the overtraining problem.A method is required to determine the tolerance in training that will maintain the best generalization capability of the neural network.
Future work will test whether the advantages of the modi?ed Cascade-Correlation hold in other areas of application as pattern recognition and signal processing. References
[1]Franzini M.A.,"Speech Recognition with Back-Propagation",Proceedings,9th An-
nual Conference of IEEE Engineering in Medicine and Biology Society,1987.
[2]Khotanzad A.and Lu J.H.,"Distortion Invariant Character Recognition by a Multilayer
Perceptron and Backpropagation Learning",in Proc.IJCNN,San Diego,CA,1988, Vol.I pp.625-631.
[3]Antsaklis P.J.,Guest Ed.,Special Issue on Neural Networks in Control Systems,
IEEE Control Syst.Mag.,Apr.1992,pp.8-10.
[4]Rabelo L.C.and Avula J.R.,"Hierarchical Neurocontroller Architecture for Robotic
Manipulation",IEEE Control Syst.Mag.,Apr.1992,pp.37-41.
[5]Sobajic D.,Lu J.and Pao Y.,"Intelligent Control of the INTELLEDEX6057Robot
Manipulator",in Proc.IJCNN,San Diego,CA,1988,pp.633-640.
[6]Josin G.,Charney D.and White D.,"Robot Control Using Neural Networks",in Proc.
IJCNN,San Diego,CA,1988,Vol.II,pp.625-631.
[7]Josin G.,"Neural-Space Generalization of a T opological Transformation",Biological
Cybernetics,1988,Vol.59,pp.283-290.
[8]Guez A.and Ahmad Z.,"Solution to the Inverse Kinematic Problem in Robotics by
Neural Networks",in Proc.IJCNN,San Diego,CA,1988,pp.617-624.
[9]Guez A.and Ahmad Z.,"Accelerated convergence in the Inverse Kinematics via Multi-
Layered Feedforward Networks",in Proc.IJCNN,Washington,DC,1989,pp.341-347.
[10]Liu H.,Iberall T.and Bekey G.,"Neural Network Architecture for Robot Arm Control",
IEEE Control Syst.Mag.,Apr.1989,pp.38-42.
[11]Bassi D.and Bekey G.,"High Precision Position Control by Cartesian Trajectory
Feedback and Connectionist Inverse Dynamics Feedforward",in Proc.IJCNN,Wash-ington,DC,1989,pp.325-332.
[12]Fahlman S.E.,"Faster-Learning Variations on Back-Propagation:An Empirical
Study",in Proceedings of the1988Connectionist Models Summer School,Morgan Kaufmann.
[13]Fahlman S.E.and Lebiere C.(1990)The Cascade-Correlation Learning Architecture,
Carnegie Mellon University report CMU-CS-90-100.
黄自艺术歌曲钢琴伴奏及艺术成就
【摘要】黄自先生是我国杰出的音乐家，他以艺术歌曲的创作最为代表。而黄自先生特别强调了钢琴伴奏对于艺术歌曲组成的重要性。本文是以黄自先生创作的具有爱国主义和人道主义的艺术歌曲《天伦歌》为研究对象，通过对作品分析，归纳钢琴伴奏的弹奏方法与特点，并总结黄自先生的艺术成就与贡献。 【关键词】艺术歌曲；和声；伴奏织体；弹奏技巧 一、黄自艺术歌曲《天伦歌》的分析 （一）《天伦歌》的人文及创作背景。黄自的艺术歌曲《天伦歌》是一首具有教育意义和人道主义精神的作品。同时，它也具有民族性的特点。这首作品是根据联华公司的影片《天伦》而创作的主题曲，也是我国近代音乐史上第一首为电影谱写的艺术歌曲。作品创作于我国政治动荡、经济不稳定的30年代，这个时期，这种文化思潮冲击着我国各个领域，连音乐艺术领域也未幸免――以《毛毛雨》为代表的黄色歌曲流传广泛，对人民大众，尤其是青少年的不良影响极其深刻，黄自为此担忧，创作了大量艺术修养和文化水平较高的艺术歌曲。《天伦歌》就是在这样的历史背景下创作的，作品以孤儿失去亲人的苦痛为起点，发展到人民的发愤图强，最后升华到博爱、奋起的民族志向，对青少年的爱国主义教育有着重要的影响。 （二）《天伦歌》曲式与和声。《天伦歌》是并列三部曲式，为a+b+c，最后扩充并达到全曲的高潮。作品中引子和coda所使用的音乐材料相同，前后呼应，合头合尾。这首艺术歌曲结构规整，乐句进行的较为清晰，所使用的节拍韵律符合歌词的特点，如三连音紧密连接，为突出歌词中号召的力量等。 和声上，充分体现了中西方作曲技法融合的创作特性。使用了很多七和弦。其中，一部分是西方的和声，一部分是将我国传统的五声调式中的五个音纵向的结合，构成五声性和弦。与前两首作品相比，《天伦歌》的民族性因素增强，这也与它本身的歌词内容和要弘扬的爱国主义精神相对应。 （三）《天伦歌》的伴奏织体分析。《天伦歌》的前奏使用了a段进唱的旋律发展而来的，具有五声调性特点，增添了民族性的色彩。在作品的第10小节转调入近关系调，调性的转换使歌曲增添抒情的情绪。这时的伴奏加强和弦力度，采用切分节奏，节拍重音突出，与a段形成强弱的明显对比，突出悲壮情绪。 c段的伴奏采用进行曲的风格，右手以和弦为主，表现铿锵有力的进行。右手为上行进行，把全曲推向最高潮。左手仍以柱式和弦为主，保持节奏稳定。在作品的扩展乐段，左手的节拍低音上行与右手的八度和弦与音程对应，推动音乐朝向宏伟、壮丽的方向进行。coda 处，与引子材料相同，首尾呼应。 二、《天伦歌》实践研究 《天伦歌》是具有很强民族性因素的作品。所谓民族性，体现在所使用的五声性和声、传统歌词韵律以及歌曲段落发展等方面上。 作品的整个发展过程可以用伤感――悲壮――兴奋――宏达四个过程来表述。在钢琴伴奏弹奏的时候，要以演唱者的歌唱状态为中心，选择合适的伴奏音量、音色和音质来配合，做到对演唱者的演唱同步，并起到连接、补充、修饰等辅助作用。 作品分为三段，即a+b+c+扩充段落。第一段以五声音阶的进行为主，表现儿童失去父母的悲伤和痛苦，前奏进入时要弹奏的使用稍凄楚的音色，左手低音重复进行，在弹奏完第一个低音后，要迅速的找到下一个跨音区的音符；右手弹奏的要有棱角，在前奏结束的时候第四小节的t方向的延音处，要给演唱者留有准备。演唱者进入后，左手整体的踏板使用的要连贯。随着作品发展，伴奏与旋律声部出现轮唱的形式，要弹奏的流动性强，稍突出一些。后以mf力度出现的具有转调性质的琶音奏法，要弹奏的如流水般连贯。在重复段落，即“小
我国艺术歌曲钢琴伴奏-精
我国艺术歌曲钢琴伴奏-精 2020-12-12 【关键字】传统、作风、整体、现代、快速、统一、发展、建立、了解、研究、特点、突出、关键、内涵、情绪、力量、地位、需要、氛围、重点、需求、特色、作用、结构、关系、增强、塑造、借鉴、把握、形成、丰富、满足、帮助、发挥、提高、内心 【摘要】艺术歌曲中，伴奏、旋律、诗歌三者是不可分割的重 要因素，它们三个共同构成一个统一体，伴奏声部与声乐演唱处于 同样的重要地位。形成了人声与器乐的巧妙的结合，即钢琴和歌唱 的二重奏。钢琴部分的音乐使歌曲紧密的联系起来，组成形象变化 丰富而且不中断的套曲，把音乐表达的淋漓尽致。 【关键词】艺术歌曲；钢琴伴奏；中国艺术歌曲 艺术歌曲中，钢琴伴奏不是简单、辅助的衬托，而是根据音乐 作品的内容为表现音乐形象的需要来进行创作的重要部分。准确了 解钢琴伴奏与艺术歌曲之间的关系,深层次地了解其钢琴伴奏的风 格特点,能帮助我们更为准确地把握钢琴伴奏在艺术歌曲中的作用 和地位，从而在演奏实践中为歌曲的演唱起到更好的烘托作用。 一、中国艺术歌曲与钢琴伴奏 “中西结合”是中国艺术歌曲中钢琴伴奏的主要特征之一，作 曲家们将西洋作曲技法同中国的传统文化相结合，从开始的借鉴古 典乐派和浪漫主义时期的创作风格，到尝试接近民族乐派及印象主 义乐派的风格，在融入中国风格的钢琴伴奏写作，都是对中国艺术 歌曲中钢琴写作技法的进一步尝试和提高。也为后来的艺术歌曲写 作提供了更多宝贵的经验，在长期发展中，我国艺术歌曲的钢琴伴 奏也逐渐呈现出多姿多彩的音乐风格和特色。中国艺术歌曲的钢琴
写作中，不可忽略的是钢琴伴奏织体的作用，因此作曲家们通常都以丰富的伴奏织体来烘托歌曲的意境，铺垫音乐背景，增强音乐感染力。和声织体，复调织体都在许多作品中使用，较为常见的是综合织体。这些不同的伴奏织体的歌曲，极大限度的发挥了钢琴的艺术表现力，起到了渲染歌曲氛围，揭示内心情感，塑造歌曲背景的重要作用。钢琴伴奏成为整体乐思不可缺少的部分。优秀的钢琴伴奏织体，对发掘歌曲内涵，表现音乐形象，构架诗词与音乐之间的桥梁等方面具有很大的意义。在不断发展和探索中，也将许多伴奏织体使用得非常娴熟精确。 二、青主艺术歌曲《我住长江头》中钢琴伴奏的特点 《我住长江头》原词模仿民歌风格，抒写一个女子怀念其爱人的深情。青主以清新悠远的音乐体现了原词的意境，而又别有寄寓。歌调悠长，但有别于民间的山歌小曲；句尾经常出现下行或向上的拖腔，听起来更接近于吟哦古诗的意味，却又比吟诗更具激情。钢琴伴奏以江水般流动的音型贯穿全曲，衬托着气息宽广的歌唱，象征着绵绵不断的情思。由于运用了自然调式的旋律与和声，显得自由舒畅，富于浪漫气息，并具有民族风味。最有新意的是，歌曲突破了“卜算子”词牌双调上、下两阕一般应取平行反复结构的惯例，而把下阕单独反复了三次，并且一次比一次激动，最后在全曲的高音区以ff结束。这样的处理突出了思念之情的真切和执著，并具有单纯的情歌所没有的昂奋力量。这是因为作者当年是大革命的参加者，正被反动派通缉，才不得不以破格的音乐处理，假借古代的