G.J. Autonomous controller design for unmanned aerial vehicles using multi-objective geneti
Autonomous Controller Design for
Unmanned Aerial Vehicles using
Multi-objective Genetic Programming
Gregory J.Barlow
Center for Robotics and Intelligent Machines,Department of Electrical and Computer Engineering,North Carolina State University,Raleigh,NC27695-7911
Abstract.Autonomous navigation controllers were developed for?xed
wing unmanned aerial vehicle(UAV)applications using multi-objective
genetic programming(GP).Four?tness functions derived from?ight
simulations were designed and multi-objective GP was used to evolve
controllers able to locate a radar source,navigate the UAV to the source
e?ciently using on-board sensor measurements,and circle around the
emitter.Controllers were evolved for three di?erent kinds of radars:sta-
tionary,continuously emitting radars,stationary,intermittently emitting
radars,and mobile,continuously emitting radars.In this study,realistic
?ight parameters and sensor inputs were selected to aid in the transfer-
ence of evolved controllers to physical UAVs.
1Introduction
The?eld of evolutionary robotics(ER)[1]combines research on behavior-based robot controller design with evolutionary computation.A major focus of ER is the automatic design of behavioral controllers with no internal environmental model,in which e?ector outputs are a direct function of sensor inputs.Most of the controllers evolved in ER research to date have been developed for simple behaviors,such as obstacle avoidance,light seeking,object movement,simple navigation,and game playing.In many of these cases,the problems to be solved were designed speci?cally for research purposes.While simple problems generally require a small number of behaviors,more complex real-world problems might require the coordination of multiple behaviors in order to achieve the goals of the problem.Very little of the ER work to date has been intended for use in real-life applications.
This paper presents an approach to evolving behavioral navigation controllers for?xed wing UAVs using multi-objective GP.Controllers should be able to locate a radar,navigate the UAV to the source e?ciently using sensor measure-ments,and circle closely around the radar.Controllers were evolved for three di?erent radar types.Despite success in evolving controllers directly on real robots[1],simulation is the only feasible way to evolve controllers for UAVs.A UAV cannot be operated continuously for long enough to evolve a su?ciently competent controller,the use of an un?t controller could result in damage to the
aircraft,and?ight tests are very expensive.For these reasons,simulation must be capable of evolving controllers which transfer well to real UAVs.
2Unmanned Aerial Vehicle Simulation
The focus of this research was the development of navigation controllers for ?xed wing UAVs.The objective is to autonomously locate,track,and then or-bit around a radar site.There are three main goals for an evolved controller. First,it should move to the vicinity of the radar as quickly as possible.The sooner the UAV arrives in the vicinity of the radar,the sooner it can begin its primary mission,whether that is jamming the radar,surveillance,or another of the many applications of this type of controller.Second,once in the vicinity of the source,the UAV should circle as closely as possible around the source. Third,the?ight path should be stable and e?cient.The roll angle should change as infrequently as possible,and any change in roll angle should be small.Mak-ing frequent changes to the roll angle of the UAV could create dangerous?ight dynamics and could reduce the?ying time and range of the UAV.
The controller is evolved in simulation.The simulation environment is a square100nautical miles(nmi)on each side.The simulator gives the UAV a random initial position in the middle half of the southern edge of the environment with an initial heading of due north and the radar site a random position within the environment every time a simulation is run.In this research,the UAV has a constant altitude and a constant speed of80knots.The low level control of the UAV is done by an autopilot;the evolved controllers navigate the UAV.
The simulation can model a wide variety of radar types.For the research presented in this paper,three types of radars were modeled:1)stationary,con-tinuously emitting radars,2)stationary,intermittently emitting radars with a period of10minutes and emitting duration of5minutes,and3)mobile,contin-uously emitting radars.Only the sidelobes of the radar emissions are modeled. The sidelobes of a radar signal have a much lower power than the main beam, making them harder to detect.However,the sidelobes exist in all directions, not just the direction the radar is pointed.This model is intended to increase the robustness of the system,so that the controller doesn’t need to rely on a signal from the main beam.Additionally,Gaussian noise is added to the am-plitude of the radar signal.The receiving sensor can perceive only two pieces of information:the amplitude and the angle of arrival(AoA)of incoming radar signals.The AoA measures the angle between the heading of the UAV and the source of incoming electromagnetic energy.Real AoA sensors do not have per-fect accuracy in detecting radar signals,so the simulation models an inaccurate sensor.In the experiments described in this research,the AoA is accurate to within±10?at each time step,a realistic value for this type of sensor.Each ex-perimental run simulates four hours of?ight time,where the UAV is allowed to update its desired roll angle once a second.The interval between these requests to the autopilot can also be adjusted in the simulation.Further details about the simulation environment can be found in[2].
3Multi-objective Genetic Programming
UAV controllers were designed using multi-objective genetic programming which employs non-dominated sorting,crowding distance assignment to each solution,and elitism.The multi-objective genetic programming algorithm used in this research is very similar to the NSGA-II [3]multi-objective genetic algorithm.The function and terminal sets combine a set of very common functions used in GP experiments and some functions speci?c to this problem.The sensors available to GP measure the amplitude,AoA,and slope of the amplitude for incoming radar signals.When turning,there are six available actions.Turns may be hard or shallow,with hard turns making a 10?change in the roll angle and shallow turns a 2?change.The WingsLevel terminal sets the roll angle to 0,and the NoChange terminal keeps the roll angle the same.Multiple turning actions may be executed during one time step,since the roll angle is changed as a side e?ect of each terminal.The ?nal roll angle after the navigation controller is ?nished executing is passed to the autopilot.The maximum roll angle is 45?.Genetic programming was generational,with crossover and mutation similar to those outlined by Koza in [4].A population of 500individuals was evolved for 600generations.The crossover rate was 0.9and the mutation rate was 0.05.Tournament selection with a tournament size of 2was used.Initial trees were randomly generated using ramped half-and-half initialization.The maximum initial depth was 5,and the maximum depth was 21.
4Fitness Functions
Four ?tness functions determine the success of individual UAV navigation con-trollers.The ?tness of a controller was measured over 30simulation trials,where the UAV and radar positions were di?erent for every run.The four ?tness mea-sures were designed to satisfy the three goals of the evolved controller:moving toward the emitter,circling the emitter closely,and ?ying in an e?cient way.The primary goal of the UAV is to ?y from its initial position to the radar site as quickly as possible.The controllers’ability to accomplish this task is measured by averaging the squared distance between the UAV and the radar over all time steps.This distance is normalized using the initial distance between the radar and the UAV in order to mitigate the e?ect of varying distances from the random placement of radar sites.The normalized distance ?tness measure is given as fitness 1=1T T i =1 distance i distance 0 2,where T is the total number of time
steps,distance 0is the initial distance,and distance i is the distance at time i .The goal is to minimize fitness 1.
Once the UAV has ?own in-range of the radar,the goal shifts from moving toward the source to circling around it.An arbitrary distance much larger than the desired circling radius is de?ned as the in-range distance.For this research,the in-range distance was set to be 10nmi.The circling distance ?tness metric measures the average distance between the UAV and the radar over the time the UAV is in-range.While the circling distance is also measured by fitness 1,that
metric is dominated by distances far away from the goal and applies very little
evolutionary pressure to circling behavior.The circling distance?tness measure
is given as fitness2=1
N T i=1in range?(distance i)2,where N is the amount of time the UAV spent within the in-range boundary of the radar and in range is
1when the UAV is in-range and0otherwise.The goal is to minimize fitness2.
In addition to the primary goals of moving toward a radar site and circling
it closely,it is also desirable for the UAV to?y e?ciently in order to minimize
?ight time to get close to the goal and to prevent potentially dangerous?ight
dynamics,like frequent and drastic changes in the roll angle.The?rst?tness
metric that measures the e?ciency of the?ight path is the amount of time the
UAV spends with its wings level to the ground,which is the most stable?ight
position for a UAV.This?tness metric only applies when the UAV is outside the
in-range distance,since once the UAV is within the in-range boundary,is should
circle around the radar.The level time is given as fitness3=1
T?N T i=1(1?in range)?level,where level is1when the UAV has been level for two consecutive
time steps and0otherwise.The goal is to maximize fitness3.
The second?tness measure intended to produce an e?cient and stable?ight
path is a measure of turn cost.While UAVs are capable of very quick,sharp
turns,it is preferable to avoid them.The turn cost?tness measure is intended
to penalize controllers that navigate using a large number of sharp,sudden turns
because this may cause very unstable?ight.The UAV can achieve a small turning
radius without penalty by changing the roll angle gradually;this?tness metric
only accounts for cases where the roll angle has changed by more than10?
since the last time step.The turn cost is given as fitness4=1
T T i=1h turn?|roll angle i?roll angle i?1|,where roll angle is the roll angle of the UAV and
h turn is1if the roll angle has changed by more than10?since the last time
step and0otherwise.The goal is to minimize fitness4.
5Results
Multi-objective GP produced controllers that satis?ed the three goals of this
problem.In order to statistically measure the performance of GP,50evolutionary
runs were done for each type of radar.Each run lasted for600generations and
produced500solutions.Since multi-objective optimization produces a Pareto
front of solutions,rather than a single best solution,a method to gauge the
performance of evolution was needed.To do this,values considered minimally
successful for the four?tness metrics were selected.A minimally successful UAV
controller is able to move quickly to the target radar site,circle at an aver-
age distance under2nmi,?y with the wings level to the ground for at least
1,000seconds,and turn sharply less than0.5%of the total?ight time.If a con-
troller had a normalized distance?tness value(fitness1)of less than0.15,a
circling distance(fitness2)of less than4(the circling distance?tness metric
squares the distance),a level time(fitness3)of greater than1,000,and a turn
cost(fitness4)of less than0.05,the evolution was considered successful.These
baseline values were used only for analysis,not for the evolutionary process.
Controllers were evolved for1)stationary,continuously emitting radars,2)sta-tionary,intermittently emitting,radars,and3)mobile,continuously emitting radars.More complete results of these experiments can be found in[2]and[5].
The?rst experiment evolved controllers on a stationary,continuously emit-ting radar.Of the50evolutionary runs,45runs were acceptable under the base-line values.The number of acceptable controllers evolved during an individual run ranged from1to170.Overall,3,149acceptable controllers were evolved,for an average of62.98successful controllers per evolutionary run.The best evolved controllers?y to the target very e?ciently,staying level a majority of the time. Almost all turns are shallow.Once in range of the target,the roll angle is grad-ually increased.Once the roll angle reaches its maximum value to minimize the circling radius,no change to the roll angle is made for the remainder of the simulation.Populations tended to evolve to favor turning left or right.
The second experiment evolved controllers for a stationary,intermittently emitting radar.The radar was set to emit for5minutes and then turned o?for 5minutes,giving a period of10minutes and a50%duty cycle.This experiment was far more di?cult for evolution than the?rst experiment,because the radar only emits half of the time.A new set of50evolutionary runs was done,and25 of the runs produced at least one acceptable solution.The number of controllers in an evolutionary run that met the baseline values ranged from1to156,1,891 successful controllers were evolved,and the average number of acceptable con-trollers evolved during each run was37.82.The?ight paths for these controllers were similar to those for the continuously emitting radars.Despite the increased complexity from the?rst experiment,GP was able to evolve many successful controllers.
The third experiment evolved controllers for a mobile,continuously emitting radar.The mobility was modeled as a?nite state machine with the following states:move,setup,deployed,and tear down.When the radar moves,the new location is random anywhere in the simulation area.The?nite state machine is repeated for the duration of simulation.The radar site only emits when it is in the deployed state;while the radar is moving,the UAV receives no sensory information.The time in each state is probabilistic,and the minimum amount of time spent in the deployed state is an hour.Of the50evolutionary runs,36 were acceptable under the baseline values.The number of acceptable controllers evolved in each run ranged from1to206,and2,266successful controllers were evolved for an average of45.32acceptable controllers per evolutionary run.
To test the e?ectiveness of each of the four?tness measures,evolutions were done with various subsets of the?tness metrics.These tests were done using the stationary,continuously emitting radar,the simplest of the three radar types presented above.Based on these tests,it was determined that all four?tness functions were necessary to evolve successful controllers.In a comparison,con-trollers evolved using only the normalized distance?tness function exhibited slightly better performance than a human-designed,rule-based controller.
Flying a physical UAV with an evolved controller is planned as a demon-stration of the research,so transference was taken into consideration from the
beginning.Several aspects of the controller evolution were designed speci?cally to aid in this process.First,the navigation control was abstracted from the?ight of the UAV.Rather than attempting to evolve direct control,only the navigation was evolved.This allows the same controller to be used for di?erent airframes. Second,the simulation parameters were designed to be tuned for equivalence to real aircraft.For example,the simulated UAV is allowed to update the desired roll angle once per second re?ecting the update rate of the real autopilot of a UAV being considered for?ight demonstrations of the evolved controller.Third, noise was added to the simulation,both in the radar emissions and in sensor accuracy.A noisy simulation environment encourages the evolution of robust controllers that are more applicable to real UAVs.
6Conclusions
Genetic programming with multi-objective optimization was used to evolve nav-igation controllers for UAVs capable of?ying to a target radar,circling the radar site,and maintaining an e?cient?ight path,all while using inaccurate sensors in a noisy environment.Controllers were evolved for three di?erent radar types. The four?tness functions used for this research were su?cient to produce the desired behaviors,and all four measures were necessary for all three cases.Meth-ods were used to aid in the transference of the evolved controllers to real UAVs. In the next stage of this research,controllers evolved in this research will be tested on physical UAVs.
7Acknowledgments
This work was supervised by Dr.Edward Grant(NCSU)and Dr.Choong Oh (NRL),?nancial support was provided by the O?ce of Naval Research,and computational resources were provided by the Naval Research Laboratory. References
1.Nol?,S.,Floreano,D.:Evolutionary Robotics.MIT Press(2000)
2.Barlow,G.J.:Design of autonomous navigation controllers for unmanned aerial
vehicles using multi-objective genetic programming.Master’s thesis,North Carolina State University,Raleigh,NC(2004)
3.Deb,K.,Agrawal,S.,Pratap,A.,Meyarivan,T.:A fast and elitist multiobjective
genetic algorithm:NSGA-II.IEEE Transactions on Evolutionary Computation6 (2002)182–197
4.Koza,J.:Genetic Programming.MIT Press(1992)
5.Oh,C.K.,Barlow,G.J.:Autonomous controller design for unmanned aerial vehicles
using multi-objective genetic programming.In:Proceedings of the Congress on Evolutionary Computation,Portland,OR(2004)
面相中的十大凶相都有这些,你知道吗,看完该注意了
面相中的十大凶相都有这些，你知道吗，看完该注意了 谓的面相‘五官’,指的就是‘耳、眉、眼、鼻、口’等五种人体器官。面相就是一个人所具有的独特气质，而成为形或色表现于面上，给人的一种感受。接下来为大家详细介绍面相算命图解大全。面相可分为三庭看，人的眉以上是上庭，人的眉至鼻头是中庭，人的鼻头以下就为下庭。面部三庭要均匀。即额头、眉眼鼻、嘴与下巴的比例要均匀，整个面部显得大方磊落。若是额形生得略高阔饱满，则代表少年运佳，但额不能太高，过高会克夫，太低则少年运差，当然没法早嫁。在面部五官之后，再细分便是十二宫。这十二个宫位囊括了面部所有的特性和吉凶。第一宫：命宫，又为愿望之宫。麻衣曰：其居两眉间，山根之上，为印堂。第二宫：财帛宫，位于土宿，包括天仓、地库、金甲、井灶。主察财运。第三宫：兄弟宫，又称交友宫。麻衣曰：位居两眉。主交友运。第四宫：田宅宫，田宅宫，位于两眼，及上眼睑。主家业运第五宫：男女宫，又称子女宫。麻衣曰：位于两眼之下，又称为泪堂。看子嗣运。第六宫：奴仆宫，麻衣说它位居地阁，重接水星。看管理运。第七宫：妻妾宫，也可以称为夫妻宫，就在眼尾。第八宫：疾厄宫，一说是山根位，一说是年寿位，建议以鼻梁统看。第九宫：迁移宫，迁移者，位居眉角。古相士，以迁移宫的位置看人阴阳宅状况。第十宫：官禄宫，
官禄者，为居中正，上合离宫。反应人的禄命官运。第十一宫：福德宫，福德者，位居天仓，牵连地阁。看福禄之运。第十二宫：父母宫，便是额头的日月角。主看父母的福祸疾厄。看面相，形体外貌、精神气质、举止情态皆可一视而察，情人、恋人、夫妻、同事、朋友之间、感情总会有变化的、是相互信任、倾慕也可以从面相看出来。额头眉毛之间只有一道纵纹。这种面相在相学中被称为天柱纹。有此面相的人个性都很顽强。是属于做事不达目的绝不会放弃，对利益也是分得很清楚。一般来讲他们是不做对自己无利的事情。这样的人不但严以律己。同时对别人的要求也非常严格。但还有就是是这种面相的人有一个特征，那就好是这道纵纹平时是不会出现。当他的身心俱疲的时候，这道皱纹才会出现。鼻子的上部这些部位若是出现了数条横纹的人。有此面相特征者对事物都会表现出十足的热情。甚至可以说是充满激情。不仅是做事情又积极又主动。待人处事也是持着一颗平常心。此外，如果是说笑时出现这种皱纹的人。一般性格都是较为温和。缺点就是比较好管别人的事情。也常常为此惹祸上身。 1、男人的眉毛中间稀疏杂乱、毛形逆生，是为乱性之相， 情绪十分不稳定，伴有较重的暴力倾向。-2、双眉过低而压眼，是为心性阴沉扭曲而走极端。-3、女子眉过粗浓，不仅一生婚姻难成，且有妨夫。-4、印堂过窄小，难容两指的人，一生运势不顺且多灾厄。-5、女子双颧露骨而突起，对夫运
14种鼻型图解
26种面型算命图解 侧面观察 1、凸面型 上停位居前额，代表十五至三十岁、父母缘分、思想智慧等事。从侧面观看，这种面型的额头是向后倾斜，表示思想敏捷，下巴向后退缩，不是行动迅速。 但一个人思想、行动都迅速，则其人是一个容易冲动的人 2、凹面型 这种面型的人额头与下巴皆凸出，形成中央鼻子部位凹入 这种人思想、行动都慢；但有忍耐力，不会轻易冲动，给人感觉城府很深，不轻易向他人吐露心声。 3、直面型 直面形是前额与下巴皆没有凸出或退缩，这种面型的人思想与行动都不会急躁或太慢，做任何事都会按部就班，从容面对。 4、额凸下巴退缩 额凸代表思想慢，下巴退缩则代表行动快。 这种面相的人思想慢而行动快，其行动往往未经深思熟虑，所以常有错误的抉择。 5、额斜下巴凸
额斜是思想迅速，下巴凸出是行动缓慢，这种人碰到任何问题都会立即得到思想是回应，但不会马上行动，而会慢慢地行动，大部分人都是这种下巴。 正面观察 正面观察面型的方法较侧面多，有西洋骨相学的三分法，中国的五分法、十分法 其实三分法与五分有许多相同之处，三分法是以人类的思想、行动、物欲享受划分种类，而十分法只是把三分发再细致分划为十种。 三分法 1、思想型 思想型的人其特点是上额广阔而高，下巴尖而小，形成一个倒三角形。 这种形格的人身材一般都属细小、腰部狭窄、手一般略长、面色带白、头发幼而密。 「思想型」这正是推动他们走向成功之路的因素；所以很多科学家、研究家在未成功之前常常会给人行为疯狂、不切实际之感觉。 这种形格的人适宜做科学研究、教育、建筑师、设计师或数学家、分析家等工作。 2、运动型
运动型的人特点就是颧骨高耸，鼻形长而鼻梁有节，腮骨显露，前额一般较低而额上有横纹、面色带黑、头发粗而多、身材高大强壮。其性格特点是忍耐力强，有冒险精神，有责任心，敢作敢为，刻苦耐劳。 这种面型的人最适合从事劳动工作，如工程师、冒险家、探险家、军人、警察或运动员等 3、享受型 享受型的人其特点是颐部园肥，前额较窄小，形成一个正三角形或圆形的面。鼻形较小，鼻头园而有肉，头发幼而疏，面色略微带红，手脚较短，脸部特别肥大，肉多骨少。 这种人处事圆滑，交际手腕强。这种人最适合经商 以上三种形质，只是基本形而已。因为每一个人同样会兼有思想型、运动型及享受型的特征，只是多寡而已，但最好是三种形质发展平衡，这样可工作不忘娱乐，娱乐不忘工作。 如果思想型过重的话，这种人每天只是充满幻想，不肯面对现实，容易引发神经衰落及头痛病等病症，如果再加上整个脸型搭配失宜，如眉粗，眼无神，鼻形短，这样的话，实际谋生能力多有问题。 如果运动型过重的话，则其人精力充沛，行事冲动，喜欢用武力解决问题。如果再加上形质配合不佳，如鼻形不端正，额骨凸露或低，眼神流露等，则其人大多从事低下的劳动工作，只能温饱而已，老来