Engineering4(2018)446–448Contents lists available at ScienceDirectEngineeringTopicInsights
Robotics:FromAutomationtoIntelligentSystems
EduardoNebot
DirectorofAustralianCentreforFieldRobotics;FellowoftheAustralianAcademyofTechnologyandEngineering,AustraliaThispaperpresentsabriefoverviewoftheprogressthathasbeenmadeinautonomousrobotsduringthepastfewyears.Itpre-sentsthefundamentalproblemsthathavebeenaddressedtoenablethesuccessfuldeploymentofroboticautomationinindus-trialenvironments.Italsodescribessomeofthechallengesfacingfutureautonomousapplicationsinmorecomplexscenarios,suchasurbanvehicleautomation.Initialimplementationsofroboticsmanipulatorsbeganinthelate1950s,withapplicationsinautomotivemanufacturing.Hydraulicsystemswerethenreplacedbyelectricalmotors,makingtherobotsmoreagileandcontrollable.Therobotswereinitiallyusedinveryconstrainedandrepetitivetasks,suchaswelding.Theywerecontrolledbasedoninternalkinematics,withnosensinginformationaboutthecurrentstateoftheenviron-ment.The?rstinnovationinthisareastartedintheearly1980s,withtheintroductionofvisualfeedbackprovidedbycameras.Severaldifferentsensormodalitieswerealsoaddedtomonitorandinteractwiththeenvironment,suchaslasersandforcesensors.Nevertheless,mostoftheworkwithmanipulatorswasperformedwithina?xedareaofoperation.Insuchcases,therewasalmostnouncertaintyregardingthelocationoftherobot,andtheexternalenvironmentwasverywellmodeledandunderstood.Averydifferentscenariooccurswhenarobotisrequiredtomovearoundwithinitsenvironment.Twonewcapabilitiesbecomeessentialtoaddressthisproblem:positioningandpercep-tion.Arobotmovingwithinaworkingareaneedstolocalize—thatis,toknowitspositionandorientationwithrespecttoanavigationframe.Inaddition,itneedstohaveaverygoodrepresentationoftheareainproximityinordertomovesafelywithoutcollidingwithotherobjects.The?rstsuccessfuldemonstrationsofmappingandlocalizationwereimplementedinindoorenvironments,andmostlyusedultra-sonicsensorinformationtoobtainhigh-de?nitionmaps[1,2].Thisprocessconsistsofbuildinganavigationmapbymovingarobotwithintheenvironmentundermanualoperation,andthenusingthismaptolocalizetherobotwhenworkingautonomously.Thenextbreakthroughdemonstratedthatthesetwoprocessescouldbedonesimultaneously,andtherebyinitiatedaveryactiveareaofresearchknownassimultaneouslocalizationandmapping(SLAM)[3,4].Thesenewalgorithmsenabledtheconcurrentbuildingofamapandlocalizationwhileexploringanewarea,andfacilitatedthedeploymentoflargeindoorautonomousapplications.The?rstmajorimpactofautonomoustechnologyinoutdoorenvironmentswasin?eldrobotics,whichinvolvestheautomationoflargemachinesinareassuchasstevedoring(Fig.1),mining,anddefense[5].Thesuccessfuldeploymentofthistechnologyin?eldroboticsrequiredtheassurancethatamachinewouldalwaysbeundercon-trol,evenifsomeofitscomponentsfailed.Thisrequiredthedevel-opmentofnewsensingtechnologybasedonavarietyofsensormodalitiessuchasradarandlaser.Theseconceptswereessentialforthedevelopmentofhigh-integritynavigationsystems[6,7].Suchsystems,asdiscussedinRef.[5],includesensorsthatarebasedondifferentphysicalprinciplesinordertoensurethatnotwosensormodalitiescanfailatthesametime.Similarprincipleswereimplementedinotherareas,suchasmining,utilizingtheconceptofan‘‘islandofautomation”—thatis,anareawhereonlyautonomoussystemsareallowedtooperate.Thisfundamentalconstraintwasessentialforthesuccessfuldevelopmentanddeploymentofautonomoussystemsinmanyindustrialoperations.Machinelearningtechniqueshavestartedtoplayasigni?cantrolein?eldroboticautomation.Duringthelast?veyears,wehaveseenasigni?cantnumberofverysuccessfuldemonstrationsusingavarietyofsupervisedandunsupervisedmachinelearningalgo-rithms.Someofthemoreimpressiveapplicationsareinagriculture(Fig.2).Itiscommonnowtotrainavision-basedsystemtoclassifyanddifferentiatecropsfromweeds,monitorthehealthofacrop,andmonitorsoilconditionsinanautomaticandremotemanner.Fig.1.FullyautonomousstraddlecarriersoperatingintheportofBrisbane,Australia.E.Nebot/Engineering4(2018)446–448447Fig.2.Applicationsofintelligentrobotsinagriculture.Theinteractionofautonomousrobotswithpeopleandothermanuallyoperatedmachinesisamuchmorecomplexproblem.OneofthehottestareasinR&Distheoperationofautonomousvehicles(AVs)inurbanenvironments(Fig.3).AnAVmustbeabletointeractwithadynamicallychangingworldinaverypredictableandsafemanner.Itsperceptionsystemisresponsibleforprovidingcompletesituationalawarenessaroundthevehicleunderallpossibleenvironmentalconditions,includingthepositionofall?xedandmobileobjectsinproximitytothevehicle.Furthermore,safeAVoperationrequirestheestimationoftheintentionsofotherdriversandofpedestriansinordertobeabletonegotiatefuturemaneuversandplanaccordingly[8,9].Mostvehiclemanufacturersandresearchinstitutionsarecur-rentlyinvestingsigni?cantresourcesintointroducingthistechnol-ogywithinthenextfewyears.Thishasacceleratedprogressinallareasrelatedtoautonomy,includingthedevelopmentofnewalgo-rithmsandthedesignoflow-costsensingcapabilitiesandcompu-tationalpower.Signi?cantprogresshasbeenmadeinperceptionbyutilizingavarietyofsensorssuchaslasers,radar,cameras,andultrasonicdevices.Eachsensormodalityhasadvantagesanddisadvantages,andanyrobustdeploymentmustuseacombinationofsensortypesinordertoachieveintegrity.Allsensormodalitieswillhavefailuremodes,whichmaybeduetovariouscircumstancessuchasweatherorotherenvironmentalconditions.Itiswellknownthatalthoughcamerascanobtainverygoodtextureinformationforclassi?cationpurposes,theydonotalwaysshowasatisfactoryperformanceunderheavyrain,snow,orheavydust.Laserscanprovideverygoodrangeinformationandaremorerobusttorain.Nevertheless,theycanhavecatas-trophicfaultsundersteam,heavydust,orsmoke.Radariswellknowntoberobusttoallweather-relatedenvironmentalcondi-tions;however,itlackstheresolutionanddiscriminationcapabil-itiesofotherperceptionmodalities.Fundamentalandappliedresearcheffortsarecurrentlydirectedatfusingdifferentsensormodalitiesinordertoguaranteeintegrityunderallpossiblework-ingconditions.Anotherareathathasseenenormousprogressisdeeplearning.Theavailabilityoflargecomputationalandmemoryresourceshasenabledthetrainingofhigh-dimensionalmodelswithalargeamountofdata.Thefundamentaladvantageofdeeplearningisthatthereisnoneedtoengineerfeaturestotrainthemodels.Oneveryimpressiveapplicationofthistechnologyisautomaticlabelinginvisionsensing,whichisusuallyreferredtoassemanticlabeling(Fig.4).Thesemethodsusealargeamountofdatatotrainaconvolutionalneuralnetworktoautomaticallyclassifyeverypixelinanimagetocorrespondtoaclasswithinapossibleset.Oneoftheadvantagesofthesenetworksisthattheycanberetrainedforuseinotherscenarioswithrelativelylowcomputa-tionaleffort.Thisisusuallyknownas‘‘transferoflearning”[10,11].Thesetechniquesarenowpartofthemostsophisticatedadvanceddriver-assistancesystems(ADASs)andautonomousroadvehicleimplementations.Fundamentalchallengesstillexistinvehicleautomation,suchaspositioning,perceptionintegrity,interactionwithmanuallydrivenvehiclesandwithpedestrians,andsafevalidationofAVtechnology.Fig.3.AutonomousconnectedelectricalvehiclesoperatingonauniversityofSydneycampus.448E.Nebot/Engineering4(2018)446–448Fig.4.OriginalImages(left),semanticlabelingofobjects(topright),andthevehicle-inferredpath(bottomright).(1)Positioning:AVsrequirealevelofpositioningaccuracythatcanonlybeachievedbyusingpre-madehigh-de?nitionmaps.Theprocessofbuildingandmaintainingthesemapsisverychalleng-ing,sincemapsmustberobustandmustbeabletoscaleforavailabilityallaroundthecountryortheworld.(2)High-integrityperception:Currentimplementationcanonlyoperateunderreasonablygoodweatherandenvironmentalconditions.Typicalsensorsusedforperception,suchasvisionandlasersensors,couldhavecatastrophicfaultswhenoperatingunderdensefog,snow,ordust.(3)Learninghowtodrive:Drivingisamulti-agentgameinwhichallparticipantsinteractandcollaborateinordertoachievetheirindividualgoals.Thiscapabilityisstillverydif?cultforrobot,sinceitrequiresinferringtheintentionsofallinteractingpartici-pantsandpossessingthenecessarynegotiationskillsinordertomakedecisionsinasafeandef?cientmanner.(4)ValidationofAVs:ThecurrentstateofAVtechnologyhasdemonstratedthatitispossibletodeployAVsforoperationinurbanroadenvironments.Itismuchmoredif?culttodemonstratethatAVscanoperatesafelyunderallpossibletraf?cscenarios.AcomprehensiveworkinthisareaispresentedinRef.[12],wheretheauthorsacknowledgethattherewillalwaysbeaccidentsinvolvingAVs;however,thoseauthorsproposetheidenti?cationofasetofnormalvehiclebehaviorstoensurethatanAVwillneverbethecauseofanaccident.Thisworkpresentedabriefoverviewoftheevolutionofroboticsautomation.Thelastfewyearshaveseentheadventofverylargecomputationalandmemoryresources,newsensingcapabilities,andsigni?cantprogressinmachinelearning.Itisveryclearthatthesetechnologiesareenablingawholenewsetofautonomousapplicationsthatwillbepartofourlivesintheverynearfuture.Thecurrentissueofthisjournalpresentsroboticautomationapplicationsinroadvehiclesandfuturebio-syncreticrobots.Italsoincludespapersaddressingactuatorsandintelligentmanufacturing.References[1]ElfesA.Occupancygrids:aprobabilisticframeworkforrobotperceptionandnavigation[dissertation].Pittsburgh:CarnegieMellonUniversity;1989.[2]MoravecH,ElfesA.Highresolutionmapsfromwideanglesonar.In:Proceedingsofthe1985IEEEInternationalConferenceonRoboticsandAutomation;1985March25–28;St.Louis,MO,USA.NewYork:IEEE;1985.[3]Durrant-WhyteH,BaileyT.Simultaneouslocalizationandmapping:partI.IEEERobotAutomMag2006;13(2):99–110.[4]GuivantJ,NebotE.Optimizationofthesimultaneouslocalizationandmap-buildingalgorithmforrealtimeimplementation.IEEETransRobotAutom2001;17(3):242–57.[5]Durrant-WhyteH,PagacD,RogersB,StevensM,NelmesG.Fieldandserviceapplications—anautonomousstraddlecarrierformovementofshippingcontainers—fromresearchtooperationalautonomoussystems.IEEERobotAutomMag2007;14(3):14–23.[6]SukkariehS,NebotEM,Durrant-WhyteHF.AhighintegrityIMU/GPSnavigationloopforautonomouslandvehicleapplications.IEEETransRobotAutom1999;15(3):572–8.[7]NebotEM,BozorgM,Durrant-WhyteHF.Decentralizedarchitectureforasynchronoussensors.AutonomousRobots1999;6(2):147–64.[8]BenderA,AgamennoniG,WardJR,WorrallS,NebotEM.Anunsupervisedapproachforinferringdriverbehaviorfromnaturalisticdrivingdata.IEEETransIntellTranspSyst2015;16(6):3325–36.[9]ZynerA,WorrallS,NebotE.Arecurrentneuralnetworksolutionforpredictingdriverintentionatunsignalizedintersections.IEEERobotAutomLett2018;3(3):1759–64.[10]SchneiderL,CordtsM,RehfeldT,PfeifferD,EnzweilerM,FrankeU,etal.Semanticstixels:depthisnotenough.In:ProceedingsoftheIEEEIntelligentVehiclesSymposium(IV);2016Jun19–22;Gothenburg,Sweden;2016.[11]ZhouW,ArroyoR,ZynerA,WardJ,WorrallS,NebotE,etal.Transferringvisualknowledgeforarobustroadenvironmentperceptioninintelligentvehicles.In:Proceedingsofthe2017IEEE20thInternationalConferenceonIntelligentTransportationSystems(ITSC);2017Oct16–19;Yokohama,Japan;2017.[12]ShwartzS,ShammahS,ShashuaA.Onaformalmodelofsafeandscalableself-drivingcars.2017.arXiv:1708.06374.Engineering 2 (2016) xxx–xxxContents lists available at ScienceDirect
Engineering
Topic Insights机器人技术——从自动化到智能系统Eduardo NebotDirector of Australian Centre for Field Robotics; Fellow of the Australian Academy of Technology and Engineering, Australia本文简述了在过去的几年中我们在自动机器人研制领域所取得的进展，并介绍了在工业环境中，为成功实现机器人自动化部署所解决的基本问题。本文还将为我们描述在更复杂的场景中，自动化技术落地应用所面临的一些挑战，如城市车辆自动化等。机器人操作器最初在20世纪50年代末期取得应用，它率先被人们引入汽车制造领域。在随后的发展中，液压系统逐渐被电动机取代，这使得机器人更加灵活可控。起初，机器人仅在极其受限的使用场景中被用于如焊接等高重复性的工作，它们的操控设计完全从机器人内部的运动学角度出发，并不涉及其所处环境状况的感知信息。该领域的首次创新始于20世纪80年代初期，我们引入了由相机提供的视觉反馈，并添加了几种不同模式的传感器以监测环境并与环境相互作用，如激光与力传感器等。然而，由于上述机器人操作器大多情况下均在固定区域进行作业，故机器人位置的不确定性几乎为零，对我们而言，其外部环境也可以被轻易模型化并深刻理解。当机器人需要在其作业环境中移动时，情况便截然不同了。为解决这个问题，定位与感知这两项新的核心能力就变得至关重要。在作业区域内移动的机器人需要被定位，即确定其相对于导航参考系的位置和方向。此外，我们更需对相邻作业域的情况进行准确的展现，以确保机器人可以安全移动而不与其他物体发生碰撞。我们在室内环境中首次成功实现了地图测绘以及机器人定位演示，并主要对超声波传感器的信息进行处理以获得高清晰度地图[1,2]。此过程包含了在手动操作下，在作业环境中移动机器人以构建导航地图，以及在随后的自动作业中通过该地图对机器人进行定位等。而下一个突破则体现在上述两个过程的同步实现中，该技术开拓了一个非常活跃的研究领域，即我们所熟知的即时定位与地图构建（simultaneous localization and map-ping, SLAM）[3,4]。这些新算法支持在探索新区域时同时进行地图构建以及定位，这在很大程度上促进了大型室内自动设备的部署。户外环境中，自动技术的首例重要应用为野地机器人，它涉及了诸如装卸（图1）、开采以及防御等领域的大型机器的自动化[5]。在野地机器人学中成功应用该技术需要确保机器始终处于受控状态，即使在某些部件出现故障时亦是如此。这需要开发基于各种不同模式传感器（如雷达和激光）的新传感技术。这些概念对于高完整性导航系统的开发至关重要[6,7]。该系统如参考文献[5]中所述，包图1. 在澳大利亚布里斯班港口运行的全自动跨运车。490Author name et al. / Engineering 2(2016) xxx–xxx含了多套基于不同物理原理的传感器，这可以确保不会有两种传感器同时发生故障。类似的设计理念在其他领域亦有应用，如在采矿机器人中，我们运用自动化孤岛的设计理念，即开发一个只允许自动化系统运行的区域以实现局部优化。这种基本约束对于在众多工业实践中自动化系统的成功开发及应用至关重要。机器学习技术已开始在野地机器人的自动化中发挥重要作用。在过去的5年里，我们已经见证了大量不同种类的有监督学习和无监督学习的机器学习算法取得成功。其中，它们在农业领域中取得的应用更令人印象深刻（图2）。现如今，基于视觉所开发的系统已经取得了广泛应用。它们能够将作物与杂草进行分类并加以区别，亦可以监测作物的健康状况并远程自动监测土壤条件。自动化机器人与人类以及其他手动操作机器之间的交互是一个更为复杂的问题。该研发领域的一个热门课题便是城市环境中无人驾驶汽车（autonomous vehicle，AV）的运行（图3）。一辆AV必须以一种可预见程度高且非常安全的方式与其周围动态变化的世界进行交互。其感知系统负责在所有可能出现的环境条件下提供车辆周围的完整信息感知，这包括车辆附近的所有固定、移动物体的位置。此外，安全的AV运行还需要估计其他驾驶员和行人的意图，以便能够对突发情况进行预测并相应地制定应对策略[8,9]。大多数汽车制造商及相关研究机构目前正在投入大量资源，以期在未来几年内引入这项技术。这加速了与自动化相关的所有领域的进展，包括新算法的开发以及低成本传感能力和计算能力的构建。通过利用各种传感器，如激光器、雷达、照相机和超声波装置等，我们在机器人的感知方面取得了重大进展。每种模式的传感器都有其优点和缺点，任何强大的设计都必须对不同的传感器类型进行组合才能实现功能的完整性。任何种类的传感器都可能会出现故障，这可能是由天气或其他环境因素等多种情况所造成的。正如我们所熟知的，尽管相机可以获取优质的适宜分类的纹理信图2. 智能机器人在农业领域的应用。图3. 在大学校园内运行的AV电动汽车。
机器人技术 - 从自动化到智能系统 - 图文
