Engineering5(2019)671–678Contents lists available at ScienceDirectEngineeringResearch
IntelligentManufacturing—Article
OnlineMonitoringofWeldingStatusBasedonaDBNModelDuringLaserWelding
YanxiZhanga,DeyongYoua,XiangdongGaoa,?,SeijiKatayamababGuangdongProvincialWeldingEngineeringTechnologyResearchCenter,GuangdongUniversityofTechnology,Guangzhou510006,ChinaJoiningandWeldingResearchInstitute,OsakaUniversity,Osaka567-0047,Japanarticleinfoabstract
Inthisresearch,anauxiliaryilluminationvisualsensorsystem,anultraviolet/visible(UVV)bandvisualsensorsystem(withawavelengthlessthan780nm),aspectrometer,andaphotodiodeareemployedtocaptureinsightsintothehigh-powerdisclaserweldingprocess.Thefeaturesofthevisibleopticallightsignalandthere?ectedlaserlightsignalareextractedbydecomposingtheoriginalsignalcapturedbythephotodiodeviathewaveletpacketdecomposition(WPD)method.Thecapturedsignalsofthespectro-metermainlyhaveawavelengthof400–900nm,andaredividedinto25sub-bandstoextractthespectrumfeaturesbystatisticalmethods.ThefeaturesoftheplumeandspattersareacquiredbyimagescapturedbytheUVVvisualsensorsystem,andthefeaturesofthekeyholeareextractedfromimagescapturedbytheauxiliaryilluminationvisualsensorsystem.Basedonthesereal-timequantizedfeaturesoftheweldingprocess,adeepbeliefnetwork(DBN)isestablishedtomonitortheweldingstatus.AgeneticalgorithmisappliedtooptimizetheparametersoftheproposedDBNmodel.TheestablishedDBNmodelshowshigheraccuracyandrobustnessinmonitoringweldingstatusincomparisonwithatraditionalback-propagationneuralnetwork(BPNN)model.TheeffectivenessandgeneralizationabilityoftheproposedDBNarevalidatedbythreeadditionalexperimentswithdifferentweldingparameters.ó2019THEAUTHORS.PublishedbyElsevierLTDonbehalfofChineseAcademyofEngineeringandHigherEducationPressLimitedCompany.ThisisanopenaccessarticleundertheCCBY-NC-NDlicense(http://creativecommons.org/licenses/by-nc-nd/4.0/).Articlehistory:Received9July2018Revised2October2018Accepted10January2019Availableonline5July2019Keywords:OnlinemonitoringMultiplesensorsWaveletpacketdecompositionDeepbeliefnetwork1.IntroductionLaserwelding—especiallyhigh-powerlaserwelding—hasbeenwidelyappliedinindustriessuchascarmanufacturing,aerospacemanufacturing,andshipbuilding[1–3].Blowouts,humping,andundercuttingaretypicaldefectsthatgreatlyreducethestrengthofthejointandlimitweldingef?ciency.Itisstillamajorchallengetocomprehensivelydepictthehigh-powerdisclaserweldingpro-cess,whichiscrucialindetectingweldingdefectsandrealizingonlinemonitoringoftheweldingstatus.Duringlaserwelding,thematerialisrapidlyheatedupandvaporizedbythehighdensityofthelaserbeamenergy[4].Akey-holeisformedinthemoltenpoolbeneaththelaserbeamduetotherecoilpressureinducedbythevaporization,Marangoniforce,gravityoftheliquidmaterial,andbuoyancyforce[5].Theexis-tenceofthekeyholeenhancestheabsorptionofthelaserenergybythematerialduetothemulti-re?ectionofthelaserbeaminthekeyhole[6].Meanwhile,ametalplumeinducedbythe?Correspondingauthor.E-mailaddress:gaoxd666@126.com(X.Gao).high-density-energylaserbeamappearsinandabovethekeyhole[7].Theplumescattersandre?ectsthelaserbeam,andfurtheraffectsthedynamicofthekeyhole[8].Somespatteringisgener-atedasaresultoftherecoilingpressureinducedbythedrasticvaporization[9,10].Thespattersdisturbthedynamicofthemoltenpoolandkeyhole,astheycarryoffsomeofthekineticenergyfromthemoltenpool.Theabove-mentionedresearchhasrevealedthatthekeyhole,plume,andspattersarethemostimportantphenom-enaduringtheweldingprocess,andthattheirreal-timefeaturescanbeusedtodepicttheweldingstatus.Agreatnumberofstudieshaveappliedvisualsensingmethodstorevealthemechanismoflaserwelding[11–13],asvisualsensorsprovidehigh-dimensionalinsightsintothespatters,keyhole,andplume.Photodiodesensorsandspectrometershavealsobeenuti-lizedtomonitorthisindustrialmanufacturingprocess[14,15]duetotheirlowequipmentcostandsimplesetupstructure.Unfortu-nately,inthesestudies,eitheronlyasinglesensorwasappliedtoobservetheweldingprocess,orthecapturedsignalswerenotrelatedtotheweldingstatusinaquantizedway.Recently,manymachinelearningmethods,suchasmultiplelinearregression(MLR)[16,17],supportvectormachines(SVMs)[18],neural672Y.Zhangetal./Engineering5(2019)671–678networks(NNs)[19,20],andsoforth,havebeenemployedinmodelingandpattern-recognitionproblems,suchasstatisticalparametricspeechsynthesis,speechemotionrecognition,andproductsmanufacturingprocessmonitoring[21,22].However,MLRhaslimitationsin?ttingthehighlynonlinearfeaturesoftheweldingprocessduetoitslinearproperty.ThemappingabilityofSVMdependsonitsprede?nedkernelfunction,andmaybeinsuf-?cientforrepresentingmultiple-sensorsignalsfromlaserwelding.TheNNmethodalsohasanintrinsiclimitation,asitiseasytobetrappedinthelocaloptimum,anddif?cultto?ndtheglobalopti-malsolution.Furthermore,thesemethodsareshallowmodelswithoneornohiddenlayers,andcannotbeutilizedinexploringeffec-tiverepresentationofhighlycorrelatedmultiple-optical-sensorsig-nals.Therefore,thisresearchintroducesadeeplearningmethodbasedonadeepbeliefnetwork(DBN)modeltosolvethischallenge.Thisresearchaimstoestablishamultiple-optical-sensorsys-temtoobtaincomprehensiveinsightsintothehigh-powerdisclaserweldingprocess.AdeeplearningmodelbasedonDBNisestablishedto?ndtheglobaloptimalresultsformonitoringtheweldingstatuswiththesignalscapturedbythemultiple-optical-sensorsystem.Theremainderofthisresearchisorganizedasfol-lows.TheexperimentalsetupisintroducedinSection2,andthefeatureextractionsofthemultipleopticalsignalsaredescribedinSection3.Section4describesthearchitectureoftheDBNmodelandthepreparationofthetrainingandveri?cationset,andpro-videsaperformancecomparisonbetweentheestablishedDBNmodelandthetraditionalback-propagationneuralnetwork(BPNN)model.InSection5,thegeneralizationabilityandeffec-tivenessoftheestablishedDBNmodelareveri?edbythreeaddi-tionalexperimentswithdifferentweldingparameters.Section6concludesthisresearch.2.ExperimentalsetupFig.1depictstheexperimentalsetupofthisresearch.Fouroptical-sensorsystems,includinganauxiliaryilluminationvisualsensorsystem,anultraviolet/visible(UVV)bandvisualsensorsystem,aspectrometer,andaphotodiodeareappliedtocapturethesignalsoftheweldingprocess.Theweldingmaterialinthisresearchis304stainlesssteel.Thedimensionsoftheworkpieceare150mminlength,10mminwidth,and50mminthickness.Opticalsignalsfromtheweldingareaareacquiredbythepho-todiodesensor.Abeamsplitterispre-equippedinthelaserhead,andhelpstocollectandtransmitthesesignalsbymeansofanopti-cal?ber,asshowninFig.1.Thephotodiodereceivesthesesignalsanddividesthemintothere?ectedlaserlightopticalsignal(wavelength1030nm)andthevisiblelightopticalsignalbymeansofadichroicmirrorinthephotodiode;bothkindsofsignalareampli?edandtransmittedtotheoscilloscope.Thesamplingratesofthetwokindsofsignalsaresetas500kHzinordertoobtainthedetailedopticalfeaturesoftheweldingprocessinhigh-temporalresolution.Aspectrometerisappliedtocollectthespectralsignal(wave-lengthfrom186to1100nm)fromtheweldingareaduringlaserwelding.AsshowninFig.1,thespectralsignalsarecapturedbyacollimatorandthentransmittedtothespectrometerviatheopti-cal?ber.Previousresearchhasshownthatspectralsignalswithawavelengthfrom400to900nmcontainthemostimportantinsightsintothesolidlaserweldingprocess.Therefore,thesignalswithinthiswavelengthrangeareselectedtoextractthefeaturesfortheonlinemonitoringweldingstatus.Thesamplingrateofthespectrometerissetas500Hz.Twohigh-speedvisualimagingsystems,includingaUVVbandvisualsensorsystem(wavelengthgreaterthan390nm)andanauxiliaryilluminationvisualsensorsystem,areappliedtoobtainthefeaturesofthekeyhole,plume,andspatter.TheUVVbandvisualsensorsystemconsistsofaUVV?lterandahigh-speedcam-era.Withthecapturedimages,thevisualfeaturesoftheplumeandspattercanbeextractedbymeansofadigitalimageprocessingmethod.A40Wauxiliarylightsourceisemployedtoproducelaserlight(wavelength976nm)toilluminatetheweldingarea,andtheauxiliaryilluminationvisualsensorsystem,coupledwitha?lterthatonlypermitslaserlightwithawavelengthof976nmtopassthrough,capturesthevisualfeaturesofthekeyhole.Thesamplingratesofthetwovisualimagingsystemsarebothsetas5000frameásà1.Fig.1.Illustrationoftheexperimentalsetup.Y.Zhangetal./Engineering5(2019)671–6786733.Featureextractionofmultiple-sensorsignals3.1.FeatureextractionfromtheauxiliaryilluminationvisualsensorsystemThreeimagescapturedbytheauxiliaryilluminationvisualsen-sorsystemareshowninFig.2.Thekeyholefeatures,includingthesizeandpositionofthekeyhole,arecalculatedandquantizedwiththecropandbinarizationoperationsindigitalimageprocessing.Fig.2showsthatthekeyholesizeandposition?uctuateatdiffer-entmoments.ThefeaturevectorXAIextractedfromthesignalsoftheauxiliaryilluminationvisualsystemisexpressedinEq.(1),wherekeyholepositiondenotesthekeyholepositionandkeyholesizedenotesthekeyholesize.spatters,plumevolumedenotesthevolumeoftheplume,andplumedegreedenotesthetilteddegreeoftheplume.??XUVV?spatterfront;spatterback;plumevolume;plumedegreee2T3.3.FeatureextractionoftwosignalscapturedbythephotodiodesensorThesignalsofthevisiblelightandre?ectedlaserlightcapturedviaphotodiodeareanalyzedbythewaveletpacketdecomposition(WPD)method.WPDisachievedbyapplyingbothlow-passandhigh-pass?lterstocalculatetheapproximationandtheircoef?-cients.ThefunctionofWPD(Snj;ketT)canbedescribedbyEq.(3),wherejisthescalecoordinate,kdenotesthelocationcoordinate,nisthemodulationcoordinate,tdenotesthesequencenumber,andZisthesetoftheintegers.Daubechieswavelets(db10)areusedasthewaveletfunction.j=2njSnj;ketT?2Se2tàkT;??XAI?keyholeposition;keyholesizee1T3.2.FeatureextractionfromtheUVVbandvisualsensorsystemFig.3showstwoimagesacquiredbytheUVVbandvisualsen-sorsystem.ThefeaturesoftheplumeareextractedfromtheimagescapturedbytheUVVbandvisualsensorsystem.Thevol-umeoftheplumeiscalculatedasthenumberofpixelsoccupiedbytheplume,asshowninFig.3.Thetilteddegreeoftheplumeisde?nedastheanglebetweenthecentroidoftheplumeandtheverticalaxisintheimagecoordinatesystem;thisfeatureistheindicatoroftheplumedirection,whichalsocanbeconsideredasthedirectionofthekeyholeopening.Thefeaturesofthespattersarequantizedaccordingtotheir?yingdirection,andthenumbersofspatters?yingforwardandbackwardarecalculatedusingdigitalimageprocessoperations.AtotaloffourfeaturesarecollectedfromtheUVVbandvisualsensorsystem;thesefeaturesformthefeaturevectorXUVV,whichisexpressedinEq.(2),wherespatterfrontdenotesthenumberofforwardspatters,spatterbackdenotesthenumberofbackwardj;k2Ze3T1The?rsttwodecomposedsignals(S00;0andS0;0)inthe?rstlayerofWPDcanbeexpressedbyEq.(4).1S00;0?/etT;S0;0?uetTe4TThefunctionsofthehigh-pass?lter(hekT)andlow-pass?lter(gekT)arede?nedbyEq.(5).hekT?h/etT;/e2tàkTi;gekT?huetT;ue2tàkTie5TInthisway,thefunctionofWPDforn>1canbedescribedbyEqs.(6)and(7).nS2j;ketT?p???X2hekTSnjà1;ke2tàkTke6Tnt1etT?S2j;kp???X2gekTSnjà1;ke2tàkTke7TWiththephotodiodesignalxp,itsWPDcoef?cients(CnjekT)canbeexpressedbyEq.(8).CnjekT?XtxpetTSnj;ketTe8TThefeaturesFj,noftheWPDcoef?cientsCnjekTrelatedtoboththetimeandfrequencyarecalculatedinEqs.(9)–(18),whereKisthenumberoftheWPDcoef?cients,Edenotesthemathematicalexpectationfunction,xistheangularvector,andiistheimaginarypart.Fig.2.Threesequentialimagescapturedbytheauxiliaryilluminationvisualsensorsystemthroughfeatureextraction.(a)Originalimage;(b)regionofinterest(ROI);(c)binarization;(d)keyhole.F1j;nK1X?CnekTKk?1je9TFig.3.ImagescapturedbytheUVVbandvisualsensorsystem,anditsfeatureextractionprocess.674Y.Zhangetal./Engineering5(2019)671–678vu???????????????????????????????KF2j;n?ut1XhKCni2jekTe10Tk?1F3max??????Cn??jekT????j;n?F2e11Tj;nF?1XK4hi2j;nKCnjekTe12Tk?1KF5j;n?1XhCni4KjekTe13Tk?1&EhCnkTàF1i3'jej;nF6j;n???F2??3e14Tj;n2F7j;n?Fj;nF1e15Tj;nF81Xj;n?Klog????x??UnjexT??????e2pxi=Ke16TF91Xj;n?Klog??????Un????4pxi=KxjexT??ee17TF10F98j;n?j;nàFj;ne18TInEqs.(16)and(17),UnjexTistheFouriertransformationoftheWPDcoef?cientsCnjekT.Inthisresearch,thevisiblelightopticalsignalSvisiblecapturedbythephotodiodeisdecomposedinto16frequencysub-bandsaccordingtotheWPDmethod.TheWPDcoef?cientsCnjekTofeachdecomposedsub-bandsignalcanbeobtained,andits10statisticfeaturesarecalculatedaccordingtoEqs.(9)–(18).Consideringallthedecomposedsub-bandsignals,thefeaturevectorXvisible-lightcanbeexpressedbyEq.(19),whereFvisibledenotesthefeatureextractedfrom2Svisible.26F1visible;j;1Fvisible;j;1...F103visible;j;16X?6F1F2visible;j;2...F107visible;j;277visible-light6visible;j;2664........e19T...7.77F1visible;j;nF2visible;j;n...F105visible;j;nThefeaturevectorXre?ected-laserextractedfromthere?ectedlaserlightsignalSre?etedisexpressedbyEq.(20),whereFre?eteddenotesthefeaturesextractedfromSre?eted.236F1reflected;j;1F2reflected;j;1...F10reflected;j;16XF2reflected;j;2...F107reflected;j;277reflected-laser?66F1reflected;j;2664........7....77e20T5F1reflected;j;nF2reflected;j;n...F10reflected;j;n3.4.FeatureextractionofthesignalcapturedbythespectrometerTheselectedspectralsignalswithawavelengthfrom400to900nmthathavebeencapturedbythespectrometeraredividedinto25sub-bands,witheachsub-bandcovering20nm.Themeanvalueoftheintensityineachsub-bandiscalculatedasthefeatureofeachsub-band,expressedbyEq.(21),whereNdenotesthenum-berofthesub-band,nN,startdenotesthestartspectralnumberoftheNthsub-band,nN,enddenotestheterminalspectralnumberoftheNthsub-band,xsdenotesthespectralintensity,andspectrumNdenotesthecalculatedmeanvalueoftheintensityintheNthsub-band.PnN;endjspectrumj?nN;startxsN?20;N?1;2;...;25e21TForeachsample,atotalof25featuresareobtainedfrom25cor-respondingsub-bands.Thefeaturevectorextractedfromthespec-trometercanbeexpressedbyEq.(22).Xspectrum??spectrum1;spectrum2;...;spectrum25??e22T4.ArchitectureandapplicationofDBN4.1.FrameworkofDBNADBNmodelconsistsofasmanyhiddenlayersasthetargetproblemsrequire,witheachhiddenlayerbeingcomposedofarestrictedBoltzmannmachine(RBM).TheDBNnotonlypossessestheadvantagesofconventionalNNs,butalsohasastrongfusingabilityformultiplesensorsduetoitsdeeparchitecture[23–26].TheglobaloptimalparametersofaDBNmodelaredeterminedthroughatwo-steptrainingalgorithm—namely,pre-trainingand?ne-tuning.Recently,DBNmodelshavebeenwidelyemployedinsignalprocessingandinthemachinelearningindustry,inareassuchasvoiceactivitydetection[27],acousticmodeling[28],andfacerecognition[29].AtypicalRBMmodelhastwolayers,asshowninFig.4;thebot-tomlayeriscalledthevisiblelayer,andthetoplayeriscalledthehiddenlayer.TheRBMmodelcanbeconsideredasaspecialMarkovrandommodel.Allneurons{v1,v2,...,vm}inthevisiblelay-ersarefullyconnectedtounitsh{h1,h2,...,hf}inthehiddenlayerbythebidirectionalweightswpq,wherepdenotestheneuronnum-berinhiddenlayerandqdenotestheneuronnumberinthevisiblelayer.TheenergyfunctionofanRBMisexpressedbyEq.(23),whereh=(w,b,c)denotestheparameterscollectioninRBM,wpqisthebidirectionalweightofthevisibleneuronsvqandhiddenneuronshp,andbqandcparethebiastermsofthecorrespondingneuronsinthevisibleandhiddenlayers,respectively.TheprobabilitiesofeachneuroninthevisibleandhiddenlayerscanbecalculatedviaEqs.(24)and(25),respectively:XfXmEev;h;hT?wpqhpvqàXmbXfqvqàcphpe23Tp?1q?1q?1p?1Fig.4.ThestructureofanRBMwithfhiddenandmvisibleneurons.Y.Zhangetal./Engineering5(2019)671–678675Pev;hT?1Xexp?àEev;h;hT??ZehTv1Xexp?àEev;h;hT??ZehThe24TPeh;hT?e25TInEqs.(24)and(25),ZehTdenotesthenormalizationfactorexpressedbyEq.(26).ZehT?fmXXq?1p?1?àá?expàEvq;hp;he26TSinceanRBMprohibitsanyconnectionsbetweenneuronsinthesamelayer,theconditionalprobabilitydistributionsPehjvTandPevjhTcanbecalculatedastheproductsoftheBernoullidistribu-tionsexpressedinEqs.(27)and(28),wherereuT?1t1isthesig-eàumoidactivationfunctionandudenotestheinputvalueoftheneuron.Acontrastivedivergencesamplingalgorithmisappliedtoupdatethemodelparameterswpq,cp,andbqinEqs.(27)and(28).canbecalculated.RBM1istrainedwithallthetrainingsamplesuntiltheterminationconditionisful?lled.Thenthetrainedparam-etersofRBM1are?xed,andthehiddenlayerofRBM1isconsid-eredtobethevisiblelayertotrainRBM2,accordingtothesamealgorithmofRBM1showninFig.5.Thepre-trainingisunsuper-visedandstopsonceallthesuccessiveindividualRBMshavebeentrained.(2)Fine-tuningstep.TheparametersineachRBMareupdatedandoptimizedbyapplyingaback-propagationalgorithmtoreducetheoverallerrorofthetrainingsamplesandenhancetheclassi?ca-tionaccuracyoftheDBNmodel.AllDBNlayersaresimultaneously?ne-tunedinthisprocess.TheoveralltrainingerrorisgeneratedbycomparingthetargetswiththeoutputoftheDBNmodel.Thesupervised?ne-tuningprocessiteratesuntiltheterminalconditionoftheDBNmodelisful?lled.4.2.DatapreparationInthisresearch,thespectralinsightsintoasampleacquiredbythespectrometeraredividedinto25sub-bandsbywavelength.Themeanvalueofeachspectralsub-bandiscalculated,and25fea-turesareextractedintotal.Boththevisiblelightopticalsignalandthere?ectedlaserlightopticalsignalcapturedbythephotodiodearedecomposedintofourlevelsbytheWPDmethod;thefre-quencybandsofinterestareconsequentlydividedinto16sub-bands.AsmentionedinSection3.3,10differentfeaturesareextractedfromeachsub-band.Therefore,320featuresintotalareacquiredfromallsub-bands,consideringthevisiblelightopti-calsignalandthere?ectedlaserlightopticalsignal.Thevolumeandtilteddegreeoftheplume,andthenumberofforwardandbackwardspatters,areextractedfromtheimagescapturedbytheUVVbandvisualsensorsystem,andthefeaturesofkeyholesizeandpositionareacquiredfromtheimagesfromtheauxiliaryilluminationvisualsensorsystem.Thecalculatedfeaturesofeachsetof1000fromtheoriginalsamplingdatafromthephotodiodearecompressedtoonepieceofsampledatainordertosynchronizewiththesamplesfromtheothersensors.Fortheauxiliaryilluminationvisualsensorsys-temandtheUVVbandvisualsensorsystem,theaveragevalueofeachsetof10sequentialsamplesiscalculatedasonepieceofsam-pledata.Therefore,thesamplingratesofallthesensorsinthisresearcharesynchronizedat500Hz,whichisthehighestsamplingrateofthespectrometer.Finally,atotalof351featuresoftheweldingprocessareacquired.Thesamplevaluesofeachfeaturearenormalizedto0–1inordertoensurethateachfeaturehasthesameweightdespitetheirdifferentscales,andthustoimproveXàáPhp?1jv?rcptwpqvqq!e27T!e28TPàvqXá?1jh?rbqtwpqhppTheoutputvectorofthehiddenunitscanbecalculatedaccord-ingtotheforwardpropagationalgorithmwiththerealinputdatainthe?rstvisiblelayer;theoutputofthe?rsthiddenlayeristhenconsideredtobetheinputdataforthesecondhiddenlayer.AtthetoplayerofDBN,aclassi?erisemployedforthepurposeofclassi?cation.Inthisresearch,asoftmaxclassi?er,whichcanconductQmultipleclassi?cationproblems,isappliedasthe?nallayeradheringtotheDBNmodel,asexpressedinEq.(29),whereDdenotestheprobabilityvalueoftheclassi?cationandddenotesthecategorynumberinQclassi?cation.Thesoftmaxclassi?ercanbeconsideredtobemadeupofanumberoflogisticmodels.exqDd?PQxpp?1ee29TADBNmodelcanbeconstructedbystackingafewRBMslayerbylayer.Inthisresearch,aDBNmodelwiththreehiddenlayersisestablished;itsstructureisshowninFig.5.ThetrainingprocessoftheDBNisconductedwiththepre-trainingand?ne-tuningsteps.(1)Pre-trainingstep.Theinputdataisdirectlytransmittedtotheneuronsinthevisiblelayer,andthentheoutputofRBM1Fig.5.ThestructureoftheDBNappliedinthisresearch.
基于DBN 模型的激光焊接状态在线监控研究 - 图文
