Astm stp 1594 2016

Ther esults ofthe performanc test meth ds, which wo ld me sur e c pabili esalo ga spe tr um,wou ldalsoprovideinsightforma nu fa tu rer a nd develo er wh co ldusetheresultstog uid ein est

ASTM INTERNATIONA L

S l cted technica l Pa PerS

Libr ary of C ngr es Catalo ing-in-Publction Data

Names: Bostelma , Ro er, e itor | Mesin , E R (Ele a R.), e itor | AS M

Interntio al

Title: Auto omo s in ustial v ehicles : f r om th la or atory to th factory

f or / ditors, Ro er Bostelma , Ele a Mesin

Descriptio : West Co sh h cken, PA : AS M Intern tio al[2016] | S ries:

S le te teh icl p p rs ; S P15 4 | Pa ers pr ese te at a worksh p h ld

Ma 2 -30, 2015, in S atle, Washin to , USA | “AS M Sto k #S P15 4.” |

“DOI 10.5 0/TP15 4-EB.” | In lu es biblo r ap ic l r e r en es

Id ntif rs: L C CN2015050 64 (print | L C CN 201505146 (e o k) | ISB

9 8080317 3 1 (p k.) | ISB 9 8080317 348 ()

S bje ts: L CSH: Auto omo s v ehicles—Co gr eses | Motor vehicles—Automatic

cntr ol—Co gr eses | Intelg nt cntr ol systems—Co gr eses

Clasif atio : L C C T 15 8 A8 2016 (print | L C C T 15 8 (e o k) | D C

6 9.0 / -d 2

L C r ecr d av aia le at htp://lc o g v2015050 64

Co yrig t © 2016 AS M INT R ATIONAL, West Co sh h cken, PA Al rig ts r eserved T is material

ma n t b r epr od cd or c pie , in wh le or in p r, in ay printe , meh nicleletr onic, flm, or oth r

distib tio a d stor ag me ia, with ut th write cnse t of h p blsh r

Photoc opy Rights

Auth rizatio to p otoo y items for intern l p rso al or e u atio al clasr oom use, or th intern l

p rso alor e u atio al clasr oom use of sp cif cle ts, is gr ante b AS M Intern tio al pr ovid d th t

th a pr opriate fee is p id to th Co yrig t Cle r ane Ce ter, 2 2 Rosew oo Driv e, Da vers, M 019 3,

T l(9 8) 646-2 0 ; htp:/w w.c p rig t.c m/

T e S ciety is n t r esp nsible, as a b d, for th stateme ts a d o inio s e pr ese in this p blc tio

AS M Intern tio al d es n t e d rse a y pr od cts r epr ese te in this p blctio

P eer Rev iew P lc

E c p p r p blsh d in this v olume was ealu te b tw o p er r eview ers a d at lest o e e itor T e

a th rs a dr ese al of h r eview ers’ c m e ts to th satisfactio of b th th teh icl e itor() a d

th AS M Interntio al Com ite o Pu lctio s

T e q alty of h p p rs in this p blctio r ef cts n t o ly th o vio s efors of h a th rs a d th

teh icl e itor(), b t also th w ork of h p er r eview ers In kee in with lo g-sta din p blctio

pr actics, AS M Intern tio al maintains th a o ymity of h p er r eview ers T e AS M Intern tio al

Com ite o Pu lctio s ak owle g s with a pr eciatio th ir d dictio ad c ntib tio of ime

a d efor o b h lf of AS M Interntio al

Citation of Paper

Wh n citin p p rs f rom this p blctio , th a pr opriate citatio in lu es th p p r ath rs, “p p r title,”

S P title, S P n mb r, b o e itor(), AS M Interntio al West Co sh h ce , PA, year, p g ran e,

p p r DOI lste in th fo tn te of h p p r A citatio is pr o vid d o p g o e of ec p p r

Printe in Ba S or e, N

Apri 2016

THIS COMPILATION OF S l ect ed Tec hnica l Papers, STP15 4, A utonomous

In-d s tri a l Vehi cles: Fr m the La boa to y to the Fa c tory Flo r, con tains pe r-reviewed

papers that were presen t ed at a wor ksh p held May 2 –3 , 2 15, in S attle, Wa

sh-ingt on, USA Te wor ksh p was sp nsored b ASTM In ternato a l Commi tt ee F4

o Dri ver les Au tomat ic G uid ed Indu stria l Vehic les

Wor ksh p Chairper so s and STP Ed i t or s:

Ro er Bost el man

El ena Mes ina

Na ti ona l Ins t ute s o Sta nda rds a nd Te ch ology

G a ithe rsb urg, MD, USAFor ewor d

Overview vi

T w ar ds Develo me t of an Automate Guid d Ve icle Intel g nc e L vel

Ro er Bostelman an Ele a Mes ina

Pr elminary Develo me t of aT st Meth d for Obsta le Detectio an Av idanc e

A dam Noro an Holy Yanco

3D S nsors o Driv rles Truck s for Detectio of Ov rhan in Objects in th Pathw ay 41

Kla He e b r g an Björn Åstr an

Multi-AGV Sy stems in S ar ed In ustial E vir onme ts: A dv anc ed S nsin an

Co tr ol T ch iq es for E hance Safety an Impr ov d Efcie cy 5

L r enzo Sabatini Ele a Car dar el Valerio Digani

Cristian S c hi an Ces r e Fantuz i

T e Safety -to-A uto omy Curve: An Incr eme tal Ap r oa h to Intr od cin

Daniel T e bald an Fr ed rik He er

Dy amic Metr olo y Perf ormance Mea ur eme t of a Six De r ees-of -Fr ee om

Ro er Bostelman, Jose h Falco, Mi S ah, an Ts i Ho g Ho g

Harmo iz tio of Resear ch an Develo me t A ctivities T war d Stan ar diz tio

Z e ko Kov ačić, Michael Butler, Paolo Lista, Gor an Va i e ić, Ivic Dr aganja ,

Damjan Miklć, Tamar a P tr ović, an Fr an P tic

Rec omme datio s for A uto omo s In ustial Ve icle Performanc e Stan ar ds 12

Ro er Bostelman

Aut omatc gu id ed vehic l es (AGVs) were o e ofthe ear l iest ap l ic t io s for mo ile

rob t s.Te fr st AGVs were d epl oyed in the 19 0s t o tr ansp rt mat eria l s in large fa

-cil i ties and wareh u ses M obile rob t c pabil i ties have a d vanc d sig nifcan tl y in the

past d eca d es Tis prog res is due in large part t o researc her s at t ec hnica l u i versi t ies

wh have ma d e tremend ou s strid es in ap l yin com put er con t rol and sensor s t o

mo-bil e platforms for u ses in ap lic to s suc h a s man ufa cturing , hea l t h care, mili tary,

and emer gency resp nse As indu stria l vehic les g ained more c pabili t ies, the “A” in

A V began t o tr ansi t io from “au t omatc” t o “aut omat ed” in informa l u sage Tis

mirror s the prog res in g u id ed vehicl es in area s suc h as safety sensin and rea ctng

F urther a d vanc men t s in mo ile rob tcs, suc h a s in more gener a l -pu rp se

sens-ing , planning, comm unic t io s, and con trol , are pavin the way for an er a where

t he “A” stand s for au t on mou s.” Tis ev l utio in on board in t eligenc ha s g reatl y

expand ed the p t en tia l sco e ofap l ic t io s for A Vs and th u s r aised the ne d for

stand ard means ofmeasurin performanc

A new commi tt ee was formed u d er ASTM In t ernato a l to d evelo these mi

ss-in standard s for measuring , d escri bing , and c har a ct eriz in performanc for thi s new

breed ofA Vs ASTM ’s Commi t tee F4 o “Dri ver l es Aut omatc G uid ed Indu st ria l

Vehic l es (h ttp:/ w ww.astm.org /COMMIT E /F4 h tm) i s sco ed t o inc lud e stand

-ard iz ed n menc lature and d ef i tio s oft erms, recommend ed pr a ctc s, guid es, t est

meth d s, spe ifc t io s, and performanc stand ard s for A Vs Tese new per

for-manc standard s wil com pl emen t the o g in wor k in A V safet y standard s b

t he Indu stria l Truc k Standard s Devel opmen t F u d ato [1] the Bri tish Standard s

Insti tu t io [2] and other s F4 i s a d d resin area s that are im portan t for p t en t ia l

A V u ser s t o u d er stand when ma kin p rcha se and ta sk ap l ic t io d eci sio s

Terefore, the commi tt ee is d i vid ed in to f e t ec hnica l su bcommi t tees that focu s o

t he key areas ofin t erest for the comm uni ty :

F4 01Environmen ta l Efe t s

F4 0 Doc kin and Navigat io

F4 0 Obje t Det ecto and Prot ecto

F4 0 Comm unic t io and In t eg r ato

F4 91Termin lo y

Te fr st even t org aniz ed b the A STM F4 Commi t tee was a wor ksh p in t end ed

to fost er comm unic to bet we n researc her s and pr a cti tio er s and was held at he

2 15 Insti tut e of Ele trica l and Ele tronic Eng ine r s In t ernat io a l Co frenc o

Ro ot ics and Aut omato (CRA) Organiz ed b Ro er Bostelman ofthe Nato a l

Insti tu te ofStand ard s and Tec hn lo y and P at Picarielo from ASTM In t ernat io a l ,

t he wor ksh p “Aut on mou s Indu str ia l Vehic l es: From the Labor at ory t o the Fa ct ory

Fl oor” so ci ted researc her in put for the d evel opmen t ofco sensu s stand ard s and

sou g h t t o educ t e researc hers abou t a standard s-based mec hanism for r apid t ec hn

l-o y tr ansfr from the labor at ory t o indu st ry

Ti s b o com prises expand ed versio s ofsele t ed papers presen t ed at he ICRA

wor ksh p Te wor ksh p and this b o fature per spe t i ves from relat ed standard s

efort s, indu stry ne d s, and cu tt ing-ed ge researc h Te fr st c hapt er, “Toward s Devel

-o men t of an Au t omat ed G uid ed Vehic le In t eligenc Level P erformanc Stand ard”

b Bost el man and M esina , set s the stage b reviewin standard s d evel opmen t for

other mo il e rob t ap l ic to d omains, suc h as emer gency resp nse, and sug gest s

ap roa c hes for ta c klin performanc mea su remen t for in t eligen t AGVs Te auth r s

d i scu ss exam pl es ofper formanc stand ard s that co l d be u sed for vehic le navigato

performanc and for percept io syst ems (whic h wo ld be key com po en t s of in

tel-l i en t vehic les)

Nort on and Yanco’s c hapter, en ti tl ed “Prel iminary Develo men t of a Test M eth d

for Obsta c le Detecto and Av idanc in Indu stria l Environmen t s,” bu ild s o the

fr st c hapt er b d ocumen tin the proc s for d evelo in a test meth d Teir

pro-c s start s wi th b il d in an u d er stand in ofthe d eplo men t en vironmen t throug h

t he d evel opmen t ofa ta xo om y ofrel evan t o sta c l es Te key c har a ct eristcs are

ab-st r a cted t o creat e reco fgur able artfa ct s for co du ct in t est s that are represen tati ve

ofrob t a sks Statstca l sig nifcanc ofperformanc data and other key aspe t s ne

-esary for uc es ful t est meth d s are a l so co sid ered

One ofthe chalenges of deplo in AGVs in u struct ured fa cili ties is the p

s-sibili ty ofo sta cles appea rin n t just o the g r ou d bu t al so abo e the f or To

broa d en o sta cle det ecto c pa bili ties, Heden er g and Å strand im plemen t ed tme

offigh t a nd struct ured ligh t sensor s o an u manned vehicle and co d uct ed

sev-era l ex perimen ts t o cha r a ct eriz e the performanc ofthis sensin combinato in

the la borat ory and in an ind ustrial set ting Te r esul ts oftheir experimen t s a re

pre-sen t ed in “3D S nsor s o Driver les Trucks for Det ecto ofOver hang in Obje ts in

the Path way,” which discuses the im plic to s ofusin these sensor s in real-wor ld

set ting s

In the c hapt er MultiAGV Syst ems in Shared Indust ria l Environmen t s: Ad vanc d

S nsin and Con trol Tec hniq es for En anc d Safety and Im proved Efciency,”

Sab-attini et a l ta c kle the com pl exi ties ofm ulti ple AGVs o er atn in u structured envi

ronmen t s Tey d so throu g h fu sio ofsensor d ata from the d iferen t vehic les.Te

fu sed data produ ces a g lo a l en vironmen t represen tato that s u pd at ed in rea l -tme

and is u sed for a ssig nin misio s t o A Vs and su pp rtn path plannin and o

-sta c l e av idanc

Teo a ld and Heger’s c hapt er co sid er s the tr ansi tio from researc h c pabili

tes t o im pl emen tato s in indu stry from an incremen ta l perspe ti ve Teir c hapt er,

en t i tl ed “Te Safety-t o-Au ton m y C urve:An Incremen ta l Ap roa c h t o In trodu cin

Au tomat io t o the Wor kforce,” prop ses g r a dua l im plemen tato ofaut omato for

rob tc syst ems Start in wi th the d eplo men t ofthe ne esary safety syst ems, whic h

inc lu d e sensin and supp rt in a lg ri thms, the au th rs a d voc t e l ever ag in the

sen-sor data rom the safet y syst ems t o a ccu m ulat e informato and k owl ed ge about the

environmen t and humans Tus, the rob t s learn h w t o navigat e and behave o an

o g in basi s, bu ild in co fd enc in the indu stry t o a llow incremen ta l a d opt io

Te cri tica l i ty of robu st sensin t o enable a d vanc d performanc and safety for

A Vs heig ht ens the im portanc ofmeasurin h w wel a sensor syst em per forms

P er formanc t est meth d s m u st have a basis for com par iso t o a refrenc —or

g rou nd truth—syst em that s ty ica ll y t en t imes bett er than the syst em u d er t est

Te c hapt er b Bostelman et a l , “Dy amic M et rolo y P erformanc M easuremen t of

a i x Deg rees-of-Freed om Tr a c kin Syst em Used in Smart M an u fa ctur ing,” d escri bes

a meth d for eva luatn the a ccu r a cy of a p t en tia l g rou nd tru th syst em

Te c hapt er “Harmo iz ato ofResearc h and Devel opmen t Act i vi ties Toward

Standard iz ato in the Au t omat ed Wareh u sin Systems b Ko a čić et a l hig

h-lig ht s the role ofstandard s in bridg in researc h and commercia liz ato Teir wor k

d escribes a E u ropean Commis io proje t in a d vancin aut omat ed wareh u sin

throu g h a set offreel y navigatn AGVs in large-sca l e fa cili ties Te auth r s d iscus

per formanc stand ard s and benc hmar ks that can enable t ec hn lo y tr ansfr from

the labor atory to indu stry

Te b o ’s f a l c hapt er, “Recommen ato s for Aut on mou s Indu stria l Vehic le

P er formanc Standard s,” b Bost elman, summariz es and syn thesiz es a d i scu ssio

sesio that wa s held at he ICRA wor ksh p Te f d ings from the wor ksh p

pre-sent ed in this c hapt er are mean t t o inform the stand ard iz ato efort s u d er A STM

Commi tt ee F4 and a cc ler ate the infu sio ofin t ell i enc so a s t o enable au ton

-mou s gu id ed vehic les

Refer enc s

[1] ANSI ITSDF B5 5:2 12, Safet y Stand ard for Dri ver l es , Au t omatc G uid ed

Indu stria l Vehic l es and Aut omat ed F uncto s ofManned Indu st ria l Vehic les,

November 2 12, htt p:/w ww.i t sd f.org

[2] Bri tish Stand ard Safet y ofIndu str ia l Truc ks—Dr i ver l es Tru c ks and Teir

Sys-t ems Tec hnica l Rep rt BS EN 15 5, 19 8

Te ICRA wor ksh p was su pp rt ed b the IE E Tec hnica l Commi t tee o P erf

or-manc Eva luato and Benc hmar kin of Ro otc and Aut omato Syst ems Te

Ed i t or s are g r at eful to the reviewer s of thi s b o wh se commen t s g reatl y im proved

i t s q a l i ty

Bostelman, R an Mesina, E “Twards Develo me t of an Automate Guid d Ve icle

Intel g nc e L vel P rf ormance Stan ard,” Au to omou s Indu strial Ve icles: From th

L b rtory to th Factory Flo r, A TMSP1594, R Bo st elman an E Mesina, E s A S TM

Int ernatio al WestC o nsho hocke ,PA ,2016,p 1–2 ,d i10.15 0/S T P 15 4 0150 5

2

Automate guid d vehicles (AGVs) ty ic ly have b e use f or in ustrial

material han ln sinc e th 19 0s n th y ear f olowin , U.S an Europ an

saf ety stan ar ds have b e evolvin to protect n arb w ork r How ever, n

p rf ormanc e stan ar ds have b e d velo e f or AG sy stems In our view,

les o s c n b learn d f or d velo in su h stan ar ds f rom th research

an stan ar ds as ociate withmo ierob tsap le tosearchan resc ean

mi tary ap lc tio s Research chale g events, tests an evalu tio s, an

intel g nce-lev el ef f orts hav also occ urr ed that c n sup ort in ustrial AG

d velo me tsintohig er-level ntel g ntsystemsan provid usef ulstan ards

d velo me t criteriaf orAGVp rf ormancetest meth ds.T is chapterprovid s

ba kgroun inf ormatio r ef erenc edf r omal of h seareasto sup ort h n e

f oranAGVp rf ormanc estan ard

K eywords

sta dards,p rf orma c e,mo ier ob t,a tomate guid dv ehicle(AGV)

Man scriptr ec ivdJu e16,2015;ace te forp blcatio Au ust1,2015

1

Natio alInstitut eofStan ards n Tcn lo y,10 BureauDr Gait hersb rg,MD208 9-8 30

2

AS MWorksh p nAu ton mo sIndus ialVe icles:Fromth La oratorytoth Fac toryFlooro

May 6–30,2015in eatle,Washin to

Copy rig tV

C

2016byASMInterntio l10 Bar HarborDriv e,POBo C70 ,Wes Cosh h k en,PA1942 -2 9

S P15 4,2016/av aiable nln atw w.a tm.org/d oi 10.15 0 S P15 420150 54

Intr oduction

Aut oma t ed gui d ed vehi c les (AGVs) ha ve ben used si nc e 19 3 In t he ye a r s

fol l owi ng,“A GV s ha veev l ved i nt o c omplexma t eri a lhandl i ngt ra nsp r t vehi c les

r angi ngfrommai lha nd l i ngAGVst ohi ghl ya u t oma t i ct r ai l erl oad i ngAGVsusi ng

l a ser a nd na t u ra l t a rget navi ga t i on t ec hn l ogies [1] P t e nti al u se r s of A V

t e c hn l og k owt ha t A Vs a re sa fe when AGV manufa c t ur ers c onform t o t he

Americ an Na t i onal St a ndar d s Inst it ut e/ ndu st r i al Tru c k Sa fet y Devel opment

F undat i on (A NSI TSDF) B5 5:2 12,Sa fet y St a nd a r d forDri ve r l es, A u t omati c

Gu i d ed Ind u st r i al V ehi c l es and Au t oma ted Fu nc t i ons of Manned Ind u st ri a l

Vehi c l es [2] However , t here a re n c u rr ent st anda rds t o di r ec t l yc ompar e A V

i nt el l i gent perfor ma nc e su c h t hat u se r s c an ful lyappr ec i a t e t hei r p t ent i a l A V

i nvestme ntwi t hou ti nd epend entt est sa ndeva l u at i ons

No i nd ust r ia l ve hi c le a ppl i c a t i ons (e.g., d ri ver l es c a rs, sea r c h and r esc u e

r ob t s, mi l i ta ryu nma nne d ve hi c les) a r e ra pi d l yi mpr ovi ngt hei r c a pabi l it i es a nd

i nt el l i genc e, t hus provi di nga c l ea r sense oft he ad va nc ed fat ur es t hat c ou l d be

i nst a l l ed i n i nd ust r i al AGV s The be nefi t s t o A GV u ser s wou l d be en r mou s i f

AGV s gai nedo b ar di nt el l i genc et ha twou l d al l owt hevehi c l est o ad a ptt o t hei r

ma nu fac t u ri ng fa c i l i t i es i nst ea d ofvi c e ver sa An i nt el l i genc e-l e vel perfor ma nc e

st a nd ar dwoul dbenc hmarkc u r rentc a pa bi l it yl evel s andst anda rd i z et estmet hods

t od ot hebenc hma r ki ng Benc hmarkspr ovi d ea ni nc e nt i veforAGVd e vel oper st o

a c hi evehi gherper for manc e ,whi chenabl e st hemt oe xpa ndt hei rma r ket st oi nc l u d e

br oa dera ppl i c at i ons,suc ha st hosewi t hi nu nst r u c t u reden i r onment swi t hworker s

presen

Thi s c ha pt e r pr op ses met hods for mea su r i ng AGV per for manc e t ha t c a n

provi d e t he fou nd ati on for a new, v l unt a ry AGV i nt el l i genc e-l evel perfor ma nc e

st a nd ar d The st a nd ar d wou l d c over a br oa d ra nge ofA GV c l a sses a nd i nc l u de

perfor ma nc e-me su rementt estmet hod s for est i ma t i ngvehi c le c a pa bi l it i esa ssoc

i-a t ed wi t h t he par t i c u l arvehi c l e c l asses Wepr ovi d e (1) ba ck rou nd i nfor ma t i on

a bou tc u rr entst a nd ar ds,i nc l u d i ngt hoseund erd evel opmentforve hi c l esi nemer

-genc yr esp nsea ppl i c a t io s;(2)exa mpl esofvehi c lec ha l l en eevent sa ndprogra ms

t ha tha vei mpr oveda ut on mou svehi c l ei nt el l i genc e;a nd(3)al i stofc apa bi l i t iest o

bec onsi d eredi nanewAGVperfor ma nc st a nd a rd

Aper for manc stan ar dwo ldprovid eAGVman fctu rer swihtest

meth-o s to estmate per formanc that cou ld be reerenc d as part oftheir pro uct

ma rket ing Ther esults ofthe performanc test meth ds, which wo ld me sur e

c pabili esalo ga spe tr um,wou ldalsoprovideinsightforma nu fa tu rer a nd

develo er wh co ldusetheresultstog uid ein estmentsinr esea rchan d

evel-o ment or to ta rget pa rtcu lar ma rket niches Be a use ther e is n cur rent

per-for manc stan a rd ,u ser s mustr elyo spe ific to spr ovidedb manu fa ctu rer s

wih, per haps, n t ra e blea nd reprod ucibleba sis This la kofstan a r ds-based

per for manc cha r acter iz a tio d isco r a g es man p tenta l AGV u ser from

atemptn toau toma teproc s sesorl ea dst hemt opu rc haseAGVst ha tm ayno tbe

a ppr o pri a fo rt h eirpart ic ul aren viro nm ent san dt asks.Mism at ch esbet weenex ec

t-a ti o nsand r esul ts mayrequ iread di ti o nal c apit ali nve st m entt o u gr adet he equ

ip-mentorenviron m en t

Asusedi nt hi s d oc u men “Atami ni mu m,i nt el l i genc erequ i r est heabi l i t yt o

senset heen i ronmen t omaked ec i si ons,a ndt oc ont r ola c t i ons Hi gherl evel sof

i nt el l i genc e ma yi nc l u de t hea bi l i t yt o r ec og i z e o jec ts and e vent s, t o represent

k owled ge i n a wor l d model , and t o rea so abou t a nd pl a n for t he fut ur e” [3]

Some i nt el l i genc e mea su rements ar e i nc l u ded i n t he ANSI TSDF B5 5 sa fet y

st a nd a rd F r exa mpl e, vehi c le spe d mu st be red uc ed when na vi gat in t hrou gh

c onfi neda rea s,a nd c ont r ol syst em perfor ma nc e t est s mustpr ovet ha tt hevehi c l e

st ops pr i ort o c ont a c twi t h human-r epr ese nt a t i ve o st a c l es when t heya re se nsed

u si ngn nc ont a c tsa fetysensors Spec i fict est s fort hel at t erexa mpl e i nt heANSI

ITSDF B5 5 st anda rd measu re per for ma nc oft henavi ga t io a nd sa fet ysensors

when i nt egra t ed i nt o t he vehi c l e c ontr ol l e r so t hat sa fe performa nc i s ensu red

pri or t o t ra nsfrr i ng t he vehi c le t o t he u ser However , most ve hi c l e c apa bi l it i es,

whi chma yorma yn ti nc l ud esa fet yfunc t i ons,l a c kst a nd ar dmea nsofc ond u c t i ng

perfor ma nc emea su rement sforrep rt i ngt op t ent i a lu ser st oenabl et hemt oma ke

i nforme ddeci si onst ha tred u c et heri skofa d opt in A GV s

Efor t s t o d evel op met hod s for t est i ngand eval u a t i on oft he sa fet y perfor ma nc e

of au t omat ed a nd semia u t oma t ed vehi c l es (S Vs) have be en o g i ng at t he

Na t i onalInst it ut eof t anda r d sandTec hn l og (NIST)a ndot heror ga ni z a t i onsfor

manyyea r s TheNIST ha s su pp rt e dt he A NSI TSDF B5 5, B5 1 6 (p wered

i ndu st r i al vehi c le o er at or vi si bi l i t y), a nd B5 1 (for kt r uc ks) st a nd ar ds [4] Thi s

projec tsh wedt hati ti sp si bl et omakest at i canddyna micmea su rementsofb t h

vehi c lesafet yand c a pa bi l i t ynee d s Mea su r ement r esu l t s havepr ovi d ed aba si st o

suggesti mproveme ntst ot heA SI TSDFB5 5st a nd a rd

Similarly, ot her vehicle programs an chalenge events have impr oved the

inteligenc a ndca pabili esofa u to omo sa ndS Vs.Ad i o aly,sta nd ar dsfor

emergency resp nse ro ot vehicles ha ve be n a nd co tnu e to be develo ed

Ex amples ofr elevant efor ts a re sh wn in this se to , ga thered fr om Inter net

se r ches a nd the ind ivid uals an or g a niza tio s sh wn in the a ck owledgments

se to

Example Related Standar ds

AS MEMERGEN YRE PONS ROBOTPERFORMA NCES ANDAR DS

SeveralA ST MInterna tionalsta ndards h avebeendevelop edfo remergencyr esp nse

rob ots, an d several others are in t he pipel ine Resp nse rob ots a re u s ed i n urban

se rc ha ndres cu e,bo m bd isp s al ,mili tary,l awenforc ment,an doth erti me -c rit ic al

a ndh azardo usap lic ati o ns Stan ardsthatm e asurevari o usperfo rmanc c apabil iti es

oft hes erob otsarebein devel opedu derAST M ’sC o m m i tt eefo rHo m elan Secu ri ty

Ap l ic at ions,Operat iona lEquipment ,Ro bo ts(E54.0 8.01).Thi sse ti o npro i desal ist

oft hos eAS TMst an a rdst hat aremo strelevant toAGVs[5] In t hede s c ri pt io ns,

a ppr o vedst an ard sa resh ownwit ht he ir t and ardn mb erpre ededb A ST M ;wo rk

i tems areprefi edwi thW K; an st at us(asoft hiswri ti ng)ofpreliminarydeve

lop-men t i s d esi gna ted b y V¼val id at in (.e., c he cki ng or pro i ng repea tabil it y) or

P¼prototy i ng(.e.,e xpe ri ment in wiharti fac tsan procdurestobestm easu reth e

part ic ul ar sys t em c apabil ity c o ncrned) The m ajo ri ty of c o m m erc i l y avai lable

rob ots fo r res ponsea pp c at ions h avel im i ted onbo ar d i ntell i enc e; h e nc e, t heyare

re m otelyo er at edb ah uman(y ic al l ares po de r)usin d ispl aystran sm i ttedbac k

from t he c ameras or oth er sensor onbo ard There a re some emergin a s si s t ive

a ut o nomyc apab il it ies (for exampl e, in st ai r c limb in ), b t general l spe kin , th e

re sp nse rob o t s req i re m u ch gre ter h u mani nt er ac tio t han AGVs d o The t est

meth odsd esc ri b edinthefo l lowi ngsect ionsa redes i gnedt obeagnos t icast owh eth er

t herob ot spro ra mm edtoru nthro g t het estsin ep endent lyo rifi th astobec o

n-t roll edb a nop erato rd uri ngt heent iret est ,mea nin t hatafu ya ut o nomous ob ot

sho u ldru nt hrou hthesam et est sasonethath ast obe“dri ven”b ano era tor

In gener al , t he t est met hods d e vel opedunderA STM E5 0 01 c onsi stoft he

fol l owi ngel ements:

•

Ap ar a t u s(orprop): Ar epea t a bl e,reprod uc i bl e,a ndi nex e nsi vereprese nt a

-t i on oft a sks t hat t he r ob ti s ex ec t e d t oper form I sh ul d c ha l l eng t he

rob twi t hi nc r easi ngd i ffic u l t yorc ompl exi t ya ndbeea syt ofa br i c a t ei nt erna

-t i ona l l yt oensu real lrob t sa r emea su redsi mi l a rl y

•

Pr oc ed u re:Asc ri ptfort het e sta dmi ni st r at ora ndt herob to er at ort ofol l ow

Theset est sa r en ti nt end edt osu rpr i sea ny ody.Theysh u l dbepra c t i c edt o

i mpr ovet ec hni que

•

Met r i c :Aqu ant i t a t i vewa yt omeasu ret hec a pa bi l i t y.F rexa mpl e,c ompl et

e-nes oft enc ont i nu ou sr epet i t i onsofat ask,ort err ai nfigu reei ght s,resul t i ng

i nac u mu l at i ved i st a nc et ra ve r sed Tog t herwi t ht heel a psedt i me,aresul t i ng

ra t ei nt asks/t i meordi st a nc e/t i mec a nbec al c u l at ed

•

Fa u l t c ondi t i ons: A fa i l u r e oft he r ob t i c syst em pr event i ng c ompl et i on of

t en ormorec ont i nu ousrepet i t i ons Thi s c ou l di nc l u dea n i nver t ed r ob t ,a

st uc kr ob t ,or ai l u rer equ i ri ngfi el dma i nt ena nc e

St ar t in t hea na l ysisof wi t ht ermi nol og ,ma nyt ermsd efi nedi nt hest anda r d

t er m i nol og foru rba nsea rc ha ndresc uer ob t sc ou l dhavea ppl i c a bi l i t yt oAGVs

F re xampl e,r amps,t owi ng,c onfi neda rea /spa c e,ma neuveri ng,o st ac l es,a ndpeak

p wer a r e ju st a fw p t e nt i a l l y rel evant a nd o erl a ppi ng t er ms ac r os t he t wo

i nd u st ri es

Stan ardT rmin lo yf orUrbanS archan Resc eRobotic

Operatio s(A STME 5 1–07a)

Mo i l i t yst a nd ar dsl i st edherema ypr over el e vantt oi ndu st r i alAGV sa ndmo i l e

r ob t sb desc ri bi ngt estme t hod sa ndd efi ni t i onsfora r eassu c hasen i r onment al

efc t s, o st a c l e d et ec ti on a nd a voi d a nc e, and t er mi nol og Ramps, spe d, t err ai n

t ypes,t owi ng,a ndsoo ar ea l la ssoc i a t edwi t hi nd u st r i a lvehi c l ei nt el l i gentsa fet y

a nd perfor ma nc c a pabi l i t ies (si mi l ar t o r esp nse rob t s) The en i r onment al

c ondi ti ons t ypi c a l l ya r e n tex ec ted t obea sha rsh i nt hei nd u st ri a lset t i ngst ha t

AGVsenc ou nt er,bu tt hec onc ept sfort est i ngmo i l it ya rest i l lt ra nsfr r a bl e

St a nd a r dTestMet ho forEval u a t i ngEmergenc yResp nseRo otC a pa bi l it i es,

Ma neu ver i ngTa sks:Towi ngHi t c hedTra i l ers(P)

Ener g y an p wer stan ards listed here a re relevant to in ustrial AGVs

an mo ile ro ots The energ yan p wer me surements a re co d cted u der

somewhat ard o sco di o s in order to ex pedie thetest pr oc s (drainin of

theba ttery)an torepresentsometypic lenerg yusa g epro lesinr esp nseap li

c to s Testmeth dsinspiredb theo esin ASTME5 b t adaptedtoAGVs

wou ld ensureal vehicle manu fa turer an user s co form to the same energy

an p werme surement e h iq ues Alth ug hn tasp tentalyc tastro hicas

losin aro ot hathaspenetratedaha ard ousa re d etoad eadbatery,user of

A Vs ne d to have predictable an known batery performanc to ensure

Pea kP werTa sks:C on nedA r eaObst ac l es:St a i r s/L a nd i ngs(P)

Vehiclecommu ic to wih themasterorwa reh u semana g ement systems

asu sedin AGVa pplic to s relatesto rescuero ot commu ic to c pabili es

Thefolowin stan ard sc npr ovidethebegin in of ommu ica tio an inter

-frenc testmeth d sforin u str ia lAGVsa ndmo iler ob ts Inpa rtcula r,bein

abletochara ter iz ethewireles commu ic to sbetwe n thevehiclean eiher

the o erator co trol stato or the c ntral fa ctor y or wa reh use co tr oler is

esental In typic lyteleo erated resp nser ob t ap lic to s, ther e is ne d for

co stant video strea min fr om the ro ot’s o b ard c mer as to the o er ator

co tr ol stato an ofmoto an other comman s from the o er ator to the

ro ot.WhenAGVshaveo b ardpathreplan in c pabili es(forex ample,tog o

ar ou dano sta le),theymayne dtoprovid eu datestothec ntra liz dco tr

ol-leriftheydeviatefromaprog rammedpa th.Theele tromagnetcenviro ment n

fctoriesan wa reh u sesmaybechaleng in towireles commu ic to s,heig ht

enin thepr ioriyofhavin ame nsof ha ra cter izin theAGV’scommu ic to

system

St and a r dTestMet ho forEva l u a t i ngEmergenc yResp nseRo otC apa bi l i t i es,

Rad i oC ommunic a t i on:

C ont rola ndPerc ept i onTa sks:Int er ferenc eSi gnalEn i r onment P)

Huma n-syst em i nt er ac ti on pe r for manc e st a nd ar ds ha ve per ha ps mi ni mal

r e l eva nc eo i nd u st r i a lvehi c l et estmet hod sbec a u seA Vsrequ i r el es andd i ffr

-ent t ypes ofhuman i nt era c t i on Never t hel es, t here ma y be some hu ma n-r ob t

i nt e r ac t i ons,p t e nt i a l l ywi t h fa c t oryorwa r eh u seworker swh ne edt oi nt e rvene

wi t h t heAGV Somec onc eptsma ybet r a nsfrra bl e from t heASTME5 hu

man-syst e mi nt er ac t i ont e stmet hodst ot hose orA GVs

St and a rdTestMet ho forEva l u a t i ngEmer genc yResp nseRo otCapa bi l i t i es,

Huma n-Syst emInt era c ti on(HSI :

S a rc hTa sks:Co fi edSpa c eVoi d swi t hC ompl exTe r ra i n(V) W K34 3 )

S nsors ar e c ommo l y used o AGVs a nd mo i l e r ob t s Resp nse r ob t

st a ndar ds, l i st ed he r e,pr ovi depe r for manc et est met hods for h wc a pabl e se nsor s

a re when i nt egr a t ed wi t h t he vehi c le Test met hods a l so eva l u at e h w wel l t he

c ont r ola l gor i t hmspl a c e sensorda t ai n ma pst ol oc a l i z et hevehi c l ea ndforu sei n

o st ac l edet ec ti onandavoi d a nc e

St and a rdTestMet ho forEva l u a t i ngEmer genc yResp nseRo otCapa bi l i t i es,

L oc a l i z at i onandMa ppi ng:Spar seFea t ur eEn i ronme nt s(P)

Thea for ement i onedst a nd a rd s c anmea su re pe r for manc eofc u rr ent t el e

oper-a t edr ob t sa swel lasemergi ngrob t swi t hau t on mo sc a pa bi l i t i es.Thest a nd ar ds

a real r eadybei ngused t oc ompa re c a pa bi l i t ie sofd i ffr entr e sc ue,mi l i ta ryr e c

on-na i ssanc e ,a ndb mbd i sp salr ob t s Basedo suc hc ompa ri so s,u sersc a nc ho se

t he be st syst em for t hei r ne d s Si mil a r st anda rd s-ba se d means of c ompar i ng

perfor ma nc eofdi ffrentc a ndi d at eAGV sa reneded

INS ITUT OFE ECTRICALA NDE ECTR ONICENGINE RSROBOTICSA ND

AUTOMA TIONSY ST MS

TheInst i t u t eofEl ec t ri c alandEl ec t ronicEn i ne er sRo ot i c sa ndA u t oma t i on

Sys-t ems Soc i et y (E E-RAS) ha s l au nc hed t wost and a rd s efor t s a i me da t a dva nc i ng

t hec a pa bi l i t ie sofr ob t s Ont ol ogi es(i ,for ma ld esc r i pt i onsoft hec onc eptsa nd

r el at i onshi pst ha tc anexi st )forrob t sa rei mp r t a ntt ot heA GVi nd u st rya svehi c l e

i nt el l i genc e i nc r e a se s St a nd ar d k owl e d ge t r ansfe r a mo gvehi c l es, ma na ge ment

syst ems, fac i l i tyequ i pment,a nd ot he r assoc i a t edsyst ems wi ll a l l owd irec ti

mple-me nt a t io ofmo i l e rob t syst ems i nt o fa c i l i ti es Si mi l a rl y, st a nd ar d represe nt

a-t i onsofvehi c l ec ont r olma pswi lla l l owfast erandsi mpl ervehi c lei nt e gra t i on i nt o

fac i l i t i es

•

On t ologi e sforRob t i csan dA ut om at i onWorki n gGrou p

TheIE E-RA SOnt ologies orRo oticsa ndAutoma tio W or kingGro pis

develo ingast and ar do t olog ya nda ssociatedmeth d ologyfork owled g e

r epresenta tio a ndr ea so inginr ob t icsa nda utoma tio ,togetherwiththe

r epr esentatio ofco c ptsinaninitia lsetofa pplca tio d omains Thefirt

ofthesest and a rd sis IE EP18 2,Cor eOntolog iesforRob ticsa ndAu t

o-matio ,which was a ppr oved b theIE ESta nda rd s As ociatio Stand ar ds

Boa rd in 2 15 Su ch stand ar dswi pro id e a u nified wayofr epresent ing

k owledgea ndwi pr ovid eacommo setofter msan d efi itio s,a llowing

foru na mbiguo sk owledge tr a nseramo g a nygro p ofhu ma ns, r ob t s,

a nd other a rtificia l systems A stud y g ro p ha s beg u n wor k to for ma liz e

an in u str i o tology This pr omises to have gre t r elevanc to AGV

ap lca t io s

•

IEEE-RA SRo bo tM apDataRe p re se nt ati o nfo rNavigat ionWorki ngGroup

The IE E-R S Ro ot Ma p Da t a Repr esent a t i on for Na vi ga t i on W or ki ng

Gr ou pa i mst ost a nd ar di z eac ommo represent a t i on a nde nc odi ngfort

wo-d i me nsi onal map d at a used for na vi gat i on b mo i l er ob t s The enc od i ng

w be u se d when exc hangi ng ma p da t a wi t hot hersyst ems orsu bsyst e ms

Thest anda r dfoc u seso t hei nt erc ha ng ofma pd at aa mo gsyst ems,par t i c

u-l a ru-l yt hoset ha tma ybesuppl i edb d i ffer entvend ors.Ina d di t i ont ot he nc od

-i ng,t hest a nd a rda i ms t ospec i fysui t abl e a ppl i c a t i onprogra mmer i nt erfac es

a nd pr ot oc ol sfort hei nt er c ha ng pr oc es sot ha t na vi gat i on-rel a t edc omp

-nent sfr ommu l t i pl evend orsma yi nt er oper at e

SOCIE YOFA UTOMOTIVEENGINE RSA S-4—A UTONOMYL VE SFOR

U MA NNEDSY ST MS

The Soc i e t y ofAu t omot i ve Engi ne rs (SAE) AS-4 U nmanned Syst e ms St eer i ng

C ommit t ees a d dr es a l l fa c et s ofunma nned syst ems—d esi g n, ma i nt ena nc e, a nd

i n-servi c e ex e r i enc e [6] The pr i ma r y g a l ofA S-4 i s t o pu bl ish o en syst ems

st a nd a rds a nda rc hi t ec t u rest ha tena bl e i nt er oper abi l i tyofunma nned syst ems for

mi l i ta ry, c i vil , a nd c ommer c i a l appl i c a t i ons Versi on 2.0 oft he A u t on myL evel s

forU nmannedSyst ems(AL US)fr ame wor k,whi c hi nc l ud est e r mi nol og ,i sev l

v-i ng a s unma nned syst ems t ec hn l og ev l ve s V ersi on 2.0 es e nt i a l l y c ove r s t he

r esu l tsoft heAL FUS efortu p t o t heen of 0 7 TheAL FUSmod ela ndh wi t

a ppl i est oA Vsandmo i l erob t si sd i sc u ssedi ngrea t erdet ai la sfol l ows

•

Inte rnat i on al Organ i zat i on for St an d ard i zat i on Pe rform anc St an dards

forSe rvi ceRob t s

TheInter nat io a lOr g a niz atio forSta nda r diz tio ( SO) 7] spla nningto

est ablsh per for ma nc sta nd ar d s for ser vic ro ot s [8] S r vic r ob ts ar e

th se used in ap lca tio s ex clud ing ind ustria l automa tio ; they ca n be

eit herpero a l(u sedb ala ypero forn ncommercia lta sks)orpr ofes

io-nal(usedb at ra inedo er at or orcommer cia ltasks).However,t hestand ar d

a nd the d evelo ment pr oc s ma y also be ap lca ble t o in ust ria l AGVs

a ndmo ie r ob ts The Internatio al Or g a niz atio ofStan a r ds/Te h ica l

C ommitt ee (SO/TC) 18 /St an ar d s Commite (SC2)Wor king Gr ou p

(WG) 8ha ssta rtedthed evelo mentofastand ar dformea su r ingper

form-a nc ofthiscla ssofr ob t s.Thecu r rentea rlyd ra ft ISO/Dr a ft nter natio al

Stand ar d[DIS] 18 4 -1)for“Per for ma nc cr iter iaa ndr ela t edtestmeth d s

forser vic r ob t—Pa r t1:L comotio forwhe ledr ob ts inclu d esinstr

uc-tio s ormea su r ingspe da ndbr a kingtimeo d iferentsur fa cestoprevent

fal ngd wn,toincre semo i tyo a slo ean o erasi /cu r b,tod etet

a nd a void o st acles, a nd t o me sur e the rela tive d ista nc /spe d betwe n

h ma nan ro ot

•

Inte rnat i on alEl e ct rot e ch ni calCom m i ss i onSt an dardsforSe rvi ceRob t s

TheInt er na t i ona lEl ec t r ot ec hni c alC ommi ssi on(EC )i sdevel opi ngsafet ya nd

per for ma nc est a nd ar d sfor yst emsa ndc omp nent susedforrob t i cc l ea ni ng

a ppl i c a t i ons[9].IECSC5 F(fl oort rea t me nta ppl i anc e s)W G5(met hodsfor

mea su ri ng t he per for ma nc e of h u seh l d c l ea ni ng rob t s) i s c onsi der i ng

per for ma nc esu c ha smo i l i t y(c over agera t e),na vi ga t i on,du stc ol l ec t i oni na

st anda r di z eda rea ,n i se,bat t eryp wer,a ndsoo IEC6 312st a nd a rd sd efi e

t estmet hodsformea su r i ngpe r for ma nc eofh u se hol dc l eani nga ppl i a nc es.A s

wi t ht hedr aft SO/DIS18 4 -1,IEC6 312ma ya l sopr oveu sefuli nd evel

op-i ngi nd u st ri alA GVa ndmo i l erob tperformanc est a nd ar dt estmet hods

EXAMPL PERFORMA NCEC AL ENGE A NDP OGRA MS

S ver alc ha l l en eeve nt s,i nv l vi ngt e l eo er at i veandfu l l ya ut on mou s unmanned

vehi c les, ha veoc c u rr ed i n t he l a st se vera l yea rs These event s and t hec ha l l en es

t hey p se a re d esc r ibe d i n t he fol l owi ng sec t io s, i nc l u di ng c ont ac t s a nd bri ef

ex l a nat i onsoft heeven W edi sc usst he seevent sbe c a u sewebel i evet ha t(1)t hey

havel edt oi mpr ovement s i nt hei nt e l l i gentper for manc ofn ni ndu stri a lvehi c l es,

a nd(2)ma nyoft hesei mprovement sc a nbeha r nesedb t heA GVmanufa c t ur ers

fort hei rvehi c l es

•

Ro oC u p[10]

*

Ro oC u pi sa ni nt ernat i onalrob t i c sc ompet i t i onfoundedi n19 7t opr

o-mot erob t i c sandar t i fic i a li nt el l i genc eresea r c h.Thec ompet i t i oni nv l ves

pu bl i c l ya ppeal i ng bu t formi d a bl ec hal l eng s forr ob t s Ini t i a l l yfoc u sed

o Ro ot i cSoc c e r ,i thasex a ndedt oot hera ppl i c a t i ons,su c ha sRo oCup

Resc ue, Ro oC u p@W or k [1 ] and Ro oC u p@Home [12] These newer

a ppl i c a t i ons foc u s o ser vi c e a nd a ssi st i ver ob t t ec hn l ogi es b be nc

h-ma rki ngper for manc ei nsevera lar eas,i nc l u d i nghu ma n-r ob ti nt era c t i on,

na vi gat i on a nd ma ppi ng i n dyna mi c e nvi r onment s, o jec t rec ogni t i on,

o jec t mani pu l at i on, a nd a da pt i ve beha vi or s A l l of t hese per formanc e

a rea sar ea l soc ri t i c a lt oi nt el l i genti nd ust r i ala ut oma t edvehi c l eswheret he

c ompe t i t i oni smo i ngt het ec hn l ogi est ometi ndu st ri a lc ha l l eng s F r

exa mpl e, Ro oC u p@Work, whi c h was i ni t i a t ed i n 2 12, a i ms t o fost e r

a dvanc e me nt si nmo i l erob t sequ i ppedwi t hma ni pul a t or sfori nd ust r i a l

a ppl i c a t i ons.Thec ompet i t i oni nc l u d esc ompl ext asksr angi ngfromma nu

-fa c t u ri ng,a u t oma t i on,andpar t sha nd l i ngt og nera ll ogi st i c s[10]

•

IE ESol u t i oni nPer c ept i onC hal l eng [13]

*

Thi sc ompet i t i onwa shel di n2 1 a spa r toft heInt er na t i ona lC onfer enc e

o Ro ot i c s and Au t oma t i on t o det er mi ne t he per for manc e c apabi l i t i es

ofal gori t hmst ha td et ec t ,r ec ogni z e,a nd l oc at ea n a rbi t ra ryc ol l ec t i onof

a rt i fac t s i nspa c e Thesea reperc ept u al c a pa bi l i t i e st ha t i nt el l i ge ntA GV s

wi l lr equi re

•

Asoc i a t i onforU nmannedVehi c l eSyst emsInt er na t i ona l[14]

*

TheAsociatio forUnman edVehicleSystemsInterna tio al(AUVSI

h lds a nn al ntel gent g ro n vehicle competitio s ( GVCs), o e of

fo r u man ed systems stud ent competitio s tha t they sp nsor The

IGVC is a multid isciplna ry ex er cise in pro u ct re lz tio that

chal-leng es coleg e enginering stu ent te ms to integrate ad va ncd co trol

theory,ma hinevisio ,vehicularele tr onics,an mo ieplatformfu nd

a-mentals to design an bu ild a n u man ed system Integ ratio of the

same comp nents is also ne esa ry for in ustrial AGVs an mo ie

r ob ts.Te ms romar ou dtheworldfocu so develo ing su iteofte

h-n logiestoeq u ipgro n vehicles ofthe fu ture with intel gent d riving

c pa bi ties

I i swor t h n t in t hat t heRo oC u p Resc u e and t he Sol ut i ons i n Per c ept i on

C ha l l en eha vec l oset i est operfor ma nc est a nd a rds.Ro oC u pResc u efat u rest est

a rena s t ha t c ompri se t est met hods based o a ppr ove d a nd d ra ft st a nd ar ds [15]

Thesei nc l udet heA STME5 t estme t hod sl i st edear l i er.Thedi ver si tyofr ob t sand

num berofc ompe t i t or sprovi des upp rtfort hed evel opmentofd r aftt estmet hod s

a nda l l owsfort hei rst rea ml i ni nga ndi mprovement.Fu rt hermor e,ex osi ngt hepa

r-t i c i pa r-t i ngr esea rc her st o t est met hods t ha t c apt u r ea ppl i c a t i on c hal l en esprovi d e s

t hem wi t h t a ngi bl e g a l s t owa rd whi ch t o en i nee r mor e a d va nc ed rob t i c s

c a pabi l it i es The Perc ept i on C ha l l en e proved out i ni t i al c onc e pts for measuri ng

t heperfor ma nc eofsyst emst ha t d e t ermi net hep se(p si t i on andori entat i on) of

o jec t s The sea pproa chesformed t hefou nd a t i ons forASTM E2 19–14,St a nd a r d

Test Me t ho for Eva l uat i ngt hePe r formanc e of yst emst ha t Measu r e St a t i c , Si x

DegreesofFr eedom(6DOF),P se

TheNISTfost eredc ha l l e ngest opr omotea c ad emi cad va nc ementofA Vi nt el

-l i genc eforfa c t or yen i ronment s Thesec ha l l en eswerei nt endedt ora i set hel eve l

ofi nt el l i gent perfor ma nc e for t asks t ha t may oc c ur i n rea l si t u a t i ons Two su c h

c hal len eswere:

•

V i r t u alMa nu fa c t ur i ngAu t oma t i onC ompet i t i ons(V MA C s)2 0 –2 0 [16]

*

Thesewereworksh psa ndvi rt ua l /rea l AGVc ompet i t i onsba sedo r eal

-worl dfa c t or ysc ena ri osd emo st r at i nga c c ur at epat hfol l owi nga ndd oc ki ng

T his virtual c h al lenge was d esigned to ad dr ess t h e needfo r o neor mo re

fac toryA GVstoop era teinunst ru ct ur edenvir o nmentst h ati nc luded yn a mi c

ob st ac les Tea msu s edt h eU ni fie dSystemfo rAutoma ti o nandRob ot

Simu-l ati o nfamewo r k[18] t od el iverc o mpl etedpal letsthro ghout asimul ated

war eho u s eenviro nm e nt,i nc ludi ngl o a di nga n du nl o a di ngofvehic leswit ha

ro b oti carm

DEPA RTMENTOFDE ENS P OGRA MS C A LL NGE

Hig hlyintelig ent,miliar yr ob tcvehiclesystemshavebe n develo ed,a

ccom-paniedb develo mentoftestsan evalu ato metricsforthesesystems.Estmat

-in miliar yvehicleinteligenc r eq u ir es d if rent kin s oftests a nd eva lua tio

meth ds,whicharebriefl describedinthefolowin se to s.Theya r etypica lly

ma tchedtoo era tio alne dsan n t ofor ma lin u strystand a rd s.However ,just

asbeor e,themeth d sd epen u o thevehicleclas a ndtheen ir onmentwhere

thevehiclewil beused.In o rview,thesemeth dsa realsoap lic bletoAGVs

An o erviewofsome ofthe keye r lydevelo ments in deense ro otcsc n be

fo n inSh emaker[19].Highlig htsaresummariz dhere

•

In 19 7,t heU.S ArmyL abor at oryC omma nd (L ABC OM) (l at erpa rtoft he

U S.ArmyResear c hL a bora t or y[ARL ])i ni t i a t edapr ogr amt oc onsol i d at et he

t ec hn l ogi esnec esar yt odevel opat estbedu nma nnedgr oundvehi c l e(UGV)

i nac oo era t i veeforta mo gt hel a bora t or i esi nt hec omma nd Thi swast he

be i nni ng oft heTec hba se En anc eme nt forAut on mou s Mac hi nes, whi c h

l a t erbec a met heRo ot i c sTestBedprogra m[2 ] a ndi tc a rr i edou tanumber

ofd emo st r at i onprogra ms:

*

DemoI 19 1–19 2)

*

DemoA(19 3)—Demo st r at edaworki ngvehi c l ea ndo era t orwor kst

a-t i on i nfra st ru c t u rea nde a r l yi nt egr at i on ofroa d fol l owi ngandt el eo era

-t i onc apa bi l i t i e s

*

DemoB(19 4)—Demo st r at edo -roa da ndof-r oadna vi ga t i on,o st a c l e

avoi da nc e,andt ar getdet e c t i onusi ngfor war d -l oo i ngi nfr ar ed

*

Demo C (19 5)—Demo st r at ed du a l c oo er at i ng sur r oga t e S V , t ar get

det ec t i ona ndt ra c ki ngc a pa bi l i t i es,mi ssi onpl anni nga ndmo i t or i ngand

exerc i se dt hesyst emi nmi l i t a ri l yrel evantsc ena r i os

*

Demo I (19 6)—Devel oped and ma t u red na vi gat i on a nd au t omat i c

t a rgetr ec ogni t i ont ec hn l ogi esc ri t i c a l fort hedeve l opmentof upervi sed

A GV sc apa bl eofpe r for mi ngmi l i t a rysc ou tmi ssi onswi t hmi ni malhuma n

o er si gh Thepr ogr amc u l mi na t edwi t hahi ghl ysuc c esfu lser i e soffiel d

exerc i ses wi t h t hr ee c oo era t i ng S V s i n a mi l i t a ry e nvi r onment a t F

Ho d,TX

*

Demo I (19 8)—The Ex e r i ment al U nma nned V ehi c l e (XU V) Ex er i

-ment wa s c ondu c t ed t o e xami ne t heresul t s ofa d di ng semi a ut on mou s

pl at formst ot he c outPl a t oo oft heArmyAft erNex

*

Ro ot i c s Col l abor at i veTec hni c a l Al l i anc e (2 0 -2 2 ) [21]—Thegoa lof

t hi s a l l i a nc e i s t o d evel op hi ghl y a dvanc ed grou nd ve hi c l es t hat a c t as

par t ner st osol d i er s.Spec i ficresear c hgoa l sar e:

n

Pe r c ept i on—Perc ei vea ndu nd erst a ndd ynami ca ndunk ownen i r

on-me nt s, i nc l ud i ng t he c rea t i on ofa c ompr ehensi ve model of t he sur

-r ou ndi ngworl d

n

Int el l i genc e—Aut on mousl ypl anandexec u t emi l i t a rymi ssi ons;rea di l y

a d aptt oc hangi ngen i ronment sa ndsc enar i os;l ear nfrompr i orex eri

-enc e ;shar ec ommo under st andi ngwi t ht e a mmembers

n

Hu ma n-Ro ot Int e r a c t i on—S a ml esl y i nt egr at e u nma nned syst ems

i nt omi l i t ar yandc i vi l i a nsoc i et y

n

De xt erou s Mani pu l a t i on and Uni que Mo i l i t y—Mani pu l at e o jec t s

wi t h near -hu ma n d ext er i t y a nd maneuver t hr ou gh t hree-di mensi ona l

en i ronment s

Ex eri ment s a rec ond uc t ed r egu l ar l yt ha teva l ua t epr ogr es i n e a c hoft hese

a rea sandfort hei nt egr at edsyst em

The ARL seri es ofunmanned vehi c le pr ogra ms l i st ed her e ha ve provi d e d

a dvanc ement si nha rd war eandal gori t hmst ha tena bl egr eat e ra u t on my,espec i a l l y

i n n a vi ga t io a nd o st a c l e a voi d a nc e The newer pr ogra ms ar e a dvanc in ot her

t e c hn ol ogi est hatwi llbeofrel evanc et oAGVc a pa bi l it i es,i nc l u d i nggr e a t eri nt el l i

-genc ea ndbet t erhu man-rob ti nt er a c t i on

An thercolab r atvete h ologyalia nc bein sp nsor ed b ARL isc led

Micr o-Auto omou s S ienc a nd Te h olog y(MAST) [2 ] I se ks to ad anc

theo b ar dintelig enc forsmal vehicles MAST develo sauto omou s,mult

fu cto al cola bora tive ensembles of a g ile, mo ile microsystems to en anc

ta tc lsiu a tio a lawa r enes in u rban a ndcomplexter a in forsmal u i o

era-to s The sc le an mis io s envisio ed for these vehicles are n t relevant to

A Vs However , siu ato a lawar enes is relevant, an the rese r ch may prove

The Defense Advanc e d Resear c h Pr ojec t Ag nc y’s (DA RPA) U GV

Pr ogr am [2 ] d evel opedkeyt ec hn l ogi esofa u t on mou sna vi gat i on a nd

a ut omat i ct ar getrec ogni t i onpri ort ot r ansi t i oni ngt hemt ot heDepa rt ment

ofDefense(DoD)

*

2 0 /2 0 Gra nd C ha l l e ng : Thi s wa s a l ong-d i st anc e c ompet i t i on for

d ri ver l es c a r st ha tweret askedt oau t on mousl ydr i ve2 0km(15 mi l es)

t hrou ght heMojaveDesertfr omL osAng l est oL a sVe a s

2 0 –2 0 DA R PA Le r ning Ap led to Gr ou d Rob ts (LA GR) Pr

o-gr am[2 ]: TheLAGRpr ogra mhadtheg oalofa cc ler a t ing pr ogr es in

a uto omou s, perc pt io -ba sed, of-r oa d na vigatio in ro ot ic UGVs

Thepr ogr amha dan velap r oa chofpr ovidinga llte mswiththesa me

ba selne plat for m an softwar e, which they wou ld au gment wit h their

speificad anc ment s Reg u la r t ria lswereheld tocompar e ea chtea m’s

r esu ltsa g a inst hebaselnean theotherte ms

*

2 0 U r ba nC ha l l eng :Thi swa sac ompet i t i ono a 6-km(6 mi l e)u rba n

ar ea c ou rse a t Geor ge Ai r F rc e Ba se, C A I r equi red dr i vi ng au t on

-mou sl ywhi l eo eyi ng a l lt r affi c r egu l at i ons a nd ne ot i at i ng ot her t ra ffic

ando st a c l e sa ndmergi ngi nt ot r affi c

*

2 12–2 15DA RPARo ot i c s C hal l eng :Int hi sc ompet i t i on,t hegoa lwas

t odevel opgrou ndr ob t sc a pabl eofexec ut i ngc ompl ext a sksi nda ng rou s,

degra ded ,huma n-engi ne reden i ronment s

The DARPA progra ms c n be lever aged b AGVs wher e a dvanc d sensor,

map ing, an behavior generato chalenges ca n c rry over into inteligent

in u stria l vehicle siu ato a warenes/av idanc an co t rols The LAGR

Pro-g r am’sevaluato ap roa hpro idedregulart rialsthatme su redprogr es a g ainst

a ba seline, simila r to h w per forma nc test meth d s sh uld be co sider ed

forAGVs

T S A NDEV LUA TIONOFMILITARYVEHICL S

Bypro vidin fu di ngfo rres earchpro r amsan c h a ll engeprob l ems,t hemil it aryh as

sti mu lat e dt h ead va nc em en toft ec h no l o gie sfo raut o no m ous,i nt el li gentveh i cl es.The

pro ra msan dc h al lengesal s oservea sago odmo d elfo rt estandeval ua ti o nofvehic les

t ha taret obefiel ded,aswel lasAGVs Duri n gt h eS i enc ofA ut o no myTec hno l o gy

Fo c usTea mstudyi n2 0 a nd20 09[2 ] t hepaneli den t ifiedt estan eval ua ti o n/ ve

r-i fic at r-io an dval i a ti o na sbei ngamajo rro a dblockt ot hefiel di ngof ut ureaut o no

-mo u s mil it aryvehic le syst em c apabil it ies I i s bel ieved t hatt h e si tuat io h a s no t

c h an gedmuc hi nt h ei nt er im

TheDoDtestan evaluato commu iyhasre ogniz dthisr oad locka nd,

as a resu lt, the Auto omou s Systems Test an Evalua tio Req u irements Stu dy

(AST RS) wassetu thr oug htheAr myTestan Evalu ato Comma nd(AT C)

[2 ] AT C lo ks for p tenta lly useul neweq uipment forsold ier s in ad di o

to verifyin t hesaetyoftha t eq uipment an lists t he “te h ic l oro erato al

limia tio s of te h olog ies that req uire fu rther investg a tio an testng

AST RS wil as es the cu rrent state ofemer g in (miliar y) AGV te h olog ies,

emergin AGVreq uirements,an cu rr ent estan eva lua tio c pa bili es

I is cl ear from th e previo s sect io ns th at “t as k co m pl exit y an adaptab il ity

to t he en iro m ent” are cri c al to im pro vin g performanc of un m an ned an

auto o m oussystems [2 ] Both oft hes ech all en es, h o weve r, involveh um an /AG V

awarenes/ intera t io n (co ni tio n) an aut o nomo usco tro llevels(auto nom y) C om

-pl exit yat h esys tem sl eveli sincreasedwh enth es esys temsareag regatedwihoth er

A GVsan dm ann eds ystemsi nm an ufc turi ngandwareh ousesc en ario

E ampleIntel g nce-L v elEf f or s

TheA L FUS [2 ] A d HocWorki ngGr ou p ha s for mu l at ed , t hr ou gh c onsensus, a

fra mework wi t hi n whi c h t he di ffrent l evel s ofa u t on myc a n be d esc r ibe d The

i ni t i al versi on oft he fra mework was pr esent ed a t t he 2 0 A SME Int er na t i ona l

Mec ha nic al Engi ne ri ngC on r es [2 ] Si gni fi c ant progres has be n mad e si nc e

t hen[2 ].Howeve r ,t hec omplexi t yoft hea u t on my-l e veli ssu eforc edt hegr ou pt o

i d ent i fy a dd i t i onal t ec hnic a l c ha l l en e s—ma ny ofwhi c h ar e a c t i ve i ssu es i n t he

r esea rc h c ommuni t i es [10] The grou p agreed t ha t t he a ut on my l e vel s for

u nmanned syst ems mu st be c har ac t eri z ed u si ng t hreed i mensi ons: mi ssi on c

om-pl exi t y,e nvi r onment ald i ffic ul t y,a ndhuman-rob ti nt er ac t i on Thegrou pd e vi sed

a t hr ee-a xi srepresent a t i on for t hosed i mensi ons Fig.1a sh wst hi srepresent at i on

a ppl i edt oi nd u st r i a lA GV s,Fig.1bsh wst hel eve l s ofau t on my,andFig.1csh ws

asu mma r y sc or ec ar do whi c ht oent er t hea u t on myl e vel a l on eac h axi s W e

bel i evet hi ssa memod elc ou l dbeusedt oi d ent i fyc ont extua lau t on myforAG Vs

FIG.1 ALF Smod lap le toA Vs(a),a tonomyle els(b),a dutonomyle el

summaryscore raphth tin orp ratesth a tonomyle elalon e c a is(c)

In spiredb A LFUS,Perfo rmanc Meas u resfo rUnmann e dSystems(Pe rMF US)

[3 ] i saframewo r kt omea s u reper form anc eofu nmann e dsys t emsort heirc o mp

-ne nts i n t he c o ntext ofm i ssi ons /ta s ks a nd enviro nmen t s U n der Per M F U S, t ests

wo u ld be c o nd c ted, d at a a na lyz d, a nd t est met h ods d evel o ped a cc o rd in t o

a ssoci at e d met ri cs An exa m ple proc s mo d el ofPe rMF US ap l ied t o emergency

r esp n se r o bo t s’ performanc eva luat io stan ard t est met h od de velo ment i s

showni nF ig.2

ThePerMFUS exa mpl ea ppl i e d t oA GVswoul di nc l ud ea setofrequ i r ement s

t od e finet heA GVperforma nc efromt hep i nt sofvi ewofu ser sorp t enti a lusers,

ma nufa c t ur ers,a ndr esear c her s.Asoc i a t e dst a nd a rdt estmet hodswou l dbe

gener-a t ed a ndAGV per formanc eeva l ua t ed a c c ord i ngl y Ou t put fr om t heefor t woul d

d ri veAGVperforma nc et ohi gherl evel s a ndgui d eusers’d ec i si ons o AGVpu r

-c hasesa ndi mpl ement a t io

Theg a lofa nAGV i nt el l i genc el evel per formanc st a nda rd i s t o d et ermi ne, for

ea chvehi c lec l a ss,t heed ge-c a sesor l i mi ts(e.g.,r e sol ut i on,a c c ur ac y,sped , sl ope,

et c ) formo i l i t y,o st a c l edet ec ti on, andna vi ga t i on i n t he i nd ust ri alset t i ng The

spec i fic c r i t er i aex ec tedt o bei nc l u ded i na pe r formanc est a nd ar di nc l u d e A GV

c l a ssesandc a pa bi l i t iesa mo got herr e l eva nti nfor mat io a sl i st e di nt hefol l owi ng

subse t i ons Thefol l owi ngc l a sses,c ri t er i a , a nd ot hera rea s’l i st s i nc l ud e a d di t i ons

fromt heAGVi nd ust r yt hrou ghbr i efi nt er vi ewswi t hma nu fa c t urer sa ndu ser sand

FIG.2 Emerg n yResp nseRo otsPerforma c Ev lu tionSta d rdTestMethod

De elo me tProc s Mod l (Ad pte f roma origin lv rsionfromth

Deparme tofH mela dS c rity ta d rdsOffic )

provi d ear east oc onsi deri ngener ict estmet ho d evel opmen F rexa mpl e,vehi c l e

c l a sses have par t ic u l a r l oa d i ng, t ype, gui d ance , and so o a nd p se qu est i ons t o

fut ur est a nd ar d st a skgrou psa st oh wbe stt oc onsi d ert heva r iet yofA GV s Si mi

l a r l y, A V a ppl i c at i ons i n d oc ki ng, pa l l et i z i ng, and so fort h ma y a l so pr ovi d e

si t u a t i ons for per forma nc e t est met hod s t ha t may or may n t fit a l l vehi c l es,

a nd pe r haps mor e t ha n o e t est met ho for each a ppl i c at io may ne d t o be

g Si mu l t a neou sL oc a l i z at i onandMappi ng

h Hy r i d(c ombi na t i onsofgui d a nc emet hods)

i P si t i onresol ut i ona nda c c u ra c y

4 Tea c hMod es

a Offli ne

b Hu ma n-l edi nsi t u

5 C ogni t i on/Aut on myL evel

a Fu l l yAu t on mous—o er at orneveri nt er venes

b S mi au t on mous—o era t ori nt er venes:

i F re a c hnewma neuver

i i Toma nu al l yc l ea rt hepa t ha ndl etmo i l i t yc ont i nu e

c Human-Ma hineIntera eCo trol—jogorpen antco trol

AP LICA TION-SPECIFICPERFORMA NCEC IT RIA

F rea c hc r i t eri onl i st ed,c onsi d ert askc ompl exit y,a d apt a bi l i t yt ot heen i r onmen

OTHERPOS IBL A REAS

Fac t or yc l ot heswor nb wor ke r s

Toa d dr es A GVperfor ma nc st a nd a rd s,ac ommi t t eehasbee n formedb A STM

Int ernat i onal , c a l l ed t he A STM F4 Dri ver l es Au t oma ti c Gui d ed Ind u str i al

Vehi c l es C ommi t t ee W orkin d oc ument s for Doc ki ng, Na vi gat i on (F4 0 ), a nd

Termi nol og (F4 91) havebee ni ni t i a t edbasedo ASTME5 mo i l it yst anda r ds

Thet a bl eofc ontent sforA STMF4 0 i ssh wni nTable1a ndr epresent st hel ayou t

ofi nformat io t ha tt heA GVpe r for manc est a nd a rdsmi ghtfol l ow

The t ask ofdevel opi ng t est met hod s for ASTM F4 fa l l s wi t hi n t he fou r

su bcommi t t eea rea s:ASTMF4 01,En i r onment a lEf c t s;A STMF4 0 ,Doc ki ng

a nd Na vi ga t i on; ASTM F4 0 , Objec t Det ec ti on a nd Pr ot ec t i on; and A STM

F4 0 ,C ommunic a t i ona ndInt egra ti on Thesea rec ompl eme nt ed b afift hc

om-mi t t ee,A STMF4 91,whi c hc over st ermi nol og st a nd ar ds

ASTM Int erna t i ona l pr ovi d es pa r t i c i pa t i ng , or gani za t i onal , a nd i nformat io al

me mbershi pswheremembert ypesa repr oduc ers(i ,equi pment manufac t ur ers),

u se r s(i ,equi pmentusers),c onsu me r s,orgener ali nt er est i e ,resea r c hers,pu bl i c ,

et c ).ASTMu sesac onsensu spr oc es t oensur ea l lc ommit t eeme mbershaveasay

i nt hedevel opmentpr oc s,a ndbala nc ei nrepresent a t io a mo gt het hreepa rt i c i

-pa t i nggr ou psi sr equ i red.AnA STMst and a rdc a nbeo eofsi xt ypes:t estmet hod s,

spe c i fic a t i ons,c l a ssi fi c a t i on,pra c t ic e , gui d e, or t ermi nol og Wec hose t od evel op

t estmet ho a ndt er mi nol og st a nd ar dsasast ar t i ngp i ntt osu pp rtt heA GVa nd

TABL 1 AS M 02workin doc me t ableofconte ts

mo i l e rob t i ndu st r i es Test me t hod s a re defi i t i vepr oc ed u rest ha t prod uc e t est

r esu l ts a nd , a s sh wn i nTable1 pr ec i si on a nd bi asand mea su rementa nd u nc er

-t ai n-t y Ther efore, sc i enti fi c r efe renc e mu st be provi ded wi t hi n ea c h t est me t ho

st a nd a rd A d d i t io a l l y, repl i c a bl e and pr opaga t a bl e ar t i fac t s ar e ex e c t ed t o be

d evel oped a nd used a s rel at i vel y ac c u ra t e and i nex pensi ve t est met ho supp r t

d evi c e s,si mi l art o,forexa mpl e,t het estpi ec esu sedforn nc ont ac tsensi ngeva l u

a-t i onwi t hi nA NSI TSDFB5 5 St a nd ard s wi t hsi mpl e,rel a ti vel yi nex ensi vea r t i

-fa c t swou l d,t herefore,n trequi r evehi c levend or sa ndu ser st opr ocu reex ensi ve

measu rementsyst emst oensu ret hei rvehi c lesc onform t ost anda r d s ort o c ond u c t

i n-h u se t est in A s suggest ed i n pr evi ou s sec t io s oft hi s c ha pt e r , met r i c s, t est

met hods, a nd t ermi nol og for t he A V a nd mo i l e r ob t i nd u st ri es may be

a d opt edfr omt hosea ssoc i a tedwi t ha ut on mousa ndi nt el l i gentc a pabi l i t ieseva l u

a-t i onsi not herdomai ns.Asu mm a ryofp t ent i alA GVr el e va nc eforeac hn n-A V

st a nd a rddesc r ibedpr e vi ou sl yi sl i st edi nTable2,andasumma r yofp t ent i alA V

rel evanc eforeac hc ha l l en eandpr ogr a mdesc ribe dprevi ou sl yi sl i st edi nTable

TABL 2 P te tialAGVr elev an ef ore c non-A GVsta d r d

Sta d rd ermin lo yforUrb n e rch n

ResceRo oticOp ratio s AS M –07)

Trmin lo y

Sta d rd estMeth dforEvalu tin Emerg ny

Resp nseRo otCa a ities:Mo i ty

E viro me talef fects,o stale etetio a d

avoid n e,a dtermin lo y

Sta d rd estMeth dforEvalu tin Emerg ny

Resp nseRo otCa a ities:E erg /Pow er

E erg /ow ermesure e tofv ehicle

f untio alty nin ln sa din th rsitu tio s

Sta d rd estMeth dforEvalu tin Emerg ny

Resp nseRo otCa a ities:Ra io

Com u ic tio

Ve icle om u ictio swithth masteror

ware o semaa eme tsy stems

Sta d rd estMeth dforEvalu tin Emerg ny

Resp nseRo otCa a ities:Huma -S stem

Interatio (HSI)

Evalu tio ofh ma fact orsin o troln ad

interatin withAG s

Sta d rd estMeth dforEvalu tin Emerg ny

Resp nseRo otCa a ities:S nsors

AGVse sorcp bity,cntolalg rithmma pin

q alty,b h viorg n ratio f oro stale etetio

IE E-RASRo otMa DataRe rese tatio for

Naig tio Work in Gro p(MD W G)

Re rese tatio ofvehicle o tolma s

SAEAS4—Auto omy ev elsforUnma n d

S stems AL US)

De nitio ofa to omylevelsf orAGVs

Intern tio alOrg nizatio forStad rdizatio

(ISO)Performa c Sta d rdsforS rvic Ro ots

(draftISO/DIS18 46-1)

Sta d rd eelo me tpro es

Intern tio alEletot ecnic lCom isio (IEC)

Sta d rdsf orS rvic Ro ots IEC60312)

Sta d rd eelo me tpro es

C urently,th ere aren perform anc m easurem ent stan ardsfo r AG Vs, o lys afety

stan dards.However,thatsiuato isl i e lyt och an ge Indevel opin suc hstan dards ,it

isim po rtan ttoreal iz th atm uchofth emobil ero b otresearc hfromdiferento rganiz

-to s an ap l ictionare sisap l ic bletoAGVs F rth erm o re ,performancestan

-ardsd exist for m o bile robots As such , performanc e standards for AGVscan b e

b ased o th eseperfo rm anc estandardsform o bilerobo tc pab il ites In thi sc hapter,

wedescrib e d a n um b ero fareas wh ere o verl aps arep s s ible The n ext steps areto

pro ce dwihth ede velop m ento fperformanc em easurementstandardsforAG V sand

m o bilerobotswi thi np tfro mm o bilerobot,sen so rs,ando t her u po rtngindustries

ACK NOW LEDG ENT

Theau t horswou l dl i ket ot ha nksevera lkeyi nd i vi d ua l sfort hei rhel pwi t ht hi sc

TABL 3 P te tialAGVr ele a c f ore c c ale g a dpr ogr a

ChlegeorProram Pte tialAGVRele vanc e

Ro oCu Resc e,Ro oCu @Work,

Ro oCu @Home

Huma-o otinteratio ,n vig tio a dma pin

in y amic n iro me ts,o jetrec g itio ,

o jetma ip latio ,a d d ptiv b h viors

IE E olutio inPer ce tio Ch le g AGVp rcptio ,p ric larlyforo jetrec g itio

a d ose etermin tio

Aso iatio forUnma n dVe icle ystems

Intern tio al(AUV SI)

Advane c ntolth ory,ma hin visio ,vehic lar

ele tronics,a dmo ie latformfun ame tals

Viru lMa uf acturin Automatio Comp titio s

(VM C)

Pathfolowin a d o kin

Mo ity n TskCompletio Ch le g MultipleAG sin nstu ture e viro me tswith

d n mic bstales

DefenseAdvane Rese rc hProjetAg ny

(DAR A)Unman dGro n Ve icle(U V)

Progra

Auto omo sn vig tio te h olo ies

U.S.Army a oratoryCom a d(LAB O ) Auto omo sn vig tio te h olo ies

Micro-Auto omo sS ie c a d eh olo y

(M S )

Situ tio alawaren s

DAR ARo oticsProgra sa dCh le g s Situ tio awaren s/void n e n c ntols,

vehicle vlu tio c mp re to aseln

Auto omo sS stemsTsta d vlutio

Re uireme tsStu y(AS ERS)

Tcnicloro eratio al mitatio s

Performa c Me suresforUnma n d ystems

(PerMFUS)

Fr a eworktome surep rf ormac ofAG s

Sa t oshi Ta dok r ooft heInt er na t i ona l Re sc ue Syst emInst i t ut e/To o uU ni

-ver si t y,Japa n

•

Jo Bornst ei n, Kel l ySwi nso , a nd Ma rsha l C hi l d ers oft he ArmyResea rc h

L a bor at ory,A ber deenProvi ngGr ou nds,MD

Ref er ences

[1] E emin,“Hist oryofAG s,”E eminA ut omatio , nc Holan ,MI 201,w ww.e emin s

com/p ag sag _e ucatio /e uc atio _ag _history.html(acc ese Ap ri 4,2016)

[2] ANSI/IT DF B5 5:2012, SaetyStandad forDriv rless, Au tomaticGu idedIndu strial

Ve icles ndAutomated u nctio sofMan edIndu s trialVe icles, n ustialTruck tan

-ar dsDevelo me tF u datio ,Wa hin to ,D ,2012,ww w.tsdf.or g

[3] Alb s,J.S “Outln fora T e ryofIntel g nc e,”IE ETrnsactio so Systems , Man,

andCy ern tics,Vol 21,No.3,19 1,p 4 3–509

[4] Bostelman,R S a klef ord,W Ch o ,G an Saidi K “Saf eCo tr olofMan fa turin

Ve iclesResearchT wardsStan ar d Tst Meth ds,” Pro e ding softh Internatio al

MaterialHandlngRes eac Coloq u iu m,Gar dan e,Franc e,Ju e 5–2 ,2012

[5] AS MInternatio al WestCo sh h ck n,PA,2012,ww w.a tm.or g

[6] Huan , H.-M, Mesina, E E glsh, R Wad , R Alb s,J an Novak,B “Auto omy

Mea uresf orRo ots,”Pro e dingsofth A ME200 4Internatio alMehanical

Engi-n eringCo g ressandEx os itio ,Americ nS ciety fMechanic lE gin ers,Anah im,

CA,Novemb r13–19,20 4

[7] ISO /DIS 18 4 -1, Rob tsandRob ticDevics—Perorman eCriteriaandRelatedTest

Meth ds for Servic Rob t, nter natio al Organiz tio f or Stan ar diz tio , Ge e a,

Sw itzerlan ,2015,ww w.so.or g

[8] Huan ,H.an Mesina,E Auto omyLevels orUnman edSystems ALFUS)Frmework

Volu meI Frme workModelsInitialVers io ,NIS S ecialP blc tio 101-I-1.0,Natio al

InstituteofStan ar ds n T ch olo y(NIS ),Gaith rsb r g,MD,20 7

[9] IEC SC 5 F, S ra e Cleanin Ap lanc es, nter natio al Electrotech ic l Com isio ,

Ge e v a, Switzerlan , 2015, htp:/w w.ec.ch/d n/ww w/f?p¼103:7:0: F P_OR _ID,

F P_LAN _ID:13 5,2 (ac cese Apri 4,2016)

[10] T eRo oCu F d ratio ,2012,ww w.rob cu or g(ac ces e Apri 4,2016)

[1] “Welc ome to Ro oCu @Work,” 2015, ww w.rob cu atw ork.org/in e html (a ces e

Apri 4,2016)

[12] “T eRo oCu @Home eag e,”2015,w w w.rob oc up ath me.org(a cese A pril4,2016)

[13] Marvel J.A Ho g,T an Mesina,E “S lutio sinP rce tio Chale g P rf ormanc e

Metic an Results,” Pro e dings of th Worksho o Perorman e Metrics for

Intelg entSystems(PerMIS

0

12),As ociatio f orComp tin Ma hin ry,NewYork,N ,

March20–2 ,2012,p 5 –6

[14] “Ab utA UVSI”Asociatio forUnman e Ve icleSy stemsInternatio al Arln to ,VA,

2012,ww w.au siorg(a cese Apri 4,2016)

[15] S e , R J cof, A Virs, A M Kimura, T P le z, J Schwer e er, S an

S thak rn,J.(J n ary).“Ad ancin th StateofUrbanS archan Res u Ro otic

T ro g th Ro oCu Res u Ro ot L ag e Comp titio ,” Field and Servic

Ro otics, K Yoshida an S Tad k ro, E s S rin er, Berln, Heid lb rg, 2014,

p 12 –14

[16] Balakirsk , S Chita, S Dimito lo , G Gorman, J Kim, K an Yim, M “Ro ot

Chale g ,”Ro oticsandAutomat io ,Decemb r2012,p 9–1

[17] Balakirsky, S an Mad avan, R “Ad ancin Man fa turin Research T ro g

Comp titio s,” Pro edings of S IE Defe s e Seu rity and Se sing , Orlan o, F ,

Apri 13–17,20 9

[18] Balakirsky,S Scrap er,C Carpin,S an Lwis,M “USARSim:Pro vidin aFra e w ork

f orMulti-Ro otP rformanc eE aluatio ,”Pro e dingsofth PerormaneMetricsfor

IntelgentSystemsWorksho ,NIS ,Gaith rsb rg,MD,Au ust21–2 ,20 6

[19] S o mak r,C “Develo me tofAuto omo sRo oticGr ou dVe icles:DoD’sGrou d

Ro otic Research Programs: Demo I thro g Demo I ” Intelg entVe icleSystems:

A4 D /RCSAp roac ,R.Mad avan,E.R.Mesina,an J.S.Alb s,E s NovaP blsh rs,

NewYork,20 6,p 2 3–315

[20] Ha s,G A David, P an Hau ,B.T “Tar getAcq isitio an E gag me tf r om an

Unman e Grou d Ve icle: T e Ro otic Tst Be of Demo 1,” Tch ic l Re or

ARL-TR-106 ,ArmyResearch ab ratory,Ad lp i MD,March19 6

[21] Army Research Lab ratory, “Ro otic Colab rativ Tch olo yAl anc e (R TA), FY

201 An ual Program Plan,” March 201, htp:/w w.arlarmy.mi/w ww/pag s3 2/

rcta.fy1.an pro plan.p f ac cese Apri 4,2016)

[2 ] “MicroAuto omo sS stems n Tch olo y(M S ),”G AS Lab ratory,University f

P n sylvania,P iad lp ia,PA,2015,ww w.ma tcta.org

[2 ] Wikip dia, “DARPA Gran Chale g ,” Def ense Advanc ed Research Project A ge cy

Chale g s, 2012, htp:/e wikip dia.org/wiki/DARPA_ ran _Chale g (a ces e

Apri 4,2016)

[2 ] S off or d, J R Rime, R D an Mu kb , S H “Overview of th U V/Demo I

Program,”L ck e dMarinAsto autic,De ver,C O,19 6

[2 ] Wikip dia,“DefenseAd ance ResearchProjectAg ncyL arnin Ap le to Grou d

Ro ots(LAG ) Progra ,” 20 9, htp:/e wikip dia.org/wiki/DARPA_LAGR Program

(a cese Apri 4,2016)

[2 ] Ratclff, A “OSD Man fa turin T ch olo y Overview,” N IA Quarerly me tin ,

De arme tofDef enseMan fa turin T ch olo y rogramprese tatio ,May14,20 9

[2 ] Swinso , K “T st an E aluatio of Auto omo s Grou d Ro ots,” N IA Gr ou d

Ro otic Capabi tiesCo f er encean E hibitio ,Ab rd e ,MD,March2 ,2012,htp:/

ww w.dtic.mi/n ia 2012 rc ce/Swinso p f ac ces e Apri 4,2016)

[2 ] Ma ia ,F “T eTstan E aluatio ofUnman e an Auto omo sSy stems,”

Interna-tio alTes tandEvalu atio Associatio Ju rnal,Vol 2 ,No.4,20 8,p 3 8–3 5

[2 ] Huan , H Pav k, K Alb s, J an Mes ina, E “Auto omy L vels f or Unman e

Sy stems (AL US) Fr amew ork: An Up ate, 20 5,” Proe dings ofth S IEDefe s e

andSe u ritySymp siu m,Orlan o,F ,March 8–Apri 1,20 5

[30] Huan ,H “P rformanceMea uresf orUnman e Sy stems,”prese te at h SAEAS D

Me tin ,SanDie o,CA,Octo er18,2010

1

a dHolyYa c o

2

in Industrial Envir onments

Citation

Noro ,A.an Yanc o,H “Pr elminar yDevelo me tofaT stMeth df orObsta leDetectio

an Avoidanc e in In ustial E vironme ts,” Auto omou s Indu strial Ve icles: From th

L b rtory to th Factory Flo r, A TMSP1594, R Bo st elman an E Mesina, E s A S TM

Int ernatio al WestC o nsho hocke ,PA ,2016,p 2 –40,d i10.15 0/S TP 15 4 0150 5

3

T ereis c re tlyn stan ardmeth d forcomparin a to omous c pabi ties

amo g systems We pro ose a test meth d f or e alu tin an a tomate

mo ie system’s abi ty to d tect an av oid o sta les, sp cific ly th seinan

in ustrial e viro me t T this e d, a ta o omy is b in g n r ate to

d termin th r elevantp ysic lchar acteristicsofo sta les othat h yc nb

a c r ately re r ese te in th test meth d Our pr elminary d v lo me t

includ s th d sig ofan ap ar atus, pr ops, proce ur es, an metrics Wehav e

f abric te a series of o sta le test pro s that re ect a variety of p ysic l

chara teristics an hav e p rforme a series of tests withasmal mo iero ot

toward valdatio of th test meth d Futur e work includ s expan in th

ta o omy, d sig in mor eo sta letest pr ops, colectin test datawithmor e

a tomatic ly guid d v hicles an r ob ts, an formalzin our work as a

p te tial stan ard test meth d thr oug th AS M F 5 Commite o

Driv rles Automatic Guid d In ustrial Ve icles, sp cific ly AS M F 5.03

ObjectDetectio an Protectio

Man scriptr ec ivdJuly1,2015;ace te forp blcatio Au ust2,2015

1

NewE glan Ro oticsValdatio an E p rimetatio (NER VE)Cet er,University fMa sc usetsLo wel

10 1Pawtuc ketBlvd Lo wel M 018 4

2

De arme tofComp terSiec e,University fMa sc usetsLo wel 1UniversityAv e.Lo wel M 0184

3

AS MWorksh p nAu ton mo sIndus ialVe icles:Fromth La oratorytoth Fac toryFlo ro

May 6–30,2015in eatle,Washin to

Copy rig tV

C

2016byASMInterntio l10 Bar HarborDriv e,POBo C70 ,Wes Cosh h k en,PA1942 -2 9

S P15 4,2016/av aiable nln atw w.a tm.org/d oi 10.15 0 S P15 420150 5

K eywords

o sta led tection,o sta leavoida ce,sta dardtestmetho ,mo ier ob t,

a tomatic lyguid dv ehicle(AGV)

Intr oduction

Automatc lyguided vehicles (AGVs) havebe omever ycommo in in u strial

manu fa ctu rin en ir onments The use of auto omo s mo ile r ob ts in this

d ma in is a lso o the r ise A ne es a ry c pabiliy ofb th systems is o sta cle

d ete t io an av ida nc Obsta cles in this d omain ra nge betwe n sta tico je ts

(e.g.,tables,palets,bar r els) an movin ag ents(e.g.,forklift s,peo le) Theloc

-to ofstatco je ts in some envir onments is fix ed, while in other i cha nges

veryfreq uentlywhen ajo r eq uir esad if r entwor kfl wan la yout If system

isc pableofd ete tn an av idin o sta les,i cre tesasaerworkenviro ment

a nda llowsforfa sterinteg ra tio intoafciliybe a useles apr ior ik owled geof

the n ir onment sne ded[1]

Cu rr ent l y,t herei sn st anda rdforc ompa r i ngt hi sc a pa bi l i t ybet we nsyst ems

TheCommi t t ee o Dri verl es Aut oma t i c Gu i ded Indu st ri al Vehi c l es (ASTM F4

[2]) has bee n formed t o a c hi eve t hi s g al , spec i fic al l y ASTM F4 0 , whi c h i s

foc u sedo o jec tdet ec t i ona ndprot e c t i on W eprop seat estmet ho d esi gnt ha t

c a na id i nt hi sefor t b a c c u r at e lysi mu l ati ngt her el eva ntp ysic al c har ac t er i st i c s

ofa n i nd u st ri al manufac t u r i ng en i ronmen In pa rt ic ul a r , t he t est met ho wi l l

r epl i c a t et hep ysi c a l qu al i t ies ofc ommo o st a c l e s a ndo jec tst ha t c a naffc ta

syst em’sa bi l i t yt od et ec tt hemwi t hi t ssensor sa nda voi dc ol l i sio s

Thereareavar ietyof fort sworkin t o wardst an a rdiz edperfo rmanc met ri csan

t estmet h odsfo rr o bo t icsys t ems.TheNat ion a lInsti tu teof tan a rdsa ndTec hno l o gy

(N IS T)hasbeenl ead in anefo rtfo rt hed eve l o pmen tofsta nd ardt estm etho d sfo r

r esp n se ro bo t s [3] for wel l o er a dec ad e t hr o ugh t he C o m mit te o Homela nd

Sec ur it yA ppli ca ti o ns ;Operati o nalEquipmen t ;Ro bo t s(AST ME4 5.0 01 [4]).Th os e

t est met hods fo c uso dif rentc apabil it ies ofmo bil it y, manip l at ion,sensor,a nd

h man-syst emi nt era ct ion,mo stpromi nen t lyfo rt el eoperat e dr o bo t s

F r AGVs, ther e is a sa fety stand ard test meth d spe ified in Amer ic n

Na tio al Stand ar d s Inst u te/ nd ustr ial Tr uck S fty Develo ment F u nda tio

(ANSI ITSDF) B5 5 [5] that verifieswhether or n ta system’ssa fetysensor(s)

ar eabletodete tap tenta lo sta leinispa th.Wihinthat estmeth d ,twotest

pie esthatmustbedete tedan a voidedar epla eda tvar yin orientato s a nd

distanc sfromthesystem.Thesurfa eoft hetestpie esar eeiherbla ck,be ause

that c n c use isues for o tc l sensor , or hig hly r efl ectve, be au se that c n

c u se is ues for u ltr aso ic sensor That test is prima rily focused o d namic

agents(e.g.,aper so ) tha ttemp ra r ilyenteravehicle’spath.Asimila r sta nda rd

for auto omou s r ob ts is ISO 13 8 [6] which includ es ap r opr iate distanc s

betwe nthesysteman a nagento je t hatenter it sspa ean emerg encysto

fu ncto s I is aimed a t per so al c r e ro ots (n t t hose in in ustr ial en ir

on-ments)a ndisla rgelyfocusedo thesystem’s aetywhenco-loc tedwithpeo le

Alth u g h i spe ifies sta nda r d per for manc , i d oes n t spe ify a stan ar d test

meth dfordeterminin performanc

Thec ompl exi t yl eve l sof n i ronmenta ndo st ac l es(C L EO)a ndpr edi c t i oni n

d ynamic n i ronment sfra meworkefor t sou t l i nedi nMa dha va neta l [7] ar ea imed

a tmeasur i ngt heperforma nc ofa ut on moussyst ems Theyn t et ha tt hec a pa bi l

-i t yof syst emt owor ki nu nstru c t u red ,d yna mice nvi r onment si sa“c r i t i ca lenabl er

for nextgener at i on i ndu st ri a l mo i l e r ob t s.” The C L EO fr amewor k pr ovi des a

me t ho for c ha ra c t eri z i ng a n a u t on mous syst em’s a bi l i t y t o navi ga t e t hrou gh

i nc r easi ngl y c ompl ex en i ronme nt s and o st a c l es a s b t h a spec ts bec ome more

d ynamic The met r i c s u se d i nc l ud e geomet r i c c or r ec t nes, d yna mi c map upd at e

me t ho s,a nda mountoft i met ou pd a t eforen i r onmentc hanges

The develo ment ofstan ard test meth ds for AGVs is also prevalent a t

NIST [8] focu sin o colab r atve workspa es amo g hu mans, u man ed

vehicles,a ndman edvehicles.Thatworkfocuseso t hedete to ofo je tsa nd

ag entsthateiher enterthepath orsto z oneofavehicleor tha ta r ebey n i

Thetest pie es fromANSI TSDFB5 5 areused for o sta les a swel asfora n

alternatve to grou nd tr uth mea sur ement c led t he grid -vid eo meth d This

meth din olvespla cin agrido theg r ou dan comp tn gr ou dtr uthloc

-to sfr omr ecor dedvid eoof test

Obst a c l e det ec ti on a nd a voi d anc ei sa c ommo c apa bi l i t yofanymo i l e,a u t on

-mous syst em Gi ven t heexi st i ngau di enc a ndefort forASTMF4 ,t hedeve l

op-me nt i si ni t i a l l yfoc used o a u t oma t ed mo i l e syst ems used i n i ndoor i nd u st ri a l

en i ronme nt s,pa rt i c u lar l yfor ma nufa c t ur i ngappl i c a t i ons Mo i l esyst emsi n t hi s

d oma i n i nc l u d e AGVs,whi ch c a nt a ket heform oft r a dit i ona l l yhu ma n-o era t ed

vehi c lest ha ta rei nst e a da ut oma t ed(e.g.,S egri dVi si onGui d e dV ehi c l es[9]) and

a u t on mousrob t s(e.g.,A d eptMo i l eRo ot s[10])

Al l oft hese syst ems a re r equ i red t o d et ec tand a voi do jec t s and a gent s t ha t

ei t herent ert hei rpat hort ha tfor mt heedgesoft hei rpa t h.Thi si nc l ud esa nyent i t y

i na ni ndu st ri a len i r onmentt ha t ,i famo i l esyst emweret oc ol l i dewi t hi t ,c ou l d

c a used a ma get ot hesyst em,i t spa yl oad ,ort o t heent i t yi t sel f F rdevel opment al

pu rposes,t hesea r e wha twer efrt o a so st a c l e s Obst a c l es c ansi to t hegr ound

orprot ru defr omawa l lorc ei l i ngi nt he n i r onmen

Depend i ngo avar i e t y of a c t ors,i nc l u di ngt hesyst e m’s c a pabi l i t i esa nd t he

l a you t oft he en i r onmen t he ma nner i n whi ch a n o st ac l e i s avoi ded var i es

Av i d a nc ec anmea nst op i ngi npl ac eu nt i lt heo st a c l ei sn l on eri nt hewayor

d ri vi nga r ound t heo st ac l et o c onti nu e o apa t h Themet r i cofper for ma nc i n

t hi st estmet ho wi l lha vet ot a kei nt oac countwha tt hesyst emi sabl et odo,gi ve n

t heen i r onmentc onst r ai nt sandi t sownmo i l i t yc a pa bi l i t ies Al so,d ependi ngo

t hesensorsusedb t hesyst emt osenset heo st a c l e,a voi d a nc ed oesn tnec esa ri l y

meann nc ont a c t Thea mountofal l owa bl ec ont ac ti nt het estmet ho sh u l dn t

c a usea nyd amaget ot hesyst emort ot heo st a c l et ha ti sbe i nghi t

Ever y in ustrial en iro ment ser ves a dif r ent p r pose an u ses dif r ent

brand s of urniur e,ma hiner y,an soo ,r esultn inavastn mberofp s ible

char acteristcs However , al ofthese o sta cles ca n be grou ped into types The

p ysic lchara cter istcsof no sta clear ewhatdetermineif nautomatedsystem

is able to det ect i pro erly To distl t he p ysic l char act er istcs ofe ch ty e

intoth setha tar e r elevant oo sta clea void anc ,thecr eato of tax on myto

g uided evelo ment spro osed AsimpleInternet magese rchofman fcturin

enviro mentswas per for med todeterminethe commo ty esofo sta les (e.g ,

tables, shelvin u nis, ca rts, palets, ba rr els, ad der s, etc.) to beinclu ded in the

ta x on my

W h e nt esti nganautonomous ora ut o mat e d c apabil it y,a dyn a mi ct esti s mo re

a ppr o pri a t ha nast at ico et oensuret hatt he systembei n gt e st edi ssensi nga nd

r eact in t oob st ac lesi nsi tu,eli mi nati nga nypo ssibil it yof no e rat o rgamin t het est

t oexh i b i tbet te rper formanc Toa chievet hi s ,ext er na lc ha ra ct eri st ic s(e.g.,l o c at ion ,

ori entat ion ) a nd i n t ernal c h ar ac teri s t ic s(e g., o eral la ndi n di vi du al fea tu re dim en

-sio s) oft heob sta cl es i nt heenviron m en tsho uldvaryduri nga t estsesi on Given

t he man po ssiblerea l-worl dob je c ts,var iabil it yofi nt ern a lc h a ract e ri st ic swil lall o w

fo rawidea rrayoft h e s eo jec tst obepro perlysi mu la ted Ado wnsel ec ti o npro c e ss

c anal sobeper formedt od et erminet h ea ppro r iat eob sta cl et ypes t ot estd ependi ng

o t hesystem’sdim en si o ns ,sensort ypes ,sensorplacemen t ,an soon

F rsomemo i l e syst ems, t he i ri mpl ement a t i on i nani nd u st r i a len i ronment

r equ i rest hea rea swheret hesyst emc and ri veandl a nd ma r kl oc at i onst obed efi ne d

W erefrt ot he dgest ha td efi nesu c hspa c esasb u nd a ri es The yc anbed efi e di n

ava riet yofwa ys Somesyst emsr equi rep ysi c alau gme nta t i onof spa c e(e.g.,t he

l a yi ngofmagnet i ct ape)i n ord erfort he mt obei mplement ed Ot hersma ket hei r

ownma poft hespa c e,a ndt heb u nd ar i esar evi r t u a l l yau gmentedi nsoft wa r e.The

i d e nt ific a t i on ofl oc a t i ons al so var i es a mo gsyst ems; some rea d ma rker s i n t he

en i ronmentt ok owwhenal oc a t i oni srea c hedandot hersha vel oc at i onsdefi ed

i nsoft wa r e Al loft hesefa t u r es(i fappl i c a bl et oasyst em)mu stbep si bl ewi t hi n

t het estmet hod,a l l owi ngforh l i st i ct e st i ngof syst em

An i mp r t a nt a spec t of t he resp nse r ob t t est met hods spec i fied t hrou gh

ASTME5 0 01 i st hea va i la bi l i t yoft hema t er i a l su se dt ofabr i c a t et het esta ppar a

-t uses and pr ops A l l oft hose t est me t ho s ar e ma de wi t h l u mber, wo d pa nel s,

p ly i nyl c hl ori de(PV C) pipes, a nd ot her c ommo bu i l di ngmat er i al s Thi shas

a l l owed t hem t o be ea sil y i mpl ement e d , d is emi na t ed , a nd u sed b t he first

r esp nd e r and rob t i c s c ommu ni t y Thi s t est met ho sh ul d fol l ow t he same

c on ent i on

Alloft hesefc to rsformaseto fre quirementst hatdefin eth esco eofth et estm et ho

soth atitmeet sth eneedso fth e m an ufcturin grobotcsdo m ain , al l wsboth tradi

to alAGVsan dm o bilerobotstob et ested,an c anb efbric atedb anyone:

•

R1:Ther el eva ntc har a c t eri st i c sofc ommo o st a c l esi na ni nd ust r i a le nvi r

on-mentmu stbep ysi c a l l yr e pr e sent ed

•

R2: Obst a c l e t estpropsmustha vevar i abl ese t t i ngst o a l l owfor ava ri et yof

r e a l -worl do jec t st oberepresent ed

•

R3:Ob s taletes tpro psettings ,b o thin ternalandextern alq alities ,mus tb evaried

d ringatestses sio ntopreven tg min gandtotest heflexib ilityo fth eauto n o my

•

R4 :Thesemi per ma nentb u nd a ri esoft heen i r onmentandl oc a t i onswi t hi n

i tmustber epr esent edi nt het estappar at ussuc ht ha tt heya rea ppr opr i at el y

d et ec t a bl eb t hesyst e mbe i ngt est e d

•

R5: Thet est a ppa ra t u s a nd pr ops mu st befabr i c a t ed u si ng r ead i l ya va i l a bl e

bu i l d i ngma t eri a l st ha ta rei nex ensi ve

TA XONOMYOFRE EVA NTC A RACT RIS ICS

In or der t o ac c u ra t ely si mu l a t e a n a ppropr i a t e l evel ofd et a i l i n ma nu fa c t u ri ng

en i ronme nt s,at a xo omyofr el eva ntc har ac t eri st i c si sbei ngd evel oped.Thet

ax-o omywi l lgui d e t hed esig oft heo st a c l e t est propsa nd wi ll pr ovi d ea u ni fied

l a ngua get odesc ribet hei rpu rpose Thec har ac t eri st i c st ha tar e t obei nc l u d e dar e

d i st i l l edt hrou ght hefol l owi ngproc es:

T4: Id e nt i fy o er l aps i n p ysi c a l c har ac t eri st i c s a mo g rea l -worl do st ac l es

(t hi si sper for medt ol i mi tt henu mberofuni qu eo st ac l et estpr opst ha tw

ne dt obed evel oped)

•

T5:Desi gn o st a c l et estpropst ha tc apt u ret hep ysi c alc har a c t eri st i c s whi l e

r e d u c i ngo e r l a pa mo got hero st a c l et estprops

Theus eo fthetax no mydevel opm entpro ces sen s uresth at herequirem entsth at

pertaintoth eob s tal etestpro psaremet.S pecificall y,R1iss atsfiedb yT1an dT2 ,R2is

satsfiedb yT3,an dR3 ndR5willh elpg ideT 5.Thispro cessh asb eenusedtode velop

aninitalseto fexam pletestpro pdesigns,wh icharedetailedin“T es tM eth o dDes ign.”

A sna psh t ofT1–T3 c a n besen i n Table1 usi nga t abl e a nd shel vi ngu ni t

a s exa mpl es Ot her o st a c l e s i nc l u de d i n t he t a xo omy ar e c on ey r bel t s,

c hai r s,pa l l ets,l a dders,ra i l i ngs,b l l a r ds,c ol umns,a ndba rr el s.Sur fa c equa l i t yi sn t

l i st ed as a va ri a bl e c ha ra c t eri st i c bec a u se t he fl at bl ac k and r efle c t i ve su rfac e

qu al i t iesu sedo t het estpiec esd esc r i bedi nANSI TSDFB5 5c a nbeusedfora l l

o st a c l et estpr opst orepresentedgec asesfor ensors.W hi l epe r for mi ngT4a c ros

t heo st a c l es,asetofhi ghe rl eve lqua l i t i eseme r gedasrel eva ntc ha ra c t er i st i c s