Iec tr 61850 90 12 2015

6 Ta le 16 – Commu ication req irements f or inter control centre commu ication.. WANs en omp s commu ication me n of dif ferent nature o tical, radio, p wer l ne car ier, co p r, etc.,

IEC TR 61 850- 90- 1 2

Editio 1.0 2 15-0

Communicat ion network s and syst ems for pow er ut il ty aut omat ion –

Part 90-12: Wide area network engineering guidel nes

THIS PUBLICATION IS COPYRIGHT PROTECTED

Copyr ight © 2 15 IEC, Ge e a, Switzer la d

Al rig ts r es r ve Unle s oth rwis s e ifi d, n p rt of this p blc tio ma b r epr od c d or uti z d in a y form

or b a y me n , ele tr onic or me h nic l in lu in p oto o yin a d microfim, with ut p rmis io in wr itin from

eith r IEC or IEC's memb r Natio al Commite in th c u tr y of th re u ste If y u h v a y q e tio s a o t IEC

c p r i ht or h v a e q ir y a o t o tainin a ditio al ri hts to this p blc tio , ple s c nta t th a dr es b low or

y ur lo al IEC memb r Natio al Commite for ur th r informatio

Th Intern tio al Ele trote h ic l Commis io (IEC) is th le din glo al org niz tio th t pr ep res a d p bls e

Internatio al Sta d r ds for al ele tri al ele tr onic a d relate te h olo ie

Abo t IEC p blc tio s

Th te h ic l c nte t of IEC p blc tio s is k pt u d r c n ta t r eview b th IEC Ple s ma e s re th t y u h v th

late t e itio ,a c rr ig n a or a ame dme t mig t h v b e p bls e

IEC Cat alo ue - webst ore.e ch/ cat alo ue

Th st an -alo e a plc t io for c n ultin th e tire

biblo ra hic l informat io o IEC Intern tio al Sta d rd ,

Te h ic l Sp cific tio s, Te h ic l R ep rts a d oth r

d c me ts A v aia le for P , Ma OS, A ndroid Ta let s a d

iPa

IEC p blc t i ns s arch - www ie ch/ se rch u

Th a v an e s arc e a le t o fin IEC p blc t io s b a

variety of crit eria (efere c n mb r, t ext, t ec nic l

c mmitt e,…) It als gives informatio o projec t s, re la e

a d wit hdrawn p blc tio s

IEC Just P blshed - webst ore.e ch/ just pu lshed

St ay u to d te o al n w IEC p blc tio s Ju t Pu ls e

d t ais al n w p blc t io s rele s d A v aia le o ln a d

als o c a mo t h b emai

Ele t ro edia - www ele t ro edia.org

Th world's le din o ln dic t io ary of elec t ro ic a d

ele t ric l t erms c ntainin more t ha 3 0 0 terms a d

d finitio s in En ls a d Fre c , wit h e uiv ale t t erms in 15

a ditio al la g a e Als k nown a t he Int ern t io al

Elec t rot ec nic l Vo a ulary (IEV ) o ln

IEC Glos ar y - st d.e ch/ glos ary

More t ha 6 0 0 ele t rot ec nic l termin lo y e t rie in

En ls a d Fre c e tra t ed fom t he Terms a d Definitio s

c la s of IEC p blc t io s is u d sin e 2 0 Some e t rie

h v e b e c le t ed fom e rler p blc tio s of IEC TC 3 ,

7 ,8 a d CISPR

IEC Cust omer Servic Cent re - webst ore.ec ch/ csc

If y ou wis to giv e u your fe d a k o t his p blc tio or

n e furt her a sista c , ple s c nta t t he Cu tomer Serv ic

Ce t re:c c@ie c

IEC TR 61 850- 90- 1 2

Edit io 1.0 2 15-0

Communicat ion networks and syst ems for pow er ut il ty aut omat ion –

Part 90-12: Wide area network engineering guidel nes

INT ERNAT IONAL

ELECT ROT ECHNICAL

FOREWORD 1

INTRODUCTION 13 1 Sco e 15 2 Normative ref eren es 15 3 Terms, def i ition , a breviation , acron ms an s mbols 2

3.1 Terms an def i ition 2

3 2 Ab reviation an acron ms 2

3.3 Network diagram s mb ls 3

4 Wide Are Commu ication in electrical uti ties 3

4.1 Exec tive s mmary 3

4 2 Use Case: ENDESA, An alu ia (Sp in) 3

4.3 Typical interf ace b twe n a s bstation an the WAN 3

4.4 WAN c aracteristic an actors 3

4.5 SGAM Ma pin 4

4.6 Network elements an voltage level 4

4.7 WAN interf aces in s bstation automation (IEC 618 0-5) 4

4.8 L gical interaces an protocols in the TC5 Arc itecture IEC TR 6 3 7 4

4.9 Network traff i an owners ip 4

5 WAN overal req irements an data tran mis ion metric 4

5.1 Traf fic typ s 4

5.2 Qual ty of Service (QoS) of TDM an PSN 4

5.3 L ten y calc lation 4

5.3.1 L ten y comp nents 4

5.3.2 Pro agation delay 4

5.3.3 Residen e delay 4

5.3.4 L ten y ac umulation 4

5.3.5 Example: laten y of a microwave s stem 4

5.3.6 L ten y an determinism 4

5.3.7 L ten y clas es in IEC 618 0-5 4

5.4 Jiter 5

5.4.1 Jiter definition 5

5.4.2 Jiter clas es in IEC 618 0 51

5.5 L ten y s mmetry an p th con ruen y 51

5.6 Medium as mmetry 51

5.7 Commu ication sp ed s mmetry 5

5.8 Recovery delay 5

5.9 Time ac urac 5

5.9.1 Time ac urac def i ition 5

5.9.2 Time ac urac clas es 5

5.10 Toleran e again t faiures 5

5.10.1 F i ure 5

5.10.2 Rel a i ty 5

5.10.3 Red n an y prin iples 5

5.10.4 Red n an y and relabi ty 5

5.10.5 Red n an y c ec in 5

5.10.7 Red n ant layout: cros - ed n an y 5

5.10.8 Maintaina i ty 5

5.10.9 Avai a i ty 5

5.10.10 Integrity 6

5.10.1 De en a i ty 6

5.10.12 Example: De en a i ty of GOOSE tran mis ion 6

6 Ap lcation analy is 6

6.1 Ap l cation kin s 6

6.2 Tele rotection (IF2 & IF11) 6

6.2.1 Tele rotection s hemes 6

6.2.2 Tele rotection data kin s 6

6.2.3 Tele rotection req irements f or laten y 6

6.2.4 Tele rotection req irements f or laten y as mmetry 6

6.2.5 Tele rotection req irements f or integrity 6

6.2.6 Tele rotection s mmary 6

6.3 Telecontrol (IF1, IF6) 6

6.4 Substation to control centre (IF10) 6

6.5 CMD (IF7) 6

6.5.1 CMD overview 6

6.5.2 CMD commu ication req irements 6

6.6 Control Centre to Control Centre (IF12) 6

6.7 Wide Are Monitorin Sy tem (IF13) 6

6.7.1 WAMS overview 6

6.7.2 WAMS to olog 6

6.7.3 WAMS commu ication req irements 7

6.8 Wide are monitorin , protection an control (WAMPAC) IF13 71

6.8.1 WAMPAC overview 71

6.8.2 WAMPAC commu ication req irements 71

6.8.3 Use case WAMPAC 7

6.9 Win turbines an win virtual p wer plants 7

6.10 Distributed Energ an Renewa les (DER) 7

6.1 Summary of commu ication req irements f or WAN 7

7 Wide-are an re l-time network tec nologies 7

7.1 Introd ction 7

7.2 To olog 7

7.3 Overview 7

7.4 L yer 1 (ph sical) tran mis ion media 7

7.4.1 Summary 7

7.4.2 In tal ation g idel nes 7

7.4.3 Metal c l nes 7

7.4.4 Power l ne car ier (PLC) 8

7.4.5 Radio tran mis ion 8

7.4.6 Fib r o tic 8

7.4.7 L yer 1 red n an y 9

7.6.1 Tele hon tec nologies 9

7.8 L yer 4 ( ran p rt protocols 15

7.8.1 Tran p rt layer en a s lation 15

7.9.4 Ses ion protocol for voice an vide tran mis ion 161

7.9.5 Ap l cation interace red n an y 161

7.9.6 Ap l cation device red n an y 16

7.10 Protocol overlay – tu nel n 16

7.10.1 Definition 16

7.10.2 Tu nel n prin iple 16

7.10.3 Tu nel n L yer 2 over L yer 3 16

7.10.4 Use Case: Tu nel n GOOSE and SMV in IEC 618 0 16

7.10.5 Circ it emulation service (CES) .16

7.1 Virtual Private Network (VPNs) 16

7.12.4 Thre t detection an mitigation 18

7.12.5 Sec rity arc itecture 18

7.12.7 Sec rity for s n hro hasor (PMU) network (IEC TR 618 0-9 -5) 18

7.12.8 Ad itional recommen ation 18

7.13 QoS an a plcation-sp cif i en ine rin 18

7.13.1 General 18

7.13.2 SDH/SONET QoS an SLA 18

7.13.3 PSN QoS an SLA 18

7.13.4 Ap l cation an priority 18

7.13.5 QoS c ain b twe n network 18

7.13.6 QoS ma pin b twe n network 18

7.13.7 QoS en ine rin 19

7.13.8 Cu tomer restriction 191

7.13.9 Cloc services 191

7.14 Config ration an OAM 191

7.14.1 Network config ration 191

7.14.2 OAM .191

7.15 Time s n hronization .19

7.15.1 Os i ator sta i ty 19

7.15.2 Mutual s n hronization 19

7.15.3 Direct s n hronization 19

7.15.4 Radio s n hronization 19

7.15.5 GNSS s n hronization 19

7.15.6 Freq en y distribution 19

7.15.7 Time distribution .19

7.15.8 PTP telecommu ication profies 2 3

7.15.9 PTP over MPLS 2 3

7.15.10 Comp rison of time distribution prof iles b sed on IEC 615 8 2 3

7.15.1 Use Case: Sy c ro hasor time s n hronization 2 5

7.15.12 Use case: Atomic Cloc Hierarc y 2 5

8 Use cases 2 6

8.1 Use case: Cur ent dif ferential tele rotection s stem (Ja an) 2 6

8.2 Use case: SDH / MPLS network (Ja an) 210

8.3 Use Case: Wide are sta i zin control s stem (Ja an) 21

8.4 Use Case: exp rimental PMU-b sed WAMPAC s stem 213 Biblogra h 216 Fig re 1 – Symb ls 3

Fig re 2 – Substation location in An alu ia 3

Fig re 3 – To olog of the An alu ia network 3

Fig re 4 – Ca inet of the s bstation ed e node 3

Fig re 5 – Commu ication interaces in a SEN 3

Fig re 6 – Commu icatin entities 4

Fig re 7 – SGAM commu ication model 41

Fig re 8 – Prin iple of grid voltage level an network tec nolog 4

Fig re 13 – Jit er f or two commu ication delay typ s 5

Fig re 14 – Precision an ac urac definition 5

Fig re 15 – Red n an y of red n ant s stems 5

Fig re 16 – Red n an y calc lation 5

Fig re 17 – Red n an y layout with single p int of f aiure 5

Fig re 18 – Red n an y layout with cros -coupl n 5

Fig re 19 – Avai a i ty definition 5

Fig re 2 – Resid al er or rate in fu ction of the BER 61

Fig re 21 – Prin iple of s n hro hasor tran mis ion 6

Fig re 2 – Target phenomena for WAMPAC 71

Fig re 2 – Example of main fu ction an general inf ormation f low 7

Fig re 2 – PMUs an data flow b twe n TSO an regional data h bs 7

Fig re 2 – Network to olog (Car ier Ethernet 7

Fig re 2 – Phase-to-grou d coupln f or PLC 8

Fig re 2 – HV PLC coupl n with s sp n ed l ne tra s 81

Fig re 2 – Phase to phase sig al coupl n f or PLC 81

Fig re 2 – Phase-to-phase sig al coupl n 8

Fig re 3 – Power lne car ier, l ne tra s 8

Fig re 31 – Ter estrial microwave l n 8

Fig re 3 – L yer 2 tran p rt on radio s stems 8

Fig re 3 – Radio network in f eeder automation 8

Fig re 3 – ADSS fib r ca le 9

Fig re 3 – ADSS in tal ation with spl cin b x 91

Fig re 3 – OPGW in grou d ca le 91

Fig re 3 – OPGW with two “C”-tub s with e c 3 fib rs 9

Fig re 3 – OPGW fib rs 9

Fig re 3 – Spl cin b x 9

Fig re 4 – WDM over one fib r 9

Fig re 41 – OCh o tical comp nents 9

Fig re 4 – Optical l n with microwave b c -up 9

Fig re 4 – Picture of p rtial y destroyed 7 5 kV lne 9

Fig re 4 – E1 an E2 c an el 10

Fig re 4 – Digital Tran mis ion Hierarc y (T – Stan ard ) 10

Fig re 4 – Digital Tran mis ion Hierarc y (E-stan ard) 101

Fig re 4 – Example of an SDH network for uti ties 10

Fig re 4 – SONET multiplexin hierarc y 10

Fig re 4 – SDH multiplexin hierarc y 10

Fig re 5 – SDH/SONET with p int-to-p int to olog 10

Fig re 51 – SDH/SONET with l ne r to ology 10

Fig re 5 – BLSR/BSHR to olog in normal con ition (f rom A to D) 10

Fig re 5 – BLSR/BSHR to olog in fai ure con ition 10

Fig re 5 – UPSR/USHR to olog in normal con ition 10

Fig re 5 – UPSR/USHR to olog in faiure con ition 10

Fig re 5 – Example of inf ormation flow relation hip in OTN 1 2

Fig re 5 – IEEE 8 2.3 (Ethernet f ame f ormat 1 3

Fig re 5 – IEEE 8 2.3 (Ethernet to olog with RSTP switc es (IEC TR 618 0-9 -4) 1 4

Fig re 5 – IEEE 8 2.1Q-tag ed Ethernet f ame format 1 5

Fig re 6 – Direct Ethernet with VLAN in s bstation-to-s bstation tran mis ion 1 6

Fig re 61 – Substation-to-s bstation L yer 2 tran mis ion tu neled over IP 1 7

Fig re 6 – PRP stru ture (within an outside a s bstation) 1 8

Fig re 6 – HSR rin con ectin s bstation an control centre 1 8

Fig re 6 – MACsec f rame f ormat 12

Fig re 6 – IEEE 8 2.1X prin iple 121

Fig re 6 – Ethernet f or s bstation-to-s bstation commu ication 12

Fig re 6 – Pac ets over TDM 12

Fig re 6 – IEEE 8 2.1Q/ad/ah network config ration 12

Fig re 6 – Case of IEEE 8 2.1Q/ad network for uti ty 12

Fig re 7 – Basic MPLS arc itecture 12

Fig re 71 – Example of MPLS f rame f ormat with IPv4 p ylo d 12

Fig re 7 – IPv6 u icast ad res stru ture 141

Fig re 8 – IPv6 ULA ad res stru ture 14

Fig re 81 – IPv6 ln local ad res stru ture 14

Fig re 8 – Ma pin of IPv4 to IPv6 ad res es 14

Fig re 8 – IPv6 evolution 14

Fig re 8 – IEC 618 0 stac with IPv4 an IPv6 14

Fig re 8 – Diff Serv code oint f ield .15

Fig re 8 – Unidirectional protocol in e en ent multicast 151

Fig re 8 – Bidirectional protocol in e en ent multicast 15

Fig re 8 – Frame format for IPsec (authenticated) 15

Fig re 8 – Frame format for IPsec (en rypted) .15

Fig re 9 – L yer 3 direct con ection within same ad res sp ce 15

Fig re 91 – Con ectin s bstation to SCADA by a NAT 15

Fig re 9 – Substation to SCADA con ection over ALG 15

Fig re 9 – Ethernet f rame with UDP tran p rt layer 15

Fig re 9 – UDP he der 15

Fig re 9 – Red n ant network tran mis ion han led by the a pl cation layer 161

Fig re 10 – Tu nel n in IEC TR 618 0-9 -1 16

Fig re 101 – L TP tran p rtin L yer 2 f ames over IP 16

Fig re 10 – Tu nel n GOOSE over IP in IEC TR 618 0-9 -5 16

Fig re 10 – Pseu o-wire prin iple 16

Fig re 10 – Non-IP voice commu ication over PSN 16

Fig re 10 – Circ it emulation over PSN 16

Fig re 10 – L VPNs VPWS an VPLS 17

Fig re 10 – L VPN 171

Fig re 10 – Emulation of L VPN by L VPN an glo al router 17

Fig re 10 – Tele-protection over VPWS, .17

Fig re 1 0 – WAMS over VPLS 17

Fig re 1 1 – VPN for IP-b sed SCADA/EMS traff i 17

Fig re 1 2 – VPN de loyment o tion 17

Fig re 1 3 – IP network se arator 181

Fig re 1 4 – Sec rity arc itecture (u in segmentation an p rimeter sec rity) 18

Fig re 1 5 – QoS c ain .18

Fig re 1 6 – Timin pulse tran mis ion method of legac tele rotection devices 19

Fig re 1 7 – Sy cE a pl cation 19

Fig re 1 8 – Sy c ronou Ethernet Arc itecture 19

Fig re 1 9 – SNTP cloc s n hronization an network delay me s rement 19

Fig re 12 – Model of GMC, two BCs in series an SC over L yer 3 2 0

Fig re 121 – Timin diagram of PTP (en -to-en , 2-ste , BCs) 2 0

Fig re 12 – Timin diagram of PTP (p er to-p er, 2-ste TCs) 2 1

Fig re 12 – Substation s n hronization over WAN 2 5

Fig re 12 – Example of s n hronization network 2 6

Fig re 12 – Cur ent diff erential 1:1 config ration 2 7

Fig re 12 – Network config ration for central zed multi-terminal l ne protection 2 7

Fig re 12 – Network config ration for distributed multi-terminal l ne protection 2 8

Fig re 12 – Cur ent diff erential tele rotection f or HV multi-terminal tran mis ion l ne

u in L yer 2 network 2 8

Fig re 12 – Config ration of wide are c r ent dif ferential primary an b c up

tele rotection s stem employin Car ier Ethernet an IEC 615 8 time s n hronization 210

Fig re 13 – Ac ievin protection f or tele rotection services 21

Fig re 131 – Sy tem config ration f or wide are sta i zin control s stem 212 Fig re 13 – Ap e ran e of typical CCE ca inet 213 Fig re 13 – Config ration f or PMU-b sed WAMPAC s stem 214 Ta le 1 – L ten y clas es in IEC 618 0-5 4

Ta le 2 – L ten y clas es in IEC TR 618 0-9 -1 4

Ta le 3 – L ten y clas es f or WANs 5

Ta le 4 – Jiter clas es in IEC TR 618 0-9 -1 51

Ta le 5 – Jiter clas es for WAN 51

Ta le 7 – IEC TR 618 0-9 -1 time ac urac clas es 5

Ta le 8 – IEC 618 0-5 time ac urac clas es f or IED s n hronization 5

Ta le 9 – WAN time s n hronization clas es 5

Ta le 10 – L ten y for l ne protection 6

Ta le 1 – Summary of o erational req irements of l ne protection 6

Ta le 12 – Summary of commu ication req irements for tele rotection 6

Ta le 13 – Commu ication req irements for CC to SS/PS 6

Ta le 14 – L ten y an timin req irements f om IEC TR 618 0-9 -2 6

Ta le 15 – Commu ication req irements for CMD 6

Ta le 16 – Commu ication req irements f or inter control centre commu ication 6

Ta le 17 – Summary of s n hro hasor req irements 7

Ta le 18 – Summary of commu ication req irements for wide are monitorin 71

Ta le 19 – Typical commu ication req irements f or WAMPAC 7

Ta le 2 – Clas ification of commu ication req irements 7

Ta le 21 – Commu ication req irements of wide-are a plcation 7

Ta le 2 – Commu ication tec nologies 7

Ta le 2 – Ph sical commu ication media 7

Ta le 2 – DSL commu ication over twisted p irs 7

Ta le 2 – Trade-of fs in co p r ca le commu ication 8

Ta le 2 – PLC commu ication tec nologies 8

Ta le 2 – PLC commu ication ad antages an disad antages 8

Ta le 2 – Microwave ln p rorman e 8

Ta le 2 – Ter estrial microwave ad antages an disad antages 8

Ta le 3 – Publ c mo i e radio tec nologies 8

Ta le 31 – Ter estrial radio ad antages an disad antages 8

Ta le 3 – Satel te radio ad antages an disad antages 8

Ta le 3 – Optical f ib rs: ad antages an disad antages 9

Ta le 3 – SONET an SDH hierarc ies 10

Ta le 3 – Summary of SDH/SONET 1 1

Ta le 3 – Ethernet ph sical layers 1 3

Ta le 3 – Paylo d ma pin u in SDH/SONET an Next Generation SDH/SONET 12

Ta le 3 – Car ier Ethernet s mmary 12

Ta le 3 – IP/MPLS c aracteristic 13

Ta le 4 – MPLS-TP c aracteristic 13

Ta le 41 – MPLS s mmary 13

Ta le 4 – Diff eren es b twe n IPv4 an IPv6 14

Ta le 4 – IPv6 vers s IPv4 addres es [RFC 4 91] 14

Ta le 4 – List of Dif fServ code oint field values 15

Ta le 4 – IP Summary 15

Ta le 5 – Typical os i ator sta i ty 19

Ta le 51 – IEC 615 8 o tion comp rison 2 2

Ta le 5 – Precision time distribution protocols b sed on IEC 615 8 2 4

Ta le 5 – Req irements f or the YONDEN IP network 21

Ta le 5 – Tec nologies for the YONDEN IP network 21

Ta le 5 – Main s stem sp cification for wide are sta i zin control s stem 213

Ta le 5 – Sp cif i ation for PMU-b sed WAMPAC s stem 215

INTERNATIONAL ELECTROTECHNICAL COMMISSION

1 Th Intern tio al Ele trote h ic l Commis io (IEC) is a worldwid org niz tio for sta d rdiz tio c mprisin

al n tio al ele trote h ic l c mmite s (IEC Natio al Commite s) Th o je t of IEC is to promote

intern tio al c -o eratio o al q e tio s c n ernin sta d rdiz tio in th ele tric l a d ele tro ic f i ld To

this e d a d in a ditio to oth r a tivitie , IEC p bls e Intern tio al Sta d rd , Te h ic l Sp cif i atio s,

Te h ic l Re orts, Pu lcly Av ia le Sp cif i atio s (P S) a d Guid s (h re fter refer e to a “IEC

Pu lc tio (s)”) Th ir pre aratio is e tru te to te h ic l c mmite s; a y IEC Natio al Commite intere te

in th s bje t d alt with ma p rticip te in this pre aratory work Intern tio al g v rnme tal a d n n

-g v rnme tal org niz tio s laisin with th IEC als p rticip te in this pre aratio IEC c la orate clo ely

with th Intern tio al Org niz tio f or Sta d rdiz tio (ISO) in a c rd n e with c n itio s d termin d b

a re me t b twe n th two org niz tio s

2) Th f ormal d cisio s or a re me ts of IEC o te h ic l maters e pre s, a n arly a p s ible, a intern tio al

c n e s s of o inio o th rele a t s bje ts sin e e c te h ic l c mmite h s re re e tatio fom al

intere te IEC Natio al Commite s

3) IEC Pu lc tio s h v th f orm of re omme d tio s f or intern tio al u e a d are a c pte b IEC Natio al

Commite s in th t s n e Whie al re s n ble ef forts are ma e to e s re th t th te h ic l c nte t of IEC

Pu lc tio s is a c rate, IEC c n ot b h ld re p n ible for th wa in whic th y are u e or for a y

misinterpretatio b a y e d u er

4) In ord r to promote intern tio al u iformity, IEC Natio al Commite s u d rta e to a ply IEC Pu lc tio s

tra s are tly to th ma imum e te t p s ible in th ir n tio al a d re io al p blc tio s An div rg n e

b twe n a y IEC Pu lc tio a d th c re p n in n tio al or re io al p blc tio s al b cle rly in ic te in

th later

5) IEC its lf d e n t pro id a y ate tatio of c nf ormity In e e d nt c rtif i atio b die pro id c nformity

a s s me t s rvic s a d, in s me are s, a c s to IEC mark of c nformity IEC is n t re p n ible f or a y

s rvic s c rie o t b in e e d nt c rtif i atio b die

6) Al u ers s o lde s re th t th y h v th late t e itio of this p blc tio

7) No la i ty s al ata h to IEC or its dire tors, emplo e s, s rv nts or a e ts in lu in in ivid al e p rts a d

memb rs of its te h ic l c mmite s a d IEC Natio al Commite s f or a y p rs n l injury, pro erty d ma e or

oth r d ma e of a y n ture wh ts e er, wh th r dire t or in ire t, or for c sts (in lu in le al f ee ) a d

e p n e arisin o t of th p blc tio , u e of, or rela c u o , this IEC Pu lc tio or a y oth r IEC

Pu lc tio s

8) Ate tio is drawn to th Normativ refere c s cite in this p blc tio Us of th refere c d p blc tio s is

in is e s ble f or th c r e t a plc tio of this p blc tio

9) Ate tio is drawn to th p s ibi ty th t s me of th eleme ts of this IEC Pu lc tio ma b th s bje t of

p te t rig ts IEC s al n t b h ld re p n ible for id ntifyin a y or al s c p te t rig ts

The main tas of IEC tec nical commite s is to pre are International Stan ard However, a

tec nical commit e may pro ose the publcation of a tec nical re ort when it has col ected

data of a dif ferent kin f rom that whic is normal y publs ed as an International Stan ard, f or

example "state of the art"

IEC TR 618 0-9 -12, whic is a tec nical re ort, has b en pre ared by IEC tec nical

commit e 5 : Power s stems management an as ociated information ex han e

The text of this tec nical re ort is b sed on the f ol owin doc ments:

This publ cation has b en draf ted in ac ordan e with the ISO/IEC Directives, Part 2.

A lst of al p rts in the IEC 618 0 series, publ s ed u der the general title Commu ic t io

n tworks a d systems for p wer utility a t omat io , can b fou d on the IEC we site

The commit e has decided that the contents of this publ cation wi remain u c an ed u ti

the sta i ty date in icated on the IEC we site u der "ht p:/we store.c " in the data related to

the sp cif i publ cation At this date, the publcation wi b

• reconfirmed,

• with rawn,

• re laced by a revised edition, or

• amen ed

A bi n ual version of this publcation may b is ued at a later date

IMPORTANT – The 'colour inside' logo on the cov r pa e of this publ c tion in ic te

that it contains colours whic are considere to be us ful f or the cor e t

understa din of its conte ts Us rs s ould th ref ore print this doc me t using a

colour printer

Uti ties u e data network to intercon ect eq ipment b twe n their premises, over distan es

f rom u der a ki ometer to thou an s of kiometers, cal ed a “Wide Are Network” of WAN

WANs en omp s commu ication me n of dif ferent nature (o tical, radio, p wer l ne car ier,

co p r, etc.), with a variety of to ologies (rin s, tre s, mes es, etc.), u in dif ferent protocols

(SDH/SONET, Ethernet, IP, MPLS, etc.), medium s arin (p c et switc in , time division

multiplex, etc.) an f or dif ferent a pl cation ( ele rotection, SCADA, voice, vide , etc.)

This contrasts with s bstation automation network as des rib d in the LAN En ine rin

Guidel nes (IEC TR 618 0-9 -4), whic are b sed on one tec nolog (switc ed Ethernet ,

make exten ive u e of L yer 2 multicast (GOOSE, SMV, PTP, etc.) an u e L yer 3

commu ication (MMS, FTP, etc.), typical y without routers within the s bstation

The IEC 618 0 s ite sets up n merou req irements on the network but do s not state how to

ac ieve them:

• IEC 618 0-5 sp cif ies the b sic req irements f or data network u ed in Power Uti ty

Automation network ;

• IEC 618 0-7 f oc ses on data model n , le vin out ph sical intercon ection detais;

• IEC 618 0-8-1 an IEC 618 0-9-2 sp cify intero era le commu ication within s bstation ;

• IEC TR 618 0-9 -1 des rib s s bstation-to-s bstation traf fic, sp cif ies the req irements

for commu ication, defines o ject models for s bstation-to-s bstation tele rotection,

models the gateway an the tu neler, but le ves the WAN u defined;

• IEC TR 618 0-9 -2

1

provides s bstation to control centre network con g ration for IEDs,

proxies an a pl cation ;

• IEC TR 618 0-9 -5 (s n hro hasor tran mis ion) ad res es the tran p rt of

s n hro hasor data b twe n PMUs an control centres an defines a tu nel n protocol

as wel as a data sec rity method;

• IEC TR 618 0-9 -4 provides g idel nes f or network en ine rin foc sed on Ethern

et-b sed re l-time an hig ly avai a le network in s bstation Some of these g idelnes

are a pl ca le to network outside of the s bstation;

• IEC 6 8 0-6 (TASE2), IEC 619 8 an IEC 619 0 (CIM) des rib the inf ormation

interc an e at the a pl cation layer without sp cif yin the network

Eac of these doc ments de ls se arately with a pl cation, tran p rt or network layer

mec anism There exist no comprehen ive en ine rin g ides for wide-are an re l-time

network f or control an protection The growin s c es of IEC 618 0 cal s for g idel nes f or

en ine rin the WANs

Complementin IEC TR 618 0-9 -4, this Tec nical Re ort pro oses g idel nes f or wide-are

an re l-time network for variou IEC 618 0-b sed a plcation in lu in tele rotection,

wide are me s rement, protection an control (WAMPAC), p wer s stem monitorin (WASA,

WAMS), o eration SCADA, an con ition monitorin an diag osis (CMD) an non

-o erational traf fic

This Tec nical Re ort is b sed on existin stan ard for semantic , services, protocols,

s stem config ration lan uage an arc itecture It is b sed on work done by variou

IEC workin groups in lu in :

• Energ management s stem a pl cation program interace;

• Data an commu ication sec rity;

• Intero era i ty within TC 5 in the lon term;

• In u trial network ;

• Hig ly Avai a le Automation Network

Contribution were in lu ed fom:

• IEEE 8 2.1 WG (Hig er layer LAN protocols);

• IEEE 15 8 WG (Precise Networked Cloc Sy c ronization);

• IEEE Power Sy tem Relayin Commite (PSRC);

• UCA International Users Group;

• The North American Sy c ro hasor Initiative (NASPI);

• CEN/CENELEC/ETSI Smart Grid Co rdination Group;

• CIGRE workin groups D2.2 , D2.2 , D2/B5.3 , D2.3 ; an

• Diff erent uti ties, providers an rese rc in titutes, in p rtic lar the Central Rese rc

In titute of Electric Power In u try (Ja an), Hy ro-Que ec [5 ]

COMMUNICA TION NETWORKS A ND SYSTEMS

FOR POWER UTILITY A UTOMA TION –

Part 90-12: Wide area network engineering guidel nes

This Tec nical Re ort is inten ed f or an au ien e f ami ar with electrical p wer automation

b sed on IEC 618 0 an p rtic larly for data network en ine rs an s stem integrators It is

inten ed to help them to u derstan the tec nologies, con g re a wide are network, def i e

requirements, write sp cification , select comp nents an con u t tests

This Tec nical Re ort provides definition , g idel nes, an recommen ation f or the

en ine rin of WANs, in p rtic lar for protection, control an monitorin b sed on IEC 618 0

an related stan ard

This Tec nical Re ort ad res es s bstation-to-s bstation commu ication, s bstation

-to-control centre an control centre-to-control centre commu ication In p rtic lar, this Tec nical

Re ort ad res es the most critical asp cts of IEC 618 0 s c as protection related data

tran mis ion via GOOSE an SMVs, an the multicast tran f er of large volumes of

s n hro hasor data

The Tec nical Re ort ad res es is ues s c as to olog , red n an y, traf fic laten y an

q al ty of service, traff i management, cloc s n hronization, sec rity an maintenan e of the

network

This Tec nical Re ort contain u e cases that s ow how uti ties tac le their WAN

en ine rin

The f ol owin doc ments, in whole or in p rt, are normatively ref eren ed in this doc ment an

are in isp n a le f or its a pl cation For dated ref eren es, only the edition cited a ples For

u dated referen es, the latest edition of the referen ed doc ment (in lu in an

amen ments) a pl es

IEC 6 0 0, Int ern t io al Ele trotec nic l Vo a ulary

IEC 6 8 4-1, Tele rotect io e uipme t of p wer systems – P erorma c a d testin – P art 1:

Comma d systems

IEC 6 8 4-2, P erorma c a d test in of teleprot ection e uipme t of p wer syst ems – P art 2:

An log e c mp riso systems

IEC 6 8 0-5-10 , Tele o trol e uipme t a d systems – P art 5-10 : Tra smis io proto ols –

Network a c s for I EC 6 87 0-5-101 usin st and ard tra sp rt profiles

IEC 615 8:2 0 , P re isio clo k sync ro izat io prot oc l for n tworked me sureme t a d

c ntrol systems

IEC 618 0-5:2 13, Commu ic t ion n t works a d systems for p wer utility a tomat io –

P art 5: Commu ic tion re uireme ts for fu ct io s a d devic models

IEC 618 0-8-1, Commu icat ion n tworks a d systems for p wer ut ility a t omation – P art 8-1:

Sp cific c mmu ic tion servic ma ping (SCSM) – Map ings to MMS (I SO 950 -1 a d ISO

950 -2) a d t o ISO/IEC 8 0 -3

IEC 618 0-9-2:2 1 , Commu ic t io n t works a d systems for p wer utility a tomation –

P art 9-2: Sp cific c mmu ic t io servic ma ping (SCSM) – Sampled v lues over ISO/IEC

8 0 -3

IEC PAS 618 0-9-3:2 15, Commu ic tion n tworks a d systems for p wer ut ility a tomation

– P art 9-3: P re isio t ime prot oc l profil for p wer ut ility a tomat io

IEC TR 618 0-9 -1:2 10, Communic t io n t works a d systems for p wer ut ility a tomat io

– P art 9 -1: U se of IEC 61850 for the c mmu ic tion b twe n su stat io s

IEC TR 618 0-9 -2:2 15, Commu ic t io n t works a d systems for p wer ut ilty a tomat ion

– Part 9 -2: U se of IEC 61850 for the c mmu ic tion b twe n su statio s a d c ntrol

c ntres

3

IEC TR 618 0-9 -4:2 13, Commu ic t io n t works a d systems for p wer utility a tomation

– P art 9 -4: Network e gin ering g ideli es

IEC TR 618 0-9 -5:2 12, Commu ic t io n t works a d systems for p wer ut ility a tomation

– P art 9 -5: Use of IEC 61850 t o tra smit sy c ro h sor informatio a c rd ing to IEEE

C37.1 8

IEC 618 9-9, Instrume t Tra sformers – P art 9: Digit al intera e for instrume t tra sformers

3

IEC TS 6 3 1-1, P ower systems ma a eme t a d as o iated information ex ch n e – Dat a

a d c mmu ic tions se urity – P art 1: Commu ic tion n twork a d system se urity –

I ntrod uct ion to se urity is u s

IEC T 6 3 1-2, P ower systems ma a eme t a d as o iated informatio exch n e – Data

a d c mmu ic tions se urity – Part 2: Glos ary of terms

IEC 6 351-3, P ower systems ma a eme t a d as o iated informatio ex ch n e – Dat a a d

c mmu ic tions se urity – P art 3: Commu ic tion n twork a d syst em se urity – Profiles

includ in TCP /IP

IEC T 6 3 1-4, P ower systems ma a eme t a d as o iated information ex ch n e – Data

a d c mmu ic t io s se urity – P art 4: Profiles includ ing MMS

IEC T 6 3 1-5, P ower systems ma a eme t a d as o iated information ex ch n e – Data

a d c mmu ic t io s se urity – P art 5: Se urity for IEC 6 870-5 a d d erivatives

IEC TS 6 3 1-6, P ower systems ma a eme t a d as o iated information exch n e – Data

a d c mmu ic tions se urity – P art 6: Se urity for IEC 61850

_ _ _ _ _ _ _

3

Tob p bls e

IEC TS 6 3 1-7, P ower systems ma a eme t a d as o iated information ex ch n e – Data

a d c mmu ic tions se urity – P art 7: N etwork a d system ma a eme t (NSM) d ata o ject

mod els

IEC TS 6 3 1-8, P ower systems ma a eme t a d as o iated information ex ch n e – Data

a d c mmu ic t io s se urity – P art 8: Role-b sed a c s c ntrol

IEC TS 6 3 1-9, P ower systems ma a eme t a d as o iated informatio ex ch n e – Data

a d c mmu ic tions se urity – P art 9: Cyber se urity k y ma a eme t for p wer system

4

IEC TR 6 3 1-10, P ower systems ma a eme t a d as o iated informat ion exch n e – Data

a d c mmu ic tions se urity – Part 10: Se urity arc ite ture g idelines

IEC 6 3 1-1 , P ower systems ma a eme t a d as o iated informatio exch n e – Dat a a d

c mmu ic tions se urity – P art 1 : Se urity for XML files

5

IEC TR 6 3 7-2 0, P ower systems ma a eme t a d as o iat ed informat io ex ch n e –

P art 2 0: Guid eln s for migration to Intern t P rotoc l v rsio 6 (IP v6)

IEC 6 4 9-1:2 10, I nd ustrial c mmu ic tio n tworks – Hig availability a t omat io n tworks

– P art 1: Ge eral c n e ts a d c lc lat io methods

IEC 6 43 -3:– , I nd ustrial c mmu ic tio n tworks – High avail bility a tomation n t works –

P art 3: P ara el Red und an y P rot oc l (PRP ) a d Hig -a ailability Se mles Redundan y

(HSR)

6

IEC TS 6 4 3-1-1, I nd ustrial c mmu ic tion n tworks – N etwork a d syst em se urity –

P art 1-1: Termin logy, c n e ts a d models

IEC 6 4 3-2-1, I nd ustrial c mmu ic tion n twork s – Net work a d system se urity – P art 2-1:

Establshing a ind ustrial a t omat io a d c ntrol system se urity pro ram

IEC TR 6 4 3-3 (al p rts), I nd ustrial c mmu ic t io n tworks – Net work a d syst em se urity

IEC TR 6 4 3-3-1, Ind ustrial c mmu ic tio n tworks – N etwork a d syst em se urity –

P art 3-1: Se urity te h ologies for ind ustrial a tomat io a d c ntrol systems

IEC 6 4 8-1, P ower li e c mmu ic t io syst ems for p wer utility a plc tions – P art 1:

P lan in of a alog e a d d igital p wer line c rier syst ems o eratin o er EHV/HV/MV

ele tricity grids

ANSI T1.4 3-19 9, Net work a d Customer I nst allat ion Intera es – DS1 Electric l I ntera e

IEEE 8 2.1ag, IEEE standards for loc l a d metro o t an are n t work; Virtu l Bridged L c l

Are Networks Ame d me t 5: Co n ct ivity Fault Man g me t

IEEE 8 2.1ah, IEEE sta dards for lo al a d metro o t an are n twork; P rovid er Ba k bo e

Bridg s

_ _ _ _ _ _ _

IEEE 8 2.1Qay, P rovid er Ba k bo e Brid ge Trafic Engine rin

IEEE 8 2.1X, P ort b sed Net work Ac es Co t rol

IEEE 8 2.3, IEEE St and ard for Informat ion tec n lo y – Telec mmu ic t io s a d informatio

ex ch n e b t we n systems – L c l a d metro o ta are n t works – Sp cific re uireme ts –

P art 3: Carier se se multiple a c s with c llision dete tion (CSMA/CD) a c s meth d a d

p ysic l lay r sp cific tions

IEEE 4 7.3, I EEE Stand ard for the Electric l P rote tio of Commu ic t ion Facilities Servin

Ele tric Su ply L c t ions Thro g t he U se ofH ybrid F aciities

IEEE 8 2.1Q, IEEE st andards for loc l a d metro o t an are n twork; Virt ual brid ged loc l

are n tworks (VLAN s a d priorities)

ITU-T G.7 3, P hysic l/electric l c ara terist ics of hierarc ic l d igital ntera es

ITU-T G.7 7/Y.13 2, Net work n d e int era e for t he sync ro o s d igit al hierarc y (SDH )

ITU-T G.8 3, Arc itecture of Tra sp rt N etworks Based o Sync ro o s Digital Hierarc y

(SDH )

ITU-T G.810, Definitions a d terminolo y for sync ro izat ion n tworks

ITU-T G.81 , Timin c ara teristics of primary refere c cloc s

ITU-T G.812, Timin Re uireme ts of SDH Equipme t Slav Cloc s (SEC)

Sync ro izat io Net work s (SEC)

ITU-T G.8 1, Eror p rorma c ofa int ern tio al d igital c n e t io o erat ing at a bit rate

b low the primary rate a d formin p rt of a Integrat ed Servic s Digital Net work

ITU-T G.8 6 , Definitio a d t erminolo y for sy c ro ization in P ack et N etworks

ITU-T G.8 6 , Arc it ecture a d re uireme ts for p c et -b sed fe u n y d elv ry

ITU-T G.8 6 1, P re isio Time P rotoc l telec m profil for fe u n y sy c ro ization

ITU-T G.8 7 1, P re isio Time P rotoc l telec m profil for p ase/t ime sync ro iz t io

ITU-T G.7 41, Ge eric Framing P ro ed ure

ITU-T G.7 4 , Lin Ca a ity Ad just me t Sc eme

ITU-T G.8 3 , Eth rn t ring prote t io switc in

ITU-T G.8 61, Timin a d sync ro iz t io asp ct s in p ck et n tworks

ITU-T G.8 6 , Timin c ara t eristics of a sy c ro o s Ethern t e uipme t slav cloc

ITU-T G.8 6 , Distribut ion of timin information thro g p c et n t works

RFC 0 6 , User Datagram P rotoc l (UDP )

RFC 0 91, Int ern t P rotoc l (IP v4)

RFC 0 9 , Int ern t Co trol Mes a e Prot oc l (ICMP v4)

RFC 0 9 , Tra smis io Co trol P roto ol (TCP ), Protoc l Sp cific tion

RFC 0 2 , An Et hern t Addres Resolution P rot oc l (ARP )

RFC 0 9 , A St and ard for t he Tra smis ion of IP Data rams o er Et hern t Networks

RFC 12 0, OSI Co n ct ionles Tra sp rt Servic s o t op of UDP , Versio 1

RFC 16 1, Th P ointto-P oint P rot oc l (P P P )

RFC 1918, Ad dres All c t io for P rivate Int ern t

RFC 210 , HMAC: Ke ed-Hashin for Mes a e Aut he tic tio

RFC 2 2 , OSP F Versio 2

RFC 2 6 , Int ern t P rotoc l, Versio 6 (IP v6) Sp cific tion

RFC 2 6 , Tra smis io of IPv6 P ackets over Ethern t N etworks

RFC 2 7 , Definition of Difere t iated Servic s Field (DS Field) in IP v4 a d IPv6 H ead ers

RFC 2 15, P oint to-P oint P roto ol o er SDH/SONET

RFC 2 6 , IP Network Addres Tra slator (NAT) Termin log a d Co sid erat io s

RFC 3 2 , Trad it ion l IP Net work Addres Tra slat or (Trad ition l NAT)

RFC 3 31, Mult iprotoc l L b l Switc ing Arc itect ure

RFC 316 , Th Ad d it ion of Expli it Co g stion N otific tio (ECN ) to IP

RFC 3 4 , An Ex edited Forward in P HB (Per-Ho Be avior)

RFC 3 4 , Su pleme tal Informat io for th New Definitio of the EF P HB (Ex ped ited

Forward ing Per-Ho Be avior

RFC 3 6 , New Terminolog a d Clarific t io s for DifServ

RFC 3 61, SIP: Ses io I nitiatio P rotoc l

RFC 3 7 , Intern t Gro p Man g me t Proto ol, Versio 3

RFC 3 8 , Pse d o Wire Emulation Edg - o-Ed ge (P WE3) Arc it ecture

RFC 419 , Uniq e L c l I Pv6 Unic st Ad dres es

RFC 4 13, Basic Tra sition Mec a isms for IP v6 H osts a d Ro ters

RFC 4 91, IP Versio 6 Addres in Arc ite t ure

RFC 4 01, Se urity Arc ite ture for the Int ern t P rotoc l (IP se )

RFC 4 0 , IP Enc psulating Se urity P ayload (ESP )

RFC 4 3 , Simple Net work Time P roto ol (SN TP ) Versio 4 for IP v4, IP v6 a d OSI

RFC 4 5 , MTU a d Fra me tat io I ssu s with In-the-Network Tu n li g

RFC 4 6 , Framework for L y r 2 Virtu l P rivate N etworks (L VP N s)

RFC 4 61, Neig b r Disc v ry for IP v rsion 6 (IP v6)

RFC 5 4 , Th Tra sp rt L vel Se urity (TLS) P rot oc l Version 1.2

RFC 5 2 , Th Syslog P rotoc l

RFC 5 41, L y r 2 Tu n ln P rotoc l Version 3 (L TP v3) Ext end ed Circ it Status Values

RFC 5 71, IANA Guid elines for IP v4 Multic st Ad dres As ig me ts

RFC 5 8 , Bid ire tion l Forward ing Detection (BFD)

RFC 5 0 , Network Time P roto ol version 4

RFC 6 5 , IP v6 Ad res in ofIP v4/I Pv6 Tra slators

RFC 6 6 , Upd ated Sp cific tion of t he I P v4 ID Field

RFC 7 1 , Sp cific tion of the I P F low Information Ex ort (IP FIX) P roto ol for th Ex ch n e

of F low Informatio

3 Terms, def initions, abbreviations, a ronyms a d symbols

3.1 Terms a d def initions

For the purp ses of this doc ment the terms an definition given in IEC 6 0 0-191 [6] as

wel as the fol owin , a ply

3.1.1

a ai abi ty

3.1.1.1

a ai abi ty

< f an item> bi ty to b in a state to p rorm as req ired

Note 1 to e try: Av ia i ty d p n s u o th c mbin d c ara teristic of th rela i ty ( 9 -01 2 ),

re o era i ty ( 9 -01 2 ), a d maintain bi ty ( 9 -01 2 ) of th item, a d th mainte a c s p ort p rf orma c

A network that con ects a s bstation or p wer plant LAN to a WAN, at a lower hierarc ical

level (in contrast to core network)

con ru nc

prop rty of a network to al ocate the same p th for b c ward an f orward traff i b twe n en

p ints, en urin that the delay are a proximatively identical

3.1.1

c stomer e ge

IP router located at the ed e of the c stomer network an administrated by the c stomer,

whic con ects to the next IP router in the provider network

3.1.12

deterministic dela

c aracteristic of a commu ication s stem that the laten y f rom en -to-en has a maximum

b u d, in e en ently of traff i , ex e t in case of ai ure or disturb n e

3.1.13

depe dabi ty

pro a i ty that a s stem wi p rf orm cor ectly when req ired to in the presen e of faults

Note 1 to e try: This d f i itio is alg e with IEC 6 8 4- a d is dif fere t f rom IEC 6 0 0- 9 :2 15 [7]

one-way time delay b twe n two en -to-en network interf aces, ex lu in the delay that the

en devices take to proces the sig al

Note 1 to e try: In this d c me t, “late c ” is s n n mo s to “c mmu ic tio d la ” or “n twork d la ”, b t n t to

“tra sfer d la ” (IEC 618 0-5), whic is th a plc tio to a plc tio d la

3.1.2

network

data tran mis ion s stem

Note 1 to e try: Th term “n twork” is re erv d f or d ta n twork , for ele tric l interc n e tio “grid” is u e

in te d

3.1.2

operational traf fic

data traff i ne ded to protect, o erate an s p rvise the electrical elements of the grid

Note 1 to e try: This in lu e traff i critic l f or th mainte a c of th grid, e.g emerg n y p o e , staf

org niz tio , me s gin or a c s to d c me tatio

< f a p wer grid> me s res to avoid damages by actin on circ it bre kers or other p wer

control devices

Note 1 to e try: This d c me t u e “prote tio ” a d “tele rote tio ” e clu iv ly for this me nin

3.1.2 2

prote tion

<in relation to a data p th> me s res taken to en ure avai a i ty in the case of faiure of the

3.1.2 3

prote tion

<in relation to a c b r sec rity> me s res taken to prevent or fen of c b r at ac s

Note 1 to e try: This d c me t u e th term “d f en e” in te d Howe er, “prote tio ” sti a p ars in

a bre iatio s a d refere c d d c me ts

node located at the ed e of the provider network an administrated by the provider, whic

con ects to the c stomer network’s c stomer ed e node

3.1.3

publ c a dre s

ad res b lon in to an ad res ran e al ocated by IANA or IEEE, whic is u iq e in the

context of the network

set of metric for the p r orman e of fered by a commu ication s stem, amon them: laten y

(delay), j t er (delay variation), delay as mmetry, throu hput (bit rate), p c et los rate, bit

er or ratio (BER), f low seq en e preservation, but ex lu in sec rity asp cts

network overlaid on to of another network, whic it p s ibly s ares with other VPNs

3.2 Abbre iations a d a ronyms

6 oWPAN IPv6 over L w p wer Wireles Personal Are Network [RFC 4 19] [RFC 6 7 ]

AAA Authentication, Authorization an Ac ou tin (sec rity)

AAAA- ecord 12 -bit IPv6 ad res record f om DNS [IPv6]

ADSS Al Dielectric Self Sup ortin (o tical fib r)

A- ecord 3 -bit IPv4 ad res record f om DNS

BFD Bidirectional Forwardin Detection

BGP Border Gateway Protocol (s c es or of EGP in Internet

BIDIR-PIM Bi-Directional-Protocol In e en ent Multicast

BITS Bui din Integrated Timin Sup ly

BLSR Bidirectional Line Switc Rin

CE Cu tomer Ed e (node)

CEM Circ it EMulation

CES Circ it Emulation Services

CESoPSN Circ it Emulation Services over Publc Switc ed Network [ITU-T Y.1413,

Y.14 3]

CFM Con ectivity Fault Management

CIA Confidential ty, Integrity, Avai a i ty (again t mal ciou atac s)

CIGRE Con ei International des Gran s Rése u Electriq es

CMD Con ition Monitorin an Diag osis

CoS Clas of Service [IEEE 8 2.1Q]

COTS Common, of f-the-s elf sof tware

CWDM Co rse Wavelen th Division Multiplexin (ITU-T G.6 1, se WDM, DWDM)

DACS Digital Ac es Car ier Sy tem (UK)

DACS Digital Ac es Cros -con ect Sy tem (US)

DANH Doubly At ac ed Node with HSR [IEC 6 4 9-3]

DHCP Dy amic Host Config ration Protocol [RFC 2131])

Diff Serv Diff erentiated Services [RFC 2 7 ]

DNP3 Distributed Network Protocol [IEEE 1815]

DoS Denial of Service (sec rity)

DPLC Digital Power Line Car ier

DS Digital Sig al (SDH c an el)

DSCP Diff erentiated Services CodePoint [RFC 4 9 ]

DSO Distribution Sy tem Op rator

DWDM Den e Wavelen th Division Multiplexer [ITU-T G.6 4.1]

EAP Exten ible Authentication Protocol [RFC 3 4 ]

EAPo Exten ible Authentication Protocol over LAN

ECMP Eq al Cost MultiPath Routin

ECN Expl cit Con estion Notif i ation

EEC Ethernet Eq ipment Cloc [G.82 1]

EIR Ex es Information rate [MEF]

E-LAN Ethernet LAN service [MEF]

E-l ne Ethernet wire service [MEF]

E-LMI Ethernet L cal Management inter ace

EoSDH Ethernet over SDH

EoSONET Ethernet over SONET

ERPS Ethernet Rin Protection Switc in [G.8 3 ]

ESMC Ethernet Sy c ronization Mes agin Chan el [G.8 6 ]

ESP Electronic Sec rity Perimeter

E-tre Routed-Multip int EVC Ethernet (p int-to-multip int service) [MEF]

EVPL Ethernet Virtual Private Line [MEF]

FCAPS F ult Config ration Ac ou tin Perorman e Sec rity

FDM Freq en y Division Multiplexin

FDV Frame Delay Variation [MEF]

FTP Fi e Tran fer Protocol [RFC 9 9]

GDOI Group Domain of Interpretation [RFC 6 0 ]

GNSS Glo al Navigation Satel te Sy tem

GOOSE Generic Object-Oriented Substation Event [IEC 618 0-7-2, IEC 618 0-8-1]

GPRS General Pac et Radio Service (2

n

generation cel phone)

GSM Glo al Sy tem f or Mo ie commu ication (1st generation cel phone)

HMAC Has -b sed Mes age Authentication Code

rd

generation cel phone)

ID IDentity

IED Intel gent Electronic Device [IEC 618 0]

IGMP Internet Group Management Protocol [RFC 3 7 ]

IntServ Integrated Services [RFC 2 10]

IP/MPLS MPLS with IP routing

IRIG Inter Ran e In trumentation Group

ISDN Integrated Services Digital Network [ITU-T]

IS-IS Intermediate Sy tem to Intermediate Sy tem [ISO/IEC 10 8 ]

KDC Key Distribution Centre

L TP L yer 2 Tu nel n Protocol [RFC 5 41]

L VPN L yer 2 VPN [RFC 715 ]

LCAS Line Ca acity Adju tment Sc eme (SDH/SONET NG) [G.7 4 ]

th

generation cel phone)

MAC Medium Ac es Control [IEEE 8 2.1]

MAC Mes age Authentication Code (sec rity)

MACsec MAC sec rity [IEEE 8 2.1AE]

MEF Metro Ethernet Forum

MFA MPLS Forum As ociation (now IPMPLS)

MIB Management Information Base (SNMP)

NAC Network Ac es Control

NASPI North American Sy c roPhasor Initiative

NCD Network Config ration Des ription [IEC 618 0-6]

NERC North-american Electricity Rel a i ty Corp ration

NPDU Network Protocol Data Unit [ISO/IEC 7 9 ]

OAM Op ration, Administration an Maintenan e

OPGW OPtical Grou d Wire (hig voltage tran mis ion ca le)

OSI Op n Sy tem Intercon ection [ISO/IEC 7 9 ]

OSPF Op n Shortest Path First [RFC 5 4 )

OSSP Organization Sp cific Slow Protocol [IEEEE 8 2.3]

OTN Optical Tran p rt Network (ITU-T G.7 9)

P2MP Point To Multip int (Pe r to Multip ers)

P2P Point-to-Point (Pe r to-Pe r)

PABX Private Automatic Bran h eXc an e ( ele hon )

PBB Provider Bac b ne Brid in ( Mac-in-Mac”, IEEE 8 2.1ah-2 0 )

PBB-TE Provider Bac b ne Brid e Traf fic En ine rin (IEEE 8 2.1Qay-2 0 )

PCM Pulse Code Mod lation

PDC Phasor Data Con entrator [IEC TR 618 0-9 -5]

PDH Plesioc ronou Digital Hierarc y (ITU-T)

PDU Protocol Data Unit [ISO/IEC 7 9 ]

PDV Pac et Delay Variation [RFC 3 9 , ITU-T G.8 6 (PSN), ITU-T Y.15 1 (IP),

ITU-T Y.15 3 (Ethernet ]

PFD Pro a i ty to Fai (dan erou ly) on Deman [IEC 615 8]

PHB Per Ho Behavior [MPLS]

PIM-SM Protocol In e en ent Multicast – Sp rse Mode [RFC 4 01]

PMU Phasor Me s rement Unit

PMU Phasor Me s rement Unit [IEC TR 618 0-9 -5]

PRC Primary Referen e Cloc

PT Precision Time Protocol [IEC 615 8]

RADIUS Remote Authentication Dial In User Service

RAS Remedial Action Sc emes (refer ed to as WAMPAC in this doc ment

RIR Regional Internet Registry

RPL Routin Protocol for L w p wer an los y network [RFC 6 5 ]

RSPEC Reservation Characteristic in IntServ [RFC 2 10]

RSVP-TE RSVP-Traf fic Engine rin

RTU Remote Terminal Unit (decentral zed me s rement an control device)

SAToP Stru ture-Ag ostic TDM over Pac et [RFC 4 5 ] [ITU-T Y.1413] [Y.14 3]

SCADA Sup rvisory Control An Data Ac uisition (control centre)

SCL Sy tem Config ration L n uage [IEC 618 0-6]

SCP Sec re Co y (UNIX service)

SDH Sy c ronou Digital Hierarc y [ITU-T]

SEM Sec rity Event Management

SFP Smal Form-f actor Plu ga le (Ethernet

SGAM Smart Grid Arc itecture Model [CEN-CENELEC-ETSI Smart Grid Co rdination

Group]

SHDSL Sin le p ir Hig sp ed Digital Line Subs rib r

SIM Sec rity Information Management

SIP Ses ion Initiation Protocol [RFC 3 61]

SIPS Sy tem Integrity Protection Sc emes (refer ed to as WAMPAC in this

doc ment

SMV Sampled Me s rement Values [IEC 618 0-7-2, IEC 618 0-9-2]

SNMP Simple Network Management Protocol [RFC 3 10]

SNTP Simple Network Time Protocol [RFC 4 3 ]

s PDC s bstation Phasor Data Con entrator

SV Sampled Values (of c r ent an voltage) [IEC 618 0-9-2]

TM Terminal Multiplexer (SDH/SONET)

TPDU Tran p rt Protocol Data Unit [ISO/IEC 7 9 ]

TSAP Tran p rt Service Ac es Point

TSPEC Traf fic des ription in IntServ

TTL Time To Live (IP, MPLS)

UMTS Universal Mo i e Telecommu ication Sy tem (3

rd

generation cel phone)

UNI User Network Interf ace [MEF]

UPSR Unidirectional Path Switc Rin

VCAT Virtual Con atenation (SDH/SONET NG) [G.7 4 ]

VCCV Virtual Circ it Con ectivity Verification [RFC 5 8 ]

VDSL Very hig sp ed Digital Subs rib r Line (ITU-T last mie)

VID VLAN identif ier [IEEE 8 2.1Q]

VL Virtual L ased Line (also cal ed VPWS)

VPWS Virtual Private Wire Service (pseu o-wire)

VRRP Virtual Router Red n an y Protocol [RFC 5 9 ]

VT Virtual Tributary (SDH/SONET)

WAMPAC Wide Are Monitorin , Protection an Control

WDM Wavelen th Division Multiplexin

3.3 Network dia ram s mbols

This doc ment u es the s mb ls s own in Fig re 1

IE C

4 Wide A rea Communication in ele trical uti ities

4.1 Ex c tiv s mmary

The electrical grid is p rt of the critical infastru ture of a cou try The commu ication

network on whic the grid rel es mu t have an even hig er avai a i ty than the grid itself, an

has to s rvive contin en ies an blac outs in order to ke p the grid stable an restore the

grid as q ic ly as p s ible, even f rom a complete blac start

In the p st, o erational commu ication ( ele rotection, telecontrol, o erational voice, etc.)

was se arate f rom enterprise commu ication (energ management, mai , internet, comp ny

tele hone, etc.) In p rtic lar, tele hones in s bstation an control centres were in e en ent

f rom the publc tele hone s stem an u ed dedicated p werl ne, microwave or radio l n s

The publ c tele hone s stem was u tru ted sin e it would bre k down in case of wide-are

emergen y, d e to the n mb r of simultane u cal s; it would ru on b teries for only a few

hours after losin the grid, whic is in uf ficient to recover f om a lan -wide blac out

The o erational network u ed ro u t tec nologies f rom the esta l s ed tele hon tec nolog

(e.g ATM, SDH/SONET) that g arante d b n width, deterministic laten y, predicta le j t er

an hig rela i ty throu h red n ant p th The o erational network c an es l t le over the

ye rs, config ration c an es are seldom The p rson el in c arge of the legac network are

proficient in this tec nolog , whic is however b comin o solete

With the de loyment of protocols u in p c et switc in s c as IEC 618 0, the o erational

tele hony network had to tran p rt varia le size, sp radic p c et traf fic in ad ition to its

f i ed-size c clc traff i , a tas that it was not desig ed f or an that it did not p r orm ef ficiently

New generation SDH/SONET NG al ows more eff i ient p c et tran p rt De loyment of this

tec nolog wi contin e for some ye rs, esp cial y to s p ort o erational traf fic, but its

f urther develo ment (e.g OTN) is u certain

New actors ap e red as Smart Grid develo ed (e.g distributed generation, deman side

management, etc.), letin the o erational p c et-switc ed traff i grow

On the other han , the enterprise network b sed on internet tec nologies (emai, bu ines

servers, internet ac es , etc.) grew exp nential y, req irin b n width wel in ex es of what

the o erational network ne d

The p rson el in c arge of the enterprise network have a b c grou d oriented toward

informatic They are more exp rien ed in managin large amou ts of u ers an data but les

ac uainted with the req irements – esp cial y temp ral an de en a i ty asp cts – of

o erational network

Therefore, a divergen e can exist b twe n the phi oso h of the de artment managin

traditional o erational network an the de artment managin the enterprise network

Mergin the de artments could b as diff i ult as mergin the inf rastru tures

Tec nology ad an es an tec nical neces ity are not the only motors f or c an e The

l f esp n of of fice eq ipment (5 to 10 ye rs) is sig if i antly s orter than the l f esp n of

tele rotection eq ipment (10 to 2 ye rs) Early o soles en e wi cau e ad itional costs

d rin the l fe c cle, with l t le b nefit f or the b sic tele rotection fu ction Total cost of

owners ip is what cou ts

The contin ou tec nological evolution le d to heterogene u network , even in the rare

cases that started on the gre n f ield Network con ist of clu ters of hardware fom the same

man facturer, inter acin over a red ced n mb r of sp cial y en ine red devices with other

clu ters f rom other man facturers, tec nologies an vintage Whi e in network within

s bstation intero era i ty of eq ipment is a major in entive, intero era i ty within WAN

clu ters is not a primary go l, an interc an e bi ty of eq ipment b twe n man facturers is

not a req irement – as lon as sp res are avai a le

Uti ties se savin s an e rnin s by mergin the o erational network with the enterprise

network With the in tal ation of o tical f ib rs with ex es b n width, they can also of fer

commercial services to Telecom comp nies an Internet Service Providers (ISPs)

Mergin the network makes sen e as lon as the non-o erational services do not

compromise o eration In e d, enterprise an commercial commu ication s ould not b a le

to influen e the o erational commu ication, either throu h in or ect mes ages or throu h

ex au tion of s ared resources In p rtic lar, the dy amic services of enterprise network

require f req ent h man intervention for fixes an updates This presents a major ris for the

grid when s arin the network, a ris the uti ties mu t weig again t the economic of u in

publ c network

The traditional ph sical se aration b twe n o erational, enterprise an commercial data can

p rtial y b enforced by virtual private network (VPNs) an by priority management However,

sin e the same ph sical medium car ies al traf fic, the o erational network wi dictate the

de en a i ty req irements of the enterprise an commercial network , p s ibly af fectin their

economic

This not only a pl es to b n width an proces in p wer al ocation, but also to al resources

s c as b tery b c up, maintenan e te m de loyment an sp res disp sition

Therefore, a uti ty has to b lan e the b nef its of mergin the inf rastru tures again t the

disad antages of imp sin the strict req irements of o erational network onto the whole

infastru ture, an may c o se to ke p the network se arated

Cost savin s also lead uti ty comp nies, esp cial y smal ones, to outsource commu ication

entirely to a Telecom comp n or Internet Service Provider u der a Service L vel Agre ment

(SLA) Even if the service provider is a tru ted entity, its bu ines go ls are dif ferent f om that

of the uti ties an its o eration is not tran p rent to the electrical uti ty Sin e its te ms are

not f ami ar with grid contin en ies, they are les aware of the imp rtan e of o erational data,

whic they of ten do not identify as s c

Here again, a uti ty has to c o se b twe n ke pin the infastru ture in the owners ip of the

uti ty an outsourcin the network

The network b comes v lnera le to c b r at ac s when it sp n over publ c grou d, b lon s

to third-p rty providers an con ects in some places directly or in irectly to the publ c internet

The protection of the infastru ture go s b yon what traditional commu ication ne d A

virtual se aration may b in uff i ient, so c b r sec rity protocols b come imp rtant, p s ibly

at several levels, de en in on the tru ted entities In b th owned an outsourced network ,

uti ties mu t en ure c b rsec rity p lcies an arc itectures sometimes ex e din the

regulatory req irements (NERC for example)

Only contin ou monitorin an sp radic exercisin of contin en ies with the network te m

can g arante grid avai a i ty Cost pres ure can ot b an ex u e for f air we ther solution ,

whic later on ne d costly exten ion Theref ore, the evaluation of network solution mu t

con ider comprehen ive Op ration An Maintenan e (OAM) an simulation to ls

These g idel nes are inten ed to give ad ice on the p tential con ict p ints:

Clau es 4, 5 an 6 de l with the network req irements of o erational protocols, most of them

are covered by IEC stan ard These are not the only critical services, as voice tran mis ion

f or o eration an maintenan e is in isp n a le for o eration

Clau e 7 des rib s the tec nologies f om the restricted viewp int of electrical uti ties It

mostly l sts what is imp rtant to k ow for proc rement an gives a preciation of their

u efulnes

Clau e 8 con ists of u e cases Some u e cases are s atered throu h the doc ment The

f irst u e case in 4.2 is an example that s ows the network asp cts of a smal grid

4.2 Us Ca e: ENDESA, And lusia (Spain)

A typical Distribution Sy tem Op rator (DSO) grid, the An alu ia (southern Sp in) grid sp n

a out a 10 km diameter an some 3 s bstation (Fig re 2)

Figure 2 – Substation loc tions in Andalusia

The network to olog is that of a p rtial y mes ed rin with sp kes an s brin s (Fig re 3)

IE C

Figure 3 – Top logy of the An alusia network

The northern half of the main rin is a traditional SDH network over VC3; the southern p rt of

the rin u es Gbits Metro Ethernet with brid es Sp kes an s brin s intercon ection u e a

variety of media: radio ln s, E1, 10 Mbit s Ethernet, etc Routers en ure IP con ectivity The

diversity of media, protocols an man facturers c aracterizes network that evolved over time

A sp cial case is the city of San Se astian with its Smart Grid infastru ture

The b sic services that the An alu ia network provides are:

Evaluation of the tec nologies con idered:

• In estments plan for the next 5 ye rs

• Life c cle f or in estments (arou d 10 ye rs)

• Priority to c oices with minor o erational costs

• Qual ty of the recommen ed solution s ould b simi ar in terms of the provided service

4.3 Typic l interfa e betwe n a s bstation a d the WAN

Substation con ect to a WAN throu h a Substation Ed e Node (SEN), whic ag regates a

n mb r of dif ferent traf fic stre ms b twe n the s bstation an the WAN It can serve as a

protocol con erter, multiplexer an as a network element, switc or router The SEN is

however not a pl cation-aware

Fig re 4 s ows a typical SEN in the eq ipment ca inet of a s bstation

IE C