This part of IEC 61850, which is a Technical Report, is intended for an audience familiar with network communication and/or IEC 61850-based systems and particularly for substation protection and control equipment vendors, network equipment vendors and system integrators.

This Technical Report focuses on engineering a local area network limited to the requirements of IEC 61850-based substation automation. It outlines the advantages and disadvantages of different approaches to network topology, redundancy, clock synchronization, etc. so that the network designer can make educated decisions. In addition, this report outlines possible improvements to both substation automation and networking equipment.

This Technical Report addresses the most critical aspects of IEC 61850, such as protection related to tripping over the network. This Technical Report addresses in particular the multicast data transfer of large volumes of sampled values (SV) from merging units (MUs). It also considers the high precision clock synchronization and “seamless” guaranteed transport of data across the network under failure conditions that is central to the process bus concept.

This Technical Report is not a tutorial on networking or on IEC 61850. Rather, it references and summarizes standards and publications to assist the engineers. Many publications discuss the Ethernet technology but do not address the networks in terms of substation automation. Therefore, many technologies and options have been ignored, since they were not considered relevant for a future-proof substation automation network design.

This Technical Report does not address network security.

This Technical Report does not address substation-to-substation communication, or substation to control centre communication. Inter-substation communication involves WAN technologies other than Ethernet, but when it uses Ethernet on layer 2, parts of this report can be applied. For inter-substation communication which uses exclusively the routable Internet Protocol, more adapted guidelines are in discussion within IEC TC 57, especially in documents IEC/TR 61850-90-1, IEC 61850-90-21, and IEC/TR 61850-90-5, which will be addressed in the WAN engineering guidelines, IEC 61850-90-122.

This Technical Report does not dispense the responsible system integrator from an analysis of the actual application configuration, which is the base for a dependable system.

PDF Catalog

PDF Pages	PDF Title
4	CONTENTS
14	FOREWORD
16	INTRODUCTION
17	1 Scope
18	2 Normative references
21	3 Terms, definitions, abbreviations and conventions 3.1 Terms and definitions
24	3.2 Abbreviations
27	3.3 Conventions 3.3.1 Network diagram symbols
28	3.3.2 Port and link symbols 3.3.3 Bridges symbols Figures Figure 1 – Network symbols Figure 2 – Port symbols
29	4 Overview of IEC 61850 networks 4.1 Logical allocation of functions and interfaces Figure 3 – Bridge symbol as beam Figure 4 – Bridge symbol as bus
30	Figure 5 – Levels and logical interfaces in substation automation systems Tables Table 1 – IEC 61850-5 interface definitions
31	4.2 IEC 61850 protocol stack 4.2.1 General 4.2.2 IEC 61850 traffic classes Figure 6 – IEC 61850 protocol stack
32	4.2.3 MMS protocol 4.2.4 GOOSE protocol Figure 7 – MMS protocol time/distance chart
33	Figure 8 – GOOSE protocol time/distance chart
34	4.2.5 SV protocol 4.3 Station bus and process bus Figure 9 – GOOSE protocol time chart Figure 10 – Example of SV traffic (4 800 Hz)
35	Figure 11 – Station bus, process bus and traffic example
36	5 Network design checklist 5.1 Design principles 5.2 Engineering flow
37	5.3 Checklist to be observed 5.3.1 Summary Figure 12 – Example of engineering flow
38	5.3.2 Environmental issues 5.3.3 EMI immunity 5.3.4 Form factor 5.3.5 Physical media 5.3.6 Substation application and network topology
39	5.3.7 Redundancy 5.3.8 Reliability, availability, maintainability 5.3.9 Logical data flows and traffic patterns 5.3.10 Latency for different types of traffic 5.3.11 Performance
40	5.3.12 Network management 5.3.13 Network supervision 5.3.14 Time synchronization and accuracy 5.3.15 Remote connectivity 5.3.16 Cyber security
41	5.3.17 Scalability, upgradeability and future-proof 5.3.18 Testing 5.3.19 Cost 6 Ethernet technology for substations 6.1 Ethernet subset for substation automation 6.2 Topology
42	Figure 13 – Ethernet local area network (with redundant links) Figure 14 – Switch with copper (RJ45) ports)
43	6.3 Physical layer 6.3.1 Data rate and medium 6.3.2 Full-duplex communication and auto-negotiation 6.3.3 Copper cabling at 100 Mbit/s
44	6.3.4 Optical cabling at 100 Mbit/s (100BASE-FX) Figure 15 – RJ45 connector
45	Figure 16 – LC connector
46	6.3.5 Optical cabling at 1 Gbit/s (1000BASE-LX) 6.3.6 Copper cabling at 1 Gbit/s 6.4 Link layer 6.4.1 Unicast and multicast MAC addresses Figure 17 – Switch with optical fibres (LC connectors)
47	6.4.2 Link layer and bridges 6.4.3 Bridging nodes 6.4.4 Loop prevention and RSTP
48	Figure 18 – RSTP principle
49	6.4.5 Traffic control in the bridges 6.4.6 Unicast MAC address filtering 6.4.7 Multicast MAC address filtering
50	6.4.8 Virtual LANs (VLANS) traffic control
51	Figure 19 – IEEE 802.3 frame format without and with VLAN tagging
53	Table 2 – Example of port ingress setting table
54	Table 3 – Example of port egress settings
55	6.4.9 Comparison VLAN versus multicast filtering 6.4.10 Layer 2 redundancy protocols Table 4 – Advantages and drawbacks of VLAN versus multicast filtering
56	Figure 20 – PRP principle
58	Figure 21 – HSR principle
59	6.5 Network layer 6.5.1 Internet protocol 6.5.2 IP public and private addresses Figure 22 – HSR and PRP coupling (multicast)
60	6.5.3 Subnet masks Table 5 – IANA private IP address blocks (copied from RFC 1918) Table 6 – IP address and mask example
61	6.5.4 Network address translation 7 Network and substation topologies 7.1 General rule
62	7.2 Reference topologies and network redundancy Figure 23 – Mapping of electrical grid to data network topology
64	Table 7 – Summary of reference topologies
65	Table 8 – Reference topologies and redundancy protocols used
66	7.3 Reference topologies 7.3.1 Station bus topologies Figure 24 – Station bus as single bridge Table 9 – Station bus as single bridge
67	Figure 25 – Station bus as hierarchical star Table 10 – Station bus as hierarchical star
68	Figure 26 – Station bus as dual star with PRP Table 11 – Station bus as dual star
69	Figure 27 – Station bus as ring of RSTP bridges Table 12 – Station bus as ring
70	Figure 28 – Station bus as separated Main 1 (Bus 1) and Main 2 (Bus 2) LANs
71	Table 13 – Station bus as separated Main 1 and Main 2 protection
72	Figure 29 – Station bus as ring of HSR bridging nodes Table 14 – Station bus as ring of bridging nodes
73	Figure 30 – Station bus as ring and subrings with RSTP Table 15 – Station bus as ring and subrings
74	Figure 31 – Station bus as parallel rings with bridging nodes
75	Figure 32 – Station bus as parallel HSR rings Table 16 – Station bus as parallel rings
76	Figure 33 – Station bus as hierarchical rings with RSTP bridging nodes Table 17 – Station bus as parallel HSR rings
77	Table 18 – Station bus as ring of rings with RSTP
78	Figure 34 – Station bus as hierarchical rings with HSR bridging nodes Table 19 – Station bus as ring of rings with HSR
79	7.3.2 Process bus and attachment of primary equipment Figure 35 – Station bus as ring and subrings with HSR Table 20 – Station bus as ring and subrings with HSR
80	Figure 36 – Double busbar bay with directly attached sensors
81	Figure 37 – Double busbar bay with SAMUs and process bus
82	Figure 38 – Double busbar bay with ECT/EVTs and process bus
83	Figure 39 – 1 ½ CB diameter with conventional, non-redundant attachment
84	Figure 40 – 1 ½ CB diameter with SAMUs and process bus
85	Figure 41 – 1 ½ CB diameter with ECT/EVT and process bus
86	Figure 42 – Process bus as connection of PIA and PIB (non-redundant protection) Table 21 – Process bus as connection of PIA and PIB
87	Figure 43 – Process bus as single star (not redundant protection)
88	Table 22 – Process bus as single star
89	Figure 44 – Process bus as dual star Table 23 – Process bus as dual star
90	Figure 45 – Process bus as a single bridge (no protection redundancy)
91	Table 24 – Process bus as single bridge
92	Figure 46 – Process bus as separated LANs for main 1 and main 2 Table 25 – Process bus as separated LANs
93	Figure 47 – Process bus as ring of HSR nodes Table 26 – Process bus as simple ring
94	7.3.3 Station bus and process bus connection Table 27 – Advantages and drawbacks of physical separation Table 28 – Advantages and drawbacks of logical separation
95	Figure 48 – Process bus as star to merging units and station bus as RSTP ring Table 29 – Process bus as star to merging units
97	Figure 49 – Station bus and process bus as rings connected by a router Table 30 – Connection of station bus to process bus by routers
98	Figure 50 – Station bus ring and process bus ring with HSR
99	Table 31 – Connection of station bus to process bus by RedBoxes
100	8 Addressing in the substation 8.1 Network IP address plan for substations 8.1.1 General structure Figure 51 – Station bus as dual PRP ring and process bus as HSR ring Table 32 – Connection of duplicated station bus to process bus by RedBoxes
101	8.1.2 IP address allocation of NET Table 33 – Example IP address allocation of NET
102	8.1.3 IP address allocation of BAY 8.1.4 IP address allocation of device Table 34 – Example IP address allocation of BAY Table 35 – Example IP address allocation of device
103	8.1.5 IP address allocation of devices with PRP 8.2 Routers and GOOSE / SV traffic 8.3 Communication outside the substation Table 36 – Example IP address allocation of switches in PRP
104	9 Application parameters 9.1 MMS parameters 9.2 GOOSE parameters 9.3 SV parameters
105	10 Performance 10.1 Station bus performance 10.1.1 Logical data flows and traffic patterns Table 37 – IEC 61850-5 interface traffic
106	10.1.2 GOOSE traffic estimation 10.1.3 MMS traffic estimation Table 38 – Message types and addresses
107	10.1.4 station bus measurements Figure 52 – Station bus used for the measurements Figure 53 – Typical traffic (packet/s) on the station bus
108	10.2 Process bus performance 11 Latency 11.1 Application requirements
109	11.2 Latency requirements for different types of traffic 11.2.1 Latency requirements in IEC 61850-5 11.2.2 Latencies of physical paths 11.2.3 Latencies of bridges Table 39 – Transfer time requirements of IEC 61850-5 Table 40 – Elapsed time for an IEEE 802.3 frame to traverse the physical medium
110	11.2.4 Latency and hop counts 11.2.5 Network latency budget Table 41 – Delay for an IEEE 802.3 frame to ingress or to egress a port
111	11.2.6 Example of traffic delays 11.2.7 Engineering a network for IEC 61850 protection Table 42 – Latencies caused by waiting for a lower-priority frame to egress a port
112	12 Network traffic control 12.1 Factors that affect performance 12.1.1 Influencing factors 12.1.2 Traffic reduction Figure 54 – Generic multicast domains
113	12.1.3 Example of traffic reduction scheme
114	12.1.4 Multicast domains in a combined station bus and process bus network Figure 55 – Traffic patterns
115	12.2 Traffic control by VLANs 12.2.1 Trunk traffic reduction by VLANs Figure 56 – Multicast domains for a combined process bus and station bus
116	12.2.2 VLAN usage 12.2.3 VLAN handling at the IEDs 12.2.4 Example of correct VLAN configuration
117	12.2.5 Example of incorrect VLAN configuration Figure 57 – Bridges with correct VLAN configuration
118	Figure 58 – Bridges with poor VLAN configuration
119	12.2.6 Retaining priority throughout the network 12.2.7 Traffic filtering with VLANs
120	12.3 Traffic control by multicast filtering 12.3.1 Trunk traffic reduction by multicast filtering Figure 59 – Bridges with traffic segmentation through VLAN configuration
121	12.3.2 Multicast/VLAN management and redundancy protocol reconfiguration 12.3.3 Physical topologies and multicast management implications Figure 60 – Station bus separated into multicast domains by voltage level
122	Figure 61 – Multicast traffic on an RSTP ring
123	Figure 62 – RSTP station bus and HSR ring Figure 63 – RSTP station bus and HSR process bus
124	12.4 Configuration support from tools and SCD files 13 Dependability 13.1 Resiliency requirements
125	13.2 Availability and reliability requirements 13.3 Recovery time requirements 13.4 Maintainability requirements
126	13.5 Dependability calculations 13.6 Risk analysis attached to “unwanted events”
127	14 Time services 14.1 Clock synchronization and accuracy requirements 14.2 Global time sources Table 43 – Synchronization classes of IEC 61850-5
128	14.3 Time scales and leap seconds
129	14.4 Epoch 14.5 Time scales in IEC 61850
130	14.6 Synchronization mechanisms in IEC 61850 14.6.1 Clock synchronization protocols Table 44 – Time representations
131	Figure 64 – Clock synchronization channels
132	14.6.2 1 PPS 14.6.3 IRIG-B 14.6.4 NTP/SNTP clock synchronization for IEC 61850-8-1 (station bus) Figure 65 – 1 PPS synchronisation
133	Figure 66 – SNTP clock synchronization and delay measurement
134	14.6.5 PTP (IEC 61588) synchronization
135	Figure 67 – PTP elements
136	Figure 68 – PTP one-step clock synchronization and delay measurement
138	Figure 69 – PTP two-step clock synchronization and delay measurement
139	14.6.6 PTP clock synchronization and IEC 62439-3:2012
141	Figure 70 – Clocks in a PRP network coupled by BCs with an HSR ring
142	14.6.7 IEEE C37.238-2011 Power profile
143	14.7 PTP network engineering 14.7.1 PTP reference clock location Figure 71 – C37.238-specific TLV
144	14.7.2 PTP connection of station bus and process bus Figure 72 – Hierarchy of clocks
145	14.7.3 Merging units synchronization 15 Network security 16 Network management 16.1 Protocols for network management
146	16.2 Network management tool 16.3 Network diagnostic tool
147	17 Remote connectivity 18 Network testing 18.1 Introduction to testing Figure 73 – Quality assurance stages (copied from IEC 61850-4)
148	18.2 Environmental type testing 18.3 Conformance testing 18.3.1 Protocols subject to conformance testing Table 45 – Standards applicable to network elements Table 45 – Standards applicable to network elements
149	18.3.2 Integrator acceptance and verification testing 18.3.3 Simple verification test set-up Figure 74 – Test set-up for verification test
150	18.3.4 Simple VLAN handling test 18.3.5 Simple priority tagging test
151	18.3.6 Simple multicast handling test 18.3.7 Simple RSTP recovery test
152	18.3.8 Simple HSR test 18.3.9 Simple PRP test 18.3.10 Simple PTP test 18.4 Factory and site acceptance testing
153	19 IEC 61850 bridge and port object model 19.1 Purpose
154	19.2 Bridge model 19.2.1 Simple model
155	Figure 75 – Multiport device model
156	19.2.2 Bridge Logical Node linking 19.3 Clock model 19.3.1 IEC 61588 datasets Figure 76 – Linking of bridge objects
157	19.3.2 Clock objects 19.3.3 Simple clock model
158	19.3.4 Linking of clock objects Figure 77 – Clock model
159	19.4 Autogenerated IEC 61850 objects 19.4.1 General 19.4.2 Abbreviated terms used in data object names Figure 78 – Linking of clock objects Table 46 – Normative abbreviations for data object names
160	19.4.3 Logical nodes Figure 79 – Class diagram LogicalNodes_90_4::LogicalNodes_90_4
161	Figure 80 – Class diagram LNGroupL::LNGroupLExt
162	Figure 81 – Class diagram LNGroupL::LNGroupLNew
163	Table 47 – Data objects of LNGroupL::LPHDExt
164	Table 48 – Data objects of LNGroupL::LBRI
165	Figure 82 – Usage of VLAN filtering Table 49 – Data objects of LNGroupL::LCCF
166	Table 50 – Data objects of LNGroupL::LCCHExt
167	Table 51 – Data objects of LNGroupL::PortBindingLN Table 52 – Data objects of LNGroupL::LPCP
168	Table 53 – Data objects of LNGroupL::LPLD
170	Table 54 – Data objects of LNGroupL::LBSP Table 55 – Data objects of LNGroupL::LTIMExt
171	Figure 83 – Usage of clock references
172	Table 56 – Data objects of LNGroupL::LTMSExt Table 57 – Data objects of LNGroupL::LTPC
173	19.4.4 Data semantics Table 58 – Data objects of LNGroupL::LTPP Table 59 – Attributes defined on classes of LogicalNodes_90_4 package
176	19.4.5 Enumerated data attribute types Table 60 – Literals of DOEnums_90_4::ChannelRedundancyKind
177	Figure 84 – Class diagram DetailedDiagram::DOEnums_90_4 Table 61 – Literals of DOEnums_90_4::LeapSecondKind Table 62 – Literals of DOEnums_90_4::RstpStateKind
178	19.4.6 SCL enumerations 19.4.7 Common data class specifications Figure 85 – Class diagram CommonDataClasses_90_4::CommonDataClasses_90_4
179	Figure 86 – Class diagram CDCStatusInfo::CDCStatusInfo Table 63 – Clock grandmaster status common data class definition
180	Table 64 – Clock port status common data class definition
182	Figure 87 – Class diagram CDCStatusSet::CDCStatusSet Table 65 – Clock ordinary settings common data class definition
184	19.4.8 Enumerated types Table 66 – VLAN filters common data class definition Table 67 – Literals of DAEnums_90_4::VlanTagKind
185	19.4.9 SCL enumerations 19.5 Mapping of bridge objects to SNMP 19.5.1 Mapping of LLN0 and LPHD attributes to SNMP Table 68 – Mapping of LLN0 and LPHD attributes to SNMP
186	19.5.2 Mapping of LBRI attributes to SNMP for bridges 19.5.3 Mapping of LPCP attributes to SNMP for bridges 19.5.4 Mapping of LPLD attributes to SNMP for bridges Table 69 – Mapping of LBRI and LBSP attributes to SNMP for bridges Table 70 – Mapping of LPCP attributes to SNMP for bridges
187	19.5.5 Mapping of HSR/PRP link redundancy entity to SNMP Table 71 – Mapping of LPLD attributes to SNMP for bridges
188	19.6 Mapping of clock objects to the C37.238 SNMP MIB Table 72 – Mapping of LCCH attributes for SNMP for HSR/PRP LREs Table 73 – Mapping of clock objects in IEC 61850, IEC 61588 and IEEE C37.238
191	19.7 Machine-readable description of the bridge objects 19.7.1 Method and examples 19.7.2 Four-port bridge Figure 88 – Four-port bridge
201	19.7.3 Simple IED with PTP Figure 89 – Simple IED with PTP but no LLDP support
208	19.7.4 RedBox wit HSR
209	Figure 90 – RedBox with LLDP but no PTP
216	Annex A (informative) Case study – Process bus configuration for busbar protection system
217	Figure A.1 – Preconditions for the process bus configuration example Table A.1 – Summary of expected latencies
220	Annex B (informative) Case study – Simple Topologies (Transener/Transba, Argentina) Figure B.1 – First Ethernet-based Transba substation automation network
221	Figure B.2 – Transba SAS architecture
222	Figure B.3 – Transener substation automation network
224	Figure B.4 – Transener SAS architecture – ET Esperanza
225	Figure B.5 – Transener 500 kV architecture – El Morejón
226	Figure B.6 – 500 kV kiosk topology
227	Figure B.7 – 33 kV kiosk topology
228	Annex C (informative) Case study – An IEC 61850 station bus (Powerlink, Australia) Figure C.1 – Example HV and LV single line diagram and IEDs
229	Table C.1 – Site categories HV Table C.2 – Site categories MV
230	Figure C.2 – HV bay and cabinet module Table C.3 – Building modules
234	Figure C.3 – Data network areas
235	Table C.4 – Network modules
236	Figure C.4 – Substation LAN topology
237	Figure C.5 – SAS Gen1 High level traffic flows
238	Figure C.6 – SCADA & gateway connection Figure C.7 – Station Core
240	Figure C.8 – Overall VLANs Figure C.9 – Three domains
241	Figure C.10 – One domain per diameter, bus zone and transformer protection Table C.5 – Domain assignment for three domains Table C.6 – Domain assignment for one domain per diameter
243	Table C.7 – Summary of expected latencies Table C.8 – Traffic types and estimated network load
244	Annex D (informative) Case study – Station bus with VLANs (Trans-Africa, South Africa)
247	Figure D.1 – Conceptual topology of substationLAN network with redundancy
248	Figure D.2 – Detailed topology of substation LAN with redundancy
249	Table D.1 – VLAN numbering and allocation
250	Table D.2 – Prioritization selection for various applications
251	Figure D.3 – Original IPv4 Type of Service (ToS) octet Figure D.4 – Differentiated Services (DiffServ) codepoint field Table D.3 – Mapping of applications to service levels
252	Table D.4 – List of DiffServ codepoint field values Table D.5 – Example of DSCP to class of service mapping
253	Table D.6 – Example of DSCP mappings Table D.7 – Typical substation IP Address map (IP range: 10.0.16.0/21)
255	Table D.8 – SNMP MIBs applicable to substation devices
257	Table D.9 – Example of device naming Table D.10 – Example of interface addressing and allocation
258	Table D.11 – Example of device access and SNMP assignment
259	Table D.12 – Example of hardware identification Table D.13 – Example of device name table Table D.14 – Example of firmware and software table
260	Table D.15 – Example of interface addressing and allocation Table D.16 – Example of network switch details
261	Table D.17 – Example of VLAN definitions Table D.18 – Example of IP routing Table D.19 – Example of QoS mapping
262	Table D.20 – Example of trunk and link aggregation table (void) Table D.21 – LAN switch port speed and duplex configuration
263	Table D.22 – LAN switch port security settings
264	Table D.23 – Example of DHCP snooping Table D.24 – Example of storm control table
265	Bibliography

Published By	Publication Date	Number of Pages
BSI	2013	268


PDF Format	Searchable	Printable	Preview Now

Status	Withdrawn
Title	Communication networks and systems for power utility automation – Network engineering guidelines
Publisher	BSI
Committee	PEL/57
Pages	268
Publication Date	2013-08-06
Withdrawn Date	2020-06-04
Replaced By	PD IEC TR 61850-90-4:2020
ISBN	978 0 580 79646 3
Standard Number	PD IEC/TR 61850-90-4:2013
Identical National Standard Of	IEC TR 61850-90-4:2013
Descriptors	Electric power transmission, Data transfer, Electric substations, Control systems, Information exchange, Telecommunication systems, Data transmission, Communication networks, Automatic control systems
ICS Codes	33.200 - Telecontrol. Telemetering

BSI PD IEC/TR 61850-90-4:2013

PDF Catalog