TRAME

TRAME (TRAnsmission of MEssages)[1] was the name of the second computer network in the world similar to the internet to be used in an electric utility. Like the internet, the base technology was packet switching; it was developed by the electric utility ENHER in Barcelona. It was deployed by the same utility, first in Catalonia and Aragón, Spain, and later in other places. Its development started in 1974 and the first routers, called nodes at that time, were deployed by 1978.[2][3][4][5] The network was in operation until 2016 (38 years) with successive technological software and hardware updates.

Beginnings
Figure I: The TRAME network in 1980

In 1974, packet switching was a technology known only in research circles. The concept began in 1968 in association with the United States' Advanced Research Projects Agency (ARPA) research project ARPANET. The idea of applying the packet switching concept to electric utilities control communication networks first appeared in 1974 when the Swedish power utility Vattenfall started to create its TIDAS packet-switching network[6][7][8] and was followed by the Spanish electric utility ENHER,[9] which aimed to telecontrol and automate its high-voltage power grid. For this purpose, ENHER created a specific team of people to develop both the packet-switching network and the supervisory control and data acquisition (SCADA) system, also called the telecontrol system. By 1978 the first four TRAME[1] routers were available[2] and by 1980, eight of them were deployed and operating.[1][5][10] The printed circuit boards (PCBs) controlling the communication lines were connected to a shared memory PCB allowing them to exchange data and messages. The project was developed together with its main initial application, the Telecontrol or SCADA system SICL (Sistema Integral de Control Local)[11][12][13] with which initially they shared a very similar hardware. The maximum link capacity was 9600 bit/s, which in 1980 was the maximum possible on a 4 kHz wide voice channel at the time. These channels were the basic unit of the then-analog communication systems in use. By that time power utilities used either telephone calls or low speed (below 1200bit/s) dedicated links for telecontrol, typically shared among ten high-voltage electrical substations.

Services

The basic service provided by the TRAME[1] network was SCADA or Telecontrol to automate the high-voltage power grid, thus improving operational efficiency, which was until then operated manually with telephone communication between human operators. Each TRAME router was associated with one or more remote terminal units (RTUs) of the SICL telecontrol system.[11][12] It also had connected screens, and later PCs, located in electrical substations to interchange messages between them and with the Control Center[13] located in the well-known Casa Fuster in Barcelona. It was a kind of predecessor to today's e-mail. Later, in the 1990s, other protocols (X.25, IP) were developed to include corporate information technology (IT) terminals, company physical surveillance systems and other services. Additionally, applications and terminals were developed for the transmission of voice and video over the TRAME network.[14][15]

Protocols

The TRAME routing system,[3] like that of the original ARPANET, was based on the Bellman-Ford algorithm but with "split-horizon"[7] as in the Swedish TIDAS network, but with an original improvement.[3] This protocol allows optimal paths to be found in meshed networks for each packet to be transmitted, allowing the shared use of the same network by multiple services. In contrast, traditional circuit-switched technology used to establish dedicated circuits for each service or communication. The addressing of routers and terminals used a proprietary system with a 16-bit address; it would be the equivalent of the well-known IP (Internet Protocol) version 4 (IPv4), still in use on the internet today, which uses 32-bit addresses. It is necessary to take into account that in 1978, the IPv4 protocol did not yet exist since the IPv4 version used on the internet did not appear until 1981, and in fact, did not reach the general public until much later.

The line protocols were also proprietary and were called UCL (Unidad de Control de Línea, 'line control unit'), which linked the routers together, and UTR (Unión TRAME-Remotas), the access protocol. They were designed to offer the highest quality of service required by the telecontrol/SCADA function in terms of data integrity and availability set by the International Electrotechnical Commission (IEC) IEC-870-5-1 and ANSI C37.1. standards, and because the protocol used at the time in corporate computer networks, HDLC (high-level data link control), did not offer enough quality for critical industrial applications.[16][17] Later on, other protocols like X.25 and IP were also made compatible with the aforementioned TRAME protocols. In 2000, the UTR protocol was replaced[18] by the international standard IEC 60870- 5-101/104.

Initially network flow control was based on the management of eight data priorities in head-of-the-line (HOL) waiting queues. Later and after some experimentation,[19] a flow control method based on a bit indicating route congestion and management of the gap between packets when accessing the network was adopted. This required measuring the capacity of the route bottleneck.[20][21][22][17][23] An end-to-end protocol was also added for some flows requiring order preservation like X.25.

Evolution

To last for 38 years, the technology had to endure intense evolution. There were essentially four TRAME generations which are summarized in the table.

ConceptTRAME 1TRAME 2TRAME 3TRAME +
Development started197419931998
Deployment started1978 (4 routers in operation)1988

1990 (fully deployed)

19951999
ProcessorZilog Z80Intel 80286Intel 80386i960CA

i960RM

Maximum link speed9600bit/s64kbit/s2Mbit/s10Mbit/s
Hardware16 processors sharing a common memory16 processors sharing a common memory16 processors sharing a common memory distributed among the 16 PCBs.A single processor with a shared multimaster 1Gbit/s bus to communicate I/O boards.
Hardware developer/manufacturerENHER/ISEL (INI)ENHER/ISEL (INI)DIMAT, S.A.DIMAT, S.A.
Addressing field14 bits · 256 routers (8 bits) · 64 terminals per router (6bits)16 bits · 256 routers (8 bits) · 64 terminals per router (6bits) · 4 interconnected networks (2 bits)16 bits, as in TRAME 216 bits, as in TRAME 2
Managed priorities8888
Inner link protocolsUCL (proprietary)UCL improved for 64kbit/s (proprietary)UCL improved for 64kbit/s (proprietary)UCL improved for 2Mbit/s (proprietary)
Access link protocolsUTR (proprietary)· IPv4 (1999) · X.25

· Proprietary: UTR

· International Electrotechnical Commission's IEC 60870-5-101/104

· IPv4 · X.25 · Ethernet

The same as for TRAME 3
Network size evolution1978: 4 routers

1980: 8 routers

1983: 27 routers

1987: 32 routers

1994: 50 routers2004: 222 routers

2014: some 3000 routers (several networks)

Router appearance
TRAME 1 router
TRAME 2 router
TRAME 3 router
TRAME + router

A description of the four generations of TRAME is provided below.

TRAME 1

The project began in 1974 and in 1978 a first network with four routers was already installed and in operation at the electric utility ENHER. In 1980, the network had eight nodes in operation (see Figure I). The hardware was based on the Zilog Z80 processor and had a multiprocessor structure with 16 processors sharing a common memory. The software was developed at ENHER's[9] headquarters located in the well-known Casa Fuster, Passeig de Gràcia, 132, Barcelona, using the Z80 assembly language. Beyond 1980 the software began to be written in C programming language and an HP64000 Logic Development System emulator was used for the purpose. The hardware was produced by ISEL, an INI (Instituto Nacional de Indústria) company.

The routing system was a variant of Bellman-Ford with split-horizon.[7] It was an improvement of the original ARPA network routing system consisting of an original update procedure which allowed for a faster reaction to changes. The distance function was the number of packets in the output waiting queues plus one.

The line protocols (UCL for internal lines linking routers and UTR for accessing the network) were designed to meet the stringent requirements set for telecontrol (SCADA) of high-voltage power networks (IEC-870-5-1 and ANSI C37.1 standards).

At the OSI transport layer, windows with a width of 1 to 8, depending on the required service, residing in the terminals were used.

Initially, addresses were only 14 bits long to address both the routers (called nodes by then) and the devices connected to them. They were made up of two fields, an 8-bit field to address the router and a 6-bit sub-address to address the terminals connected to it. The node address was assigned to the nodes and not to the ends of the links as in the internet.

The basic advantages of TRAME[1] over other technologies used in electric utilities at the time were in part due to the packet technology itself: ability to manage any network topology, automatic adaptability to topological and traffic changes, integration of different link technologies (digital or analog) and capacities in a single network, open and decentralized intercommunicability between users and devices, simultaneous communication with several users and locations from a single physical connection, and integrated network supervision. In fact, the network was provided from its inception with a supervision center[24] consisting of a computer and a synoptic board located at the company's headquarters (see Figure II).

Figure II: Synoptic panel used to monitor the TRAME network in 1987

But other advantages were due to the specific design of TRAME: high data integrity,[16][17] priority support for packets, and ease of including special protocols such as the many SCADA protocols in use at that time. All of the above resulted in improved quality of service, especially with respect to data availability and data integrity, and in the integration of services in a single network. Part of the evolution of its deployment can be seen in Figures II to IV.

Figure III: Topology of the TRAME network in 1986

TRAME 2

In 1990, TRAME 2 was fully deployed and TRAME 1 was replaced. The processor of the new hardware was Intel 80286 and the hardware structure and external appearance of the routers was very similar to that of TRAME 1. The software was written in C and the above-mentioned emulator continued to be used.

Improvements over TRAME 1 were the introduction of the standardized X.25 access protocol to enable the connection of corporate terminals to the network, the ability to handle the 64kbit/s of the new digital lines, increased switching capacity, and the introduction of an end-to-end protocol to avoid packet loss and clutter as required by X.25.

An important improvement was the possibility of using dual homing to increase terminal availability; they could be connected to the network by two access points. For the purpose, the terminals had two addresses, a primary and a secondary one.

Regarding addressing, in 1991 two bits were added to the addressing to indicate the network. The address space was thus increased to 16 bits and, in this way, up to four networks could be freely meshed as in a single one. This addressing scheme was maintained in subsequent versions of TRAME.

Figure IV: Topology of the TRAME network in 1994

TRAME 3

The hardware was again a multiprocessor structure with 16 processors sharing a common memory but the latter was not a separate PCB but instead was distributed among the 16 PCBs to avoid single points of failure. The interconnection of PCBs was done with a shared 40Mbit/s capacity multimaster bus designed and manufactured by DIMAT, S.A.. It also included a serial channel for maintenance, monitoring, reprogramming and resetting of the different modules through a terminal connected to them. The software was developed by ENHER in collaboration with DIMAT, S.A..

The routing algorithm remained the same, but the distance function was changed to a less dynamic one. A flow control procedure based route congestion metering and backwards indication to the source was introduced.[21][23]

Improvements over TRAME2 were IPv4 support, the introduction of an SNMP monitoring agent, a new flow control system, an improved distance metric that made the system less dynamic, and an autoexec task to periodically check hardware and software.

TRAME+

The hardware design was radically modified by moving to a single processor per node architecture as opposed to the traditional TRAME hardware. It had two alternative base modules of different capacity based on Intel i960CA and i960RM processors with a 1Gbit/s bus to communicate the different router boards. The number of physical interfaces was only ten (eight serial + two Ethernet (10BASE2 or 10BASE-T)) since Ethernet allowed for the connection of several devices on a single LAN. It also had a front service serial channel. By losing redundancy (a single processor per router) the node lost some availability over previous versions of TRAME. This was done for economical reasons stemming from the fact that the network was being extended to smaller substations where cost constraints are higher. Dual homing could help in places with more stringent availability requirements.

Improvements over TRAME 3 were the ability to handle 2Mbit/s capacity links, smaller and less expensive routers, access by Ethernet and standard protocols, and the change from the proprietary UTR protocol to the internationally standardized ones for SCADA systems (IEC 60870-5-101 and IEC 60870-5-104) with an original adaptation to packet-switched networks.[18][25]

References
  1. Selga, J.M. « TRAME: A Packet Switching Computer Network for Power Systems». Proceedings of CIGRE (Conference Internationale del Grandes Reseaux Electríques), Paris Session 1978, Paper 35-03, September 1978.
  2. Hoffmann M.G. and Selga J.M. «Line Control Unit for a Packet Switching Network». Proceedings of MIMI. Zurich, June 1978.
  3. Selga J.M. and Xampeny J. «Flow-Adaptive Updating Procedure for Dynamic Routing. Comparative Simulation Results». Proceedings of IEEE International Conference on Communications (ICC'80), pages. 23.6.1 to 23.6.6., Seattle (WA), USA, 1980.
  4. Selga J., Rivera J., Xampeny J. «Modelos analíticos y de simulación utilizados para el diseño de la red de conmutación de mensajes de ENHER». I Symposium Nacional sobre Modelado y Simulación en la Indústría y Servicios Públicos. Sevilla., 7–9 May 1980.
  5. Selga, J.M., Rivera.J., Xampeny, J. «Red TRAME de conmutación de paquetes». Revista Novática, Vol. VII, num.37, 1981.
  6. Jerlhagen, T. and B. Leander, B. , “A Message Switching Network Designed for Data Communication and Remote Control”, Proceedings of CIGRE, Paris France, Paper 35-01, August 1974.
  7. Cegrell, T. «[A Routing Procedure for the TIDAS Message-Switching Network]». IEEE Transactions on Communications, Volume 23, Issue 6, pag. 575-585, June 1975
  8. Kaijser, Arne (2010). "The Use of Computers for Controlling Electricity Flows in Sweden 1950-1980" in "History of Nordic Computing 3: Third IFIP WG9.7 Conference, HiNC3". Stockholm, Sweden: Springer. pp. 28–33. ISBN 978-3-642-23315-9.
  9. Sánchez i Vilanova, Llorenç. L’aventura hidroelèctrica de la Ribagorçana-ENHER i la seva influència en la transformació sòcio-econòmica de l’Alta Ribagorça”, Història i Cultura de l’Alta Ribagorça-Volum I (in catalan). La Pobla de Segur: Printer: Casa Torres S.A., Dipòsit legal: L-679-1991, Edits: Associació d’Amics de l’Alta Ribagorça., 1991. ISBN 84-604-0570-2.
  10. Selga, J.M., Rivera, J.,Xampeny, J. (Book coordinated by A. Alabau and J. Riera) (In Spanish. Red TRAME de conmutación de paquetes (Teleinformática y redes de computadores). Barcelona: Marcombo, p. 95 to 101, First Edition: 1982. Second Edition: 1984, ISBN 84-267-0427- 1.
  11. Ventosa, J.; Sanchez, M. ; Casals, A.; Rivera, J.; Xampeny, J. «An Integrated Station Local Control-Telecontrol System in the Scenario of an Overall Telecontrol System». IEEE Power Engineering Review, Volume PER-6, Issue:10, pages. 39 - 40, 1986
  12. Ventosa, J.; Sanchez, M. ; Casals, A.; Rivera, J.; Xampeny, J. «An Integrated Station Local Control-Telecontrol System, in the Scenario of an Overall Telecontrol System]». IEEE Transactions on Power Delivery, Volume: 1, Issue: 4, pages. 159-165, 1986
  13. Equipo TAC de ENHER «Sistema de Telecontrol integral para redes eléctricas. Una avanzada realización española». In Spanish. Mundo Electrónico. num.110., 1981.
  14. F. Álvarez-Cuevas Figueroa, M. Bertran, Oller, F., J. M. Selga, «Voice Synchronization in Packet Switching Networks». IEEE Network Magazine, Volume 7; Issue: 5, p. 20-25, September 1993
  15. J. Dalmau and J.M.Selga «Slow-Video Network». CIGRE SC35 Colloquium in Madrid, September 1995.
  16. Selga, J.M. and Rivera «HDLC reliability and the FRBS method to improve it». IEEE Seventh Data Communications Symposium Mexico, 1981.
  17. Selga, Josep Maria «Contribucions al disseny i optimització de xarxes d’ordinadors» (Contributions to the design and optimization of computer networks). In Catalan. Doctoral Thesis, September 1985.
  18. Cabezas,R.;Selga,J.M.;Samitier,C. «Experience in the implementation of a Telecontrol network based on the IP technology». Proceedings of CIGRE (Conference Internationale des Grandes Reseaux Electriques) Paris Session, Paper 35-201, September 2000.
  19. Selga J., Xampeny J. «Congestion Control Method for the TRAME network. Description and Simulation Results». IEEE Proceedings of the Mediterranean Electrotechnical Conference MELECON'83. Athens(Greece), 23–25 May 1983.
  20. Selga, J.M. «Optimum Control of Packet Switching Networks». Electronic Letters. Vol.19., pag. 794-795,15 September 1983.
  21. Selga, J.M. «Flow Control Method for Packet Networks». Proceedings of the 8th International Conference on Computer Communications ICCC 1986. München, p. 625-630,15-19 September 1986.
  22. Selga, J.M., Altemir, S. «Uso de redes de conmutación de paquetes en telecontrol» (In Spanish). Proceedings de les Primeres Jornades sobre Telecontrol Industrial. Escola Universitària d'Enginyeria Tècnica Industrial de Barcelona, 3–5 April 1984.
  23. Selga, J.M. «Optimización del uso de redes de conmutación de paquetes» (In Spanish). VI Congreso de Informática y Automática. Madrid., 15–18 October 1985.
  24. Selga J.M., Rivera J., Urquizu M., Sirvent J.C. «Centro de supervisión de una red de computadores: aplicación a la red TRAME de ENHER» (In Spanish). Mundo Electrónico. Num 130, June 1983.
  25. Cabezas,R.;Selga,J.M.;Samitier,C. «A new generation of packet switch designed for the Integration of operational services». CIGRE Colloquium in Krakow, Poland, 13 October 1999.
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.