A Short History of Internet Protocols at CERN

Ben Segal / CERN IT-PDP-TE
April, 1995

Now that the Internet has exploded in popularity on a world wide scale, with a major component of its success (the World Wide Web) being developed at CERN, it seems a good time to look back and trace the history of the Internet at CERN. Even before the Web allowed Internet penetration in the most unexpected places, the presence of the Internet protocols at CERN had already encouraged their adoption not only in many other parts of Europe but also in such influential organizations as the ITU and ISO in Geneva.

Another reason for writing this history today is that it is almost exactly ten years ago that CERN named me as its first "TCP/IP Coordinator". The TCP/IP protocols (as Internet protocols were then called) had actually entered CERN a few years earlier, inside a Berkeley Unix system, but not too many people were aware of that event. In the computer networking arena, a period of 10-15 years represents several generations of technology evolution. Readers of this history will perhaps be surprised that in a period of only three years there can be developments that radically change the whole way that people think about computer communications. This has just happened with the Web (prototyped in 1990-1, fully accepted over 1993-4), but several related steps needed to take place beforehand to enable the Web's emergence. First of all, standards had to emerge in computer systems themselves, and in programming techniques. Next, standards were needed in network and computer hardware, with the accompanying price reductions. Finally there had to be a change in mentality, among both manufacturers and computer users, for them first to allow and then to insist that their systems should be able to communicate freely.

Another interesting element, apart from the rapidity of change, is the factor of accident or coincidence, often traceable to a personal event or a meeting of one or two people in critical circumstances. Bringing the Internet to CERN was not a simple business, although similar events probably occurred at other pioneer sites. Being very well acquainted with the people involved, the present author was ideally placed to observe the interplay of technical, personal and political elements at CERN that helped bring about a major part of today's Information Revolution.

In the Beginning - the 1970's
In the beginning was - chaos. In the same way that the theory of high energy physics interactions was itself in a chaotic state up until the early 1970's, so was the so-called area of "Data Communications" at CERN. The variety of different techniques, media and protocols used was staggering; open warfare existed between many manufacturers' proprietary systems, various home-made systems (including CERN's own "FOCUS" and "CERNET"), and the then rudimentary efforts at defining open or international standards. There were no general purpose Local Area Networks (LANs): each application used its own approach. The only really widespread CERN network at that time was "INDEX": a serial twisted pair system with a central Gandalf circuit switch, connecting some hundreds of "dumb" terminals via RS232 to a selection of accessible computer ports for interactive login.

CERNET, beginning in 1976, offered a fast file transfer service between a number of mainframes and minicomputers via 2Mbit/s serial lines using packet switching in a network of gateway nodes. Remote login (known as "virtual terminal service") was only supported to a single system, the central IBM mainframe. At the end of its ten year life CERNET supported 100 systems, including its own version of a LAN bridge, connecting some of CERN's first Ethernets. However, even though architecturally CERNET resembled ARPAnet, all its protocols had been developed independently. It was therefore doomed, though this was of course unknown at the beginning. Even if its designers had been in contact with Vint Cerf and company, there was no efficient way to run a transatlantic collaboration. Imagine a period without electronic mail... but no, this was only introduced at CERN to any extent at the beginning of the 1980's.

This was also a period without standards for computer systems themselves, not just communication systems. There are therefore few computer files from this period and we must rely on the written record. We must look back into our paper files, typed by our typists, back into a time when there were no PC's, no Macintoshes, no Unix and no C programming at CERN...

The Stage is Set - early 1980's
To my knowledge, the first time any "Internet Protocol" was used at CERN was during the second phase of the STELLA Satellite Communication Project, from 1981-83, when a satellite channel was used to link remote segments of two early local area networks (namely "CERNET", running between CERN and Pisa, and a Cambridge Ring network running between CERN and Rutherford Laboratory). This was certainly inspired by the ARPA IP model, known to the Italian members of the STELLA collaboration (CNUCE, Pisa) who had ARPA connections; nevertheless the STELLA internet protocol was independently implemented and a STELLA-specific higher-level protocol was deployed on top of it, not TCP. As the senior technical member of the CERN STELLA team, this development opened my eyes to the meaning and potential of an Internet network protocol.

Ethernet made its appearance at CERN at about that time (1983), when an initial stretch of the soon-to-be-famous yellow cable arrived to support a demonstration of the very advanced Symbolics machine from MIT, which actually ran ChaosNet, XNS and TCP/IP protocols if I remember correctly. Before that, starting in 1982, we had installed some Apollo Domain coaxial cables for a 12 Mbit/s token ring network running Apollo's proprietary protocol, the first to offer a real distributed file system as well as network virtual memory paged over the ring.

The Apollo workstations brought CERN firmly into the distributed computing business. They had no equal for large scientific and graphics applications. The fact that the system was proprietary seemed unimportant at that time: Unix based competitors like Sun and HP were outclassed, and Unix itself had little appeal for CERN physicists. A file-exchange gateway was made (by this author) between the Apollos and the home-grown CERNET network, sufficient to exchange data and program files with the central CERN mainframes and other CERNET hosts. Remote login was made, if desired, by terminal emulation via RS232 cables; other connections were improvised by other techniques, in the disjoint spirit of those times.

In 1983, for the first time, a Data Communications (DC) Group was set up in the CERN computing division (then "Data-handling Division" or "DD") under David Lord. Before that time, work on computer networking in DD had been carried out in several groups: I myself belonged to the Software (SW) Group, which had assigned me and several others to participate in DD's networking projects since 1970. All my work on STELLA had been sponsored in this way, for example. The new DC Group seemed to have a mandate to unify networking practices across the whole of CERN, but after a short time it became clear that this was not going to be done comprehensively. DC Group decided to leave major parts of the field to others while it concentrated on building a CERN-wide backbone infrastructure. Furthermore, following the political currents of the time, they laid a very formal stress on ISO standard networking, the only major exception being their support for DECnet. PC networking was ignored almost entirely; IBM mainframe networking (except for BITNET/EARN) as well as the developing fields of Unix and workstation-based networking all remained in SW Group. So did the pioneering work on electronic mail and news under Dietrich Wiegandt, which made CERN a European leader in this field. (From the early 1980's until about 1990 CERN acted as the Swiss backbone for USENET news and gatewayed all Swiss e-mail between the EUnet uucp network, BITNET, DECnet and the Internet). As these were precisely the areas in which the Internet protocols were to emerge, this choice led to a situation in which CERN's support for them would be marginal or ambiguous for several years to come.

It was from around 1984 that the wind began to change.

TCP/IP Introduced at CERN
In August, 1984 I wrote a proposal to the SW Group Leader, Les Robertson, for the establishment of a pilot project to install and evaluate TCP/IP protocols on some key non-Unix machines at CERN including the central IBM-VM mainframe and a VAX VMS system. This was to decide if TCP/IP could indeed solve the problem of heterogeneous connectivity between the newer open systems and the established proprietary ones. It also proposed to evaluate Xerox's XNS protocols as a possible alternative. The proposal was approved and the work led to acceptance of TCP/IP as the most promising solution, together with the use of "sockets" (pioneered by the BSD 4.x Unix system) as the recommended API.

In early 1985 I was appointed the "TCP/IP Coordinator" for CERN, as part of a formal agreement between SW Group (under Les Robertson) and DC Group (under its new leader, Brian Carpenter). Incorporating the latter's policy line, this document specifically restricted the scope of Internet protocols for use only within the CERN site. Under no circumstances were any external connections to be made using TCP/IP: here the ISO/DECnet monopoly still ruled supreme, and would do so until 1989.

Between 1985 and 1988, the coordinated introduction of TCP/IP within CERN made excellent progress, in spite of the small number of individuals involved. This was because the technologies concerned were basically simple and became steadily easier to buy and install.

A major step was taken in November 1985 when the credibility of the Internet protocols as implemented within CERN was sufficient to convince the management of the LEP/SPS controls group that the LEP control system, crucial for the operation of CERN's 27 km accelerator LEP then under construction, should use TCP/IP. This decision, made by P-G. Innocenti, Jacques Altaber and Pal Anderssen, combined with a later decision to use Unix-based systems, turned out to be essential for the success of LEP. The TCP/IP activity in LEP/SPS included a close collaboration with IBM's Yorktown Laboratory to support IP protocols on the IBM token ring network that had been chosen for the LEP control system.

Other main areas of progress were: a steady improvement of the TCP/IP installations on IBM-VM/CMS, from the first University of Wisconsin version (WISCNET) to the fully-supported IBM version that still runs today; the rapid spread of TCP/IP on VMS systems, using third-party software in the absence of any DEC product; the first support of IBM PC networking, starting with MIT's free TCP/IP software and migrating to its commercial descendant from FTP Software. This latter work, by Mike Gerard and Brian Henningsen (who would later move to ISO in Geneva), led directly to the very comprehensive support of PC and Macintosh networking that exists today at CERN using TCP/IP, Novell and Apple protocols. All this was accompanied by a rapid change from RS232 based terminal connections to the use of terminal servers and virtual Ethernet ports using TCP/IP or DEC-based protocols. This permitted either dumb terminals or workstation windows to be used for remote login sessions, and hence to today's X-Windows. In particular, starting from 3270 emulator software received from the University of Wisconsin and developed by myself for Apollo and Unix systems, a full-screen remote login facility was provided to the VM/CMS service; this software was then further developed by Mike Gerard and used as the basis for the Terminal Access Gateway service (TAG) which became a standard way for CERN users to access VM/CMS systems world-wide.

As late as September 1987, DD's Division Leader P. Zanella would still write officially to a perplexed user, with a copy to the then Director of Research, J. Thresher: "The TCP-IP networking is not a supported service." This illustrates the ambiguity of the situation (already referred to above) as these words were written at essentially the same time as another major step forward was made in the use of Unix and TCP/IP at CERN: the choice to use them for the new Cray XMP machine instead of Cray's well-established proprietary operating system COS and its associated Cray networking protocols. Suddenly, instead of asking "What use is Unix on a mainframe?" some users began to ask "Why not use Unix on everything?".

The Cray represented CERN's first "supercomputer" according to US military and commercial standards and a serious security system was erected around it. As part of this system, in 1987 I purchased the first two Cisco IP routers in Switzerland (perhaps in Europe?), to act as IP filters between CERN's public Ethernet and a new secure IP segment for the Cray. I had met the founder of "cisco systems", Len Bosack, at a Usenix exhibition in the USA in June 1987 and been very impressed with his router and this filtering feature. Cisco was a tiny company with about 20 employees at that time, and doing business with them was very informal. It was hard to foresee the extent to which they would come to dominate the router market, and the growth that the market would undergo. Unfortunately I did not purchase any Cisco shares when a little later they went public...

Birth of the European Internet
In November 1987 I received a visit from Daniel Karrenberg, the system manager of "mcvax", a celebrated machine at the Amsterdam Mathematics Centre that acted as the gateway for all transatlantic traffic between the US and European sides of the world-wide "USENET", the Unix users' network that carried most of the email and news of that time using a primitive protocol called "uucp". Daniel had hit on the idea of converting the European side ("EUnet") into an IP network, just as major parts of the US side of USENET were doing at that time. The news and mail would be redirected to run over TCP/IP (using the SMTP protocol), unnoticed by the users, but all the other Internet utilities "telnet", "ftp", etc. would become available as well, once Internet connectivity was established. Even better, Daniel had personal contacts with the right people at the NIC who would grant him Internet connect status when he needed it. All he was missing was a device to allow him to run IP over some of the EUnet lines that were using X.25 - did this exist? I reached for my Cisco catalogue and showed him the model number he needed. Within a few months the key EUnet sites in Europe were equipped with Cisco routers, with the PTT's, regulators and other potential inhibitors none the wiser. The European IP network was born without ceremony.

CERN Joins the Internet
In 1988, the DC Group in DD Division (later renamed CS Group in CN Division) finally agreed to take on the support of TCP/IP, and what had been a shoestring operation, run out of SW Group with a few friendly contacts here and there, became a properly staffed and organized activity. John Gamble became the new TCP/IP Coordinator, performing this task until quite recently; he had just returned from extended leave at the University of Geneva where he had helped to set up one of the very first campus-wide TCP/IP networks in Europe. A year later, CERN opened its first external connections to the Internet after a "big bang" in January 1989 to change all IP addresses to official ones. (Until then, CERN had used an illegal Class A address, Network 100, chosen by myself).

CERN's external Internet bandwidth flourished, with a growing system of links and routers managed by Olivier Martin and Jean-Michel Jouanigot. Concurrently with the growth of the new European IP network (later to be incorporated as "RIPE" within the previously ISO-dominated organization "RARE"), many other players in Europe and elsewhere were changing their attitudes. Prominent among these was IBM, who not only began to offer a good quality mainframe TCP/IP LAN connection product of their own but also began to encourage migration of their proprietary BITNET/EARN network towards IP instead of the much more restricted RSCS-based service. They even began a subsidy programme called EASINET to pay line charges for Internet connection of their European Supercomputer sites of which CERN was one. In this way, the principal link (1.5 Mbit/sec) between Europe and the USA was located at CERN and funded by IBM for several years during the important formative period of the Internet.

By 1990 CERN had become the largest Internet site in Europe and this fact, as mentioned above, positively influenced the acceptance and spread of Internet techniques both in Europe and elsewhere. Brian Carpenter, still the leader of CS Group, is today a member of the Internet Architecture Board and a well known speaker at Internet gatherings. Experts like Olivier Martin and Jean-Michel Jouanigot are world authorities on Internet traffic and routing questions. CERN also facilitates the efforts of some staff members, including myself, who take time to teach Internet technology in developing countries.

The Web Materializes
A key result of all these happenings was that by 1989 CERN's Internet facility was ready to become the medium within which Tim Berners-Lee would create the World Wide Web with a truly visionary idea. In fact an entire culture had developed at CERN around "distributed computing", and Tim had himself contributed in the area of Remote Procedure Call (RPC), thereby mastering several of the tools that he needed to synthesize the Web such as software portability techniques and network and socket programming. But there were many other details too, like how simple it had become to configure a state of the art workstation for Internet use (in this case Tim's NeXT machine which he showed me while he was setting it up in his office), and how once on the Internet it was possible to attract collaborators to contribute effort where that was lacking at CERN.

The above is a short and personal record and I may have missed out some people or events that deserve mention. I would be happy to hear from colleagues whose memories can add to this history, which does not have to be considered as my private property. Of course in the end I take responsibility for what appears under my name.

The key words that came to my mind while writing this history were: synergy, serendipity and coincidence. Many of life's most propitious happenings are very deeply "a question of timing". It is my personal belief that the Web could have emerged considerably earlier if CERN had been connected earlier to the Internet: the first Web proposal was written immediately after the opening of CERN's first external connection and it is known that Tim had been working with hypertext ideas since 1980, influenced by Ted Nelson's work on Xanadu among other things. But this remains in the realm of speculation. What is certain is that the Internet has provided a unique opportunity for some of us at CERN to take part in a series of events which are helping to change the world for countless people, hopefully for the better.