Northern Grid for Learning - School Broadband, eSafety, Learning Resources and More

LAN Network Design Considerations

The IP design of a network is critical in ensuring easy administration and maximising performance. All schools have been assigned a public IP range if you keep to this, your network addressing is simple to manage and efficient in data transfer.

To maximise use of bandwidth, consideration must be given to the concepts and application of broadcast and collision domains. Modern switched networks have reduced some problems but created new issues that need to be considered otherwise they will impact on user satisfaction.

Broadcast Domain

When a host PC communicates with resources such as a printer or fileserver, it needs the destination MAC address of the device. If the host does not have this information, it has no option but to broadcast to all nodes on the network asking the resource to reply. If the device is switched on, it will reply to the client and the client will record the address for future reference.

The host PC holds destination addresses for a short period of time: 5-10 minutes being typical. If the PC needs to communicate with the same resource again, after 10 minutes it will send out another broadcast, which again goes to every node on the network. A fully switched network can help reduce the impact of the broadcasts but remember broadcasts and multicasts by default will flood to all nodes on your network. The best policy is to reduce the broadcasts at source and then to manage them at network device level.

If your network has 400-450 nodes this includes PCs, printers, servers, wireless access points etc. Then consider segmenting your network using subnets or VLANs.

Collision Domain

A collision domain is a part of the network where collisions can occur. When a device sending a packet on the network, detects another device on the same segment starting a transmission, it stops and waits for the traffic to clear before resending. This results in a substantial loss of throughput on that segment causing considerable delays.

Switches provide a collision domain between port and device if dedicated cabling is used. Servers and some devices on the network can double data transmission rates by being configured as full-duplex device. This prevents the possibility of the collisions.

Switches and Hubs

Hubs are very simple physical layer devices allowing multiple computers to share the same network. They are suitable for very small networks only. Given the recent fall in the cost of switches there are no sound reasons to continue to use hubs within a school. Network switches are much more efficient at moving information around.

Switches are considered intelligent devices as they store the addresses of hosts attached. The host PC sends out a broadcast, which goes first to the switch, where it checks its own internal database of known resources i.e. MAC addresses. If the switch database knows the location of the device, the computer looking for it forwards the request directly and replies to the computer letting it know where it is. If the switch does not know about the device in question, then it will broadcast to the entire network.

Implementing a switched network

A switch located between the users and the server will not necessarily increase network speed. If everyone is accessing the server and that server is connected to the switch by only one port, they will have to wait. The switch is ineffective because only one individual gets information at a time as shown below.

If the switch has three ports dedicated to the server, providing an aggregate bandwidth of 30Mbps, it can conduct three conversations with three different clients at the same time. All modern network operating systems can use multiple network cards to provide increased bandwidth in this way.



The bottleneck in any client/server architecture is normally between the service (resource) and the switch. Multiple network cards provide one solution. However, installing a Gigabit network card may be easier and more cost effective. A Gigabit module will be required for the switch if this solution is chosen.

How a router works

A router essentially performs two functions. Firstly it identifies a suitable link between the source system or source network and the destination system or destination network, and secondly it transports data packets along this link. If the destination system (destination network) is directly connected to the router, i.e. router and destination systems are on the same subnet, the data packet sent by the source system goes directly to the destination system.

Figure 1: routing

If the destination system (destination network) is not directly connected to the router, then the router transmits the data packet to a neighbouring router that is closer to the destination system (or destination network), and is known as the next hop. The last router in this chain is always directly connected to the destination network and transmits the data packet to the destination system.

Figure 2: routing

The function of a router is either to pass incoming data packets directly to the specified recipient or else to forward them to the next network. The routing metric determines the network to which the data packet should be forwarded if it cannot be delivered directly. The metric is a measure of the quality of the link between the originator and the router or destination of the packet. The router uses the metric to decide to which next hop it should forward the packet. Routing metrics are not just concerned with the length of the path between sender and recipient, but other features, such as the quality of the lines, the bandwidth or loading, can also be taken into account in the decision. Which criteria are used depends on the routing protocol used.

The routing information is managed in routing tables. Routing tables contain information about which neighbouring routers can serve as the next hop for particular destination networks. The decision as to the next hop to which an incoming data packet will be forwarded is made solely on the basis of these routing tables. Hence it is particularly important to protect these tables from tampering. A number of attacks are known to exploit the vulnerability of routing tables to tampering. The table below illustrates the possible content of a routing table.

Table 1: example of an extract from a routing table

	Next Hop	Hop Count
210.23.125.98	210.23.122.4	3
	127.200.45.123	5
	203.2.67.187	8
...	...	...
...	...	...

In this example, the router would direct a packet with destination address 210.23.125.98 to the next hop 210.23.122.4. The hop count specifies how many intermediate stations the packet still has to pass to reach its destination, assuming that it is sent to the relevant next hop. If more than one neighbouring router is available for use as the next hop for a given destination, then the hop count can be used as a routing metric for determining the "most favourable" next hop. The hop count is also used as a routing metric with the RIP routing protocol.

Static and dynamic routing

There are two kinds of routing, static and dynamic routing. These two methods differ from each other with regard to the updating of the routing tables.
Under static routing, these tables are updated manually by inputting system commands.
Under dynamic routing, updating of the routing tables is automated, and is performed with the aid of routing protocols. A further distinction has to be made here between Interior Gateway Protocols (IGP) and Exterior Gateway Protocols (EGP). IGP is used within networks that are under a single administrative authority. The set of networks operated under a single administrative authority is referred to as a routing domain. EGP, on the other hand, is used to exchange routing information between different routing domains

How routers are deployed within Northern Grid

Northern Grid and Easynet have redesigned the network and added a resilient ring connecting all LA hub sites together with at least one backup connection for all hub sites. In addition a backup ring has been added to the two POP’s located at Middlesbrough and Newcastle.
When a router at the hub site identifies a fault has occurred it has to examine its routing table to calculate the best alternative path to re-route data. This is carried out automatically and within seconds. If a firewall cluster in one of the POP’s identifies that Internet connectivity has been lost all data will be automatically rerouted to the alternative POP by a backup trunk link. This is all possible by routers talking together exchanging routing information and making intelligent decisions based on their routing database.

The following advice should be considered when redesigning the infrastructure or reviewing its design and performance.

Cabling

The cable type used for networks, Ethernet, is commonly known as CAT 5 cable. This cable is UTP, which stands for Unshielded Twisted Pair. A typical UTP cable used for network data cabling has 8 wires covered in a sheath. As the cable is unshielded it can be prone to the reception of radio interference noise from a variety of sources – fluorescent lights, machinery, other adjacent cables carrying data to other computers, or mains voltage cables.

Therefore cable runs in ceiling cavities should stay well away from fluorescent lights, electric motors or fans, parallel runs with mains cables and large areas of metallic building infrastructure.

When contracting a company to cable a network, ensure that it complies with the CAT5e standard in accordance with the EIA/TIA 528A standard. This will ensure that the network has a cost effective and easy upgrade in the future. Any work carried out on site should be guaranteed for a minimum period of 15 years.

Category 5e cable will enable data rates of 100Mbps to the desktop, as recommended by BECTA: it also offers 1000Mbps (1Gbps) over short distances. Categories above 5e involve increased costs Cat6 offers gigabit to the desktop. At present Cat6 offers no operational advantages unless you plan to deliver gigabit to the desktop my advice is to use the money saved to build a more powerful scalable-switched core.

The type of cable sheath chosen and the covering for the 4 twisted pair internal cables is important. It should be graded as a Low Smoke Zero Halogen (LSZH or LSNH) type. For cable runs between patch panels in the equipment cabinet and the desktop or wall data outlets, the cable used should be solid core 24 AWG (Belden 1583ENH for example), it should be no longer than 90 metres from the patch panel termination to the wall data port connection. Patch cables, used to connect active components to your patch panels and your computer systems to the wall data outlets, should be a 24 AWG multi-core flexible cable. The patch cables used for the wall port to computer connection should be no greater than 5 metres in length.

For data cable runs within ceiling cavities, you should use low-voltage electrical service baskets. These minimise the amount of metal contact with the cable. Cables can be kept in place with cable ties. Care should be given when tightening. Excessive force may damage the cable’s outer protective sheath and can crush the inner twisted pair cables together. This can affect the cables’ characteristics and lead to poorer network performance.

If the data cables from the ceiling to the wall mounted data points are buried within the wall structure, ensure that plastic conduit pipe is used. Additional mechanical security is required to protect the cable, so use PVC boxed trunking or conduit. All external building-to-building connections should be run in fibre optic cable to protect against lightning strikes and to ensure that maximum segment lengths are not exceeded.

If the site requires a high degree of flexibility for location of data sockets then consideration should be given to “flood wire”. However, the current cost of flood wiring restricts its use in schools.

Data outlet ports should be CAT5e or above with good physical strength and ideally with a shutter to protect the RJ45 socket from dust and dirt when not in use. The best solution is to use a modular RJ45 socket within a suitable mounting frame. This will allow the easy replacement of a defective socket without disturbing any other terminated and functioning data ports.

For a standard “single” sized patress fixing, recessed into a wall or surface mounted, up to 3 x RJ45 data outlets can be provisioned. Accordingly “double” sized wall plates can accommodate up to 6 x RJ45 outlets. Alternatively, if a single RJ45 data outlet is required, this can be provisioned with additional blanking plates to provide a professional and safe presentation.

TCP-IP is the prominent protocol on all Ethernet based systems. Other protocols i.e. IPX or AppleTalk can coexist on a school LAN but for all web-based traffic IP must be used. All modern desktop or network operating systems such as Mac, OS, Windows 89/Me/NT/2000/XP, Novell Netware 4/5/6 and Linux variants fully support IP.

The Windows 98 operating system is not supported and should be phased out quickly. It is difficult to secure, has limited hardware compatibility and generates regular broadcast traffic. NT 4 is no longer supported and should be phased out as part of a school’s ICT strategy.

For further detailed information on the product lifecycle of Windows operating systems check out the following URL www.microsoft.com/windows/lifecycle

To configure the TCP-IP settings of any device, a thorough understanding of the following terms are required.

• IP Address.
• Subnet Mask.
• Default Gateway (Router).
• DNS Server.
• DHCP Server.
• Site (School) Domain Name.
• Proxy Server Address.

IP Address

An IP address comprises two parts, a unique device identifier referred to as the host and a network identifier referred to as the network. The network address is used to identify the LAN that can receive or send information. The host component uniquely identifies individual hosts.

A typical IP address: 192.168.10.10

The address is analogous to a postcode. The 192 could represent the UK, 168 Newcastle upon Tyne and the third number, the street name Westgate Road. The last number (octet) 10 would identify the individual house number.

Subnet Mask

The mask is used to split the Network from the host address. In the above example the default mask of 192.168.10.10 is 255.255.255.0. The host is identified as the last octet. The default mask of a class C IP address permits the unique identification of 254 hosts. 192.168.10.0 represents the network and 192.168.10.255 is reserved for broadcasts.

Default Gateway (Router)

This IP address must be on the same network as the device and points to a router that allows packets out of the local network. If this is not correctly set, the client can never leave their network and therefore cannot browse the Internet.
DNS (Domain Name Server)

DNS Servers are used as a look-up service. Computers only communicate by using numbers. Humans are much better at remembering names. Because of this, the domain name system was created to make it easier for humans to remember network resources (www.northerngrid.org for example).

When a website address, e.g. www.northerngrid.org is entered into a computer’s web browser the device will query a DNS server (local or remote) and find out what number (IP address) to use. The DNS server will return an IP address number for the named website and the computer will then connect using that number.

DHCP (Dynamic Host Configuration Protocol Server)

DHCP automates the distribution of the IP address information discussed above. Client computers on the network, when first switched on, will query the DHCP service for a valid unique IP address as well as other relevant information to identify and locate services required to operate on the network. All servers should have their IP address statically assigned.

To test, release, or renew a leased DHCP delivered IP address run the following commands from the command line.

Ping < IP address> Ping 127.0.0.1 checks TCP-IP stack .

IPConfig This will display current IP information.

IPConfig/Release This will delete any DHCP delivered IP
address information.

IPConfig/Renew This will request a new IP address from the DHCP server.

Site (School) Domain Names A school may already have a domain name registered e.g. stjosephs.s-tyneside.sch.uk

As part of a the “duty of care” to pupils and staff, the domain name format, domain names such as. co.uk or .gmail give the wrong impression to recipients of email or other hosted services. This may attract unwanted replies or data traffic.

Proxy Server IP Address

All Internet access should go through Equiinet’s Cachepilot. This cache server performs a proxy role and provides URL filtering to protect pupils and staff from unsuitable websites. To ensure that the host access is filtered, the proxy setting of every workstation browser should point to the Cachepilot. As an additional security measure, it might be wise to enforce a policy rule that inhibits any web access from your network except that sourced by the Cachepilot.

Recommendation

All schools within the Northern Grid are assigned an IP address range that is suitable for their size. Unless there are unusual and specific circumstances it is advisable to use this. All schools within Northern Grid are on their own dedicated VLAN isolating them from each other. If two or more schools wish to allow direct connectivity a firewall policy can be configured on request.

There are many network switch manufacturers to choose from. As a guide, you are advised to purchase devices from known manufacturers such as: Cisco, HP, 3Com, D-Link or Nortel.

As a general rule, look for the device’s management connectivity. This is usually either a dedicated port identified for the purpose, or a 9 way ‘D’ socket on the front or the rear of the device. Management of the device may be via a web-style or text-style command-line interface.

The devices should support a minimum of Layer 2. For larger networks, where specific IP routing requirements are needed, the minimum level required is a Layer 3 core grade switch.

Primary Schools or small business units with less than 20 computers should select a switch that has a switching capacity in excess of 8Gbps for 24 port switches ports or greater than 13Gbps for 48 port density switches.

Secondary schools or sites with larger network topologies using a tiered network bandwidth design should choose a “core” grade switch that has a switching capacity of more than 50Gbps.

All switches should be “stackable” to produce larger port density network infrastructures without the need to cascade to downstream switches. Ideally Switch-to-Switch links should be Gigabit.

When purchasing a Network Switch other key requirements that should be looked for are:

• IEEE 802.1q Virtual Private Networks (VPN).

• IEEE 802.1p Quality/Class of Service standard.

• Backplane speed (throughput) greater than 8Gbps for 24port and 13Gbps for 48port devices.

• Stackable switch device connectivity.

• Large network (more than 400 hosts) – “core” switch required.

• Web based GUI – SNMPv2.

IEEE 802.1p is especially important should a school wish to use video conferencing services or operate a “VoIP” (Voice Over IP) system across the network infrastructure.

Designing your switched LAN for the future

It is recommended that a managed switch be used at the backbone of your network. Managed switches are also advisable at the workgroup and access level. A HP Procurve 24 port managed switch cost as little as £200.

A secondary school should incorporate a hierarchical topology into their switched network design. At the top layer the core and distribution functions should be combined in a powerful 1000M 24 port fully managed switch. Additional core switches can be added as necessary. Gigabit copper can be used in many installations to keep costs low. The local server’s and proxy/filtering service need to be attached at the core level. The access switches should support 802.1p and 802.1Q standards. Primary schools can use a cost effective option by utilising stackable switches to create a high-speed backplane. The stackable switches have limited scalability but can deliver adequate services for up to 150 hosts without any significant management overhead.

The following network diagrams show suitable primary and secondary school designs The secondary school model shown uses a stackable switch configuration, if this is your school’s topology start looking into incorporating a two level structure in any upgrade plans. The connection’s from access to core switch must be 1000M by utilising link aggregation multiples of this value are possible.

The switched network topology has to fit existing and future user demand, if the number of hosts on your network exceeds 200 than start migrating to a two level structure with a minimum configuration of 2000M backbone and 1000M uplinks.

For further information check out the resources at the bottom of the page.

It can be seen from the Northern Grid WAN diagram Fig 2 that the core infrastructure is routed at layer 3 and switched at layer 2. The Local Authority hub sites were previously switched but due to a number of security requirements, routing is now carried out at the Local Authority hub site. Cisco multilayer switches perform both routing and switching functions. Cisco Modular Routers are being installed at all schools upgrading their LES connections to ensure LES termination and to provide clear demarcation support. Individual schools are on their own dedicated VLAN using a LES 2/10/100 connection and are aggregated and routed at their local hub site. Local Authorities in the North and South of the region have their own Checkpoint clustered firewall at the POP. If a POP fails, a resilient megastream link between the two POPs will reroute data as necessary.

A 100M IP Transit to the WWW & NEN from each POP creates a total bandwidth of 200m. A resilient ring will link all core hub sites to ensure that an individual link failure between hub site and POP is bypassed to ensure 24/7 connectivity for all schools and Local Authorities.

A central clustered firewall infrastructure ensures that individual school firewall deployment is not necessary. The firewall cluster comprises an active switch and firewall with backup systems to ensure equipment remains online in case of device or equipment failure.

Network Topology Overview

All schools in the Northern Grid use Ethernet as the primary LAN protocol. This uses a media detection system known as CSMA/CD (Carrier Sense, Multiple Access/Collision Detection) to detect another computer’s communication and automatically delays sending data until the wire is quiet.

If two or more computers try to communicate simultaneously on a hub network as shown below then a “collision” status occurs and all computers concerned wait predetermined but different periods of time before trying to communicate again.

 

Fig 1 Ethernet hub network


Fig 2 Northern Grid Wide Area Network

Issues with performance

Most Ethernet networks used hubs in the early years. These devices heralded the start of structured cabling infrastructures. Cables were installed at precise locations up to 90m away from the network hub device. These provide some resilience to network cable failures. However, they operate using the same “shared” communication methods as the serial “daisy chained” process of early networks. With faster processing speeds, modern computers process received network data more quickly and increase requesting for data. This leads to more “collisions” on the network, degrading the network performance to an unusable state.

Inevitably communication speeds using hub-based networks have increased from 10Mbps to 100Mbps. This has helped to alleviate some of the congestion. Larger network infrastructures of 24 or more hosts still become unreliable and throughput is decreased unless switches are used. Some network hardware manufacturers may simply represent a “switch” as a device that can select between 100Mbps and 10Mbps bandwidth types to provide legacy support. However, this provides no real advantage.

Solutions to speed problems

Network switches are the answer to modern network infrastructures. A basic switch will simply channel data to the port and cable between the origin computer and the destination device on a per packet (data transmission) basis.

Network switches that are classed as “managed” can be programmed in order to fully accommodate the configuration requirements of the network. They are highly recommended and cost little more than an unmanaged switch. A managed switch allows for additional IP address ranges, secure virtual network creation across the physical wired infrastructure and bandwidth control for attached devices. It ensures the prioritisation of data from attached devices in order to ensure that they receive appropriate bandwidth. Examples are real-time video conferencing. One of the easiest and most useful features of a managed switch is to utilise SNMP to monitor the network’s performance. The switch management software will allow you to troubleshoot by identifying errors and problems.

Managed Switches handle the data between the source and the destination in an advanced way that allows for maximum efficiency of the data transfer. They do this by using a system known as “store-and-forward”. This stores received data from a computer until the destination device is able to receive it. In real terms this is usually shorter than 1 second.

This allows multiple computers on a network to “talk” to a network server or other attached device without the “collision” effects experienced when using hub or pseudo switch type components.

As the data traffic is stored it can then be channelled and directed by the switch in a more fluid data stream to the destination device or network server. Switches that use “store-and-forward” can also handle more than one data stream to different destination devices. As an example, one group of computers could be communicating with the site’s network server, whilst another group are using the Northern Grid Broadband Service via the router. At the same time, another group of computers could be using a CD-ROM server attached to the network, all of which would be achievable without impact on the bandwidth and speed of operation of the network infrastructure.

The type of switch selected will depend on the type of topology used. There will eventually come a point when a large network topology using small capacity switches will start to slow down. The switches will struggle to keep up with the demands. In this situation, the topology design needs to be reviewed and consideration should be given to a tiered approach to the problem by the introduction of “core” grade switch devices.

The difference between a standard workgroup switch for the site and a core grade switch is that core switches have the capacity to handle vast amounts of network data throughput. When a packet of data is received from a computer to a port on the switch, at a speed of 100Mbps for example, it will be switched internally to its destination device’s port. This is referred to as “wire speed”.

The data packet, once switched, will proceed to the destination device from a port on the switch at the correct speed for that device. The speed that this data is passed through the switch is called the “backplane speed”. The backplane speed of a “core” or standard switch is the key to achieving a good network performance.

The Easynet / Northern Grid Broadband Router

The network router supplied by Easynet & Northern Grid is a Cisco 1700/2600 model. All new installations are being fitted with a Cisco 1721. The primary role of the router is to terminate the LES circuit and to provide a demarcation point on the network.