Cabling Business Magazine, September 2007

The modern Ethernet network combines data traffic with voice and video services along with management traffic in a complex aggregate that traditional testing methods do not address. With the disparate traffic loads and service requirements, ensuring quality of service while maximizing throughput is critical. Network providers use a variety of tools, including VLAN-, MPLS and IP-based class of service, to keep service quality and revenues up. Traditional testing methods are no longer adequate for these differentiated services: a multi-stream, multiport, multi-service model is needed.

As triple-play services reach more end users and the demand for commercial Ethernet traffic continues to grow, network providers are under more pressure to update aging architecture. Competition between traditional telephony and cable providers in the Ethernet space means networks must be turned up and generate revenue quickly and as cost-effectively as possible. One tool to maximize revenue is over-subscription of existing Ethernet networks, adding customers and services without adding network elements or lighting more fiber. Proper network pre-qualification and monitoring is required to ensure that service quality is maintained from the head end, through the network backbone, and to the customer premises.

Evolution of Ethernet Services

For the past five years the bulk of data services provided by telecommunications companies have been point-to-point, running from one business campus to another, from one business data center to another, or similar configurations. These early Ethernet services were relatively straight forward to turn-up, monitor and maintain, as the majority of testing was isolated to a single dark fiber. Performance was verified by completing hard loop back tests running at data rates of 10 Mb/s, 100 Mb/s, or even 1 Gb/s, based on the bandwidth sold. A variation of the dark fiber implementation was to provide a wavelength over an existing DWDM network. Similarly, an existing TDM architecture, based on SONET or SDH, could be adapted to run Ethernet traffic.

The technology that forms the basis of these networks has evolved, and presently Generic Framing Procedure (GFP) over Next Generation Networks (NGN) provides an efficient means for a traditional TDM network to carry Ethernet traffic. Virtual concatenation (VCAT) and link capacity adjustment scheme (LCAS) provide rate adaption to allocate bandwidth based on network loads. Even with GFP and NGN, testing is a straightforward process; though more flexible and sophisticated, it is not much more complicated than a traditional point-to-point service over dark fiber. The absence of switches or routers directing traffic simplifies the testing procedure to an exercise of verifying overall throughput. In rare instances where this equipment exists, it belongs to customers, and technicians are required only to verify connectivity and the dedicated bandwidth.

Service verification in a traditional data Ethernet network is also a simple task. Many services were sold as a fixed bandwidth, such as 100 Mb/s, without any strata or class of service designation, only a promised of bandwidth, which in most instances was not guaranteed. Service providers did not provide traffic grooming or policing to ensure the Quality of Service (QoS); they simply installed the network, verified throughput and maintained it when customers identified issues. From a testing perspective, these networks are uncomplicated. Technicians simply turn up the service using a hard loopback at the far end to verify performance.

These traditional Ethernet technologies have performed satisfactorily, and in some cases, are still adequate. However, as the demand for Ethernet services and bandwidth increases, network providers will be under pressure to add more capacity without impacting existing services.

With the advent of triple play, the topology is no longer point-to-point as service providers are delivering retail customers with voice, video, and Internet access. The requirements for triple play are significantly more complicated than traditional data services. First, instead of simply providing a mechanism to transport data, they are providing data, video and voice connectivity that they had not previously been responsible for. Additionally, these three types of traffic have very different requirements, and stress the network in a unique manner.

Voice, for instance, requires extremely low latency as it travels – usually in about 50 milliseconds – from one speaker to another. Any delay or gap in transmission is instant aneously noticeable by the customer. In a generation of packet-based telephony, latency is perhaps even more important than the sound quality of voice, meaning a few dropped packets is more acceptable than any form of packet delay.

Video presents a completely different set of challenges. With video, extremely large amounts of data are transported from a central video server or head end to the customer premise and ultimately to the television set. Small changes in latency go unnoticed and are typically smoothed over by constant buffering on the set. Home video delivery is not a real-time application like a video conferencing, which has the same latency requirements as voice calls. The biggest challenge with sending large amounts of data over a long period of time is ensuring every frame is delivered. If not, the picture can pixelate, freeze or disappear altogether.

Internet or data is the least stringent in terms of service level parameters when compared to voice and video. If a few frames are lost, traditional TCP/IP mechanisms manage data restoration adequately. In many cases, the bottleneck is not with the Ethernet service, but with the ISP and Internet servers themselves. With data services, the challenge is ensuring minimum download speeds and service up time. If customers expect download speeds of 1.5 Mb/s, the test technicians must groom the network to ensure delivery from the customer site to the ISP, on to the Internet, of 1.5 Mb/s, even if the actual download rate of a web page is slower.

In addition to voice, video and data, network management traffic must be verified. This traffic signals devices, such as routers and switches, and directs communications between them. In all this traffic may only account for about one percent of all network traffic, yet it is critical that it is transmitted reliably, and typically has the highest priority of all traffic.

As Ethernet services becomes more popular, networks become oversubscribed and service providers increase the data that is transmitted along a given path segment; a 100 Mb/s pipe may be loaded with 200 Mb/s of data or more. Since not every user will download at once, this typically does not present a problem if service providers are constantly grooming and policing their networks to verify they can handle the excess load. This is especially important when deploying Carrier Class Ethernet as defined by the Metro Ethernet Forum (MEF). Service providers are required to deliver the class of service promised in their service level agreements (SLA). This can only be guaranteed by testing different types of traffic (voice, data, and video) simultaneously with Ethernet Differentiated Service Testing. (See Figure 1.)

Figure 1. M.2301 - IP QoS class definitions and network
performance objectives for an end-to-end IP flow.

A single network element, especially those closer to the core of the network, may aggregate traffic from dozens of sources, each arriving into a different port. The traffic is then combined into a single egress port as it heads deeper into the core. Similarly, video traffic may be continuously sub-divided from a 10 Gigabit connection at the video headend down to Gigabit or Megabit interfaces as it approaches the customer. The network element doing the aggregating and / or subdividing should maintain the quality of service for each upstream or downstream traffic stream. Thus, testing differentiated services involves not only putting test traffic on a single interface, but adding traffic onto multiple ports simultaneously.

Ethernet Service Top Ten Test Tips

1. Generate and measure test traffic for every service provided. There may be up to four different types of traffic with different class of service settings, different bandwidth usages, and different packet loss, latency, and jitter requirements. The test set must not only simultaneously generate each traffic type, but also be able to measure and monitor each one independently.

2. Set up the test set to exactly mirror the network architecture and services provided. If the network runs VLAN 85 with priority 4, the test set must have the same structure and values or the results are meaningless. Only if this is done for the myriad of services on the network can all them be certified.

3. Always complete thorough QoS tests for lost frames. To obtain the most accurate measurement of lost frames, a sequence number tag should be added to the frame payload.

4. Measure latency relative to the QoS standards. This can be difficult to accomplish with most test sets, as typically only round-trip delay measurements are captured. While this is adequate for a first-order approximation, latency is not necessarily symmetrical and taking a round-trip measurement may give misleading results. Newer test sets use unidirectional delay measurements that through GPS technology is synchronized on each end with millisecond resolution. This set up gives a true measure of latency between point A and point B.

5. Measure packet jitter and verify it does not exceed the QoS requirements. This is particularly true for voice and video services.

6. Conduct BER measurements, but do not rely on them as a sole indicator. When testing Ethernet services over TDM and DWDM networks, BER is a useful metric when communicating problems to network managers, who may not fully appreciate the nuances of Ethernet quality measurements.

7. Don’t be limited by the expected service bandwidth. If the service is 10 Mb/s, the test should be run at both lower and higher speeds to monitor changes in performance, verify frames are dropped in the order of priority if needed, and ensure data is delivered at promised speeds. A customer who purchases a 10 Mb/s service may attempt to use more bandwidth, and only by testing at higher rates can the service provider verify traffic policing is working properly.

8. Capture test results, establish a performance baseline and compare to standards, specifications and future tests. As more customers are added to the network, and utilization approaches or exceeds 100 percent, the impact on quality of service should be tracked and compared to the baseline taken when the service was turned up.

9. Conduct regular in-service monitoring. Once the network is up and running, monitor performance and perform preventative maintenance. Understand the utilization and identify potential problems before they occur.

10. Retest quality of service as part of normal network maintenance, especially as the network capacity approaches or even exceeds 100%. Adding new revenuegenerating customers does not improve the bottom line if existing customers become dissatisfied and leave because of degraded service.

Once the service as been certified and handed off, monitoring is required to identify potential spikes in customer usage and help service providers adjust policing so the highest priority data is given preferential treatment, understanding that if lower priority data is dropped, it will cause the least noticeable disruption of service and less likely to result in customer dissatisfaction. As more customers are added to the network, the existing customer traffic must be monitored to verify they receive the same quality of service as when the service was first turned up.

The network architecture plays a crucial part in determining which tests are required to guarantee delivery of high performing services: stacked VLAN or Q-in-Q, Multi-Protocol Level Switching (MPLS), or Internet Protocol (IP). Each of these network architectures employs different systems to differentiate traffic based on priority. VLAN uses a priority field with a number from 0 to 7, as does MPLS. In the header of IP there is a type of service field with 256 potential values (though the DSCP standard only uses 6 of the 8 bits). These architectures can be used separately or in combination to determine which type of traffic is most important. For example, providers may add service VLAN tags, or map IP-based TOS into VLAN- or MPLS-based priorities. Understanding these architectures is vital for testing as they have a direct impact on the quality of service. Testing without the proper setting can deliver invalid results.

Evolution of Testing Methodology

Traditional test sets have been focused on testing one piece of this puzzle at a time, testing each type of service separately. With new triple-play services and the need to generate multiple types of traffic simultaneously, these test sets no longer provide adequate test coverage. Because differentiated services now exist side-by-side on the network rather than on their own dedicated fibers, wavelengths, or TDM tributaries, they must be tested simultaneously.

The most basic test for Ethernet services is to generate traffic in the same format or formats (VLAN, MLPS, IP) used by the network architecture. Not only do technicians need to configure their test sets to match the network architecture, they also must set the priority field to match the type of service being turned-up. This is required in order to verify that data is successfully transmitted from the video head end to the DSLAM, from the data center to the network, from the central office to the customer, etc. To do this, a single test set must be able to generate multiple types of traffic simultaneously, all with different frame configurations, classes and priorities.

Once this important first step is complete, the test is run from the customer premise to the central office, video headend, etc. with another tester to make QoS measurements. (See Figure 2.)

Figure 2. To make QoS measurements, using two test sets, tests are run from
the customer premise to the central office, video headend, etc.

Three key metrics can be used to characterize the network capability: lost frames, latency, and packet jitter or packet delay variation.

When one or more packets of data traveling across a network fails to reach its destination, the effect upon the service may be minor, but may lead to further service degradation as packets are resent. Packet loss can be caused by a number of factors, including signal degradation over the network medium, oversaturated network links, corrupted packets rejected in-transit, faulty networking hardware, or normal routing routines. Lost or dropped packets can result in highly noticeable performance issues and will affect all other network applications to a degree. Most IP-based services assume there will be some packet loss. However, service testing should not only verify that the ratio of lost frames falls within the acceptable limits defined by the class of service, but also verify that higher-priority services have a lower loss ratio than lower-priority ones.

Latency is the delay between the time a frame is transmitted and when it is received. Low latency is critical for voice as described above, as well as for SAN over Ethernet, where increased latency requires larger buffer-to-buffer credits. It also negatively impacts TCP sessions, where increases in latency have a profound effect on throughput.

Packet jitter or packet delay variation is the differences in the time of arrival of the packets. For classic data applications, jitter is easily managed and not a key parameter. But for real-time applications such as voice, jitter becomes a critical parameter that must be tested and verified to ensure quality of service.

In some cases, bit error ratio (BER) is used as a QoS metric due to its traditional importance to TDM networks. BER is calculated by taking the ratio of errored data bits to the number bits transmitted. While an interesting measurement, it can be misleading as it is possible to measure a BER of zero on all received frames and still have a data loss of 97%. For this reason, Ethernet service metrics do not refer to BER.

With the increasing importance of Class of Service standards, Carrier Class Ethernet certification, and real-time applications; assuring QoS is a critical element in offering revenue-generating Ethernet services. This assurance comes from properly testing these differentiated services using techniques of multi-stream traffic generation and prioritization that has not played a large role in traditional point-to-point services. Furthermore, special attention must be paid to the key QoS metrics and matched against the SLA. By adopting test procedures focused on differentiated services testing, network providers can be confident they are satisfying the needs of their customers today and will have a reliable source of revenue into the future.