First Draft of Final Paper

Anywhere it is not noted Chris put together the rough version of these sections

CSC402/ECE470 - (Inter)Networking Technology & Projects

Test Tool Development for Access Concentrator Devices

Team 3:

Dalat Bui

Chris Brezovec
Shadi Eskamaei

Tyler Humphrey

K. Fritz Lehr

North Carolina State University

November 12, 2002

(to table of contents)

Abstract

Our general over-arching goal this semester is to define a test methodology for Access Concentrators. To test and verify our methodology, we run the tests against many Access Concentrator brands being used in the field today. Examination of these results and recommendations for the future, are also to be done. While examining the tests, we determine the ranges where these particular Access Concentrator brands fail and succeed during the test.

While researching various Access Concentrators, we are looking for new features that the vendors are advertising, that might make a good and reasonable test for the future. Based on this and results from the tests, we will proceed to make recommendations for following semesters.

Table of Contents

Cover page

Abstract or Executive Summary

Table of Contents

Introduction

Problem description.

Approach to solution and Results

Verification and Validation

Schedule

References

Appendices

(to table of contents)

Introduction

Today's Internet Service Provider (ISP) is use to connections and sessions used in the PPP protocol suite. This is a result of early dial-up connections. The ISP is use to this Peer to Peer model, that when mediums that have greater bandwidths came along, PPP was run over them, to give the ISP this familiar model. This is why PPPoE, PPPoATM, and PPPoATMoE came into view, and are used in this project.

An Access Concentator is an Edge Device. This means that it sits on the edge of the IP cloud. It handles all of the connections for the ISP. It terminates these connections, and send the data into the IP cloud to be handled however the routers inside of it decide. An Access Concentrator acts as both a switch and a router, so its behavior under different circumstances is not well known or standardized.

Our project is to examine the Access Concentator and create a standard test metholodology and terminology that can be used by all.

(to table of contents)

Problem Description

Through the examination of what the purpose of an Access Concentrator is, its location in the network, and various known attributes, we have derived four main aspects that should be tested on the Access Concentrator. These four catagories of tests focus on the Access Concentrator's Throughput, Sessions Capacity, Sessions Establishment Rate, and Stability. The other problem is, what other test catagories should be added to test for specific functionality that vendors think are important and advertise.

(Tyler)

1. Vendor Specific Functionality:

Each vendor advertises different attributes including but not limited to maximum throughput, session capacity, the rate at which sessions can be established and stability. Stability is in other words, what causes the DUT to reach an unstable state. Many other attributes of each vendor's Access Concentrator are advertised that are rather difficult to test. However, the problem is the aforementioned quantitative attributes can be either overestimated to make their particular seem better than it actually is or underestimated in order to assure that an unstable state will never be reached. Tests will be run to test the actual capabilities of each vendor's product to determine how close the advertised specifications come to the actual values.

(Shadi)

2. Throughput Tests:

Access Concentrator throughput tests check how well an AC network device transmits and receives data. Vendors are able to report a single value that has been proven to have use in the marketplace from the throughput figure determined by the tests. It is useful to know the actual maximum data rate that a device can support, since even the loss of one frame in a data stream can cause significant delays while waiting for the higher level protocols to time out. In addition to testing the rate of transmission and reception of data, throughput tests should ensure the data was transmitted and received correctly.

(Fritz)

3. Session Capacity Tests:

Provide a meaningful determination of the session capacity of a DUT. Session Capacity is the maximum number of PPPoE and PPPoEoA sessions that a DUT can establish and maintain. The established sessions will be validated by the use of Echo Requests sent to the DUT.

(Dalat)

4. Session Establishment Rate Tests:

In order for an ISP to provide more services to more customers, an Access Concentrator must be able to handle more connections. That is, it needs to be able to handle a huge number of connections at any certain time. Usually, the AC device vendor may claim that the AC device they sold can handle more connections than it really can. So our job here is to determine the maximum rate at which an AC device can establish connections.

To establish a connection between a user device and an Access Concentrator, the two devices have to go through four stages: Discovery stage, Data Link Layer setup, Authentication, and Link Layer setup. A brief description of these stages is below:

Discovery Stage

To establish a connection between a host and an Access Concentrator, the host starts with broadcasting an initiation packet (PADI) to look for an Acess Concentrator. The Access Concentrator then sends an Offer packet (PADO) to the host. Upon receiving the PADO, the host sends back a unicast Session Request packet. When the Access Concentrator receives this Session Request packet, it sends the host a Confirmation packet with the session ID and the MAC address. As soon as the host receives this Confirmation packet, it can proceed to the Data Link Layer setup.

Data Link Layer Setup

At this point, the Access Concentrator and the host are seen as peers instead of server and clients. To establish a connection, one peer starts with sending an LCP Config-Req message to the other peer to negotiate options. The other peer looks at the options, then sends back either a Config-Nak, a Config-Rej, or a Config-Ack message. A Config-Nak means that the second peer does not agree to some of the options. A Config-Rej means that it does not agree to any of the options. A Config-Ack means that it agrees to all of the options. Based on the reply of the second peer, the first peer may need to revise the options and then keep negotiating until it receives a Config-Ack. A worse case scenario of this process is illustrated below:

Config-Req ---->

<---- Config-Rej

Config-Req ---->

<---- Config-Nak

. . . .

. . . . .

Config-Req ---->

<---- Config-Ack

Authentication

Once the Config-Ack message is received, the data link layer has been set up. Now the peers MAY need to go through an authentication process. In this process, the second peer sends a Config-Req message requesting the first peer to identify itself using a certain authentication protocol (see RFC 1661 for some examples of authentication protocols). The first peer then responds to with a Config-Ack that contains the identification. The second peer then looks at the Config-Ack and decide whether it should grant the permission or not.

Network Layer Setup

Even when the identification process is completed, the peers still cannot transfer data yet. Now they will need to set up the network layer. To do this, they exchange IPCP configure packets, just like they did during the Data Link layer setup, to negotiate network layers option (see section 3.3.1).

(Chris)

5. Access Concentrator Stability Tests:

These tests measures the traffic patterns that cause the DUT to reach an unstable state. After this state is reached, the test measures the severity of the state by the time it takes for the DUT to become stable again on its own. The longer it takes, the more sever the instability.

When a DUT reaches a state where the number of valid requests and the number of current sessions out number the maximum number of sessions, the behavior of the Access Concentrator is unknown, and therefore unstable since some of these connections will have to be denied.

When a DUT reaches a state where the sessions that are up and working start to be torn down when they should not be, the behavior of the Access Concentrator is unknown, and therefore unstable since some of these sessions are in use and now need recreated.

When a DUT reaches a state where the throughput is under 10% of the requested throughput, the behavior of the Access Concentrator is unknown, and therefore unstable since some of these connections will have to be denied. This assumes that the requests are not unreasonable, and nobody is trying to send beyond the maximum throughput.

When a DUT reaches a state where the jitter of the sessions is varying by more then X%, the behavior of the Access Concentrator is unknown, and therefore unstable since some of these connections will have to be denied.

Many DUTs can pass individual tests in the lab, but in a “real world” traffic setting, the DUT may reach an unstable state. Since the goal of the DUT, is to be installed, and never be touched again, unless some new configuration is wanted, the DUT needs to be stable. If an unstable state is reached, the DUT MUST has to become stable in a very short amount of time. Anytime the DUT is unstable in the field, it costs the buyer money.

(to table of contents)

Solution

Approach to Solution

The Change In Tasks and Solution Halfway Through the Semester:

As a group, initially, there were 5 different tests to be run. These tests were session capacity, throughput, sessions establishment rate, Session Stability, and Session Forward Stability. We each picked on of the topics and began research and started to write a section of what will be the RFC. Once we met with Kumar to go over the first drafts of the RFCs, we decided that through the research that was done, and based on general session knowledge with PPP, it was decided that Session Stability and Session Forward Stability were not really important tests. The work that was done for those tests were scratched, and we replaced the both of them with Access Concentrator Stability as of 11/7/2002. Chris and Tyler are both working on the Access Concentrator Stability, however Chris has taken more of the visual aspects of the test. To make up for Tyler not having anything to put in the RFC draft, he is now responsible for the helping Chris with the Access Concentrator Stability, but he also is creating the definitions section of the RFC and possible changes that we might want to add based on functionality that is being advertised by the different providers.

(Tyler)

1. Vendor Specific Functionality:

In order to test each vendor's product, test methodologies and algorithms to run these tests have been designed. The following details each individual test and a way to quantitatively compare the vendor's "estimated" specifications to the actual specifications found by running these tests.

(Shadi)

2. Throughput Tests:

Conceptually measuring throughput is a simple task that involves issuing a certain number of requests, counting the number of replies received and dividing that number by the time it took to complete the test. The software being used to count the number of packets received should also evaluate each packet for content and bit count, and return Cyclic Redundancy Check error if there is a defect within the packet. In order to get a quantitative idea of the robustness of a particular measurement, it is necessary to run the same test several times, which requires a fair amount of time to obtain just a single data point, since each test run is likely to take several minutes. Also, if only one throughput estimate was computed for the entire test, the variations that may occur at time scales shorter than that of the entire test will be hidden. Also, the algorithm allows many packets to be sent in a sequence without synchronizing between the sender and the receiver, resulting in the throughput algorithm for the test in pseudocode.

This approach to a solution was taken because the algorithm ensures getting a quantitative idea of the robustness of a particular measurement resulting in accurate calculations of throughput measure on the Access Concentrator device.

(Fritz)

3. Session Capacity Tests:

Develop a methodology that can test session capacity for any DUT. The test methodology must produce the same session capacity results for the exact same machines. This means that the methodology must be repeatable, and all of the parameters must be clearly defined. This does not mean that the session capacity determined by this methodology will be the same as that listed in the DUT's specifications.

The methodology first needs to define an established session. Once the PPP protocol determines the Session-ID and the authentication stage is completed, we have an established session. After we have established a session, we want to see if the DUT can maintain the connection. Sending an echo-request to the DUT and receiving back an echo-reply will do this. The echoes will be sent encapsulated in an LCP packet.

Some parameters that we need to control are keep-alives and network traffic. Since the DUT acts as a server in a client-server relationship, want to turn off keep alives, which are echo-requests in an LCP packet. Typically the server, the DUT in this case, sends a keep-alive and creates traffic. Since the DUT doesn't need this information for this test, it will be turned off. Also we want to eliminate all traffic other than what is needed for session establishment, and echo requests to the DUT, and echo replies from the DUT.

(Dalat)

4. Session Establishment Rate Tests:

To determine the session establishment rate, two tests can be performed. One is to determine the maximum session establishment rate when the PADI’s are sent at a constant back to back rate. The other one is to determine the maximum session establishment rate when the PADI’s are sent at a bursty rate.

The test set up is shown in the diagram below:

Here, the test port represents the user devices. This is where the connection requests begin. The DUT represents an Access Concentrator device. In between is the connection media that connects the two devices. So basically, the test port will generate a bunch of PADI packets and send them to the DUT at a certain rate. Then it measures how many connections the DUT is able to handle by counting the network acknowledgement packets from the DUT.

(Chris)

5. Access Concentrator Stability Tests:

In order to write a test methodology for Access Concentrator Stability, one must know how to measure instability. This was the first thing that was looked at, stability vs. instability (cite source here). Eventually it was decided that the instability of an Access Concentrator should be measured based on how long it takes for it to reach a stable state. Once an unstable state is reached, this is defined as a state where things are happening that should not be, the Access Concentrator is configured in such a way that it backs off to a known stable state whatever was being done to push the DUT. The severity of the instablity is measured by the time it takes for the DUT to recover without outside interference. An example of an unstable state is where sessions are being torn down that should not be.

Once instability and stability were defined and quantified, one must now consider how to get the DUT into an unstable state. In order to test the stability, one MUST be able to get into an unstable state. Nothing is perfect, the DUT does not have infinate memory, or infinate bandwidth at its command. The DUT, just like all systems have their breaking points. The breaking points need to be found and used to test instability.

These are defined as:

sessions are no longer being created

sessions are being torn down early

session throughputs have been reduced below an acceptable range

jitter associated with each of the existing sessions becomes outside of tolerated limits

For more on how to test the instability and the various ways to get the DUT unstable, please refer to the appendice on Access Concentrator Stability for a more in-depth examination on how to do all of this.

Another situation an Access Concentrator may reach instability is in an internet based traffic model. This is where the traffic being simulated is being simulated as if it were on the internet. Many DUTs may be able to pass through the lab with no problems, however, the internet based traffic is much different then the traffic patterns normally used. One of the key differences is the length of time it takes to run a test. Normally these types of tests run for at least a day. Testing various traffic patterns that may be experienced during the day at the time they might be witnessed.

For more information of this, please refer to the appendice on Access Concentrator Stability.

Results

(Tyler)

1. Vendor Specific Functionality:

Many of the vendors give a quantity for attributes of their access concentrator such as net.com's claim that their product (SCREAM 100) has a "maximum throughput of 5Gb per second per Network Data Plane (NDP)" and "can handle up to 20 Gb per second of bursty traffic per NDP" (net.com). Depending on which devices are available for testing, many of these company's products will be tested using the following algorithms in order to compare actual findings to vendor advertised quantities.

(Shadi)

2. Throughput Tests:

Throughput algorithm for the test in pseudocode.

CONSTANT

Test_Duration = Time the test is to run; determines the max number of frames sent.

Packet_Size = Certain size of packets.

Resolution_percent = The percentage of traffic within which the acceptable loss percentage is still met.

Maximum_Loss = Test can be adapted to find the optimal throughput rate with an acceptable packet loss ratio defined by the user.

BEGIN

*Certain number of packets of a certain size are sent and the receiver signals

when all packets have been received.

//Sending Side

For(i=0; i < Test_Duration; i++)

Send(Packet_Size); //Send small packet

//Receiving Side

For(i=0; i < Test_Duration; i++)

Receive(Packet_Size); //Receive small packet

END

(Fritz)

3. Session Capacity Tests:

Please refer to the Appendice on Session Capacity Test to see some of the results so far.

(Dalat)

4. Session Establishment Rate Tests:

Measuring Session Establishment Rate with Constant Back to Back PADI

From the test port, initiate the establishment sessions by sending out an N number of PADI packets back to back (e.g. 25 PADI packets/second).

Measure the number of sessions the DUT is able to establish by counting the number of of IPCP Config-Ack packets received from the DUT. Also measure the total time it took to establish the connections.

If the number of established sessions is equal to N, increase N by ? and restart the test.

If the number of established sessions is less than N, decrease N by ? and restart the test.

Keep going until you can find the highest number of N that still result in successful session establishments.

Measuring Session Establishment Rate with Busty PADI

From the test port, initiate the establishment sessions by sending out an N number of PADI packets in a bursty fashion (e.g. 25 PADI packets per burst per second)

If the number of established sessions is equal to N, increase N by ? and restart the test.

If the number of established sessions is less than N, decrease N by ? and restart the test.

Keep going until you can find the highest number of N that still result in successful session establishments.

(Chris)

5. Access Concentrator Stability Tests:

Please refer to the Appendice on Access Concentrator Stability to see some of the results so far.

(to table of contents)

Verfication and Validation

Testing Plan

(Tyler)

1. Vendor Specific Functionality:

Through the use of Spirent Com’s Adtech Network Tool v4.31 and the algorithms designed for each attribute to test, the actual specifications will be found. These specifications will be compared individually to the specifications advertised by the vendor for the DUT. The comparisons and any conclusions drawn from this comparison will be reported when completed.

(Shadi)

2. Throughput Tests:

Basic throughput analysis involves generating a pre-determined data stream in accordance with specific application’s encoding rules. The data stream can be sent over any Access Concentrator under test and throughput can be assessed at the receiver end using protocol analysis software. This software counts the number of packets received, evaluates each packet for content and bit count, and returns error if defects are found. Modifications to the algorithm that simulate the effects of adverse operating conditions such as signal jitter, noise, delay, and amplitude changes can be programmed to evaluate and assess the system under study. The software being used in the throughput testing of the DUT is Spirent Com’s Adtech Network Tool v4.31.

(Fritz)

3. Session Capacity Tests:

This test is fairly straightforward and should closely match the manufacturer's specifications. I would be surprised to see a session capacity more than 10% below what the manufacturer specified. I actually expect to see a number above the manufacturer specifications. The session capacity is determined by the amount of memory, so results should be consistent.

Some problems could arise would be from a large number of sessions generating high traffic. This traffic may cause session protocol errors. Also, having the keep-alives turned off could cause sessions to be dropped.

(Dalat)

4. Session Establishment Rate Tests:

The graph we get from the test should be similar to the one below:

By comparing to the graph from the experiment and the one above, we should be able to know whether the test is right or not.

(Chris)

5. Access Concentrator Stability Tests:

By testing the data that is returned to the test tool by the DUT. Verifying that the correct Identification values, protocol values, and options are in the packet is the major way of testing if the test worked or not. Stability is a very difficult, but important aspect to test. By looking at the results sections in the appendice for Stability, graphs are given to compare your data against. If the graphs do not match up, that does not mean that the test did not work, it just means that one should look closer at the responses and requests to make sure that the test is running as designed.

Results

1. Unknown

2. Unknown

3. Unknown

4. Unknown

5. Unknown

(to table of contents)

Schedule

Schedule as of Tuesday November 12th 2002

(to table of contents)

References

Brader, S. "RFC 1242: Benchmarking Terminology for Network Interconnection Devices." Network Working Group. July 1991.

Bradner, S. and McQuaid, J. "RFC 2544: Benchmarking Methodology for Network Interconnect Devices." Network Working Group. March 1999.

Mandeville, R. "RFC 2285: Benchmarking Terminology for LAN Switching Devices." Network Working Group. February 1998.

Mandeville, R. and Perser, J. "RFC 2289: Benchmarking Methodology for LAN Switching Devices." Network Working Group. August 2000.

W. Simpson, W. "RFC 1661: The Point-to-Point Protocol (PPP)." Network Working Group. July 1994.

Mamakos, L., Lidl, K., Evarts, J., Carrel, D., Simone, D., and Wheeler, R. "RFC 2516: A Method for Transmitting PPP Over Ethernet (PPPoE)." Network Working Group. February 1999.

Gross, G., Kaycee, M., Lin, A., Malis, A., and Stephens, J. "RFC2364: PPP Over AAL5." Network Working Group. July 1998.

Bradner, S. "RFC2026: The Internet Standards Process -- Revision 3." Network Working Group. October 1996.

Agilent Technologies. "Journal of Internet Test Methodologies Edition 2.2." 2002.

More to come

(to table of contents)

Appendices

Will be lumped into one appendix when the RFC draft is put together, but for now, the sections will be individual Appendices.

(Tyler)

1. 0.1 Introduction

This document provides methodologies for the performance benchmarking of access concentrators. It provides methodologies in four areas: throughput, session capacity, session establishment rate, and device stability. In addition to defining the tests, this document also describes specific formats for reporting the results of the tests.

0.2 Definitions

The following section includes definitions of terms used throughout this rfc.

0.2.1 Definition Format

Definition: Specific definition of term

Discussion: A brief discussion about the term, its application and any restrictions on measurement procedures.

Measurement Units: The units used to report measurements of this term, if applicable.

0.2.3 Throughput

Definition: The maximum rate at which none of the offered frames are dropped by the device under test.

Discussion: The throughput will be tested with different numbers of sessions, packet lengths, and lengths of time.

Measurement Units: Gb per second

0.2.4 Session Capacity

Definition: The maximum number of PPPoE and PPPoEoA sessions a DUT can
establish and maintain.

Discussion: Session capacity is a function of memory and should be consistent from machine to machine.

Measurement Units: Number of sessions

0.2.5 Session Establishment Rate

Definition: A session establishment rate is the rate at which a DUT can establish connections without dropping any connection request packet.

Discussion: For these tests this is the maximum rate at which PPP session establishment requests can be sent to the DUT resulting in successful sessions.

Measurement Units: Session Capacity over time

0.2.6 Device Stability

Definition: Device’s ability to handle traffic patterns. Some will cause the DUT to reach an unstable state meaning that this is beyond the threshold of the DUT’s stability..

Discussion: There are many ways to test the stability of an access concentrator. The following RFC uses five test cases to test stability.

Measurement Units: N/A

(Shadi)

2. Throughput Tests:

1.0 Throughput

1.0.1 Objective

To determine the maximum rate at which none of the offered frames are dropped by the device under test (DUT).

1.0.2 Setup Parameters

The following parameters MUST be defined. Each variable is configured with the following considerations.

· Packet Size: The following IP packet sizes should be used: 40, 64, 128, 512, 1024, 1500 bytes, as implied in RFC 2544.

· Test Duration: Length of time the test is to run. This variable determines the number of frames sent.

· Number of Sessions: Large number of sessions established might cause throughput to decrease and vice versa.

· Resolution %: The % of traffic within which the acceptable loss % is still met.

1.0.3 Procedure

· Create traffic with N streams as uni-directional flows from one or more of the test device’s source ports, through the DUT and to one or more destination ports on the test device.

· Transmit a specific number of frames at a specific rate through the DUT from all sources to all destinations. Once an iteration of transmission has been completed, measure the number of received frames.

· If the count of offered frames is greater than the count of the received frames, then fewer frames are received than were transmitted. The rate of the offered stream is reduced lower than the failed rate.

· The third trial and subsequent trials make use of binary search to continue to increase or decrease the offered load.

· If all the transmitted frames are received by the receiving port, no further trials are attempted and maximum frame rate is recorded as throughput.

1.0.4 Algorithm

CONSTANT

Test_Duration = Time the test is to run; determines the max number of frames

sent.

Packet_Size = Certain size of packets.

Resolution_percent = The percentage of traffic within which the acceptable

loss percentage is still met.

Maximum_Loss = Test can be adapted to find the optimal throughput rate

with an acceptable packet loss ratio defined by the user.

BEGIN

*Certain number of packets of a certain size are sent and the receiver signals

when all packets have been received.

//Sending Side

For(i=0; i < Test_Duration; i++)

Send(Packet_Size); //Send small packet

//Receiving Side

For(i=0; i < Test_Duration; i++)

Receive(Packet_Size); //Receive small packet

END

1.0.5 Results

Each trial iteration results in a record of the total number of packets transmitted and received. For the successful and last trial, the packet rate and percentage of maximum rate for a given packet size is recorded. The results of the throughput test SHOULD be reported in graph format where the x coordinate SHOULD be the frame size, and the y coordinate SHOULD be the frame rate. There SHOULD be at least two lines, actual and theoretical on the graph. Text accompanying the graph SHOULD indicate the protocol, data stream format, and type of media used in the

(Fritz)

3. Session Capacity Tests:

1. Session Capacity

1.1 Objective

Determine the maximum number of PPPoE and PPPoEoA sessions a DUT can
establish and maintain.

1.2 Session Capacity Test Description

Find the maximum number of sessions an Access Concentrator can establish and
test the connections to determine if it can maintain these connections. Session
capacity is a function of memory and should be consistent from machine to
machine. In this experiment, my turn off keep-alives or control when they are
sent. This will help with benchmark testing by controlling unnecessary traffic
that may affect results.

1.3 PPPoE and PPPoEoA session establishment

PPPoE enters a discovery stage to set up a session that is a client-server type of
relationship. A PADI packet is broadcast to the Access Concentrator requesting
service. The access concentrator responds to the PADI with several different
PADO packets that contains information on the Access Concentrator. The host
then chooses one PADO packet and sends a PADR packet. The Access
Concentrator generates a unique Session-ID once it receives the PADR packet
and responds with a PADS packet. After this the PPPoE or PPPoEoA session
begins and there is an option authentication phase. Once authenticated, the
session is established and this is what we will define as an established session.

1.4 Session Establishment Rate

Establish sessions at different rates to see if this has any effect on the maximum
number of sessions that can be established. First, establish a set number of
connections one at a time in a specified amount of time. Second, establish a set
number of connections in several bursts in a specified amount of time.

1.5 Established Connection Testing

Test currently established connections to see if the Access Concentrator can maintain all of the established connections. This test will be performed in conjunction with the establishment test. Sending Echo Request and Echo Reply messages through LCP packets can test this. Typically, server sends Echo Request, in this case that would be the DUT. Since the keep-alives are off and the Access Concentrator will not send Echo Request, the test machine will have to send an Echo Request and the DUT will send an Echo Reply.

1.6 Assumptions

The keep-alives will be turned off, so the DUT will not send echo-requests and generate this traffic. Traffic will be controlled, only traffic used to establish sessions and check session connection will be allowed.

(Dalat)

4. Session Establishment Rate Tests:

3 Session Establishment Rate

3.1 Objective

To determine the maximum rate at which PPP session establishment requests can be sent to the DUT resulting in successful sessions.

3.2 Definition

A session establishment rate is the rate at which a DUT can establish connections without dropping any connection request packet.

3.3 Discussion

3.3.1 Discovery Stage

3.3.2 Data Link Layer Setup

Config-Req ---->

<---- Config-Rej

Config-Req ---->

<---- Config-Nak

. . . .

. . . . .

Config-Req ---->

<---- Config-Ack

3.3.3 Authentication Setup

3.3.4 Network Layer Setup

3.4 Setup Configuration

The setup should be as in the figure below:

3.5 Setup Parameters

The following parameters affect the outcome of the test and they must be defined.

Connection Request Rate - The rate at which configure request packages are sent to the DUT to request connections. The rate SHOULD be set at or lower than the maximum rate at which the DUT can handle. That is, the number should be less than the maximum session capacity of the DUT.

Buffer - The DUT SHOULD have an amount of buffer to store connection requests while it is processing the other requests. The capacity of the buffer plays an important role in determining how many connections a DUT can set up. The bigger the buffer, the more connection it can set up. However, a bigger buffer also introduce more delay.

3.6 Procedure

Two types of tests will be performed. One is to figure out the maximum session establishment rate when the PADI’s are sent at a constant back to back rate. The other one is to figure out the maximum session establishment rate when the PADI’s are sent at a bursty rate.

3.6.1 Measuring Session Establishment Rate with Constant Back to Back PADI

From the test port, initiate the establishment sessions by sending out an N number of PADI packets back to back (e.g. 25 PADI packets/second).

If the number of established sessions is equal to N, increase N by ? and restart the test.

If the number of established sessions is less than N, decrease N by ? and restart the test.

Keep going until you can find the highest number of N that still result in successful session establishments.

3.6.2 Measuring Session Establishment Rate with Busty PADI

From the test port, initiate the establishment sessions by sending out an N number of PADI packets in a bursty fashion (e.g. 25 PADI packets per burst per second)

If the number of established sessions is equal to N, increase N by ? and restart the test.

If the number of established sessions is less than N, decrease N by ? and restart the test.

Keep going until you can find the highest number of N that still result in successful session establishments.

3.7 Measurements

Establishment Request Rate - This is the number of PADI packets sent over the period of time it takes to send these packets.

Percentage of Sucessful PPPoX sessions – This number is obtained by measuring the number of successful sessions and the number of connection requests sent. The number of successful sessions over the number of connection requests sent is the percentage of successful PPPoX sessions.

3.8 Reporting Format

The format should be in the form of graphs. There should be two separate graphs: One for constant back to back PADI rate and the other one for the Bursty PADI rate. The x-axis is the Establishment Request Rate, and the y-axis is the corresponding percentage of successful established connections. The point where the set up rate is highest and the percentage of successful connection is still 100% is the maximum establishment session rate.

(Chris)

5. Access Concentrator Stability Tests:

Access Concentrator Stability

Purpose:

This test measures the traffic patterns that cause the DUT to reach an unstable state. After this state is reached, the test measures the severity of the state by the time it takes for the DUT to become stable again on its own. The longer it takes, the more sever the instability.

Description:

There are 5 test cases to run to test the stability of the DUT. Four of the five test cases test different forms of instability. The fifth test case tests the DUT’s stability with an internet/”real-world” traffic model.

Types of Traffic (General):

LCP Traffic:

New Session Establishments –

These session establishment requests can be valid, invalid, bursty, or steady state requests.

Session Teardowns –

These session teardown requests can be valid, invalid, bursty, steady state requests, or be due to keep-alive timeouts.

Re-Negotiation of Established Sessions –

These session Configure-Requests can be valid, invalid, bursty, or steady state requests.

NCP Traffic:

Configuration of Session’s Network Layer Protocols –

These NCP configure requests can be valid, invalid, bursty, or steady state requests on valid or invalid LCP sessions.

Teardown of Session’s Network Layer Protocols –

These NCP teardown requests can be valid, invalid, bursty, and steady state requests on valid or invalid LCP sessions, or be due to keep-alive timeouts.

Re-configuration of Session’s Network Layer Protocols –

These re-configurations can be valid, invalid, bursty, or steady state requests on valid or invalid LCP sessions.

Normal Network Layer Traffic –

Run varying amounts of valid and invalid NCP traffic on established sessions. Starting at the minimum frame size up to the maximum frame size. Varying the throughput on all established sessions upstream and downstream.

Types of Traffic (Specific):

LCP Traffic:

New Session Establishments –

To configure a new sessions, the host will send out a Configure-Request Packet. Before this is done, the host needs to find out some information about the server. This is called the discovery stage of PPP. PPP will send out a PADI waiting for a PADO response. After the response it will send out a PADR packet waiting for a PADS packet to respond.

To communicate you need to very important things, a MAC Address (for Ethernet), and you need a Session ID number. These packets are to get just those. When sending bad data, you can look at changing these two fields.

The following are the Discovery Stage packet formats and Options:

New Session Establishments (cont) -

The following are PPP session stage packets and options:

Parts of the Packet:

Code –Which LCP Packet is being sent:

Code	Name
0x1	Configure-Request
0x2	Configure-Ack
0x3	Configure-Nak
0x4	Configure-Reject

Identifier – Must be changed when the data inside of request changes, or a response with that number is received. Used to match requests with replies.

Options – List of Configurations option:

Type	Name	Value
0	RESERVED	xxxxx
1	Maximum-Received-Unit(MRU)	1484 octets
3	Authentication-Protocol	C023 = PAP, C223 = CHAP
4	Quality-Protocol	C025 = Link Quality Report
5	Magic-Number	xxxxx
7	Protocol-Field-Compression	xxxxx
8	Address-and-Control-Field-Compression	xxxxx

Session Teardowns –

There are three ways to teardown a session:

1. If you the host does not respond to the keep alive message

2. If a PADT is sent (Discovery Stage)

3. If a Terminate Request and Terminate Ack are sent/received

Everything is the same as in Session Establishment, except there are no options, and the codes are as follows:

Code	Name
0x5	Terminate-Request
0x6	Terminate-Ack

Re-Negotiation of Established Sessions –

This is the same as the Session Establishment PPP session stage. Send the following LCP packets with the appropriate codes and options to re-negotiate different parameters.

NCP Traffic:

Normal Network Layer Traffic –

Configuration:

Configuration being simulated (logical) -

Configuration in the lab (physical) -

Test case 1-

Purpose:

This test measures the unstable state of the DUT where sessions are no longer being created. After this state is reached, the test measures the severity of the state by the time it takes for the DUT to become stable again on its own. The longer it takes, the more sever the instability.

Description:

Procedure:

Goal – Create Sessions so fast that some the sessions no longer get created that are requested

Run the Max Establishment Script

Once the limit is found, back off to the maximum value that allowed traffic through.

Start time, at the first back off.

Stop time, when all sessions are created and torn down as thought.

Script:

VARIABLES

VALUES [2] = {MAX_SESSIONS, MAX_RATE}

BEGIN

VALUES = MAX_ESTABLISHMENT_SCRIPT(NO_COOL_DOWN)

WHILE(SESSIONS_ARE_NOT_BEING_CREATED)

BEGIN

TEARDOWN so the number does not exceed MAX_SESSIONS

CREATE sessions at MAX_RATE

TIME = TIME+1;

END

Variables:

Total Number of Sessions

Maximum Rate To Create Sessions

Time To Get Back To A Stable State

Results:

Record the time it takes to get back to a steady state. You SHOULD run this test 20 times to get a good average of time. Plot the session creation rate vs time. You MAY also run this test and plot the number of sessions created vs. time (getting rid of max limit).

The graphs SHOULD look something like this:

NOTE: This is an approximation

The ridge in the graph shows where stability was lost. We backed off to the max right before that, so the rate should go back up to this max, and be a straight line the rest of the time. The difference in the x-axis is the recovery time.
Test case 2:

Purpose:

This test measures the unstable state of the DUT where sessions are being torn down early. After this state is reached, the test measures the severity of the state by the time it takes for the DUT to become stable again on its own. The longer it takes, the more sever the instability.

Description:

Procedure:

Goal – Bog down the line, so the keep-alives get lost, and sessions start to get torn down early.

Create 10% of the maximum number of sessions on the DUT

Start with a small amount of traffic on these sessions

Increase traffic as time goes on until sessions start to get dropped

(Restart/Create a new session for each one that gets dropped)

Back off to the point right before this happened

Start timer

Stop timer when the sessions are not dropping

Script:

VARIABLES

NUMBER_OF_SESSIONS = 10% MAX SESSIONS (defined earlier)

CURRENT_TRAFFIC = 5% LINE RATE

BEGIN

UNTIL(SESSIONS_ARE_DROPPED)

BEGIN

CURRENT_TRAFFIC = CURRENT_TRAFFIC+ 5% LINE RATE

END

CURRENT_TRAFFIC = CURRENT_TRAFFIC – 5% LINE RATE

WHILE(SESSIONS_ARE_DROPPED)

BEGIN

CREATE session to equal NUMBER_OF_SESSIONS

TIME = TIME+1;

END

Variables:

Traffic rate

Burstiness

Time To Get Back To A Stable State

Results:

Record the time it takes to get back to a steady state. You SHOULD run this test 20 times to get a good average of time. Plot the traffic rate vs time. You MAY also run this test and plot burstiness vs. time..

The graphs SHOULD look something like this:

NOTE: This is an approximation

Purpose:

This test measures the unstable state of the DUT where session throughputs have been reduced below an acceptable range. After this state is reached, the test measures the severity of the state by the time it takes for the DUT to become stable again on its own. The longer it takes, the more sever the instability.

Description:

Procedure:

Create 50% of MAX Sessions

Send invalid requests, and so valid requests to the DUT to be processed

Send traffic throughout at a rate of 5% max throughput

Increase the throughput until, the DUT can not handle all of the data on the line, and the throughput drops down dramatically.

Back off the throughput to the MAX accepted value.

State timer

Stop timer when the throughput is within acceptable ranges again

Script:

VARIABLES

NUMBER_OF_SESSIONS = 50% MAX SESSIONS (defined earlier)

CURRENT_TRAFFIC = 5% MAX THROUGHPUT

BEGIN

UNTIL(THROUGHOUT < 50% of CURRENT_TRAFFIC)

BEGIN

CURRENT_TRAFFIC = CURRENT_TRAFFIC+ 5% LINE RATE

END

CURRENT_TRAFFIC = CURRENT_TRAFFIC – 5% LINE RATE

WHILE(THROUGHOUT < 50% of CURRENT_TRAFFIC)

BEGIN

TIME = TIME+1;

END

Variables:

Traffic rate

Burstiness

Max throughput

Current throughput

Time To Get Back To A Stable State

Results:

Record the time it takes to get back to a steady state. You SHOULD run this test 20 times to get a good average of time. Plot the throughput vs time.

The graphs SHOULD look something like this:

NOTE: This is an approximation

Purpose:

This test measures the unstable state of the DUT where jitter associated with each of the existing sessions becomes outside of tolerated limits. After this state is reached, the test measures the severity of the state by the time it takes for the DUT to become stable again on its own. The longer it takes, the more sever the instability.

Description:

Procedure:

Same as throughput, except this time measure jitter instead of throughput

Script:

Same as throughput, except this time measure jitter instead of throughput

Variables:

Traffic rate

Burstiness

Max throughput

Current throughput

Time To Get Back To A Stable State

Results:

The results should look very similar to the throughput test case.

Test case 5:

Purpose:

This test measures the stability of the DUT under “real world” traffic circumstances. The DUT should never reach an unstable state, but if it does, the severity should be very low.

Description:

Procedure:

Goal is to mimic Internet traffic. Internet traffic, from the perspective of sessions, and the DUT, will look like this:

The x-axis is time of day starting at midnight and ending at midnight.

The general procedure is to allow 10% of the links to time out randomly because of no, purposeful, keep-alives. This is simulating users logging off, but the session not getting torn down until the keep-alive response is missed for a while.

We will traverse time, creating sessions and tearing them down based on the given chart. The traffic running on these sessions will be a variety of small queries like e-mail to big downloads. The size of the traffic will be randomly generated.

Run the test for 24 hrs, to see at what points the DUT becomes unstable and/or unable to recover.

Record the traffic patterns being sent at the time of instability so that more research can be done later on this.

Scripts:

Each of the traffic patterns will have its own script running in parallel. The scripts will share the clock, and all the actions will be based on the time of day the system thinks it is.

EVERY MINUTE:

Do not reply to 10% of the keep-alives

Create or Destroy number of sessions required to get to the current level for this time

Renegotiate 10% of the active sessions paramenters

Send random mix of traffic over the sessions

Varying Sizes

Valid/Invalid Protocols

Send random mix of incorrect LCP traffic to the DUT

Varying Codes and options

Variables:

Number of Connections

Number of Connection retries

Burstiness of source

Throughput on sessions

Jitter through the sessions

Time that stability was lost

Time stability was regained

Results:

Plot Variables vs. time for the entire 24 hr period. You MAY wish to run this testcase 7 times back to back to sim

Soon to Come:

Scripts for 1.

Scripts for 2.

Scripts for 3.

Scripts for 4.

Scripts for 5.

Results for 1.

Results for 2.

Results for 3.

Results for 4.

Results for 5.