P4: Traffic Monitoring

Many network applications depend on being able to monitor the state of the network. There are a variety of questions we mighht wish to ask including:

What is the topology of the network?
How much traffic is flowing across a particular path?
What are the largest sources of traffic?

Given this information, there are several actions that a network application might take such as:

Computing the traffic matrix – i.e., the amount of traffic flowing between each source-destination pair.
Blocking unauthorized traffic flows – i.e., according to the access control policy.
Implementing traffic engineering – i.e., shifting load away from congested links.

In this lecture, we will see how a control-plane can collect information from the data plane using counters.

P4 Counters

The V1 Model provides a counter extern object that can be used to implement a wide range of monitoring tasks. The declaration for counter objects,

extern counter {
    counter(bit<32> size, CounterType type);
    void count(in bit<32> index);
}

includes a constructor counter and a method count for actually counting packets. Counters are allocated similarly to arrays in general purpose languages. Accordingly, the constructor is parameterized on a size parameter that specifies how many counters to create, as well as a CounterType parameter that specifies the granularity of the counters: packets, bytes, or both.

The granularity of each counter, packets, bytes or both, can be specified using the following enum:

enum CounterType {
    packets,
    bytes,
    packets_and_bytes
}

Example

As an example to illustrate the usage of counters, suppose that we have an existing control that forwards IPv4 packets. We can add fine-grained monitoring functionality as follows:

control MyIngress(..., inout headers hdr, ...) {
  counter(64, CounterType.packets) c;
  
  action tally() {
    c.count((bit<32>) standard_metadata.ingress_port);
  } 

  table monitor {
    key = { 
      hdr.ipv4.srcAddr: lpm;
    } 
    actions = { tally; NoAction; }
  } 

  apply { 
    ...
    if(hdr.ipv4.isValid()) {
      ...
      monitor.apply();
    } 
  } 
}

There are several things going on in this example code. First, the declaration creates a packet counter c with 64 entries, one for each physical port on the switch. The tally action increments this counter, using the ingress port as the index into the array. Finally the monitor table matches on the source IPv4 address and either executes tally action or NoAction, which is a no-op. Hence, the control plane can selectively monitor the amount of traffic being sent from specific IPv4 prefixes, aggregated by ingress port.

Run-Time API

Counters have no effect on the execution of the data-plane program. That is, it is impossible to read the value of a counter in a P4 program.

However, the value of a counter can be queried by the control plane. For example, the simple_switch_CLI utility provides an operation counter_read that takes the name of a counter and an index and returns the packet/byte counters associated with that counter. If we populate the monitor table with a forwarding rule that matches on packets coming from 10.0.1.1,

{
  "table": "MyIngress.monitor",
  "match": {
    "hdr.ipv4.srcAddr": ["10.0.1.1", 32]
  },
  "action_name": "MyIngress.tally",
  "action_params": { }
}

and send a packet from 10.0.1.1 to any other host, then querying the counter will return a value such as the following:

RuntimeCmd: counter_read MyIngress.c 1
MyIngress.c[1]=  BmCounterValue(packets=1, bytes=59)

Note that the Bmv2 implementation of counters keeps track of both packet and byte counters by default, even though the counter itself was created with CounterType.packets.

Direct Counters

In many applications, it is useful to associate counters directly with the entries in a match-action table. In paricular, rather than having to specify the length of the array and the index on each count operation, we can use the size of the table and the index of the table entry.

The V1Model architecture includes a second counter-like primitive, direct_counter that provides this functionality:

extern direct_counter {
    direct_counter(CounterType type);
    void count();
}

Note that compared to the counter object, the size and index arguments have been elided.

Example

To use a direct counter a program, we can simply add the counters attribute to a table. The example from above can be rewritten as follows:

control MyIngress(..., inout headers hdr, ...) {
  direct_counter(CounterType.packets) c;
  
  action tally() {
    c.count();
  } 

  table monitor {
    key = { 
      hdr.ipv4.srcAddr: lpm;
    } 
    actions = { tally; NoAction; }
    counters = c;
  } 

  apply { 
    ...
    if(hdr.ipv4.isValid()) {
      ...
      monitor.apply();
    } 
  } 
}

The run-time API for direct counters is identical to the API for standard counters. Note, however that the Bmv2 API has a small bug – it does not provide a way to access the counter associated with the table’s default action.

Discussion

Although counters are quite limited as a primitive, when combined with a dynamic control plane, they can be used to implement more sophisticated monitoring tasks while staying within the resource constraints of network devices such as hardware switches.

In particular, using a table like monitor above, the control-plane can “slice and dice” into aggregates of varying-size, using the semantics of LPM tables on IP prefixes. For example, the control-plane could install an initial set of forwarding rules that monitor relatively coarse-grained aggregates, but then “drill down” to finer-grained aggregates as the mix of traffic evolves. In the limit, it can monitor the traffic from individual hosts.

Reading

For detailed case study showing how counters can be used to implement more sophisticated monitoring tasks, see the (HotICE ‘13 paper)Jose13 “Online measurement of large traffic aggregates on commodity switches.”