Peering at Peerings: On the Role of IXP Route Servers

Philipp Richter, Georgios Smaragdakis, Anja Feldmann, Nikolaos Chatzis, Jan Böttger, Walter WillingerInternet Measurement Conference 2014 · DOI

TL;DR

This paper provides a deep dive into how Internet Exchange Points (IXPs) and their route servers (RSes) shape modern internet routing. By analyzing unique control and data plane data from two major European IXPs, the authors reveal that while RSes dramatically simplify connectivity, the bulk of actual internet traffic still flows over direct, bilateral peerings. This work highlights the complex interplay between ease of connection and traffic volume in shaping internet infrastructure.

What Problem Does This Paper Address?

Internet Exchange Points (IXPs) are crucial hubs where networks connect directly to exchange traffic. Historically, this involved setting up individual BGP sessions between each pair of networks (bilateral peering). As IXPs grew, this became unmanageable, leading to the adoption of "route servers" (RSes). RSes act as a central point, allowing a network to establish a single BGP session to reach potentially hundreds of other networks. This simplifies connectivity immensely, but it's unclear how this ease of connection translates to actual traffic flow and how it impacts routing decisions. The paper aims to bridge this gap by empirically analyzing the role of RSes in IXP peering, correlating control plane information (who can connect) with data plane information (who is connecting and exchanging traffic).

Methods

The authors employed a novel approach by combining two types of data:

  • Control Plane Data: Detailed BGP data from the IXP route servers, including peer-specific routing information bases (RIBs) for one IXP (L-IXP) and master RIBs for another (M-IXP). This allowed them to reconstruct the multi-lateral peering fabric.
  • Data Plane Data: sFlow traffic records from the IXPs' switching infrastructure. This enabled them to identify actual traffic flows between networks and infer bilateral peerings, as well as verify which connections were actively carrying traffic.

By correlating these two views, they could analyze not just who could peer (via RSes) but also who did peer and how much traffic was exchanged over different types of connections (bilateral vs. multi-lateral via RS). They also leveraged public BGP data (from RIPE RIS, Routeviews) to assess the visibility of IXP peerings in external datasets.

Datasets

The study utilized proprietary datasets from two European IXPs (referred to as L-IXP and M-IXP) that operate route servers. These datasets included:

  • IXP-internal Route Server Data: Weekly snapshots of peer-specific RIBs (L-IXP) and master RIBs (M-IXP) from their BIRD route server deployments.
  • IXP-internal Traffic Data: 4 weeks of sFlow records from the IXPs' public switching infrastructures.
  • Public BGP Data: Data from route collectors like RIPE RIS, Routeviews, and PCH, as well as data from IXP-specific Looking Glasses (LGes).

Key Insights and Ideas

  • RSes Simplify, But Don't Dominate Traffic: While RSes dramatically increase the number of potential peerings (multi-lateral), the majority of actual internet traffic still flows over direct, bilateral peerings. This suggests that while RSes are great for establishing broad connectivity, high-volume traffic often necessitates dedicated, direct links.
  • The "Hidden Path Problem" is Largely Solved: Early concerns about RSes hiding optimal routes seem to be mitigated by modern route server software (like BIRD) that can maintain peer-specific routing tables.
  • IPv6 Connectivity vs. Traffic Disconnect: Many IPv6 peerings exist, but very little actual IPv6 traffic flows over them, highlighting a significant gap between connectivity and utilization.
  • Traffic Patterns Reveal Peering Intent: The authors found that networks with high traffic volumes tend to establish bilateral peerings, even if they also use RSes. This suggests that traffic volume is a key driver for direct peering.
  • RSes are Essential for IXP Growth: The availability of free RS services is a significant factor in the growth and attractiveness of IXPs, creating a positive feedback loop.

Key Results

  • Connectivity vs. Traffic: At L-IXP, multi-lateral peerings outnumbered bilateral peerings by a ratio of 4:1, but bilateral peerings carried twice as much traffic. At M-IXP, this ratio was closer to 1:1, but still, bilateral peerings were highly significant.
  • RS Route Coverage: The RSes at the studied IXPs advertised routes that covered 80-95% of traffic and included many popular destinations, making them valuable for new members.
  • Traffic Distribution: While a large percentage of peerings carry some traffic, a small number of bilateral peerings are responsible for the vast majority of traffic volume.
  • Peering Type Dynamics: Changes from multi-lateral to bilateral peering were often associated with significant increases in traffic, supporting the idea that traffic volume drives direct peering.
  • Common Members' Behavior: Networks that were members of both IXPs tended to have consistent peering strategies across both, indicating a deliberate choice in how they manage their connectivity.

Limitations and Future Work

  • Proprietary Data: The study relies heavily on proprietary datasets from two IXPs, which limits generalizability to all IXPs worldwide. The authors acknowledge that RS adoption and usage can vary significantly.
  • Inferring Bilateral Peerings: Bilateral peerings were inferred from traffic data, which might miss some connections or misattribute traffic in edge cases.
  • Future Work: The authors suggest that future research should focus on developing better monitoring infrastructures to track changes in the peering ecosystem and investigate the impact of initiatives like Open-IX on the US interconnection market. They also highlight the potential for Software Defined Networking (SDN) to further innovate IXP route server functionalities, addressing issues of flexibility, trust, and security.

Summary updated

BibTeX
@inproceedings{richter2014peering,
  author = {Philipp Richter and Georgios Smaragdakis and Anja Feldmann and Nikolaos Chatzis and Jan Böttger and Walter Willinger},
  title = {{Peering at Peerings: On the Role of IXP Route Servers}},
  booktitle = {Internet Measurement Conference},
  year = {2014},
  doi = {10.1145/2663716.2663757},
}
Figure