“To work jointly and cooperatively with leading Internet service providers (ISPs), peer-to-peer (P2P) software distributors, and technology researchers to ascertain appropriate and voluntary best practices to accelerate distribution of content and optimize utilization of ISP network resources in order to provide the best possible performance to end-user customers.” -- DCIA P4PWG

Questions & Answers

We compiled a list of questions that are often asked and our answers to them. This list of questions and answers can help understand

  • what P4P is designed for,
  • how P4P is designed,
  • why P4P is designed in its current format.

This list keeps becoming longer and longer, as the more progresses we are making, the more interesting questions are asked and addressed. Please keep updated!

Sample Tags

An example of an ordered list:

  1. Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
  2. Phasellus nec erat sit amet nibh pellentesque congue.
  3. Cras vitae metus aliquam risus pellentesque pharetra.

Heading Level 3: Followed by an Unordered List

  • Cras vitae metus aliquam risus pellentesque pharetra.
  • Maecenas vitae orci vitae tellus feugiat eleifend.
  • Etiam ac tortor eu metus euismod lobortis

 

<-->

General Questions

Q: Today, does peer to peer technology use a distributed model for file distribution? Can you give us an overview of how this distributed model works and how it differs from traditional P2P?

[clarification] It is my editor's impression that today's peer to peer technology is somehow different from the initial iterations of peer to peer technology. If there are no differences between "traditional" peer to peer and today's technology, then please point that out. I think by "traditional" we mean peer to peer technology circa 2001 - 2003/2004, popularized by the file sharing applications used to exchange music tracks, videos, etc.

Traditional P2P such as KaZaA was just used for file sharing (i.e. search and download), but modern P2P applies to a wide range of applications including not only file distribution but also live streaming, collaborative research, and many others. For example, there is increasing usage of P2P for video content distribution by peer-to-peer television (P2PTV) services such as the BBC's iPlayer and Joost. This wikipedia article provides a survey on file sharing.

The most popular P2P file-sharing protocol is BitTorrent. Although there have been many revisions since the initial release of BitTorrent in 2001, the basic BitTorrent protocol has been relatively stable. BitTorrent-based P2P file-sharing applications use swarming, which breaks files into thousands of component parts, to improve efficiency. In such applications, a peer, upon joining, is given a random set (~40-50) of peers with which it can exchange file contents.

Q: We understand today's peer to peer distributed model is more efficient and cost effective than the traditional file sharing model. How and why is it more efficient and cost effective?

There have been many advances. Examples:

  1. Swarming, which breaks files into many small pieces, improves the efficiency of distribution over whole-file sharing. Modern swarming techniques are also becoming more sophisticated (e.g. tit-for-tat)
  2. The data delivery is more reliable (e.g. KaZaA was highly susceptible to corruption, BitTorrent is not).
  3. Many P2P p2p networks (e.g. Pando, Kontiki) have added a variety of techniques to work through firewalls and home NAT's, and to try to infer when peers are topoligically close.
  4. Decentralized look-up tables, which represent an advance over centralized P2P applications from prior generations, have also boosted efficiency.
All of the preceding techniques improve P2P performance and reliability over first generation P2P.

Q: We understand that users must manage their use of P2P in order to make use of it. How must users manage their use of P2P?

Many P2P software programs allow users to configure their behavior. For example, a user can specify how much upload bandwidth the user would like to contribute. In addition, users of certain P2P applications can select which files and folders to make available to the P2P network for sharing with other peers.

P4P Interfaces and Funtionalities

Q: What interfaces and functionalities does P4P exports to P2P and other applications?

Here is a picture showing the potential entities in the P4P framework: iTrackers owned by individual network providers, appTrackers in P2P systems, and P2P clients (or peers for short). Not all entities might interact in a given setting. For example, trackerless systems do not have appTrackers. P4P does not dictate the exact information flow, but rather provides only a common messaging framework, with control messages encoded in XML for extensibility.

A network provider may choose to implement a subset of the interfaces. The richness of information conveyed is also determined by the network provider. Note that a network provider may also enforce some access control to the interfaces to preserve security and privacy.

P4P presents three classes of interfaces:

  1. info interface
  2. The info interface allows others, typically peers inside the provider network, to obtain network topology and status. Specifically, given a query for an IP address inside the network, the interface maps the IP address to a (ASID, PID, LOC) tuple, where ASID is the ID of the network provider (e.g., its AS number), PID is an opaque ID assigned to a group of network nodes, and LOC is a virtual or geographical coordinate of the node. Note that the opaque PID is used to preserve provider privacy at a coarse grain (e.g., a network provider can assign two PIDs to nodes at the same point of presence or PoP). Note also that LOC can be used to compute network proximity, which can be helpful in choosing peers. When sending an info query, a peer may optionally include its swarm ID (e.g., info hash of a torrent). The iTracker may keep track of peers participating in a swarm.

  3. policy interface
  4. The policy interface allows others, for example peers or appTrackers, to obtain policies and guidelines of the network. Policies specify how a network provider would like its networks to be utilized at a high level, typically regardless of P2P applications; while guidelines are specific suggestions for P2P to use the network resources. To name a few examples of network policies: (1) traffic ratio balance policy, defining the ratio between inbound and outbound traffic volumes, for interdomain peering links; (2) coarse-grain time-of-day link usage policy, defining the desired usage pattern of specific links (e.g., avoid using links that are congested during peak times); and (3) fine-grain link usage policy. An example of network guidelines is that a network provider computes peering relationships for clusters of peers (e.g., clustered by PID). The policy interface can also return a set of normalized inter-PID costs, which indicate costs incurred to the provider when peers in two PIDs communicate.

  5. capability interface
  6. The capability interface allows others, for example peers or content providers (through appTrackers), to request network providers' capabilities. For example, a network provider may provide different classes of services or on-demand servers in its network. Then an appTracker may ask iTrackers in popular domains to provide such servers and then use them as peers to accelerate P2P content distribution.

    P4P Information Flow

    Q: We understand that P4P uses basic subscriber information from ISPs. What types of information does P4P use, and how is this information used to reduce P2P bandwidth consumption?

    There are multiple ways that P4P uses basic subscriber information from ISPs. One particularly useful piece of information about a subscriber is its location in the topology of an ISP network. An ISP network may spread at multiple locations. For example, a backbone ISP can have points of presence (PoPs) in many major cities.

    P4P reveals two types of information to a P2P-based content distributor who is trying to distribute content to subscribers of an ISP. The first is the PoPs of the subscribers, and the second is the "peering cost" between each pair of PoPs. If the cost between PoPs A and B is higher to the ISP than that between PoPs A and C, then the P2P application should direct its traffic so that subscribers at A are more likely to communicate with subscribers at C than with those at B.

    The preceding information allows the content distributor to make a more intelligent selection of peers than it would make without this data, and thus enable it to reduce transaction bandwidth cost to ISPs. Typically, an ISP will indicate that intra-PoP cost is lower than the cost to transfer a file among multiple PoPs. Therefore, it is more economical for subscribers at the same PoP to communicate with each other. This reduces long-haul inter-PoP bandwidth charges. Note that P4P can seamlessly handle the case where the opposite is true, i.e., intra-PoP is more expensive (see the BT example below in Q6), if the ISP indicates that it has a higher cost for such connectivity.

    Overall, getting P4P info from the ISP is that it can be driven by ISP business policies, which by definition cannot be reverse engineered by observing the network. For example, while route A may be "best" from a network perspective (latency, etc.), the ISP may prefer p2p data to use route B, because it is lower actual cost (e.g. peering link vs. paid transit) or has more available bandwidth, or A is reserved for other purposes.

    When using the locations of the subscribers, P4P assigns opaque numbers, which we call PIDs, to preserve ISP and subscriber privacy. For example, P4P may assign multiple PIDs to the same PoP or multiple PoPs the same PID. Thus, P4P isn't violating anyone's privacy, and isn't tied to individual identity.

    P4P can also use the bandwidth availability information of subscribers. For example, it may try to match subscribers with similar bandwidth availability to reduce under-utilized subscriber capacity (think of balancing of supply and demand, where the commodity is the bandwidth).

    Q: What is exactly is the "peering cost"? How is this information communicated from ISPs to subscribers?

    The "peering cost" between a pair of PoPs of an ISP can be quite generic and transparent to anyone except the ISP. These costs are not to be confused with payments. Instead, they are virtual costs reflecting the costs that an ISP associates with application-level peering over these links. These costs reflect ISPs' preferences regarding P2P connectivity, and can be used to capture a number of important ISP metrics, such as peak backbone utilization, congestion level, preferred connectivity links to other networks, and so on.

    A particular novelty in P4P is that it applies a powerful optimization technique called primal-dual decomposition to derive and update the costs. Through this decomposition technique, neither do the ISPs need to know the specifics of P2Ps, nor do P2Ps need to know the specifics of ISPs. The only coupling is the virtual cost.

    Let me give an analogy. Think of the connectivity between each pair of PoPs as a commodity. Some commodities have high cost to manufacture/procure and/or are in short supply (popular); others are not. An ISP uses higher costs to signal the first category. Just as price in real life provides a transparent interface to decouple producers and consumers (as a consumer we do not care how a producer produces a product, which can be quite complicated), the peering costs provide such an interface to guide the usage of ISP networks by P2Ps.

    At this point, it's important to note that the "peering cost" interface is only one of several types of possible information flow between ISPs and P2Ps. P4P allows other types of information flow as well. Information flow in P4P is specified in the Web Services Description Language (WSDL; http://www.w3.org/TR/wsdl) and is quite flexible. For example, another possibility is that the P2P content distributor makes the first round of selection on how subscribers should be peered, and the ISP makes the second round of selection from the first round.

    Information needed by P4P

    Q: Why is this information needed to make P4P work?

    It should be clear by now that the aforementioned information flow helps to improve network efficiency. Such information is communicated explicitly in P4P because it is difficult to obtain in the current Internet architecture.

    Specifically, in the current Internet, if a P2P seeks to explore its peering flexibility to improve network and application efficiency, it will have to probe the network to *reverse engineer* the aforementioned ISP and subscriber internal information. This is rather challenging in spite of significant progress in network measurement techniques. In addition to the redundant and wasteful network probing that the measurements might require, a fundamental hurdle would remain -- the ability to perform the necessary reverse engineering in an accurate manner. New technologies, such as MPLS, and routers that do not respond to measurement probes make it difficult to infer network characteristics. Available bandwidth and loss-rate estimation from end-host subscribers are difficult because their views are obscured by last-mile bottlenecks; it is difficult for a subscriber to identify which links are under-utilized or over-utilized.

    More importantly, cost and policy information of ISPs, which is communicated in P4P through the peering costs, are difficult, if not impossible, to reverse engineer without P4P. For example, it is difficult for a P2P to determine which peers are accessible through lower-cost inter-domain links (where the cost takes into account factors such as ISP policies, traffic balance ratio between peering providers, and 95-percentile based billing).

    Let me give two more examples:

    1. in a wireless network where P2P nodes communicate through a base station, peering using local peers sharing the same base station may require more wireless bandwidth than through the base station to other non-local peers.
    2. a common issue that exists in the UK is that network providers buy their DSL "last mile" connectivity via a BT central pipe, which provides connection from a DSL customer to its network provider. This connection can be many orders of magnitude more expensive than IP transit. Thus, it can be much more expensive for a network provider to have a customer retrieve a file from another customer on its network, than it would be to go off the network for the file. Such information is better communicated explicitly, as it is quite difficult, if not impossible, to infer.

    Overall, the objective of P4P is based on the observation that the lacking of such information in the current Internet exposes a fundamental issue of the current Internet: emerging applications such as P2P can have tremendous flexibility in how data is communicated, and thus, they should be an integral part of network efficient control. However, if subscribers are to participate in network resource optimizations, then the networks cannot continue to be opaque but need to provide a communication channel for collaborative traffic control. The objective of P4P is to provide such a communication channel.

    Q: We understand that for the P4P approach to work, P2P content distributor will have to share general information about network topology and customers and work with service providers. What specific types of information will P2Ps have to share?

    For many types of P2Ps, they do not need to share specific information with the ISPs. They obtain the cost information from the ISPs, and then conduct local computation to shape P2P connectivity and choose ISP-friendly communication patterns if possible.

    P4P Algorithm

    Q: How does P4P's approach work to reduce bandwidth consumption and speed up P2P downloads? What are the technical details behind these improvements?

    In previous Q&A, we have discussed how P4P reduces bandwidth consumption to ISPs. For more technical details, please see the paper.

    Let me add a few words on why P4P can speed up P2P downloads. There are several reasons that P4P can speed up P2P downloads:

    1. by peering subscribers at close locations and avoiding congestioned links, P4P reduces the latency between communicating subscribers and therefore speeds up their transmission. On the other hand, P2P without P4P may be network oblivious: a subscriber might be connected to peers at the opposite side of the planet while there are other subscribers close by; thus P2P without P4P may take unnecessarily long routes and consume more than necessary bandwidth.
    2. P4P tends to peer subscribers at the same ISP instead of between ISPs, because ISPs typically will assign higher costs to inter-ISP connectivity. Some ISPs provide higher bandwidth if a subscriber communicates to other subscribes at the same ISP. (Think about the family/friend plan in cell phones; you get more minutes or no charge if you talk to others belonging to the same cellphone carrier.) Thus, in P4P, subscribers are connected more to subscribers with higher bandwidth in such settings. On the other hand, P2P without P4P may not be aware of this synergy.
    3. P4P provides a capability interface for P2Ps to request ISP support. One support is that the ISPs provide, on-demand, a few stable, powerful servers to participate in content distribution. This can speedup content distribution. It also provides ISPs with another revenue source.

    Q: We understand the P4P approach would add network intelligence so P2P systems get content more efficiently and from the closest possible peers. Exactly how will this work? What are the technical details involved?

    see previous Q&A.

    Q: How will P2Ps work with ISPs to make this work?

    see previous Q&A. <-->

    Why P4P is so different?

    Q: Are ISPs and P2Ps not sharing this information and working together this way now?

    No, they are not.

    Before P4P, there was no way for P2P application distributors and ISPs to share this information. The current Internet does not have a protocol to allow explicit communications between applications and ISPs. The communications in the current Internet is implicit: Traditional ISP feedback/controls to application traffic is only through congestion feedback (packet drops) and implicit routing. These are ineffective for P2P due to highly dynamic, scattered traffic patterns caused by dynamic, unguided (network-oblivious) peer selection.

    Q: We understand that, historically, these groups have been reluctant to work with each other. Why is this so, and what effect could this reluctance have on P4P?

    Historically, P2P content distributors and ISPs have not collaborated much. In fact, the relationship some ways has been an adverse relationship.

    One concern is the legality of the content. It should be noted that the P2P technology itself is not under attack; rather, inappropriate usages of it are. The issue is being addressed. There is wide recognition that the Internet is the media delivery platform of the future. In June 2007 at the D5 Conference, Steven Ballmer, CEO of Microsoft, stated that "Within five years, all media will be delivered across the Internet." P2P presents a disruptive technology that can substantially reduce distribution costs while supporting trackable content. Starting from 2007, we have seen that content owners prefer to buy integrated P2P + CDN solutions, and major content and CDN players have selected P2P technology partners. Thus, P2P is getting integrated into distribution of legal and copyrighted content. This provides good motivation for the success of P4P.

    Another issue between them is that largely ignoring network efficiency issues, many P2Ps impose a substantial load and cost on ISPs. According to CacheLogic, 60-70% of Internet traffic is contributed by P2P. This may also lead to degraded performance of other applications of ISPs. P4P is trying to address this issue.

    P4P Performance and Experiments

    Q: By what specific percentage has the P4P approach reduced bandwidth consumption, and by what specific percentage has P4P sped up P2P downloads?

    The exact benefits of P4P will depend on many factors: the topology of the ISP networks, the bandwidth usage policies of ISPs on subscribers (e.g., lower rates to other ISPs), the distribution and capacity of subscribers, the nature of the content to be distributed (e.g., file or streaming), and so on.

    So far, in a wide range of scenarios, compared with P2P without P4P, P4P typically can reduce aggregated P2P bandwidth consumption at least by a factor of 2; it often can reduce bandwidth consumption on bottleneck links (the link with the highest level of congestion) by a factor of more than 2. We observe more than 60% reduction in ISP transit costs (which are costs that an ISP pays to its service providers).

    In terms of P2P download speedup, we have observed typical speed-up from around 20% to 30%. For ISPs with higher intra-ISP bandwidth than inter-ISP bandwidth, the speed-up can be much more substantial, reaching 50% or much higher.

    Q: It is our understanding that Verizon and Pando are either planning to test or have already tested this technology in Verizon's network. Can you share details on these plans and the status of the testing today?

    We will be conducting the field tests in Feb. and Mar.

    Please check out the results of field tests here.

    P4P Deployment and Future

    Q: When is the P4P technology going to be ready for commercial use?

    This is difficult to predict. We are pushing as hard as we can.

    The P4P Working Group (P4PWG), under the leadership of P4PWG Co-Chairs Laird Popkin (Pando) and Doug Pasko (Verizon), with the hosting of the Distributed Computing Industry Association (DCIA), is doing an excellent job at fostering collaboration between ISPs and P2Ps. The P4P Working Group now has 18 core members (AT&T, Bezeq International, BitTorrent, CacheLogic, Cisco Systems, Grid Networks, Joost, LimeWire, Manatt, Oversi, Pando Networks, PeerApp, Telefonica Group, VeriSign, Verizon, Vuze, Univ of Washington, and Yale University) and 18 observers (Abacast, AHT International, Akamai, Alcatel Lucent, CableLabs, Cablevision, Comcast, Cox Comm, Juniper Networks, Microsoft, MPAA, NBC Universal, Nokia, RawFlow, Solid State Networks, Thomson, Time Warner Cable, and Turner Broadcasting). It is quite encouraging to see this level of collaboration.

    Q: Do you think P4P technology will be widely adopted? Why or why not?

    Yes. I think so. It is addressing an important problem.

    When we were presenting the P4P idea at NYC P2P Workshop in May 2007, Haiyong Xie met Laird Popkin and learned from industry feedback that the problem was widely recognized. The P4P solution is a clear win-win for all involved. It is simple, effective and extensible. For example, we can envision some sort of revenue-sharing agreement between ISPs and P2P for improved QoS. It addresses not only the current P2P problem but also other applications. So it is not just a temporary fix.

    Q: What is your opinion about the future of P2P technology? Are there any dark clouds or potential obstacles?

    I am quite optimistic about the future of P2P technology. As previous Q&A already discussed, it presents a disruptive technology and can substantially reduce content distribution costs. There are still quite interesting technical issues to be addressed. For example, how to achieve truly carrier-grade P2P content distribution. The future is promising.

    Other Concerns

    Q: By singling out some applications, does the P4P scheme violate network neutrality?

    The objective of P4P is to provide a portal for applications and network providers to communicate. In our current deployment model, ISPs and P2P applications mutually agree to participate in P4P.

    Q: Compare P4P with Pricing or Priority Schemes. One possibility to address P2P problem is to introduce multiple levels of traffic priority or non-flat-rate pricing. Will this eliminate the need for P4P?

    P4P will be complementary with such schemes, as these schemes will only impose constraints on the access bandwidth from the network edges. They will not resolve the network-wide efficiency problem.

All News

Initial P4P Field Tests Completed

March 10, 2008

The key component in the tests is the P4P algorithm proposed by Haiyong Xie et al. in a recent SIGCOMM paper. Xie implemented the algorithm and iTracker server. The test also uses clients of Pando Networks, and network topology of Verizon[…]

Read more…

X-ray Vision: P4P Winning Over ISPs

Computer Power User Magazine, Apr 01, 2009
Full Article Here

Supercharged File Sharing: Cooperating with file-sharing networks could avert congestion

MIT Technology Review, Dec 15, 2008
Full Article Here

A Look Ahead: P4P & DigiMeld

DigiMeld, Dec 15, 2008
Full Article Here

2008: The Year ISPs Got Real About P2P Video

New TeeVee, Dec 13, 2008
Full Article Here

Comcast P4P Results 'Better Than Anyone Expected'

Contentinople, Nov 04, 2008
Full Article Here

Comcastic P4P trial shows 80% speed boost for P2P downloads

ARs Technica - The Art of Technology, Nov 03, 2008
Full Article Here

Internet Congestion: ISPs Don Traffic Cop Uniforms

E Commerce Times, Oct 18, 2008
Full Article Here

Verschnellerung oder Ausbremsung?

Telepolis, Sep 08, 2008
Full Article Here

P4P can make P2P more efficient, if ISPs help

Guardian, Aug 25, 2008
Full Article Here

Throttling P2P traffic is shortsighted, experts say

ITBusiness.ca, Aug 22, 2008
Full Article Here