Friday, October 28, 2005
Level3 and Cogent End Peering Dispute!
Congratulations to Cogent's Dave Schaeffer and Level3's Jim Crowe for putting first things first! Their peering dispute seems to be resolved. [Cogent statement] [Level3 statement]
Last October 7, when Level3 abruptly ended its peering arrangement with Cogent in a business dispute, leaving customers in the lurch, I was puzzled. I've spent enough time with Crowe and Schaeffer to respect both of them deeply. Why would they let a silly business dispute blow up into an event that might potentially weaken the whole Internet?
Last week, Daniel Berninger kindly invited me to help draft a letter expressing our concern for the effect of such acts on the health of the Internet. Our aim was to get a bunch of important Internet architects, builders, users and policy people to sign it. We were going to release it on Monday, November 7; Jim Crowe had threatened to escalate on November 9th. We didn't want to twist anybody's arm or dictate business terms, but we did want to focus some sunshine (and maybe a little heat) on the process, give voice to customer frustration, and advocate for 'net goodness.
Fortunately, we didn't need to revise our letter beyond Draft 0.3.1!
But I did want to share the last version of the letter we were working on:
Last October 7, when Level3 abruptly ended its peering arrangement with Cogent in a business dispute, leaving customers in the lurch, I was puzzled. I've spent enough time with Crowe and Schaeffer to respect both of them deeply. Why would they let a silly business dispute blow up into an event that might potentially weaken the whole Internet?
Last week, Daniel Berninger kindly invited me to help draft a letter expressing our concern for the effect of such acts on the health of the Internet. Our aim was to get a bunch of important Internet architects, builders, users and policy people to sign it. We were going to release it on Monday, November 7; Jim Crowe had threatened to escalate on November 9th. We didn't want to twist anybody's arm or dictate business terms, but we did want to focus some sunshine (and maybe a little heat) on the process, give voice to customer frustration, and advocate for 'net goodness.
Fortunately, we didn't need to revise our letter beyond Draft 0.3.1!
But I did want to share the last version of the letter we were working on:
Mr. Crowe:In the process, I sure learned a lot about peering, for example, from Susan Crawford's blog post on Why De-Peering Matters. And Dan and I (mostly Dan) put together a nice FAQ which said, in part,
Mr Schaeffer:
De-peering threatens the universal connectivity of the Internet. We ask that Level 3 and Cogent resolve their dispute without de-peering their networks.
Universal Internet connectivity is in your larger interest and ours. Peering is necessary for universal connectivity, which is an essential component of the more-faster-cheaper-better virtuous circle that powers info tech industry growth, expands the Internet's usefulness to everybody, and puts more traffic on the Internet to grow carrier revenues.
Light-handed Internet regulation and governance is in your interests and ours. But as de-peering disrupts both Level 3 and Cogent end users, people are beginning to demand heavier regulation via US, EU, and/or UN based intervention. Renewed de-peering, such as that announced by Level3 for November 9, will make such demands louder. In addition, de-peering might set a dangerous precedent for the behavior of even larger carriers.
Therefore, we ask that Level3 and Cogent consider other ways to resolve the current dispute in ways that do not threaten the well-being of the Internet as a whole.
Yours truly,
The consolidation of backbone providers with the acquisition of AT&T and MCI by SBC and Verizon threatens more scarcity and rising prices . . . the loss of settlement free peering among Tier 1 providers threatens to reduce inter network performance and raise costs that get passed on to end users. The Internet ecosystem will suffer to the extent a toll collecting ethic replaces the value creation mode of competition.and
The decisions of Level 3 and Cogent will affect more than just their direct customers. The resolution of this peering dispute has important implications for everyone presently benefiting from the role of settlement free peering.But Jim Crowe, himself, actually said it best. Here are his own words from year 2000 in contemplation of a Worldcom-Sprint merger:
"The free, open and competitively neutral interconnection of competing networks is the real key to competition. In the emerging Internet industry, interconnection is called 'peering'. Whatever you call it, it is the single most important gatekeeper to competition. It lets information flow unimpeded from one company's network to another's, allowing users of one network to communicate with users of another network.
"We have consistently told the Justice Department, the European Commission, the FCC and others, that competition in communications . . . depends on open, cost effective and non-discriminatory connection of the networks of competing companies.
snip
"We at Level 3 believe that open and fair interconnection is just as important to the development of a competitive Internet backbone industry as it was to the development of the competitive telephone services market."I'm glad Jim and Dave exercised wisdom and perspective in this matter. We have more important Internet threats to worry about!
Technorati Tags: Cogent, DontBeEvil, Internet, Level3, peering
Comments:
Dave,
I think you're missing the point on this one. This is not just a "come on guys, can't we all get along" sort of comment. All things have to have an economic justification. Peering is justified because every network is a customer of every other network, AT SOME LEVEL. Hence, instead of dealing with the enormous OSS issues related to contracting usage prices and billing, firms decided that if the levels of traffic were more or less equal, they would call it a wash. Hence, we have free peering.
That's NOT what was going on with Level 3 and Cogent. Cogent has been systematically abusing the peering system, knowing that network operators like you would whine to Level 3 to put Cogent back on. Cogent's business model makes no sense, so they terminate traffic on "peers" because otherwise having to pay transit fees destroys their idiotic business model.
Unlike other ISPs who know that their traffic levels are too small and agree to pay to peer or buy transit services, Cogent "blackholes" companies like Level 3 who de-peer them, and then puts on a marketing blitz blaming Level 3 for cleaving the Internet.
Companies have a right to de-peer anyone they don't deem a "peer". They aren't in the business to provide a social service. They are in business to make a profit on their investments. If peering becomes too costly, then they have a right to opt out.
Lastly, I can't believe that you are putting Jim Crowe and Shaeffer in the same category. Shaeffer is a charlatan and master of hyperbole who makes absurd statements like
1 -- Cogent represents 12% of the internet and
2 -- Cogent has more than 90% of its capacity free to sell at no incremental cost to itself.
I think you're missing the point on this one. This is not just a "come on guys, can't we all get along" sort of comment. All things have to have an economic justification. Peering is justified because every network is a customer of every other network, AT SOME LEVEL. Hence, instead of dealing with the enormous OSS issues related to contracting usage prices and billing, firms decided that if the levels of traffic were more or less equal, they would call it a wash. Hence, we have free peering.
That's NOT what was going on with Level 3 and Cogent. Cogent has been systematically abusing the peering system, knowing that network operators like you would whine to Level 3 to put Cogent back on. Cogent's business model makes no sense, so they terminate traffic on "peers" because otherwise having to pay transit fees destroys their idiotic business model.
Unlike other ISPs who know that their traffic levels are too small and agree to pay to peer or buy transit services, Cogent "blackholes" companies like Level 3 who de-peer them, and then puts on a marketing blitz blaming Level 3 for cleaving the Internet.
Companies have a right to de-peer anyone they don't deem a "peer". They aren't in the business to provide a social service. They are in business to make a profit on their investments. If peering becomes too costly, then they have a right to opt out.
Lastly, I can't believe that you are putting Jim Crowe and Shaeffer in the same category. Shaeffer is a charlatan and master of hyperbole who makes absurd statements like
1 -- Cogent represents 12% of the internet and
2 -- Cogent has more than 90% of its capacity free to sell at no incremental cost to itself.
Strong words, Shaitaangul, and fascinating! I plead ignorance and would like to learn more. But first let me completely agree that peering is a business arrangement that requires economic justification, and that companies have a total right to de-peer with companies they don't deem a peer.
But citizens also have rights to take issue with legitimate business decisions, and such citizen protests have sometimes caused beneficial changes. The boycott of companies doing (legitimate, economically well-motivated) business in South Africa, which ended Apartheid, is a good example.
Nevertheless, I am intrigued by some of your comments. Could you please help me understand what *was* going on? For example, how is it that, "Cogent's business model make no sense"? (After all, they have been in business a long time!) Are you saying that Cogent only appears to makes money because it is fundamentally deceptive?
Also, you say, at the bottom of your comment that, "Shaeffer is a charlatan and master of hyperbole who makes absurd statements like
1 -- Cogent represents 12% of the internet and
2 -- Cogent has more than 90% of its capacity free to sell at no incremental cost to itself," I'd like to know how you know these are false statements. They might be, but could yu let us knowhow you know, and what the "real" data from your sources are?
Also, you refer to "network operators like you," meaning me, I think. Minor correction. I am simply a commentator. The only network I operate is the wi-fi hotspot in my house . . . :-)
More please, Shaitaangul!
David I
But citizens also have rights to take issue with legitimate business decisions, and such citizen protests have sometimes caused beneficial changes. The boycott of companies doing (legitimate, economically well-motivated) business in South Africa, which ended Apartheid, is a good example.
Nevertheless, I am intrigued by some of your comments. Could you please help me understand what *was* going on? For example, how is it that, "Cogent's business model make no sense"? (After all, they have been in business a long time!) Are you saying that Cogent only appears to makes money because it is fundamentally deceptive?
Also, you say, at the bottom of your comment that, "Shaeffer is a charlatan and master of hyperbole who makes absurd statements like
1 -- Cogent represents 12% of the internet and
2 -- Cogent has more than 90% of its capacity free to sell at no incremental cost to itself," I'd like to know how you know these are false statements. They might be, but could yu let us knowhow you know, and what the "real" data from your sources are?
Also, you refer to "network operators like you," meaning me, I think. Minor correction. I am simply a commentator. The only network I operate is the wi-fi hotspot in my house . . . :-)
More please, Shaitaangul!
David I
Dave,
in my enomous ignorance, I forsee a Future where big expensive backbones will charge to "peer" and small ones will not.
But,as it happens in other market fields like TV, this WILL bring the majority of customers to the "free" backbones...
We will have different levels and different prices.
But I really do not see it as a negative side.
We will have the "commercial, low level" Internet and the "pay, higher level".
Then somebody will realize that the numbers of users makes the value of a Network, and the Value of the Network means higher revenues...
Is it better one dollar out of 1 million or 100 dollars out of 100?
At the end THIS is going to be the real matter...
in my enomous ignorance, I forsee a Future where big expensive backbones will charge to "peer" and small ones will not.
But,as it happens in other market fields like TV, this WILL bring the majority of customers to the "free" backbones...
We will have different levels and different prices.
But I really do not see it as a negative side.
We will have the "commercial, low level" Internet and the "pay, higher level".
Then somebody will realize that the numbers of users makes the value of a Network, and the Value of the Network means higher revenues...
Is it better one dollar out of 1 million or 100 dollars out of 100?
At the end THIS is going to be the real matter...
Dave,
For a long time now, most companies have been playing by the informal rules of Internet network management. By and large, interconnection negotiations in the backbone were done in an atmosphere that encouraged and practiced cooperation. However, if one party enters into this context and decides to abuse the system, that could obviously bring the atmosphere of cooperation to an end. I am claiming that this is Cogent's M.O. Their business plan simply would not work otherwise.
Let's just review. Cogent has been pursuing a $1000/10 0 Mb/Sec / month strategy for quite a while. Their theory was that if they can get enough users, at some scale, the price they charge will be higher than costs of providing service to the average customer. The reason for this is that ISPs have increasing returns to scale at almost all times, so marginal costs are pretty low. However, when you are building up users, AVERAGE COSTS will be high, because the fixed costs of setting up a network are pretty large and play a larger role in average costs when the business is starting out.
This approach seems like it would work to a naive observer, and many telecom businesses have pursued this strategy right into bankruptcy within the last 10 years, including Cogent. However, if you look at other industries that exhibit network effects and increasing returns to scale, they do not behave in this manner. Software is the most striking example. The marginal cost of producing the incremental unit of Windows or Oracle 11g is almost 0. That's why the gross margins of software companies are in the 90-95% range. But, you don't see any software companies pricing irrationally just to get market share, nor any investors willing to finance such shenanigans.
Telecom, unfortunately, has many unsophisticated investors that have little clue as to how complex the business really is, so there are SUCKERS that are willing to finance such businesses. The same thing can be said about the US airline industry, which when set up as a hub-and-spoke model, also displays such increasing returns to scale characteristics. Also, just like the airline industry, one or two irrational competitors who don't understand pricing can screw things up for the whole industry.
The beneficiary of such irrationality in both the airline industry and IP telecom industry has been the consumer. They have benefitted quite a lot, at the expense of a number of investors who've lost all their money supporting these businesses. Now, in all industries, you have companies that can't make it and go bankrupt. However, in the airlines and telecom, this is the RULE not the EXCEPTION. That tells you something is seriously wrong in the design of the industry altogether.
In my opinion, Level 3 was doing the absolute right thing by looking at their peering arrangements and deciding to de-peer those that didn't make sense. If one party, like Cogent, doesn't like it and wants to whine to the market the Level 3 is the bad guy, then Cogent is just using the ignorance of the IP world to hold everyone in the network hostage.
If this is allowed to go on, in the end, this industry will wind up like the airline industry -- a crappy industry for who no one wants to work, with lousy service, and no real innovation.
For a long time now, most companies have been playing by the informal rules of Internet network management. By and large, interconnection negotiations in the backbone were done in an atmosphere that encouraged and practiced cooperation. However, if one party enters into this context and decides to abuse the system, that could obviously bring the atmosphere of cooperation to an end. I am claiming that this is Cogent's M.O. Their business plan simply would not work otherwise.
Let's just review. Cogent has been pursuing a $1000/10 0 Mb/Sec / month strategy for quite a while. Their theory was that if they can get enough users, at some scale, the price they charge will be higher than costs of providing service to the average customer. The reason for this is that ISPs have increasing returns to scale at almost all times, so marginal costs are pretty low. However, when you are building up users, AVERAGE COSTS will be high, because the fixed costs of setting up a network are pretty large and play a larger role in average costs when the business is starting out.
This approach seems like it would work to a naive observer, and many telecom businesses have pursued this strategy right into bankruptcy within the last 10 years, including Cogent. However, if you look at other industries that exhibit network effects and increasing returns to scale, they do not behave in this manner. Software is the most striking example. The marginal cost of producing the incremental unit of Windows or Oracle 11g is almost 0. That's why the gross margins of software companies are in the 90-95% range. But, you don't see any software companies pricing irrationally just to get market share, nor any investors willing to finance such shenanigans.
Telecom, unfortunately, has many unsophisticated investors that have little clue as to how complex the business really is, so there are SUCKERS that are willing to finance such businesses. The same thing can be said about the US airline industry, which when set up as a hub-and-spoke model, also displays such increasing returns to scale characteristics. Also, just like the airline industry, one or two irrational competitors who don't understand pricing can screw things up for the whole industry.
The beneficiary of such irrationality in both the airline industry and IP telecom industry has been the consumer. They have benefitted quite a lot, at the expense of a number of investors who've lost all their money supporting these businesses. Now, in all industries, you have companies that can't make it and go bankrupt. However, in the airlines and telecom, this is the RULE not the EXCEPTION. That tells you something is seriously wrong in the design of the industry altogether.
In my opinion, Level 3 was doing the absolute right thing by looking at their peering arrangements and deciding to de-peer those that didn't make sense. If one party, like Cogent, doesn't like it and wants to whine to the market the Level 3 is the bad guy, then Cogent is just using the ignorance of the IP world to hold everyone in the network hostage.
If this is allowed to go on, in the end, this industry will wind up like the airline industry -- a crappy industry for who no one wants to work, with lousy service, and no real innovation.
Dave,
Shaeffer's statements were publically repeated by an analyst who works for Lehman Brothers. He published comments in a report soon after the Level 3 / Cogent thing went public. Furthermore, I believe Shaeffer has made these statements in investor conferences many times. I also remember hearing him claim that their network is more efficient than anyone else because they are all IP, whereas everybody else is running IP over SONET, basically claiming that Layers 1 and 2 of the OSI stack were totally unnecessary and a source of unnecessary expense.
Post a Comment
Shaeffer's statements were publically repeated by an analyst who works for Lehman Brothers. He published comments in a report soon after the Level 3 / Cogent thing went public. Furthermore, I believe Shaeffer has made these statements in investor conferences many times. I also remember hearing him claim that their network is more efficient than anyone else because they are all IP, whereas everybody else is running IP over SONET, basically claiming that Layers 1 and 2 of the OSI stack were totally unnecessary and a source of unnecessary expense.
