Broadband "always on" connections are taken for granted. But demand spikes can slow down the whole network. Are the days of cheap, plentiful bandwidth numbered?

During the first months of 2007 a select few million people were fortunate enough to be offered early access to Joost, a website that delivers high-quality video over the internet, developed by Niklas Zennström and Janus Friss.

The duo, also behind online telephony provider Skype, are signing up content providers for the official launch of their new offering later this year, and the feeling is that it could do as much damage to the business models of commercial television providers as Skype did to traditional telecoms companies, forcing them to offer VoIP (voice over internet protocol) services, and reducing their revenues in the process.

Joost's real impact, however, may be felt elsewhere. In order to deliver high quality, full-screen video it has to pump vast amounts of data over the internet, and experts such as Eric Openshaw, principal and leader of the Technology, Media and Telecoms practice at Deloitte America, fear the network may not be able to take the strain.

Joost downloads about 320 megabytes of data per hour, and the peer-to-peer model it uses means that each viewer is also sending up to 150 megabytes to other users, the equivalent of 50 iTunes songs.

Joost is not the only high-bandwidth service targeting home users. BT Vision, the internet TV service from BT, requires fast connectivity, as do popular video chat services offered by Yahoo!, Microsoft and Apple. Games players who plug their consoles into the network for interactive play on Microsoft's Xbox Live or Sony's PlayStation Network expect fast responses and reliable links, while the millions of people uploading and downloading videos from YouTube or putting photos on Flickr, Photobucket, MySpace and Facebook are quick to complain if the network slows down or their connection is lost.

Businesses have long been heavy bandwidth users, and new services will add further pressure. Thousands of small firms are adopting online collaborative tools such as Basecamp to manage projects, or Amazon's S3 (simple storage service), to host data online instead of running their own servers. Some are even offloading their entire IT infrastructure, from email to word processing, onto Google.

Simon Appleton, the Brighton-based entrepreneur who founded the successful online recruitment service PlanetRecruit, is now CEO of job search engine WorkCircle. His small team uses a variety of online tools, including Central Desktop for sharing documents and Campfire to host virtual meetings, keeping the need for office attendance to a minimum.

The same is true at Cambridge-based data protection company nCipher. CEO Alex van Someren says the firm uses collaborative and IP-based tools such as NetMeeting, Skype and WebEx regularly "to minimise travel and maximise productivity". These services all depend on fast, reliable networks, and can use a lot of bandwidth when large files are being shared.

The Net is also used to maintain contact with customers and van Someren admits his company is "strongly dependent on good internet bandwidth to maintain our dialogue with customers, employees and partners". Simon Appleton points out that "as a distributed company, we need the network for both external and internal communication, as well as the operation of our website, which is our business".

It might seem there is more than enough capacity on the internet to meet even the most ambitious projections for growth. New fibre-optic links are being installed to extend the "long-haul" backbones that move data between cities, countries and continents, and breakthroughs in signal transmission hold out the prospect of faster data speeds.

The Internet2 project, a university-led research consortium developing new network technologies, has announced a 100 gigabit per second link between Washington and New York, sufficient to download a DVD movie every second. This is an astonishing technical achievement and one which indicates that we are unlikely to be short of backbone bandwidth.

But fast backbones may not be enough. Other network components must also work harder. In February 2007, an attack on the servers that keep track of computer names, the Domain Name System, slowed the whole network significantly and the routers, which direct data around the network, require constant upgrades to keep up with traffic growth.

It's largely a problem of capacity. At the end of every high-speed cable there needs to be a switch that looks at each of the billions of data "packets" being transmitted to decide where it goes next. The internet is a vast collection of connected networks, and the routers decide how data flows efficiently between them.

Keeping up with network growth poses a serious challenge to companies like Cisco Systems, which manufactures the routers that glue the network together. Speaking recently to the BBC, Phil Smith, Cisco's head of technology and corporate said: "Routers have come a long way. Those we're talking about now can handle 92 terabits per second. We have enough capacity to do that and drive a billion phone calls from those same people who are playing a video game at the same time as they're having a text chat."

Cisco's confident assertions won't mollify everyone. Alex van Someren of nCipher is concerned about new services, pointing out that  "the growth in spam, VoIP and video social networking sites is chewing up bandwidth at an accelerating pace." His concern is that today's Web 2.0 services require more than simply sending Web pages to clients. Services like Google Maps continuously exchange data between the user's browser and Google's map server, and this, too, is straining the network.

The end result is that users notice when the connection slows down, because the mapping service stutters and the near-instant response time becomes a few seconds, breaking the illusion of a seamless and real-time interaction. This isn't just an irritant to baseball-capped home gamers. Delays in updating a shared, online whiteboard can severely disrupt a business meeting, while pixellation or lost audio in a critical video conference could jeopardise a deal.

There's potential for a major overload, too. Andrew Coburn is vice-president of Catastrophe Research at Risk Management Systems, which provides risk assessment for insurance and financial companies. He says that while we can cope at current traffic levels, an external event like a terrorist attack or a flu pandemic could easily overload today's strained network.

"A massive increase in traffic that resulted in overloads of router buffers and caused localised and progressive failure through the network is one of our major concerns," he warns. "Packet loss leads to degradation of service, increased waiting times and reduces reliability to the point of unusability."

There isn't enough information to determine how real the risk is. "Each of the many thousands of individual service providers knows their own demand-supply relationships and the reliability that they expect," adds Coburn. "But these data are not public, and each sees only their own small part of the overall network. Nobody has more than a glimpse into the total network that comprises today's internet."

Fast, reliable backbone links and a new generation of routers supporting network protocols designed to make traffic flow more efficiently may solve the long-haul problem, but they do little to tackle the "last mile", the distance between the high-speed backbone and the end user. In April, telecoms regulator Ofcom reported that over half of UK adults now have a broadband internet connection from home, with an average speed of just under four megabytes per second. It also found that 29 per cent of them are listening to online audio, and 26 per cent are watching video at least once a week. And over half of us are now also making phone calls over the internet.

The rapid rise in domestic internet use is an opportunity for businesses to trade online, but it also creates problems, especially for small businesses connected to the network using the same ADSL (asymmetric digital subscriber line) or cable links offered to home users. Larger companies are more likely to have dedicated data connections over leased lines or even fibre. nCipher has one permanent, high-speed, leased line and one ADSL back-up line at each major trading location, while Workcircle relies solely on ADSL.

One indication that ISPs (internet service providers) are aware of the problem is the introduction of data capping and "bandwidth shaping," where the capacity of a connection is temporarily reduced to help manage demand. In the UK, Virgin Media has started doing this, as has AT&T in the US.

But limiting network use is a short-term solution; providers need incentives to change their offerings. According to Appleton, "it's time for ISPs to stop hiding behind caveats about speed, and stop tying us into long contracts." He believes that once customers can move more easily between suppliers then the pressure  will be on to find better solutions.

The network is not going to fall over, at least not unless we have a disaster of the scale of the 2006 earthquake in the Indian Ocean that severed nine undersea cables and cut most of southern China off from the  internet for several days. But the assumption that ISPs will deliver the bandwidth they promise is now questionable. And businesses that rely on fast, trouble-free network access should be aware of the risks. 

The days of cheap and plentiful bandwidth, like the days of cheap and plentiful oil, may be drawing to an end. Our reliance on the network to support our businesses, our social lives and the wider economy means we may have to accept that the current price of a high-speed internet connection is too low, and that a market correction is due.

A correction could come through differential service provision. At present, ISPs charge customers according to the speed of their connection and not by the amount of data sent and received. In the US, network providers want to charge heavy users like YouTube and Google Video for delivering their content. Google and other providers argue that the network should not discriminate in this way and that "network neutrality" must be preserved and the current model maintained.

Network neutrality sounds desirable, but it creates major difficulties for providers trying to service home user demands for streamed video and still offer reliable service to businesses.

Some businesses are willing to abandon neutrality and pay more for a better service, including Alex van Someren at nCipher. "With the consumption of IP bandwidth rising fast, commercial traffic and private traffic are almost certainly going to have to be separately managed to maintain service levels. I would be happy to pay for this kind of service distinction," he says. Other companies may be less receptive. But they may have no choice if they want an internet connection they can rely on.

Bill Thompson is an independent writer who appears regularly on Digital Planet and writes for the BBC News website. For more by Bill Thompson, visit http://news.bbc.co.uk/1/hi/technology/6958615.stm