On 2 February 2016, productivity across the UK ground to a halt, and social media lit up in indignation, after a faulty router on BT’s network brought down broadband services to tens of thousands of consumers and businesses.
Although engineers were swiftly on scene, and the fault with the service was largely resolved within a few hours, the cost to the UK economy in terms of lost business and man-hours of a major failure is likely to run into tens of millions of pounds.
For BT, this was the latest in a string of outages. A swift look at its service status website gives the casual observer some idea of the sheer size of the task its engineers face in keeping everyone online 24/7 – but such a widespread and pronounced failure is far beyond what it normally has to deal with.
Thankfully, large service outages of this type are rare, which for BT is probably something of a relief, according to Joshua Raymond, chief marketing officer at contract for difference (CFD) and foreign exchange trader XTB, for whom broadband connectivity is critical to operate in a global market 24 hours a day.
“Given the sheer intensity of competition within the telecom and broadband sector, this outage is bad news for BT and could cause short-term damage to its brand reputation,” said Raymond.
BT’s network hiccup happened on the same day that internet service provider TalkTalk admitted it had lost more than 100,000 customers following a cyber attack on its systems in October 2015. Analyst Imran Choudhary at Kantar Worldpanel said that, as a result, users had “lost faith” in the business.
Although BT suffered a technical fault, rather than a damning security breach, Raymond feels its outage serves as a “clear and present risk to the retention of its [BT’s] customer base”.
BT could be forced into paying small compensation claims Joshua Raymond, XTB
“BT could also be forced into paying small compensation claims to keep its customers happy,” he added.
Martin Li, chief market strategist at XTB, says BT is dealing with a lot of disgruntled customers in the financial services industry who are highly dependent on broadband.
“Other large-scale outages could really hurt its performance and growth prospects,” he said.
Kantar Worldpanel’s Choudhary says that in contrast to TalkTalk’s security breach, which left customers confused and bewildered because of a perception that nobody at TalkTalk really knew what was going on, BT handled its downtime well.
“They were honest in terms of what was happening,” he said. “They put out a consistent message, and once it was repaired, there was a genuine explanation for it.”
This sort of approach would serve BT business customers well, too, he added.
Paul Evans, CEO of Boosty, a recently launched provider of solutions that enable users to supplement their broadband and boost their network speed using mobile broadband services on their smartphone, said the reputational damage to BT may be substantial.
“An outage impacts the experience for your customers,” he said. “Even a few minutes can cost thousands of pounds because the phones simply stop ringing.”
With angry customers increasingly likely to approach brands on social media, it makes sense for organisations to brush up on their strategy for engaging with them effectively, making sure staff are well-informed about what is going on, keeping clear lines of communication open and, above all, not losing their cool on the internet.
Have a failsafe
Risk management, coupled with the usual elements of a coherent disaster recovery plan, such as setting objectives around recovery processes and procedure, and establishing clear service-level agreements (SLAs) with external providers should, of course, be second nature to the average IT department.
But there is another way in which organisations can protect themselves: businesses without their own corporate networks or datacentres, SMEs and web-dependent outfits should think about deploying multiple options where possible.
For Hubert Da Costa, vice-president of EMEA at 4G long-term evolution (LTE) solutions supplier Cradlepoint, the BT outage highlights a dangerous over-reliance by businesses on fixed networking solutions.
“Tuesday’s nationwide BT broadband outage revealed how vulnerable UK businesses are to fixed-line service interruptions, and underlined just how fundamental this technology is to our working lives,” he said.
“Organisations need to build much greater levels of protection into their communications infrastructure to ensure they can maintain ‘business-as-usual’ operations.
Da Costa urges businesses to adopt a telecoms strategy that employs failover capability which can mitigate risks around lost sales and employee productivity.
Protection against fixed-line service failure should figure more actively Hubert Da Costa, Cradlepoint
“As businesses assess the cost of this major service interruption, and think again about their ability to overcome a similar situation in the future, protection against fixed-line service failure should figure more actively in their technology strategy,” he says.
Boosty’s Evans also agrees on the importance of having some form of failsafe. But this should not necessarily mean resorting to tethering smartphones to connect their other devices to the internet, he says.
“This is a technical and time-consuming process and one that not all consumers may be aware of and there are cost implications,” he said.
“Hybrid access solutions that combine 4G signals with fixed-line broadband are resilient enough to maintain a broadband connection in the event of an outage. Configured to enable faster internet access, hybrid solutions can function even when the fixed network fails by automatically switching over to the mobile broadband connection.
“This provides customers with reliable internet access, reducing disruption while the operator traces the cause of the outage.”
During the 2 February outage, Evans says Boosty received a number of reports from customers with BT digital subscriber line (DSL) services who said they had barely even noticed the outage, because their connections had automatically failed over to mobile broadband.
“We now have quite a few people using our technology for resilience because if they are using voice-over-IP [VoIP], social media or cloud services, they are incredibly reliant on their connection,” he added.