The PopVote attack aftermath

As the voting period draws an end on PopVote -- the website hosting Hong Kong's unofficial online referendum on political reform--Computerworld Hong Kong talked with its cloud provider CloudFlare and other security vendors to find out the lesson learned from dealing with massive DDoS attacks.

With traffic reaching 300Gbps at the peak of the attack, PoPVote experienced a massive scale of DDoS attack last month, paralyzing the system. Though the scale is larger than the regional average and the largest in Hong Kong, according to the site's organizer The University of Hong Kong, it is not a scale unheard of.

"On average, DDoS attacks are about 20 Gpbs in traffic," said Sudeep Charles, product marketing manager at Akamai Technologies. "[This attack] was far larger than the ones observed in S.E. Asia."

"The attack against PopVote was a very large and sophisticated attack," said Matthew Prince, CEO of CloudFlare. "However, we've seen other attacks at similar scale."

Prince told Computerworld Hong Kong, one of the victims was the European-based spam-fighting group Spamhaus, who experienced similar scale of DDoS attack in March 2013. At the time, the scale was considered the largest DDoS in history, causing "a widespread congestion and jamming crucial infrastructure around the world," according to the New York Times.

Source of attack

Although local media has widely reported that mainland companies and organizations launched these attacks through botnets in Hong Kong, security experts noted it is unable to prove so.

"We have no technical evidence that points to the attacker being located in any particular country," said Prince from CloudFlare.

He added the botnet traffic of this attack was found from nearly every country in the world, with a large amount from Brazil, Indonesia, Turkey, the US and China. The infected machines are also running on a network around the world, including in Hong Kong.

"In general, DDoS of this size are launched from global botnet, it is not likely that this attack originated from within Hong Kong," said Phil Rodrigues, director of security architecture for Asia-Pacific, Middle East and Africa, BT Global Service.

"One cannot really say for certain that one entity is attacking another from a single location or region, without inspecting and analyzing the logs," added Coden Hau, technical director at Trend Micro.

Unique attack with sophistication

With experience of seeing thousands of large scale DDoS on a weekly basis, Prince added that the PopVote attack was a sophisticated one compare with the other.

"What was unique about this attack was the sophistication of the attacker," he added. "The attacker did not just use a limited number of techniques in the attack but instead tried a number of different strategies."

Apart from using multiple botnets comprised of hundreds of thousands of infected machines, Prince said the attacker also used DNS reflection--an amplification technique to magnify the size of attack--and leverage other cloud servers like Amazon Web Services and Softlayer to launch the attack.

Computerworld Hong Kong on June 24 reached out to Amazon Web Services, and UDomain to inquire about their services for PopVote. There has been no response from either of these companies as at press time.

Industry experts also noted the technologies to launch DDoS attack has evolved, creating massive attack with less effort.

"DDoS attack technology is constantly advancing, and with the increase in reflected and server-launched attacks the size of the attack may not always correlate to the number of bots launching the attack," said Rodrigues from BT.

"In the past, DOS attacks were primarily generated from compromised home computers or by willing participants," said John Jellema, global security manager, Verizon Enterprise Solutions. "Now, we are seeing attackers scanning for and exploiting vulnerable websites and content management systems (CMSs). Then placing specific DOS attacks scripts onto these sites."

He added many of the recorded 1,100 DDoS cases in 2013 were targeted off-the-shelf CMSs to gain control of servers for use in DDoS campaigns.

Cloud computing for DDoS

To tackle DDoS attack in similar scale, industry experts agreed cloud computing plays a significant role.

"The attacks actually make the case for the move towards cloud computing because you have someone guaranteeing your services will be available, even a DDoS attack!" said May-Ann Lim, executive director at Asia Cloud Computing Association.

Although using cloud computing limited the users' ability to deploy specific defense technologies within the data center, Rodrigues from BT said cloud providers with a global network of traffic scrubbing centers could help to mitigate the attack. "[Popvote] appears to use CloudFlare, which is one of the providers of specialized global scrubbing centers, which is a good defense," he said.

Prince from CloudFlare said the company's global network was setup to automatically detect and mitigate large scale attacks. In addition, the company is committed to protect politically or artistically important speech via a program called Project Galileo.

"Through that initiative we work with a number of NGOs and civil society organizations to identify sites in need of protection. Popvote was referred to us by one of our Project Galileo partners," he said. "We stepped in to shield them from the attack even though Amazon and other cloud service companies had terminated them as a customer."

He added "unless you have a global network like Google's defending yourself against these attacks is virtually impossible. CloudFlare allows any organization to have a global network and sophisticated network operations team to help defend against these attacks."

Charles from Akamai also added cloud security service providers could provide the scale and skills to protect enterprise from a typical attack, which measured in the 10Gbps range. "Some vendors who currently offer a multi-perimeter strategy in the cloud," he added.

Other defending strategies

In addition to relying on cloud providers, Jellema from Verizon suggested enterprise to also check with their ISPs and anti-DDoS service providers.

"You should be able to test it quarterly without charge," he said. "Understand that all ISPs will, at some point, protect their general network over your company's specific traffic. Ask your anti-DDoS provider about its upstream peering capacity."

"While there are no specific or bullet-proof preventive measures against DDoS attacks, organizations must still plan ahead and prepare for these types of attacks by investing in appropriate solutions and infrastructure," said Hau from Trend Micro.

He said investment in redundant servers, data monitoring and log inspections tools, together with training IT can empower enterprises' ability to mitigate impact from the attack.

CloudFlare in Asia

Despite the large scale of DDoS in Hong Kong, it did not appear to change CloudFlare's development plan in Asia.

"CloudFlare already runs a number of data centers in the Asian region, including Hong Kong, Singapore, Tokyo and Seoul," said Prince. The company is currently planning to open an engineering office within the next 12 months and is considering Singapore, Sydney and Hong Kong as the potential cities.

"We were surprised that the [local] attention the [Popvote] attack brought to CloudFlare," said Prince. "We've received hundreds of applications from engineers based in Hong Kong who are interested in working for our team if we located there."