GitHub clobbered by record-breaking 1.35 Tbps ‘memcached’ DDoS

Code hosting company GitHub was knocked offline yesterday by massive distributed denial of service (DDoS) attack that peaked at record-setting 1.3 Terabits per second. 

The attack is slightly larger than the Mirai-botnet attack that downed the now Oracle-owned Dyn in 2016, taking with it access to dozens of the world’s largest websites, from Amazon to Spotify.

The source of the attack are memached servers that someone is abusing as reflectors to drastically amplify traffic directed at a target website. 

Cloudflare, Arbor Networks and Akamai this week reported a surge in DDoS attacks using the technique, which can amplify a response to a request by 50,000 times. 

The attack on GitHub relies on memcached servers that shouldn’t be left open on the internet but are, and that support UDP, an internet protocol that’s prone to IP address spoofing. According to Rapid7, there are over 100,000 exposed memcached servers at any given time.  

Surprisingly, despite the size of the attack on GitHub, the site was only unavailable for about 10 minutes on Wednesday. 

The company handed off inbound traffic to Akamai’s DDoS protection service, Prolexic, shortly after traffic to one of its facilities exceeded 100Gbps. 

The first attack wave peaked at 1.35 Tbps, arriving at 17:21 UTC, and a second wave that hit about 30 minutes later peaked at 400Gbps.   

memcached servers are used by websites for caching memory to optimize performance of sites that rely on external databases.      

As Arbor Networks explains, memcached servers are ideal for DDoS amplification. Besides the size difference between request and response, memcached servers often have high-bandwidth access links and sit inside high-capacity cloud networks.  

The attack works by spoofing the IP address of the target using available memached servers that reflect larger responses to a request from the spoofed IP address. The attacks come from UDP port 11211, the default port used by memcached. 

 “The attacker typically ‘primes’ a given set of memcached reflectors/amplifiers with arbitrary-length key/value pairs, and then issues memcached queries for those key/value pairs, spoofing the IP addresses of targeted hosts/networks,” explained Arbor Networks’ Roland Dobbins.  

“Both the priming queries and the attack-stimulus queries can be directed from source ports of the attacker’s choice to UDP/11211 on abusable reflectors/amplifiers, meaning that the attacker has full control of which destination port is targeted on the destination hosts/networks.”

Security researchers have known about insecure memcached servers for nearly a decade but as Dobbins points out, it was only in November that researchers at a China-based security team called 0kee 360 spotted potential for their use in DDoS amplification attacks. The technique was likely used by skilled attackers initially, but quickly became commoditized through DDoS rental services known as stressers.

In other words, expect more of these attacks in the short term until network operators can close off exposed memcached servers to mitigate their impact.