What changed
1 fix0 additions3 changes0 removals
fixedThis is a story about the internet, and how it doesn’t work like it should, when it works at all, except when it doesn’t. It’s a story about how the internet stopped working for our customers, and how we fixed it.
changedBack in 2014, the method those trolls were using was distributed denial-of-service (DDoS) attacks. “Distributed” refers to a large number of internet hosts maliciously flooding a particular target IP with traffic in an effort to overwhelm the network stack. This is called a volumetric attack, and the intent isn’t to try to get into the network. It’s just trying to deny service. A distributed denial of service means that legitimate people who want legitimate service are crowded out by the bad traffic.
changedIn August 2015, The International was disrupted with DDoS attacks. Although the pros playing the match weren’t affected, for more than two hours, the broadcasters couldn’t get into the matches to give play-by-play and color commentary. Sending out the stream as a TV broadcast became an issue. The players were suddenly playing in a void. This was a professional gaming event with millions watching and millions of dollars on the line, and it was being disrupted by random people with five-dollar software. It was a problem Valve couldn’t ignore.
changedWe tried several solutions to deal with DDoS attacks before we arrived at one that worked. Initially, we attempted to filter the traffic with a powerful network switch. Unfortunately, this type of filtering is inherently difficult to do with game traffic. It is the nature of game servers to receive unsolicited UDP (User Datagram Protocol) traffic from arbitrary IP addresses. Imagine you had a post office that weeded out unwanted junk mail for you. But now imagine your job is as an advice columnist, and you receive tons of legitimate mail from random strangers all the time. For you, the post office doesn't know what's junk mail and what isn't. That's how traffic to game servers tends to look. Furthermore, the source IP in UDP packets is not secured, and can be easily spoofed. Our
Dota 2 changes
fixedThis is a story about the internet, and how it doesn’t work like it should, when it works at all, except when it doesn’t. It’s a story about how the internet stopped working for our customers, and how we fixed it.
changedBack in 2014, the method those trolls were using was distributed denial-of-service (DDoS) attacks. “Distributed” refers to a large number of internet hosts maliciously flooding a particular target IP with traffic in an effort to overwhelm the network stack. This is called a volumetric attack, and the intent isn’t to try to get into the network. It’s just trying to deny service. A distributed denial of service means that legitimate people who want legitimate service are crowded out by the bad traffic.
changedIn August 2015, The International was disrupted with DDoS attacks. Although the pros playing the match weren’t affected, for more than two hours, the broadcasters couldn’t get into the matches to give play-by-play and color commentary. Sending out the stream as a TV broadcast became an issue. The players were suddenly playing in a void. This was a professional gaming event with millions watching and millions of dollars on the line, and it was being disrupted by random people with five-dollar software. It was a problem Valve couldn’t ignore.
changedWe tried several solutions to deal with DDoS attacks before we arrived at one that worked. Initially, we attempted to filter the traffic with a powerful network switch. Unfortunately, this type of filtering is inherently difficult to do with game traffic. It is the nature of game servers to receive unsolicited UDP (User Datagram Protocol) traffic from arbitrary IP addresses. Imagine you had a post office that weeded out unwanted junk mail for you. But now imagine your job is as an advice columnist, and you receive tons of legitimate mail from random strangers all the time. For you, the post office doesn't know what's junk mail and what isn't. That's how traffic to game servers tends to look. Furthermore, the source IP in UDP packets is not secured, and can be easily spoofed. Our
Welcome back to Between the Lanes, a blog feature where members of our development team walk through some of the challenges, bugfixes, and occasional happy accidents we encounter while working on a game as unique as Dota.
This is a story about the internet, and how it doesn’t work like it should, when it works at all, except when it doesn’t. It’s a story about how the internet stopped working for our customers, and how we fixed it.
The internet is more of a wild frontier than we give it credit for being. Stray from the safe confines of your two-factor identification and trusted cookies, and it can be a bit of a wilderness out there, full of random trolls with the maliciousness—and, increasingly, the ability—to make your life pretty miserable for no other reason than because they can.
Back in 2014, the method those trolls were using was distributed denial-of-service (DDoS) attacks. “Distributed” refers to a large number of internet hosts maliciously flooding a particular target IP with traffic in an effort to overwhelm the network stack. This is called a volumetric attack, and the intent isn’t to try to get into the network. It’s just trying to deny service. A distributed denial of service means that legitimate people who want legitimate service are crowded out by the bad traffic.
The problem with DDoS attacks was that, by late 2014, they weren’t being committed by data-exfiltrating super-hackers with advanced computer science degrees. They were being committed by just about anybody who wanted to pay a service or a bot to do it for them. And it’s pretty obvious why. Although some people were happy to do it purely for vandalism’s sake, others had a motive: A DDoS attack was a surefire way to shut down a match that either you or someone you were rooting for were losing. This wasn’t just an occasional irritation anymore. It was turning into an outright assault on any game where players competed.
By the opening months of 2015, we were seeing a huge uptick in DDoS attacks on Dota and CS:GO, with other companies reporting a huge jump as well. Someone had, very suddenly, made it very easy for anyone to do this.
In August 2015, The International was disrupted with DDoS attacks. Although the pros playing the match weren’t affected, for more than two hours, the broadcasters couldn’t get into the matches to give play-by-play and color commentary. Sending out the stream as a TV broadcast became an issue. The players were suddenly playing in a void. This was a professional gaming event with millions watching and millions of dollars on the line, and it was being disrupted by random people with five-dollar software. It was a problem Valve couldn’t ignore.
We tried several solutions to deal with DDoS attacks before we arrived at one that worked. Initially, we attempted to filter the traffic with a powerful network switch. Unfortunately, this type of filtering is inherently difficult to do with game traffic. It is the nature of game servers to receive unsolicited UDP (User Datagram Protocol) traffic from arbitrary IP addresses. Imagine you had a post office that weeded out unwanted junk mail for you. But now imagine your job is as an advice columnist, and you receive tons of legitimate mail from random strangers all the time. For you, the post office doesn't know what's junk mail and what isn't. That's how traffic to game servers tends to look. Furthermore, the source IP in UDP packets is not secured, and can be easily spoofed. Our
