How do the FBI and Secret Service know your network has been breached before you do?

Knock, knock! Secret Service here.  "Is this your customer payment card data?"

By all accounts, many of the massive data breaches in the news these days are first revealed to the victims by law enforcement, the Secret Service and Federal Bureau of Investigation (FBI). But how do the agencies figure it out before the companies know they have been breached, especially given the millions companies spend on security and their intense focus on compliance?

The agencies do the one thing companies don't do. They attack the problem from the other end by looking for evidence that a crime has been committed. Agents go undercover in criminal forums where stolen payment cards, customer data and propriety information are sold. They monitor suspects and sometimes get court permission to break into password-protected enclaves where cyber-criminals lurk. They have informants, they do interviews with people already incarcerated for cybercrime, and they see clues in the massive data dumps of information stolen from companies whose networks have been breached.

They are constantly investigating, says Shawn Henry, president of CrowdStrike Services, a subsidiary of security firm CrowdStrike, describing how law enforcement follows the digital trail of cybercrime. He should know. Until two years ago Henry was executive assistant director of the Criminal, Cyber, Response and Services Branch of the FBI.

+ ALSO ON NETWORK WORLD Target says it investigated but dismissed early signs of breach | Slideshow: 20 infamous hacker security vendor break-ins | Security firm Trustwave named in Target-related lawsuit +

In the course of all of this monitoring, Henry says, law enforcement often finds itself in the odd position of having to show companies evidence they have been victimized. And they aren't always thanked for their efforts. Sometimes, Henry says, companies say "'Please just go away.'" He adds, "It happens all the time."

The FBI acknowledged the reluctance issue when James Comey, FBI director, said during his keynote at the RSA Conference in February, "We come knocking on your door to say you're under attack," and "we totally get you're reluctant to report intrusions because you fear government rummaging in your network or that competitors will hear about it." Law enforcement "asks for a lot but doesn't seem to offer much in return," he said, but the knowledge is critical for the industry at large.

The companies presented with evidence of stolen data don't have to work with law enforcement investigators, Henry points out, but many do, sometimes providing forensics reports to show how intruders got into their network to exfiltrate sensitive information.

How frequently do the Secret Service and FBI come calling? "About 40% to 50% of our customer base have regular conversations with the FBI and other agencies that have warned that they have been breached," says Simon Crosby, chief technology officer at security vendor Bromium. Law enforcement is very actively trolling the Internet to discover things, he says.

A source at the U.S. Secret Service admitted that agents sometimes do go undercover masquerading as hackers to get information but declined to say much else. The FBI didn't comment on the topic, but Henry says that's just one way FBI agents work to ferret out cybercrime. He notes the FBI and Secret Service have "concurrent jurisdiction" in cybercrime and may work together on certain cases.

The FBI is "sometimes way deep undercover," says Stan Stahl, president of Citadel Information Group in Los Angeles. A few months ago Citadel was called in by a corporate customer that had been contacted by the FBI about a possible breach. In the course of that investigation it was discovered a laptop had malware on it that eluded anti-virus tools and the malware had been in contact with a botnet command-and-control server on the Internet.  "The FBI happened to be monitoring the C&C center" for that botnet, Stahl says.

According to a report entitled "Markets for Cybercrime Tools and Stolen Data: Hackers' Bazaar," published March 25 by the RAND Corp., the online black markets where the purloined spoils are sold and traded get shut down from time to time, as Silk Road was last October, but "substitutes appear almost immediately as competing forums vie for market share."

According to the report, law enforcement is getting better at the cybercrime effort because agents are getting more "technologically savvy" and they are making it a priority to go after cybercrime suspects because their crimes keep getting bigger.

The report claims the cybercrime marketplaces today are run like consolidated, highly organized criminal businesses. The so-called "freelance" criminals that once represented 80% of black-market participants a decade ago are now thought to be closer to 20% today.And these organized cybercrime groups have turned to newer technologies. "For instance, ICQ chats have been replaced by participants hosting their own servers, sharing email accounts where content is exchanged by saving draft messages, and using off-the-record messaging, the encryption scheme GNU Privacy Guard (GPG), private Twitter accounts, and anonymizing networks such as Tor, Invisible Internet project (I2P), and Freenet," the RAND report claims. "Participants frequently alter their communication tactics hoping to stymie law enforcement." The main language of cybercrime is Russian or Ukrainian, the report contends, though spear-phishing campaigns, for example, are typically done in English because the majority of potential victims they're going after speak English.

The RAND report speculates that the reason takedowns of various crime-markets have "not seriously dented the market is that many countries condone hacker activity that is illegal in the United States. One Russian hacker was arrested, let out on a technicality, apologized to, and is now connected to the government. Although Russian officials may have a good idea of what is happening, as long as they can point to fraud in other parts of the world -- especially in the West -- they tend to let things slide." The report claims China "tends to turn a blind eye" as well, but some countries, including Vietnam, have actually been "helpful." And Romania, Ukraine and Poland are "selectively helpful" in pursuing cybercrime.

Of course the government agencies aren't the only ones discovering cybercrime. Gartner analyst Avivah Litan says in the retail sector the banks that issue payment cards notice fraud starting on customer accounts and call Visa and MasterCard, for example. When they get multiple banks calling about payment fraud, it might be tracked down to a specific time and place. "Sometimes the fraudsters sort the cards and hit one bank at a time to avoid the detection early on," she says.

Independent security reporter Brian Krebs' articles on his site Krebsonsecurity.com have also alerted the public of major breaches. Among his many major stories, Krebs broke the news about the Target breach, which had the company scrambling to issue a public statement about how it was investigating the report. Krebs says most of his tips come from sources in the financial industry, not law enforcement.

Krebs says he's not sure if his reporting may have impacted companies' decisions about whether and how much to work with law enforcement. "My sense is that in cases where the news breaks before the victim is ready to go public, there is more pressure on the victim to sync up with law enforcement agencies that may be involved, at least initially just to get some lay of the land and to inform an official statement for the press. Whether that communication continues in earnest after that is anyone's guess."

Krebs adds that he's seen cases where "law enforcement will reach back to a known victim organization after reading some details published in the press that appear to draw connections for law enforcement that perhaps they didn't see before, and the law enforcement agency will try to reconnect to gather more info in the hopes of testing those theories."

But how law enforcement regards Krebs and his ground-breaking stories about data breaches? "I don't know how law enforcement views me frankly," says Krebs. "My guess is as an impediment; they usually prefer to keep things under wraps until people are in silver bracelets."

What happens after the FBI or Secret Service show up with evidence of a breach?

Bromium's Crosby points out that law enforcement typically shows up at the business they think was compromised with concrete evidence, such as the stolen data itself and technical information like IP addresses.

And one of the main questions then becomes, are the companies victimized ready to investigate it? Unfortunately, often they are not, say security experts at Solutionary, which last year became part of NTT's security group. Rob Kraus, Solutionary's director of the company's security engineering research team, who has participated in forensics investigations at the behest of corporate customers who've had the "bad news" visits from FBI and the Secret Service, says every case is different.

CrowdStrike's Henry also says there's no one way that law enforcement conducts investigations.

The FBI no longer seems to have the reputation for grabbing servers as evidence or otherwise disrupting networks, as was the case a decade or so ago. However, law enforcement may place specialized devices on a corporate network to to see if suspects that grabbed data return to try for more, Kraus says.

Don Gray, chief security strategist at Solutionary, adds that some national-security-oriented investigative forces will still show up without advance notice and grab entire storage drives on national security grounds. Solutionary, as a managed security services provider, accounts for that possibility in its network design for customers with sensitive information, he suggested, without offering more detail.

At companies that have been breached, the house attorney is often the one appointed to first receive the forensics report before it's handed to law enforcement or anyone else at the company, Gray notes. Sometimes the forensics report is requested in "draft form," and attorneys draw up the written report. "They only want the house lawyer to see it," says Gray. This is a way to try and keep control over the breach from a legal vantage point, especially if the case ends up in court. That may limit how much the IT department initially knows. Law enforcement typically commences its interactions with upper management, not the IT department directly.

Solutionary last year was hired by a bank to conduct a forensics examination after the FBI showed up with evidence of a major breach that turned out to have been caused by SQL Injection attacks on the bank's website and had been going on for months. One difficulty, says Kraus, is the bank's logging system was weak and only stored log data for 2 ½ months. Solutionary believes incident response capabilities remain tepid at best in companies today.

This raises the all-important question of how well companies defend their networks and whether their logging capabilities are sufficient to give them a clue about anything after a breach.

Target, whose CIO Beth Jacob "resigned" in the wake of the data breach, recently acknowledged the security team at the retailing giant missed clues about the breach, even after spending well over a million dollars on threat-detection software from FireEye and Symantec endpoint protection software. The Target breach has stirred up the debate over whether to blame the security software, the security staff, or both.

In spite of all the FBI and Secret Service visits, there's still the perception that the cyber-criminals -- sometimes in faraway places like Eastern Europe -- are not being brought to justice. After all, it is each enough for cyber-criminals to reach across the world to break into a network, but nabbing them in a foreign country and bringing them to trial remains a tough proposition.

Ellen Messmer is senior editor at Network World, an IDG website, where she covers news and technology trends related to information security. Twitter: MessmerE. E-mail: emessmer@nww.com

Read more about wide area network in Network World's Wide Area Network section.

Tags endpoint securityWide Area NetworkFederal Bureau of Investigation

Show Comments