In 1995, when I was a university student in Helsinki, I developed a security protocol to protect data-in-transit as it moved throughout our network. I named it the "secure shell," or SSH for short. Today, SSH is used by organizations of all types and sizes as a secure method to move data from machine to machine and provide remote administrator access. From the perspective of an attacker or malicious insider, SSH is an artery that carries vital organizational data.
[RELATED: Tatu Ylonen, father of SSH, says security is 'getting worse']
However, most organizations do not have control over how their SSH keys are created or managed, leaving them vulnerable to attack and at risk of costly fines for regulatory noncompliance. To remediate this issue, every organization should audit their SSH key management systems.
SSH: A Primer
SSH works by creating a cryptographic key pair - one for the server, and one for the user - and securing the data that passes between those two points. Unfortunately, most organizations have no process for managing, removing, and changing access-granting "keys" once they are created and deployed. As a result, thousands of keys are distributed at will throughout the network environment, typically without any way to find or remove them, creating untraceable access paths to sensitive data. Mismanagement of SSH keys, therefore, presents a significant security vulnerability.
The scope of the problem is as pervasive as it is shadowy. Our experience in working with major enterprises has shown that most organizations have on average eight to over 100 SSH keys configured granting access to each Unix/Linux server. Some keys even grant high-level administrative access.
"High-risk" insiders can use these poorly-managed SSH keys to create permanent backdoors to production servers, bypassing all normal privileged access management and session auditing systems. These insiders include anyone who has ever had access to a server, including former system administrators, consultants, and anyone with access to backups or decommissioned hardware.
This problem has flown under the radar because it is deeply technical and obscure, residing in the domain of system administrators. Each system administrator typically only sees a small corner of the IT environment, and therefore does not have a full picture. Administrators and their managers are often so busy that while they may recognize that there is a problem, they simply have not had time to analyze its scope, or its implications.
Unmanaged SSH Keys - An Attack Vector for Viruses
The chances of such a breach occurring are growing by the day. News reports on network breaches are commonplace as attacks become more widespread and sophisticated. Implementing SSH keys as an attack vector in a virus is quite easy, requiring only a few hundred lines of code. Once inside an organization, a virus can use improperly-managed SSH keys to spread from server to server.
In fact, the mesh of key-based access is so dense that it is highly likely that an attack can spread to nearly all servers in an organization, especially if the virus also utilizes other attack vectors to escalate privileges to "root" (high-level administrator) after penetrating a server. With so many keys, odds are the virus will infect nearly all servers in a manner of seconds to minutes, including disaster recovery and backup systems that are typically also managed using such keys.
In the worst case, a virus or cyberweapon using multiple attack vectors could spread globally, Internet-wide, enterprise-wide in a matter of minutes and, combined with destruction technologies, could wipe out massive amounts of data.
Compliance at Risk
Major organizations without proper SSH key management in place aren't just at risk from security breaches; they also appear to be non-compliant with mandatory security regulations and laws. Industry requirements such as SOX, FISMA, PCI and HIPAA demand proper control of access to servers and termination of that access. Furthermore, these organizations might also find themselves running afoul of internal security policies (including in some cases policies mandated by their customers).
How did we get to this point? The situation is not a result of any vulnerabilities or flaws in the SSH protocol itself or in the most commonly used implementations of the protocol. Rather, it is the confluence of a number of factors:
- Years without clear guidelines or policies relating to SSH keys.
- Lack of understanding of the scope and implications of the problem.
- Insufficient time and resources to dig into the issue to gain understanding or develop solutions.
- Reluctance on the behalf of auditors to flag issues for which they don't have effective solutions.
These masking symptoms are pervasive, yet the issue of SSH key mismanagement cannot be ignored forever. Without properly controlling, auditing or terminating SSH key-based access to IT systems and data, most enterprises, government agencies, and healthcare providers are leaving themselves vulnerable to attack.
Best Practices
Before the problem can be remedied, its existence must be recognized. Several teams within IT operations might be involved in the remediation project and proper support and endorsement within the organization is needed.
Remediation of the problem involves several steps:
" Discovering all existing trust-relationships (who can access what).
" Monitoring the environment in order to determine which keys are actually used and removing keys that are no longer in use.
" Enforcing proper processes for all key setups and other key operations.
" Automating key setups and key removals, eliminating manual work, human errors, and reducing the number of administrators from possibly several hundred to virtually none.
" Rotating keys, i.e., changing every authorized key (and corresponding identity keys) regularly, so that any compromised (copied) keys cease to work.
" Controlling where each key can be used from and what commands can be executed using the key.
Nearly all Fortune 500 companies and major government agencies appear to be non-compliant and facing major security issues (existential virus risk, hacking risk and rogue employee risk). Most of the next 10,000 companies (there are about 10,000 companies having over 10,000 employees in the world) are also exposed to the same issues. We have seen this issue in companies with as few as 20 servers, but when scaling to 100 servers and above, the benefits of automated systematic key management are clear.
It will take several years to fully address the issue and thousands of people must be involved and properly trained. Awareness must be created among CISOs, CIOs and enterprise IT risk management professionals so that they can work to ensue SSH user keys are properly managed in their organizations.
While SSH remains the gold-standard for data-in-transit security, the current threat landscape requires organizations to take decisive steps to improve how they are managing access to their SSH networks.
About the Author:
Tatu Ylönen is the CEO and founder of SSH Communications Security. While working as a researcher at Helsinki University of Technology, Tatu Ylönen began working on a solution to combat a password-sniffing attack that targeted the university's networks. What resulted was the development of the secure shell (SSH), a security technology that would quickly replace vulnerable rlogin, TELNET and rsh protocols as the gold standard for data-in-transit security.
Tatu has been a key driver in the emergence of security technology, including SSH & SFTP protocols and co-author of globally recognized IETF standards. He has been with SSH since its inception in 1995, holding various roles including CEO, CTO and as a board member.
In October 2011 Tatu returned as CEO of SSH Communications Security, bringing his experience as a network security innovator to SSH's product line. He is charting an exciting new course for the future of the space that he invented.
Tatu holds a Master of Science degree from the Helsinki University of Technology.
Read more about wide area network in Network World's Wide Area Network section.