Balancing speed and security to avoid technical debt

We all know how easy it is to rack up a financial debt in business – just by trying to stay afloat and meet the demands of customers, partners and staff. Most of us don’t realise we’re probably accruing other kinds of debt too – like technical debt.

Technical debt is a concept that encompasses the cost of choosing a quick and easy software development solution, instead of using a more considered and security-minded approach. The cost could be in time spent redressing oversights, or it could be financial or reputational damage caused by an incident.

Many businesses fall into the tech debt trap. Your IT team is focused on keeping things up and running, your DevOps team is in a constant cycle of creating new applications and rolling out updates to meet the expectations of both management and customers. You’re making decisions about your systems and applications on the fly that are nothing more than band-aids.

You may be increasing business efficiency and able to keep up with commercial demand, but a security problem lies under the surface, particularly when using open source software as many companies now do. You’re accumulating cruft over the long term. And that cruft can come back to bite you.

In part, it comes down to the fact that the development cycle within business has changed. It used to be that changes were slowly and methodically rolled out in software, carefully checked for bugs and tested by the usability team before being made generally available.

Modern business has thrown that careful nurturing out in favour of rapid, continuous software iteration. Take Etsy, the well-known online artisan marketplace. In the past it had a development process that saw two full site deployments per week, when they would add new products and services to their online offering. This was a fast development cycle, but in recent times Etsy has revved up that approach, and now does 50 micro changes to its core site every single day of the week.

That’s not to say that Etsy has a tech debt. But one company that the tech debt bit was ridesharing company Uber. It suffered the consequences of its debt in 2016, with a data breach that compromised the details (including credit card numbers and other credentials) of 57 million riders and drivers.

And it was tech debt that caused the breach. Put simply, Uber was growing so fast that it needed to frequently ‘spin up’ (as the terminology goes) new database instances on Amazon Web Services. In doing so, it was writing automation scripts that had a ‘hard coded’ AWS Access Key – the key to the AWS Kingdom – embedded within it. This hard coding of secrets is relatively common in the software development community. It saves time, increases efficiency, and many members of a development team can rapidly make changes to the software and then ‘commit’ it to production without having to fuss with constantly remembering new secrets or having to manually change the secret in the code when passwords or other credentials are changed.

The mistake Uber made was that it then posted this software code on a popular code repository called GitHub. Everything was in plain text, and nothing was encrypted. Although they were using a private repository, two-factor authentication was not enabled to prevent someone from brute forcing or just guessing the password to authenticate into the repository. Because the hard-coded Access Key was in plain text, the attackers used the key to access the software databases they were spinning up on AWS where customer data was stored.

Then the worst happened: their GitHub account was compromised by hackers, who used the credentials to steal the details of 57 million individuals.

These days, software engineers are under so much pressure from business leaders that, in the interests of pure efficiency, they leave business-critical credentials totally unprotected. With business executives and managers in the driving seat, security is often overlooked in favour of making things as fast as possible and getting them into production.

The reality is that it’s not security that makes a company money. But a lack of security can cost a company a lot of money, as well as reputational and institutional damage.

The answer to this technical debt is to embed best practice, as well as the proper security tools, to allow the rotation of passwords and other credentials from a secure vault. Doing this means that passwords don’t get hard coded into software, and hackers then can’t use those keys to access vital corporate or customer data.

It’s the preventative measures that are taken which can eliminate technical debt and keep the business and its customers secure.