Software Supply Chain Security

July 17, 2023

What is Software Supply Chain Security ?

In this day and age most of us are looking for trust and security. This is also true for our IT infrastructure as challenges from the outside grow more and more dangerous to the continuation of undisturbed business. Attacks on the IT infrastructure of companies not only rise because of individuals trying to gain an advantage but also for countries trying to upend the daily lives of their perceived enemies.

Of all current known types of attack vectors, Software Supply Chain Attacks are the most difficult to find and to protect against. This report done by IBM shows about 1/5 of all breaches were Software Supply Chain Attacks. And that it took on average 26 days longer than the global average over all attacks (277 vs 303 days) to find and contain them. Also that the average cost was higher to remediate this type of attack.

But why is it this way? What actually is a Software Supply Chain Attack and what factors reduce the Software Supply Chain Security?

An application providing services for your customer usually comprises of several components, services. Some of those services and used Frameworks will not originate in your development department but will come from suppliers specializing on certain aspects of the services. If you then look at those suppliers they also have an amount of frameworks they are using in their libraries/applications. This chain of suppliers will most likely go on for some time. Thus resulting in a Software Supply Chain. Some of those suppliers deliver components which are used in several other components. This 1:n relationship enhances the result of an attack. An attacker only needs to infect a component deep in the Software Supply Chain to reach a lot of other components thus making it a better target.

The picture shows a Software Supply Chain, where a component A is reused by components B - D. Furthermore Components C and D are part of Component E. This component is used by an Endcustomer

How an attacker selects targets and infects your company

In the next few paragraphs we are trying to shed some light on Software Supply Chain Security using a simple example. In this example our protagonist George is upset. A company (Broken Products Ltd) sold him a product, which was broken. Although he should have anticipated this, looking at the name of the Corporation, he bought it anyway as the price was too good to be true. He sent it back but as expected they refused to refund him. He expects to get his money back. What is he going to do?

He already invested some time into this and is looking to get a lawyer for help, but the lawyer told him it would cost him more to get the refund than the product was worth. But then he gets a great Idea. Why shouldn’t he, as a Software Developer, try to get his money in another way? Maybe by using a Cryptominer who generates his money on their hardware? Or by carrying out a Denial of Service Attack on their Systems to waste their time, like they wasted his? Or… why not both?

His mind is set. Ok, it is not really legal but they wanted this. The plan is set and now he tries to get into their WebShop System, but they seem to have done a very good job in securing it. He is unable to get in. Or… can he? Using some search-engines he finds a company (SupProd Ltd), which lists Broken Products Ltd as a customer reference. He registered for a trial product, downloads and installs it.

Using his knowledge he finds enough information to determine that the products use some components, which seem to be proprietary to SupProd Ltd. Since it is a npm package, he registers the name on npm.js and uploads his very own modified version of the package including a crypto miner, and a small code-piece which deletes every file it can reach if is run in Broken Products Ltd domain (in the Picture this is the Use Bad Dependency Vector).1

A few days later he sees the first results. His wallet receives some mined crypto. And several weeks later he reads about an availability problem at Broken Products Ltd, and that they had to shut down, because their Webshop, their only distribution channel, was not working for too long.

Of course this Story is fictional and simplified. But it has elements from the real world. In 2021 a researcher used a dependency confusion attack and infiltrated over 35 companies, because public packages have a precedence over private packages in python and npm. Targeting specific companies via the Software Supply Chain was shown in the M.E. Doc Hack and also in the notorious Solarwinds Software Supply Chain attack.

The Picture shows a Supply Chain starting left with the Developer. Developer submits code (in git), Code goes into a Buildsystem (CI/CD) which moves further to a Distribution point as a package. Afterwards it goes over into a use stage. Package also interlocks with dependency which may or may not be part of the build. In every step are different Attackvectors: Between Developer and Git is Submit bad code”. On Git is Compromise Source Control. Between Git and Build is modify code. On Build is Compromise Build Platform. Between Build and Distribution is bypass CI/CD. On Distribution is compromise package repository. Between distribution and use is use bad package. And lastly between dependency and build is use bad dependency.

What can you do to protect against it?

The Software Supply Chain is typically not only wide (by using multiple software suppliers/dependencies in one component) but also can have an almost unlimited length (because suppliers supply your suppliers). You can get attacked in every step in the process of building and distributing an application from source code to package. So what are the options?

Of course you could simply tighten the Software Supply Chain and reduce the number of your suppliers. You can do so by establishing strategic partnerships with some of your suppliers and rely on products of companies you already have a relationship with. Furthermore you can analyze the portfolio of your suppliers and see where they overlap and maybe consolidate services. By doing so, you will reduce width and length of the dependencies and reduce complexity.

This will on the one side make it easier for you to manage the Software Supply Chain, to evaluate risks carried in it and maybe make some commercial deals possible. On the other side you might lose some features of products you used, have to migrate the software/data and re-train users. Also you will have less suppliers but may be more dependent on them. You can also standardize on a specific platform (Operating Systems, Kubernetes Platforms, …) to homogenize your standard operation environment and thus reduce the number of packages, suppliers and different infrastructure you have to secure.

Another possibility is to not rely on suppliers for providing software, but to develop and/or build it on your own utilizing your own resources. This gives you full control of the process. But this also has disadvantages. You need to have the people and competencies to do this. And you have to evaluate if you really can do it better and in a more secure way than your suppliers. You might even lack the access to the source code to build the supplied software on your own, since it might be proprietary. 

A third option is to contribute upstream in the open source projects that are most important to or most widely used by you. Help them by providing (secure) Infrastructure or with pull requests which help to increase the security of their build process. Of course always consult with the community before tampering with the process. A good start can be to reach specific SLSA Levels.

SLSA Levels

Whooow hold a minute. What’s that SLSA you are talking about? I don’t have any tacos! SLSA stands for Supply-Chain Levels for Software Artifacts and is a specification around how to securely build and deliver software. It is divided in different tracks and enables developers to check against a specification to identify areas of improvement. It is also useful for software consumers to evaluate risks.

The SLSA Levels also allow you to challenge your suppliers to implement specific security levels which are defined there. You can make this a selection criteria for software you buy. Also even if your vendors/suppliers implement specific measures, always follow the guidance: “Trust is good, control is better” and verify all incoming artifacts from your suppliers.

You should furthermore understand that your Software Supply Chain is not only the output or the artifacts, but also the process and involved infrastructure. And a single tool or check is not enough. When the Software Supply Chain gets infected in an early step you may even get signed artifacts from your supplier including malware you don’t want. Every tool or measure you use to verify the artifact has blind spots. Antivirus might only see known malware. Signatures may have been applied after compromised packages or code has been added. In this case also SBOMs and checksums won’t show errors.

Red Hat Trusted Software Supply Chain

In this article you have seen what a Software Supply Chain is, how it is endangering your company and where in the process of developing an application the danger is constituted. Those different attack points make it difficult to defend against as there are countless tools focusing on how to fix the issues / problems. In May ’23 Red Hat announced its Trusted Software Supply Chain platform. The goal of this platform is to consolidate the defense against those types of attacks while staying open to customer preferred environments and tools. It concentrates on the different areas where an attack is possible. 

3 Areas of Interest

Coding an applications does not start on a greenfield

Picture shows the word code on top. Under it is Software Composition Analysis at the same stage as Digitally signed & verified. On the bottom is a product logo of Red Hat Trusted Content with a new sign on the top right.

When creating an application on the average 600 components have open source included. This is an important entrypoint for malicious attacks (i.e. frameworks, libraries, …). In a development project you need to make sure that components are not infected. Red Hat Trusted Content is supplying Content Repositories which have been signed and certified. Also as the development process becomes more and more complicated through a multitude of tools and frameworks, ensuring that the developed software is free of malicious components is becoming a growing sometimes too complicated task. With Red Hat Trusted Content integration of security checks is integrated into the Developer IDE. Within those plugins the developer is able to check on each component and see if there are vulnerabilities known. With this knowledge the developer can then remove the use of unsecure components.

Endless and dynamic Building

Picture show the word build on top. Under it is Artifact Building, Image Building, Image Scanning and Deployment Gates. On the bottom is a product logo of Red Hat Trusted Application Pipeline with a new sign on the top right.

Building an application is not a one time effort. But it will stay with an application through its entire lifetime. Even when development has finished, libraries and components need to be updated to incorporate new fixes for new vulnerabilities. In a CI/CD process this can be done with a trusted application pipeline. This includes automated security checks to ensure that known malicious components cannot enter an image build. Also a verified and immutable Software Bill of Materials is created to ensure an up to date list of used components and remove the chance of adding malicious components to a build application.

Regulations, policies and the likes

Picture shows the word monitor on top. Under it is OSS Risk Profiles and Images, Containers, Cluster network. On the bottom is a product logo of Red Hat Advanced Cluster Security Cloud Service with a new sign on the top right.

The third area of interest is deploying and monitoring an application. In the Red Hat Trusted Software Supply Chain this is done by Red Hat Advanced Cluster Security. It can be utilized to monitor applications across the lifecycle phases build, deploy and run. You can use policies to enable admission control over packages, enforce signatures and overview behavior of your applications. In this way you are able to enforce your companies strategy on Software Supply Chain on a technical level.

Conclusion

With more available software frameworks, diverse set of available tooling and software suppliers, the risk of a Software Supply Chain attack rises. In the development process displayed above, measures have to be taken at each step to minimize the risk of such an attack. With the Red Hat Trusted Software Supply Chain the trust of your customers can be rebuilt.

Footnotes:

  • 1: npm is known to download packages of public repositories over the one in a private, local repository due to default proxy settings. This behavior can be changed with configuration but is a known source of errors.