EM360 Podcast: Software Supply Chain Attacks and the Best Defense


Listen to Suresh Bhandarkar, Director of Product Solution Architecture at Beyond Identity, speak with the Enterprise Management 360 podcast about the best defense against software supply chain attacks.


Enterprise Management 360, your main source for tech news, analysis, podcasts, and videos for the enterprise. 


Hello, and welcome to the "EM360 Podcast." I'm Richard Stiennon, chief research analyst at IT-Harvest. I write books in IT security, work with IT security technology providers on their go-to-market, and I'm a trusted advisor to CISOs and their teams. IT-Harvest is an industry analyst firm that covers over 3000 vendors in the IT security industry. 

In today's episode, I'm joined by Suresh Bhandarkar who is Director of Product Solution Architecture at Beyond Identity. We're going to talk about software supply chain attacks and the best defenses. Suresh, welcome. 


Hello, Richard. How are you? 


I am great. Suresh, first let our listeners know about Beyond Identity in general. 


Sure. So, Beyond Identity provides modern passwordless authentication backed by the principles of zero trust. What that means is, as an enterprise, your employees, contractors, and partners, when they authenticate to your SSO or they try to access any SaaS applications, they get authenticated through the Beyond Identity passwordless authentication, which uses asymmetric cryptography behind the scenes. 

It takes you away from password-based authentication, which is based off of Shared Secret. And because we are using asymmetric cryptography, we are able to bring in device trust backed by a number of device telemetry signals, which gives you an increased assurance level when you're authenticating to your applications. 

And that provides you phishing resistance, MITM resistant secure authentication. 


Awesome. And if it includes phishing, it probably includes business email compromise too. 


That's true. These days emails are provided through Office 365 and Gmail and both of these are backed by an authentication directory vendor. So, in the case of Office 365, typically you use Azure Active Directory as your directory source. 

And in the case of Gmail it could be Google Workspace. We integrate with both of these directory sources so you can get secure phishing-resistant authentication. 


Awesome. So, let's talk about supply chain attacks. This is top of mind for me because I just finished writing a chapter for a new book, you know, how to counter them. And I don't come up with very many solutions, at least, from the end user's perspective. And we've had dramatic cases of supply chain attacks going all the way back to the attack on RSA security when they were actually a supplier to Lockheed Martin, and then more recently NotPetya, which was easily the most devastating attack in history to date. 

And then SolarWinds. And both NotPetya and SolarWinds use software updates in order to infect the end customers of those products. So, talk about that and how passwordless identity can help solve the problem. 


Sure. So, Richard, I think both of us have been in this industry long enough to see that software supply chain attacks is not a new phenomena. Way back, if you go back a couple of decades, the attacks that happened against Target, the attack that exploited software supply chain vulnerability in the case of Eastern European ATMs, and even Stuxnet. 

These are all classic examples of software supply chain attacks. But you're right, increasingly we are seeing more incidents of it. And the impact caused by these supply chain attacks including the one that happened with SolarWinds are so enormous that it really serves as a wake-up call to the whole industry. Specifically, you mentioned SolarWinds, and that caused about 18,000 customers to be impacted, including Fortune 500 companies like Microsoft, NASA, U.S. 

Justice Department, U.S. State Department. And obviously, after SolarWinds, there have been a few other breaches that have happened. In each of these cases the impacts are enormous, reason being in a typical data breach, the attacker is able to go after one specific customer, but when a breach is caused due to a software supply chain attack, the bang for the buck for the attacker is that they can go after hundreds and thousands of victims. 


Yeah, it is frightening, especially when the entire security community such as we are have been telling people that the solution to all their problems is to update as soon as possible whenever there's a new software update. And that's turned into very few checks on updates, right? You still have to test them to make sure they don't break everything. 

But most small organizations don't do any testing at all. They rely on not being patient zero for bugs. But that's the case also for back doors that have been installed in these things. So, how do we… If you're a end user, you're a big bank or something, how do you determine and protect yourself from a malicious software update? 


Yeah, that's a great question, Richard. As you rightly said, as the attackers are shifting their bases, shifting left, as they call it, the cybersecurity industry is not sitting idle. Collectively we are upping our ante. We are also shifting our defenses to the left so that we are able to fight the attackers on their own turf. 

And this is what has given rise to shift left security. But if you double-click on it, shifting left and specifically providing defense against software supply chain attacks involves quite a bit of controls and frameworks to be in place. There are a number of industry initiatives that are out there which are trying to put in standards and frameworks to control how and where we can fight the attackers. 

And that actually covers the entire software supply chain ecosystem, starting from the developer accessing the source code, modifying the source code to building the final product, packaging it, and delivering it to a consumer. And if you inspect the different aspects of this whole chain, there are various points within which a vulnerability could get introduced. 

So, there could be threats at the source level, there could be threats within the build process, there could be threats when you're building your dependencies. And based on that, specific standard called SLSA, Supply-chain Levels for Software Artifacts, has defined the entry points where you need to have your defenses in place. 

Obviously, in order to have the right level of security posture, any company has to have defense in depth. However, what Beyond Identity is doing is looking at a specific set of these vulnerabilities and providing defense against those. 

The good news is we are guarding the front door. We are making sure that the threats don't get introduced in the first place, and that's a good place to start for most organizations. So, to your specific question about what could a typical SMB or a small customer start off with, the first thing to do is to make sure that you have code provenance in place. 

What do I mean by that? If you look at a software supply chain, it typically involves maintaining a git repo, a distributed version control system where you would be hosting all your source code. And typically, developers access this repo using a command line interface which is protected through SSH. And based on our conversations with global customers and based on our analysis of the threats that are out there, it is very evident that this command line interface backed by SSH is the primary culprit or primary channel attack vector that is exploited by attackers. 

So, if you look at SSH, the SSH keys are typically created by developers using open-source tools. And these keys are lying in the open on their file systems. There have been a number of incidents and you would probably come across them every other week where these SSH keys can be easily stolen. 

The second vulnerability, and it is not really vulnerability per se, Richard, but it is a feature of git which typically lends itself to be easily exploited, is you can take a git config file and change the user details, the email address to be any value. And git did that for a reason. 

However, the hackers, the threat actors, are smart. They're trying to exploit that. What they typically do as an external attacker, if they get their hands onto the private SSH keys, they're able to establish tunnel, but then modify the email address so that they can spoof as an internal user and then get unnoticed the code that they commit if they poison the repo, then the developers can unwittingly trust that code and merge that code into their final build. 

So, that is from the external attacker standpoint. You also have internal malicious insiders who can also be compromised in the sense they don't have to steal the SSH keys, but they could also spoof the git config entry and pretend to be someone else who has a higher level of trust. 

Now, the solution for this problem is not new. It is a well-known solution that is understood and known to the industry. It's called code commit signing, which provides you code provenance, it allows you to have code integrity, authenticity, and non-repudiation qualities. The problem is the products that are out there to provide code commit signing are open-source tools, and this is exactly where Beyond Identity comes into picture. 


Cool. Thanks for that explanation because that goes into some deep detail. Okay. So, how does Beyond Identity step into the breach? 


Yeah. So, we have a very simple and elegant solution that does quite a bit of complex things behind the scenes, but from the developer experience standpoint, it's simple, elegant, transparent, and non-intrusive. We have the Beyond Identity lightweight authenticator app that runs on each developer machine. 

At the beginning of the call, we talked about Beyond Identity being used for MFA passwordless authentication. It's the same authenticator that provides you passwordless access that can also be used to create this GPG key pair. 

The difference between Beyond Identity solution and open source solution is we create this GPG key pair in the trusted platform module of the device. So, it's a secure enclave that is part of an individual developer machine. So, if you are using a Windows machine, you have a TPM chip. 

If you have a MacBook it is the same concept, it's just called T2. 

So, we provide the ability to create public-private GPG key pair within the TPM of a Windows, Mac, or Linux machine. And we back that using our zero-trust principles. So, we have a policy engine that an administrator can configure to define corporate policies. 

And the corporate policies could be the developer machine should have hard disc encrypted, should have firewall turned on, should have an antivirus system, should have crouch strike EDR running, should potentially be domain joined, should have certain registry entries present, should have certain services running. 

So, you can define all these policies and you can govern the creation of this public-private key pair backed by the policies. Once you have created the keys and you have defined the repo that should use those keys, you are all set. As a developer, you then go about doing your business with your standard git operations. 

So, you might be doing a git clone, you might be doing git config, git checkout, modifying the files, git status, git add. Finally, when you run git commit, behind the scenes we invoke our authenticator to extract the private key from the TPM and sign that code commit. The net effect of that is all the commits that you execute on your machine before you are ready to push the code to the git repo are all signed using the Beyond Identity generated GPG private key. 

So, that is one side of the equation where you now have all your commits signed using Beyond Identity, and then we have a check at the backend of the equation where we can configure the git repo, git remote to invoke a specific action using the Beyond Identity API, which in turn makes a real-time call to the Beyond Identity cloud to confirm a bunch of things. 

So, we want to make sure that the commit is signed. We want to make sure that the commit is signed using a key that was generated using Beyond Identity, but we also want to see that the individual who committed the code is still part of the corporate repo. So, you might have a situation where the user was let go, the user was fired, or the user left voluntarily, but he's no longer with the company. 

And using this API key, we can confirm that the key corresponding to that individual is active, the device is not stolen. And using all these checks, we then finally give a green light and your CI/CD pipeline can proceed. So, to summarize, Beyond Identity provides a mechanism to have code provenance through code commit signing. And the unique value proposition is the keys are created using the TPM. 

The key creation and commit signing is backed by zero-trust policies. And we have a GitHub action that fires at the backend as part of your CI/CD pipeline to confirm that the individual is still part of the Beyond Identity corporate directory. 


Beautiful. Thank you for that very explicit and detailed explanation because I understand it. You mentioned non-repudiation, and I can see how that would be a defense against a malicious insider, even if they were introducing back doors or anything else, command and control, you would have a record of who had signed that commit and that would be a good dissuasion from actually doing it because it would be tracked back to them and they'd lose their jobs or be prosecuted. 


That's a great point, Richard, because there could be malicious insiders who think that they can get away because there is no auditable record, but because of the non-repudiation qualities of code commit signing, this can be used as a concrete evidence in the court of law. 


Yep. Okay. So, it sounds like a no-brainer solution to a really big problem. How complicated is it to… If you've got thousands of developers, like a lot of the big banks I talk to have, how complicated would it be to introduce this to your entire CI/CD pipeline? 


Yeah. That's a great question and one that comes up quite often when we typically talk to large enterprises with a distributed user base. And many of these large enterprises also have outsourced or offshore software development, so they have other entities in place. 

The good news is this solution can be introduced in phases. The solution itself also is broken down into multiple components as I explained. So, the first step is for Beyond Identity to enable code commit signing at the backend, and then the administrator gets to decide who can be included as part of this code commit signing process. So, as an administrator, you choose a set of users either individually or based on their group, based on their geolocation, based on the business unit that they're part of. 

Those users who have been brought under the umbrella of code commit signing will see that their authenticator lights up with a new link that they can click to create this GPG public-private key pair. Those users will then just click on that link and then create the key and then set repo. 

That takes less than a few minutes. And once that part is done, the individual user on his specific desktop is all set to use Beyond Identity. Now, from this point onwards, those developers will start committing code and those code commits will be signed using Beyond Identity. 

At this time, only a small set of users are actually using Beyond Identity, but it does not cause any inconsistencies in your overall process. You then start putting in some native checks within your git remote, so, for example, GitHub, GitLab, Bitbucket, Azure DevOps, and AWS CodeCommit all support native checks to confirm that the commit is signed using a valid GPG key and they show a green light. 

So, you could turn on those control points. At this point as an administrator when you go to your git remote, you can see that certain commits are signed, certain commits are not. Again, it does not disturb any of your CI/CD pipeline. You just get increased visibility and an assurance that things are working as expected. 

Once you have now extended that same functionality to additional users by letting them know that they can start creating keys and start committing the code, you have entire user population that is now onboarded to Beyond Identity code commit signing process, now is the time for you to go ahead and turn on that final check on the backend where we can start making API calls to Beyond Identity and accepting or rejecting the code commit. 

So, by the time you come to this stage, you have to ensure that all your users are now complying and committing the code. And this gives you the ability to now declare victory and confirm that Beyond Identity has been rolled out enterprise-wide. 


Brilliant. As you're describing that, it seems like you also solve one of the biggest problems that I've seen for large groups is the onboarding process when a new person is added to a project or a new hire comes on board, you know, as depicted in gruesome detail in Gene Kim's book, "The Phoenix Project." 

It takes forever to get everybody to get a new person their credentials for each different repository and to set their authorizations for what they can do, commit or change or download or view. It sounds like you're giving your customers one place to do all of that. 


Yes. Yes. That's a great point. And Richard, we come across this challenge every day. One of the first requirements for this solution is to have a smooth developer experience. The security teams definitely find a great value in this product, but they want to make sure that developers do not see this as a friction point. 

And as you rightly pointed out, the ability to have a decentralized way to generate the keys at the same time have a centralized way to enable and disable this functionality gives us a unique advantage that we can roll this out in a low friction way. And once you have already put the keys in place, the developers, you know, don't even know that the commits are getting signed behind the scenes. 

They don't have to make any changes. And that has been a huge differentiator in order to embrace, adopt, and succeed in this effort. 


Thank you so much for this detail, Suresh. I know I've gained a much better understanding of both the CI/CD process and Beyond Identity's ability to make it more secure. 

And thanks to all of our listeners who came in to hear this conversation. If you would like more information on what we've discussed today, make sure you head on over to beyondidentity.com. We'll be back next week with another episode in our podcast series. Until then, make sure you subscribe to this podcast on all major platforms. Follow the conversation on our socials at EM360 Tech on both Twitter and LinkedIn. 

And for more great daily content, head on over to em360tech.com