(© Alexander Limbach - Shutterstock)

When the Log4j vulnerability news first came out, it seemed like a problem for overworked security experts. But as the patching crisis unfolded, many ERP managers spent their holidays on the job instead, addressing Log4j vulnerabilities in mission-critical systems.

Avantra’s customers were no different. We were called upon for help — and to push our software to its limits to automate SAP system patching. We are now evolving our own security automation capabilities. Along the way, we learned plenty of lessons on how to cope with Log4j — and to get ready for what could be coming next.

On Monday February 7th, SAP released HotNews note 3123396. This security vulnerability has a CVSS of 10. SAP's recommendation is to update the SAP kernel to the latest patch level listed in the note and to upgrade any SAP Webdispatcher systems. For more information on this vulnerability see SAP note 3123396

By now, many SAP customers have addressed this kernel update. If you haven’t — time is of the essence.

So how do SAP operations teams make security part of their team DNA? How do they get out of fire fighting and security crisis mode? To answer these burning questions, I asked the team at Avantra to put together a quick Q&A.

How relevant is Log4j in the SAP world?

Many SAP products are based on the Java programming language. Log4j is used in quite a number of the Java based products, but not in all. By time of this writing, SAP has released 23 SAP HotNews (i.e. urgent news often concerning security) documenting log4j vulnerabilities. But even in the products where it is not used by default, Log4j can still be contained in 3rd party applications which are deployed on the SAP products, for example SAP NetWeaver Java or the SAP cloud platform. — Answered by Bernd Engist, CTO Avantra.

How could a vulnerable Log4j version impact my business?

Very severely. The problem with this vulnerability is the simplicity of the attack. If, for instance, the content of an input field of an web application is logged with Log4j, it is enough to just manipulate the input value so the backend application could be infected by remote malicious code. Once getting to such a level, further malicious code can be infiltrated and bring a whole business’ IT landscape down. — Answered by Bernd Engist.

Where should I look for more information about the impact to my landscape?

Bernd Engist: First priority is to check all internet facing systems if they make use of Log4j. You may contact your software vendors. Is the system running on Java?

How can I be sure that I’ve updated all Log4j installations within my landscape?

Put measures in place to search for vulnerable libraries.

A quality SAP automation platform should allow you to see what the current kernel patch release you have installed. In Avantra’s case, our Enterprise Edition can perform the automated SAP kernel updates. — Answered by Bernd Engist.

How can I stay ahead of future vulnerabilities like this and ensure I can react quickly?

There are a couple of things to consider. It starts with making sure you know your estate. What are the most business critical applications your business is relying on? Which of those are SaaS products where you only need to track what the vendor is doing, which are the ones that fall into your operations responsibilities.

Make sure for each and every one of those you understand how you will get aware of any vulnerabilities. Not every vulnerability will become as popular as log4shell or shellshock or heartbleed. But that doesn’t mean they are less critical to your business. Some vendors may send out emails if they uncover vulnerabilities, some may publish information on their web sites. You can also subscribe to tailored RSS feeds of CVE database web sites if you don’t want to rely purely on the vendor’s communication channels. However, as we have seen with log4j, the vulnerability may be located in one of the libraries, and you don’t even know that some product is using these. So, at the end of the day you more or less have to rely on the vendors.

If you are using home grown software, make sure your development teams understand and identify the third-party libraries they make use of. There are services out there that automatically scan these libraries for known vulnerabilities. Make sure you take advantage of that.

The next crucial part is that all this information about vulnerabilities comes together at one central point where it can be monitored around the clock. Having them buried in several people’s overcrowded inboxes will not get you moving. Ideally you have established some incident response process or event team to take it from there. One way or the other, this is the point where you should perform your risk assessment. The CVE score will be a major indicator of the implied risk. But even if the vulnerability is critical and widely popular, it may turn out the probability to actually exploit it in your particular environment is rather low. Or the exact opposite may be true. There is no way around your own risk assessment.

Based on the result of the assessment, you may either handle the issues within your regular patch management operations. Or you invoke an emergency change. Again, it is helpful to have an established protocol for this, including a change advisory board that has the authority to make the required decisions.

When it comes to the technical implementation of these changes or patches, automation may be key to success. A high impact of a vulnerability often means many systems are affected, as with log4shell. And while you may have an established and scalable patch management process in place, your first response can as well be to implement some workaround. And you don’t want to do this manually on perhaps hundreds or thousands of systems.

The automation aspect is not restricted to remediation actions: at Avantra we were able to search for vulnerable log4j libraries on hundreds of systems in development, production, and on end user devices. Simply by executing a simple shell or PowerShell command in parallel on all devices. Solutions like Ansible and Avantra, in particular in SAP environments, can become crucial to incidence response. And not only were we able to identify vulnerable systems internally, but we also provided an add-in to the Avantra customers to help them identify vulnerable systems. At scale.

Having said that, the short answer to the question is: maintain a proper inventory, process vulnerability notifications centrally, establish an incident response process, have a role in place to authorize emergency changes, and automate remediation actions. — Answered by Heiko Mannherz, Chief Innovation Officer, Avantra.

How do I make security part of the team’s DNA?

The slightly cynical answer would be: rest assured, this will come anyway. However, you have the choice whether or not it will be the hard way. After the SAP Recon vulnerability was discovered, the first one related to SAP marked with a highest CVS score, we prognosed that it’s not a question if the next one will come, but only when (see: How to prevent SAP security vulnerabilities). And this question has been answered in the meantime.

The best thing you can do is integrate security vulnerabilities into your Business Continuity Planning. Although BCP is an integral part of Information Security Management Systems, it often focuses primarily on service disruptions or hardware failures. Make a severe security vulnerability the center of your next desktop exercises: assume you have to urgently patch systems at large scale. Or prevent access except for a very limited number of people. Or implement a workaround by executing a bunch of operating system commands. This is a way to make your team familiar with common remediation measures, without the pressure of an actual threat. Do this twice a year, or even once a quarter.

There are also lots of excellent trainings out there. You might want to consider making security training as mandatory as ERP operation training. But don’t just focus on the ones your ERP vendor offers. Attackers don't either.

If you’re looking for ideas on how to make this feel more real, just have a look at what's already happening. Ransomware attacks, Denial of Service attacks, they are all out there. Don’t wait until they hit you. — Answered by Heiko Mannherz.

Want more insights? Bernd Engist reviews how you can stay ahead of attacks with intelligent automation. Read his latest article on the Avantra blog (Protecting your SAP systems from new vulnerabilities), or watch the live session replay, SAP HotNews and CVE kernel patch: Securing your SAP systems.