consider LibreSSL as default OpenSSL provider again

This should not be considered a formal system change proposal yet, but more a brain dump regarding my thoughts concerning the recently aborted OpenSSL 3 migration, the future of the OpenSSL project governance, and how trying LibreSSL again might be a better path for us. This would be targeted at 3.17 or perhaps 3.18 release if implemented, as there are surely things that would need implementation in LibreSSL.

Current situation

The current situation is that we ran into a serious regression with OpenSSL 3 involving apk-tools, which was caused by the implementation of the new providers API. Implementation of the new providers API also caused fallout in other packages, such as wpa-supplicant, which use deprecated protocols such as MD4 as required by the WPA-PSK specification.

As we did not have good answers on how to move forward, we hit the timeout period for the system change proposal and the contingency plan automatically went into effect, leading to the revert which has concluded today.

Current status of OpenSSL

The aforementioned regression has been acknowledged, but no work has been done to fix it, despite being a serious regression and open for a month as of this writing. It seems like it will be difficult to fix due to the way the providers API is implemented.

Meanwhile, there are more problems: it turns out that OpenSSL 3.x will not have LTS releases of the same length as past branches of OpenSSL. In addition, there are governance problems, as outlined by Rich Salz in his email to the openssl-project mailing list: the OpenSSL developers appear to want to focus on developing new features rather than cleaning up the mess of regressions they have created with OpenSSL 3.

Finally, my own review of how the OpenSSL providers API has been implemented makes me doubt the performance impact will be negligible. There is, for example, no caching of provider requests, and lazily loading provider modules on demand means that initial connections serviced by a TLS server/client using OpenSSL 3 may have significant latencies.

In short, based on the real-world deployment of OpenSSL 3 in Alpine, I have a lot of doubt about it. The governance-related discussions have done nothing to assuage these doubts. I also have doubt that the EVP_md_null regression, which is known to be related to the introduction of the providers API, is the only regression, there could be other crash issues, or worse yet, the possibility of a cryptographic downgrade attack (consider downgrade to NULL cipher, for example).

Revisiting the rationale for switching to OpenSSL 3

There were two areas of concern which made OpenSSL 3 desirable to Alpine:

Licensing

OpenSSL 3 was relicensed under the Apache 2.0 license. Many copyleft licenses are known to be explicitly compatible with the Apache 2.0 license, such as the GPLv3.

However, it is the opinion of the Alpine license review community that the Apache 2.0 license is not compatible with GPLv2. It is also the opinion of the Alpine license review community that the OpenSSL 1.x license was already compatible with both GPLv2 and GPLv3 due to the system library exception: it is generally not possible to install an Alpine system without having an OpenSSL implementation, so it clearly qualifies as a system library.

This means that the originally perceived benefit of OpenSSL 3's Apache licensing is moot: from our perspective, which we believe would be defensible, OpenSSL is a system library and therefore the GPL requirements do not apply to it.

There are also some which have doubted the methods used to support the relicensing effort. This was not a direct concern of anybody dealing with legal issues in Alpine, but one could raise questions about indemnification.

Providers API

It is known to us (and presumably everyone else) that the OpenSSL ENGINE subsystem is a mess. We had an explicit interest in the providers API as a replacement to the ENGINE subsystem. However, the providers API has introduced regressions, and many ENGINE modules were not ported over as providers. It is unknown whether these modules will ever be ported over, before they are removed in OpenSSL 3.1 -- initially I was hopeful, but now I am a bit more doubtful.

Meanwhile, the hardware acceleration capabilities enabled by the ENGINE modules was one of the primary reasons we switched back to OpenSSL from LibreSSL to begin with. So, if we are doomed to lose support for Padlock and other AES accelerators in OpenSSL 3, then this is not a compelling reason to stick with it.

By comparison, the LibreSSL developers do not plan to reintroduce the ENGINE API, but I have been told that they would be willing to consider an alternative implementation, as long as it is clean and passes review. This means that basically, either way, we are probably going to be stuck porting the ENGINE modules for the AES accelerators we want to support, therefore I consider LibreSSL and OpenSSL 3 equal here.

Accordingly, regardless of which path we take, figuring out what happens with Padlock should be considered an action item.

Current status of LibreSSL

While the OpenSSL project was busy missing the OpenSSL 3 release date by several years, firing multiple project managers in the process, the LibreSSL developers have started to replace large swaths of the OpenSSL codebase with new ISC-licensed code, while maintaining compatibility with the majority of OpenSSL 1.0 and 1.1 APIs. LibreSSL also has not fired any project managers or missed any key deadlines. Also, Alpine already has a productive working relationship with the OpenBSD community relating to other initiatives.

While it is laudable that the OpenSSL developers are starting to clean up the OpenSSL API, one could argue that we should really migrate applications to using the libtls API instead, which is already a clean, high-level API that is well understood and audited.

And, as noted above, the LibreSSL developers are willing to collaborate on missing functionality as needed. I do not think we can expect such levels of collaboration with the OpenSSL team, even if their project had healthy governance: they appear to have intentionally installed multiple layers of red tape between themselves and the community.

FIPS mode

One major issue that would require addressing is that LibreSSL has removed FIPS mode, while we have end users who require FIPS mode for compliance reasons. One possibility could be to reintroduce FIPS mode as a set of configurations which restrict ciphersuites to ones that have been approved for use under FIPS. In the past, however, this led to OpenSSL being used for some packages in lieu of LibreSSL so that those users could make use of a FIPS module.

Discussing FIPS compliance functionality with LibreSSL developers would be considered a task item if we looked in this direction.