xz/liblzma Backdoor: Open Source Nuke? Maybe Not That Bad!
xz/liblzma Backdoor: Open Source Nuke? Maybe Not That Bad!
Story Background
On March 29, 2024, a report exposing a backdoor in the upstream source code of the controversial open-source project, the xz software package, was made public on the oss-security mailing list. This backdoor affected the liblzma library, which is a part of the xz software package. Following the initial report, further research was conducted, and the key findings are as follows:
-
The backdoor was present in its entirety in the released xz source code packages (versions 5.6.0 and 5.6.1). However, in the upstream Git repository, it was disguised as test data and not inserted as payload into liblzma. The payload injection occurred separately by adding the wake-up code to the source code package before packaging it. Therefore, the backdoor does not exist in the liblzma compiled from the Git repository or the source code packages generated by GitHub.
-
The injected code modified the build system to include malicious files, which were executed during the build process, leading to further payload injection.
-
The payload targeted x86-64 GNU/Linux systems using gcc and the GNU linker, specifically as part of Debian or RPM software package builds.
-
The malicious code aimed to hijack OpenSSH servers to achieve remote code execution but significantly slowed down the login process. It accomplished this by hijacking and modifying certain functions in the liblzma library, to trick the dynamic linker to override RSA_public_decrypt() of OpenSSL with a malicious version, with the payload indirectly loaded into sshd. The inclusion of liblzma was due to its dependency on other parts of libsystemd, as sshd implemented support for systemd-notify. The complexity of systemd once again became an actual security concern.
-
Immediate upgrading of any potentially affected systems is advised, as these compromised versions have not yet been widely integrated into GNU/Linux distributions.
-
A script is provided to detect whether a system may be affected.
-
The backdoor operator disabled the LANDLOCK sandbox, which has since been fixed by the main maintainers.
However, this is a serious supply chain attack that affects the xz tool and could have a widespread impact on the GNU/Linux ecosystem. Immediate action is required to mitigate this risk. The above is a brief summary of the story, and interested readers can refer to the relevant reports for more information.
Who did this? WTH is Jia Tan (JiaT75) ?
The maintainer of xz-utils is Lasse Collin (Larhzu). He has been the primary maintainer of this open-source project since 2009. Larhzu mentioned in an email in June 2022 that he has mental health issues, so most of the work is taken up by Jia Tan. Jia Tan started contributing to the maintenance of xz-utils in March 2022 and released his first version in January 2023. In March 2023, he took over the responsibility for the signing and release of the tar package.
Is Jia Tan a Chinese APT group?
There have been speculations suggesting the possibility of Jia Tan being associated with a Chinese APT organization. These speculations are primarily based on the use of a Chinese Pinyin name and the submission timestamps on GitHub indicating the use of GMT+8 time zone.
It would be hasty to conclude that Jia Tan and the organization behind them are from a Chinese APT based solely on these two factors. The following reasons support this:
- APT groups typically do not use naming conventions associated with their own country.
- The extensive two-year-long infiltration and the meticulous refinement of certain technical aspects do not align with the characteristics commonly observed in Asian APT actors.
- Based on Jia Tan’s GitHub activity, the majority of their active hours fall between 11 AM to 6 PM UTC time. This timeframe is less likely for a regular worker with a day job in the Asian region.
Is the xz/liblzma backdoor the nuke of the open-source world?
It should not be that catastrophic. The backdoor was discovered early in the implantation process, so it did not affect GNU/Linux systems in production environments. It is likely that Jia Tan and the organization behind them had additional activity, such as infiltrating the Linux kernel community. Nevertheless, we should be grateful for engineers like Andres Freund who exist on this planet.
It feels like this world is full of crises. Can we still trust open-source projects?
Yes, welcome to the harsh reality. This world is indeed filled with dangers, but that doesn’t mean it’s the end of the world. Attackers face many obstacles when launching attacks, and even in cases of supply chain contamination, backdoor attacks can be detected and prevented through a combination of technology and other factors. The most important aspect of free and open-source software is transparency, which increases the likelihood of public auditing. The availability of source code, including in the case of this backdoor incident, contributed to its discovery.
Can SBOM and SLA completely eliminate such problems?
In simple terms, they cannot. The industry has spent billions of dollars over the past few years trying to solve supply chain issues, but the results have not been significant. To mitigate risks at the supply chain level, leveraging technologies related to AGI (Artificial General Intelligence) can be helpful. Additionally, treating and encouraging engineers to participate in the development of best practices for open source is crucial.
Are there any technical solutions to defend against supply chain attack?
Different scenarios require different technical approaches. In addition to implementing standard security measures, addressing the technical aspects involved in the xz/liblzma backdoor, such as hooking the dynamic linking process, can be detected and prevented by ensuring the integrity of the relevant code sections in processes. VED (Vault Exploit Defense) attempted to implement specific code segment integrity measurement for user-space processes to protect against process injection.
There’s another open source ref proc-integrity. However, at the time, users didn’t show much interest in this feature. Perhaps the xz/liblzma backdoor incident will lead to a change in industry perceptions and priorities.
Do you have a conspiracy story?
We have prepared a story for tomorrow’s holiday, and the mastermind behind it is actually M$. Their intention is to destroy the GNU/Linux ecosystem. First, they secretly bought out Lennart Poettering and had him develop systemd. Under M$’s guidance, Lennart gradually infiltrated systemd into GNU/Linux distributions, paving the way for future actions. In March 2022, M$ believed it was time to execute the next phase of their plan and deployed the Jia Tan team to negotiate with the xz-utils community. Lennart Poettering’s mission came to an end. In June 2022, M$ publicly recalled (hired) Lennart. Since then, M$ has planned to support systemd’s ecosystem with openssh in 2023, allowing Jia Tan to take control of the entire GNU/Linux ecosystem. It’s just that the perfect plan was disrupted by an engineer who was obsessed with performance and pursued computational aesthetics. M$ immediately ordered Github to close the relevant accounts to destroy evidence. Fortunately, in the cyberspace of this blue planet, there is a way-back machine, which allowed crucial information on Github to be preserved. Warning: This is a story only suitable for tomorrow’s telling.
Reference
- XZ Backdoor Analysis and symbol mapping, https://gist.github.com/smx-smx/a6112d54777845d389bd7126d6e9f504
- FAQ on the xz-utils backdoor, https://gist.github.com/thesamesam/223949d5a074ebc3dce9ee78baad9e27
- modify_ssh_rsa_pubkey.py, https://gist.github.com/keeganryan/a6c22e1045e67c17e88a606dfdf95ae4