On 17.05.19, CVE-2019-11328, a vulnerability in Singularity was publicly announced.
The vulnerability affects Singularity 3.1.0 to 3.2.0-rc2. It allows a malicious user with local or network access to the host machine cause namespace privilege escalation, privileged corruption file and possibly privilege escalation.

What is Singularity

Before I start explaining the vulnerability, let’s go over the basics –
Singularity is a free, cross-platform and open-source container platform that started as an open source project created by a team in Lawrence Berkeley National Laboratory and now is maintained by Sylabs.

Singularity was designed for the scientific community. It’s supposedly used by universities along with commercial HPC centers and as HPC becomes more and more popular, so does Singularity.

Singularity supports high-performance communication architectures and standards, such as InfiniBand and Intel Omni-Path Architecture which make it fit the HPC world.

The vulnerability

When a Singularity container starts running, the directory – `/run/singularity/instances/sing//` gets insufficient permission.
The permission of this directory is set to “:root” with mode 550 and since the
unprivileged user is the owner of the directory, he may change its content.
In this interesting sensitive directory, there are 2 things –

  • .json – A configuration file for the container
  • ns directory – directory that can contain the namespaces for of the container

This directory is our pandora’s box and now that we know that it is accessible and can be open, out troubles begin.

Exploitation

There are several ways to exploit this vulnerability which will lead to several impacts.

  • The easiest way to exploit this vulnerability is to create symlinks to some desired namespaces which we are not allowed to access and put them inside “ns”.
    Then, they will be followed when joining the container instance and allow an unprivileged user to access the desired namespace.

  • Another way to exploit the vulnerability is by changing the .json, which is a configuration file for the instance.
    “Starter-suid” (the component that is in charge of starting Singularity containers) rely on this file so by changing it we can cause a lot of trouble.
    For example, We can configure any namespace by changing the JSON and the “ns” directory, or we can even set the noNewPrivileges flag to false allowing the user to run without the PR_SET_NO_NEW_PRIVS bit, which normally restricts the process from gaining more privileges than his parent,

  • Furthermore, after the creation of an instance, the unprivileged user can place a symlink in the path of /.json and the next time the user will try to create the same instance he will get an error, but before that `starter-suid` program will follow this symlink and create and truncate an existing file in the target location. This allows to create or overwrite arbitrary files in the system with root privileges. The content written to the file is only partially controlled by the attacker because the beginning of the JSON has a fixed structure but he is still able to do a lot of damage.
  • On top of that, it may be possible to gain root access on the host by using a combination of the exploitations above and by some way to manipulate some components on the host or in Singularity, but so far I and other researchers haven’t found a way of doing it.

I’ve created a demo to demonstrate how serious and dangerous the vulnerability is.
In this demo, you will be shown that any file in the system can be corrupt with the vulnerability and nothing is safe.
https://www.twistlock.com/wp-content/uploads/2019/04/Singularity_exploit.mp4

Mitigation

Sylabs released Singularity 3.2.0 which contains the security fix.

They commented on the new release that in order to fix this issue – “Instance files are now stored in the user’s home directory for privacy and many checks have been added to ensure that a user can’t manipulate those files to change the behavior of instances when they are joined”.

The full release notes can be found Github: https://github.com/sylabs/singularity/releases/tag/v3.2.0

The Bottom Line

This vulnerability is a severe issue and can lead to various impacts given the fact that an attacker has access as an unprivileged user on the host and I would recommend firmly to all of Singularity users to update their software to the latest release.

Lastly, I would like to add that users should pay attention and watch closely for the latest releases of any container runtime because lately there are many issues that are being revealed thanks to a growing amount of researchers who are looking for vulnerabilities in those platforms, including us in Twistlock.