Category Archives: Lessons Learned

Valheim Dedicated Server on Kubernetes

Posted on by

What is Valhiem?

Valheim official Graphic

Valheim is a brand new early access games that just hit Steam. It’s a brutal exploration, survival, and crafting game inspired by Viking culture. The game world is generated based on a random or provided seed value and allows up to 10 people to play together by default.

Out of the box, the game also does support sharing your server over steam cloud, so you don’t actually need a dedicated server or forwarding to play with your friends. However, forcing one person to leave their game open and active to share the world, can caused its challenges.

So after going through the trial and tribulations from the Norse gods. Here is what I’ve learned about how to run a Valhiem Dedicated Server on Linux, Docker, and Kubernetes. I will try my best to provide updates to this post for as long as I’m able and playing the game.

Where are the world saves?

Since the world is entirely procedurally-generated, each world is saved every 30 minutes to the following folders.

  • Windows -> C:\Users\<user>\AppData\LocalLow\IronGate\Valheim\worlds
  • Linux -> ~/.config/unity3d/IronGate/Valheim/worlds

If you want to share your game saves, these files should be portable. If you want to move an existing save to a dedicated server, just move the world files over to your server and make sure the world argument is set to the same as the world file you are targeting when you start the server (example: -world “myworld” correlates to myworld.db and myworld.fwl).

A Quick note on Port Forwarding

You will need to Port Forward on your router and/or firewall for your dedicated server to work! Due to the complexity and diversity involved in port forwarding, I’m not going to include direction for it in this guide. By default these ports are 2456/UDP and 2457/UDP.

How to Run Valheim Dedicated Server Linux

Start by setting up steamcmd and downloading the game files (based on Ubuntu/Debian based systems, see official docs for other distros). 

useradd -m steam # create steam user for security and isolation
cd /home/steam # move to the home directory to keep files clean
sudo apt install steamcmd # install the thing

Next we can download the game files with steamcmd.

steamcmd +login anonymous +force_install_dir ./valheim +app_update 896660 +quit

Then we can modify the server start script (start_server.sh) that comes with default server files. Changing the server name and password are a definite must, but changing the port or world name (reference to a world save) are not required. Then simply run the script to start the server.

bash ./valheim/start_server.sh

Running Valheim Dedicated Server with Docker

Without getting too far into the weeds on the details behind docker container, we can get a server up with two fairly simply commands. First we just need to download the app data to a local folder.

mkdir ${PWD}/valheim-server # make the directory if its not there
docker run -it -v ${PWD}/valheim-server:/data steamcmd/steamcmd:latest +login anonymous +force_install_dir /data +app_update 896660 +quit

Now that we have the server files, we need to modify the start script severname and password as before. But we also need to restructure the file, because the world will be saved to the ~/.config and not the /data volume we mounted.

Very Important: If you don’t capture or link your world saves to your data volume, your world could be lost because containers are ephemeral.

To make up for the lack of control of where the world data is being stored, we can just utilize symbolic links to redirect the ~/.config files to our /data volume. To do this we can use a script like the following.

export templdpath=$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/data/linux64:$LD_LIBRARY_PATH
export SteamAppId=892970

mkdir -p /root/.config/unity3d/IronGate/Valheim
ln -s /data/adminlist.txt /root/.config/unity3d/IronGate/Valheim/adminlist.txt
ln -s /data/bannedlist.txt /root/.config/unity3d/IronGate/Valheim/bannedlist.txt
ln -s /data/permittedlist.txt /root/.config/unity3d/IronGate/Valheim/permittedlist.txt
ln -s /data/prefs /root/.config/unity3d/IronGate/Valheim/prefs
ln -s /data/worlds /root/.config/unity3d/IronGate/Valheim/worlds

# Tip: Make a local copy of this script to avoid it being overwritten by steam.
# NOTE: Minimum password length is 5 characters & Password cant be in the server name.
# NOTE: You need to make sure the ports 2456-2458 is being forwarded to your server through your local router & firewall.
/data/valheim_server.x86_64 -name "Hackersvanguard" -port 2456 -world "Dedicated" -password "CHANGEME" -public 1

Finally we can just reuse the steamcmd container to run the new startup script and launch the server.

docker run -it -v ${PWD}/valheim-server:/data -p 2456:2456/udp -p 2457:2457/udp steamcmd/steamcmd:latest bash /data/start_server.sh

Running Valheim Dedicated Server On Kubernetes

To build on the ideas and method laid out in the docker section. Instead of running the docker container locally, we can create a quick deployment and service file to run on Kubernetes instead.

To start we can create simple app deployment yaml file with a volume which contains our server data and modified start script from the docker sections. Here I use a simple hostPath volume with a node selector, but a PVC would work all the same.

Very Important: If you don’t capture or link your world saves to your data volume, your world could be lost because containers are ephemeral.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: valheim-deployment
  name: valheim-deployment
  namespace: valheim
spec:
  replicas: 1
  selector:
    matchLabels:
      app: valheim-deployment-pod
  template:
    metadata:
      labels:
        app: valheim-deployment-pod
    spec:
      containers:
      - image: steamcmd/steamcmd:latest
        name: valheim-server
        ports:
        - containerPort: 2456
          protocol: UDP
        - containerPort: 2457
          protocol: UDP
        command: ["sh /data/start_server.sh"]
        volumeMounts:
        - name: valheim-data
          mountPath: /data
        lifecycle:
          preStop:
            exec:
              command: [" echo","1",">","/data/server_exit.drp"]
      volumes:
        - name: valheim-data
          hostPath:
            path: /opt/valheim-data
            type: Directory
      nodeSelector:
        kubernetes.io/hostname: kubenode1

Note: Make sure you have a copy of your server files and/or world data in your hostPath, on the node targeted by the NodeSelector. In the example I have a folder of “/opt/valheim-data” on a node with hostname kubenode1.

Next we can just create the nodPort service for the deployment so that we can portforward directly to our Valheim server.

Note: In this example we are using nodePort 32456 and and 32457. Therefore you would need to port forward to the nodes IP address and the nodePorts, not the default dedicated server ports.

apiVersion: v1
kind: Service
metadata:
  labels:
    app: valheim-deployment-svc
  name: valheim-deployment-svc
  namespace: valheim
spec:
  ports:
  - name: port-1
    nodePort: 32456
    port: 2456
    protocol: UDP
    targetPort: 2456
  - name: port-2
    nodePort: 32457
    port: 2457
    protocol: UDP
    targetPort: 2457
  selector:
    app: valheim-deployment-pod
  type: NodePort

Now all we have to do is made the resources within Kubernetes using kubectl.

kubectl create -f valheim-deployment.yaml
kubectl create -f valheim-service.yaml

Valhiem Server Access Lists

The Valhiem Sever also maintains a set of access list files to control what role users have on the server. These files can be found one directory up from your world saves, within the Valheim base directory. The files are all structured with one player ID per line. The Player ID can be found within the F2 server status menu, next to each players name.

  • adminlist.txt – list of server admin who can issue sever commands
  • bannedlist.txt – list of users who are banned form the server
  • permittedlist.txt – list of users who are allowed to join the sever when not set to public

Valhiem Server Basic Commands

Here are the basic console commands (opened by pressing F5) used to administer a Valhiem dedicated server.

  • help – Show all available commands.
  • kick [name/ip/userID] – Kick the user.
  • ban [name/ip/userID] – Ban the user.
  • unban [ip/userID] – Unban the user.
  • banned – Shows a list of banned users.
  • ping – Send a ping to the server to get your latency.
  • info – Print system info

If you run into issues and need to spawn in items or would rather play in a pseudo-creative mode. You can type “imacheater” in the console to get access to a full suite standard admin console commands.

Skal!

I wanted to get this information out as quickly as possible to help those who may be struggling. Let me know any questions, comments, or feedback on any of the socials @sleventyeleven.

Sometimes You Just Have to Proxy Your Socks Off

Posted on by

Problem

Sometimes during assessments sensitive systems are significantly segmented from other networks. Therefore its very important for penetration testers to know how to proxy your socks off in order to move across network.

Solution

To gain access to other networks, whether it’s the internet or a protected subnet. We can use putty on windows and the native ssh client on Linux to preform port forwarding and create Socks proxies to bypass access controls.

Proxy Your Socks Off - Web server Post Exploitation with SSH Tunnels and Socks Proxy

Proxy Caveats

SOCKS proxies only work for TCP traffic and with applications that support using a transparent proxy. Applications that use their own proxy settings, require forward secrecy, or check session integrity likely won’t function correctly.

All ports from 1-1024 require administrative rights to allocate on both windows and Linux systems.

Port 0 is used to represent a randomly generated port number, in both windows and linux systems.

How to Proxy Your Socks Off

Proxy Traffic in Windows

In Windows simply open putty and enter the IP address you want to connect  in as the Hostname/IP address.

Proxy your socks off - configure Putty SSH connections

Next we have to tell putty that we want it to open a port on the localhost to be used to forward all traffic to our remote host. To do that, we go to the connections -> SSH -> Tunnels section, add a source port, choose the Dynamic option, and click the add button.

Proxy your socks off - configure Putty SSH for dynamic port forwarding options

At this point you can click the open button and authenticate as if it were a normal SSH connection. Just be sure to leave the terminal open once authenticated, to ensure traffic is being passed from the local port to the remote host.

To tell windows to use the socks proxy, open internet options from the control panel or the start menu search. Then go to the connections tab and open LAN Settings.

Proxy your socks off - configure Control Panel Internet Properties for Socks Proxy

Once LAN settings opens, select the “use a proxy server for your LAN” check box and click the advanced.

Proxy your socks off - configure Windows Lan Settings for Socks Proxy

In the Socks box add localhost or 127.0.0.1 and the port you set as dynamic in putty. Then click OK three times to save all the settings.

Proxy your socks off - configure Windows Advanced Proxy Settings for Socks Proxy

Proxy Traffic in Linux

If you need to proxy your Kali system, the process is fairly similar. Start by using the ssh client to dynamically forward traffic from a local port. This can be done with a command similar to the following, where 9050 is our dynamic port.

ssh -NfD 9050 root@159.246.29.206

Next we need to tell proxy chains where to send traffic from our programs. This can be set globally be using a command like the following.

echo "socks4\t127.0.0.1\t9050" >> /etc/proxychains.conf

To run an application through the socks proxy, simply prepend it with the proxychains command, like the following.

proxychains iceweasel

There is not built in means to setup a system wide socks proxy. However the BadVPN package has a package tun2socks that can tunnel all traffic over a local socks proxy.

Proxy Your Socks Off with Metasploit

Sometimes, while doing an assessment you may even want to run some tools such as nmap or even SQL Management studio (ssms.exe) over an established shell. Metasploit has a post module (auxiliary/server/socks4a) that can be used to create a socks4 proxy on an existing session.

However, to start off we need to tell metasploit how to route traffic to each of our shell’s networks before running the socks proxy. This can either be done manually with the route command or if your session is on a windows host with the autoroute module (post/windows/manage/autoroute).

To add a route manually you can use the built in route command with options similar to the following.

route add 10.0.0.0 255.255.255.0 1

To add routes with autoroute, either use the post module or run autoroute from a meterpreter shell. For the autoroute module (post/windows/manage/autoroute) just set the session ID and run. For autoroute from meterpreter use a command similar to the following.

run autoroute -s 10.0.0.0

Once routes are established within metasploit to your target networks, you can run the socks proxy module (auxiliary/server/socks4a) and note the SRVPORT.

Using Proxychains to Proxy Traffic through Metasploit Meterpreter

Next we need to tell proxychains what port to send traffic to within the global configuration file (/etc/proxychains.conf), just like in the Linux example above. There should be a line like “socks4 127.0.0.1 1080” at the bottom of the file, change the port 1080 to whatever your SRVPORT was in metasploit.

Once the configuration file is updated, proxychains can be used to issue commands through metasploit shell(s). Like with the following nmap example.

proxychains nmap -v -sS 10.0.0.0/24

If we want to make this socks proxy available to a windows host for programs like SQL Server Management Studio, perform a local port forward  to the socks port on the Linux system. To do this we can use putty and follow steps similar to those presented above.

Start by creating a local port forward of a local port on our windows system, to the local socks port on the Linux system with putty. Start by allocating a source port for connection on the local system and forward to a destination of 127.0.0.1:1080; where 1080 is your metasploit SRVPORT.

Proxy your socks off - configure Putty SSH to allocate local port to connect to remote Socks Proxy

We can then just configure a system wide proxy by adding our forwarded port as the socks port, instead of using a local socks proxy.

Proxy your socks off - configure Windows Advanced Proxy Settings for forwarded Socks Proxy

Once those settings are changes, we should be able to use the majority of our tools within windows without issue.

Using SSH to Provide Remote System Internet Access via local Socks Proxy

An SSH tunnel can be used to forward traffic from your local system to a port on a remote system. This can be done in Linux by switching the -L option with -R. Or in putty by choosing the Remote option under tunnels instead of Local. For example if you wanted to share your local socks proxy with a remote system to provide internet access, putty can be used with a remote forward like the following.

Proxy your socks off - configure Putty SSH to allow remote host internet access via a remote port forward to a local socks proxy

Using Compromised Linux Webserver to Access Internal Network and Database

It’s also worth noting that SSH port forwarding can be performed on the network socket level and does not require an interactive session be established; only valid authentication is required. For instance, say you wanted to log into a restricted database of a webserver. But you only have access to the webserver account. The webserver user is not allowed to log into the server interactively by default, but that doesn’t mean it can’t authenticate. In many cases SSH can be used as described in my post on SSH for post exploitation to get around limited user shells.

Using Linux Native Tools to Proxy Your Socks Off

Tools natively built-in to windows and linux can also be used to preform port forwarding. Just note that this methodology simply makes a port to port translation and does not manipulate the traffic in any way. Netcat (nc) is found in almost every single Linux distribution and can be used to easily preform port forwarding with commands similar to the following.

First we have to make a named pipe so that any response from the server aren’t dumped to standard out.

mkfifo backpipe

Then we can use a command similar to the following to send traffic from 8080 on the localhost to a remote host on a different port utilizing the named pipe. This could help get around a firewall or help send traffic to another system to be caught by another port translation or process.

nc -l 8080 0<backpipe | nc example.com 80 1>backpipe

Similarly the netsh (commandline windows firewall editor) command in windows can be used to create a local port forward as well. In this cause we can follow the same example and create a port translation from localhost 8080 to example.com on port 80.

netsh interface portproxy add v4tov4 listenport=8080 listenaddress=127.0.0.1 connectport=80 connectaddress=example.com

Windows 7 and above will likely require administrative privileges to make changes to the windows firewall. But you can likely still utilize the windows version of nc or netcat to redirect traffic all the same.

Leveraging Pillaged SSH keys

Posted on by

TLDR; These days when you run into a production Linux or cloud environments, they use public key authentication. Making lateral movement as easy as leveraging pillaged SSH keys.

Level Settings

SSH (Secure Shell) is the primary means of managing Cloud Instances, Linux, Unix, OSX, Networking Devices, Vendor Devices, and even some embedded devices. It’s also worth noting that Microsoft has received glowing reviews and support for its roll out of SSH into current builds, but it is not enabled by default. Generally speaking SSH uses the servers local user base and corresponding passwords to authenticate remote connections. However, SSH can also be configured to use Public Key authentication.

How SSH Public Key Authentication Works

Since SSH is designed to use a RSA or DSA Public (Encryption) key and Private (Decryption) key combinations to encrypt traffic. A user can add a Public key to their authorized keys file, to allow the use of the corresponding Private key for authentication. This allows the user to attempt to establish a secure connection by sending their username and the fingerprint of the Public key to the SSH Server. If a Public key with the given fingerprint is within the requested users authorized keys file, then the SSH server responds with an encrypted challenge. This challenge is encrypted with the users Public key and can only be decrypted with the corresponding Private key. If the challenge is successfully answered with an encrypted respond using the SSH Servers Public key, the client and server are successfully authenticated.

What is the Inherit Problem

These days when you run into a production Linux or Cloud environment, more than likely SSH services are going to use Public Key authentication. The traditional rapid guessing won’t work if only public key authentication is enabled. If a Public key fingerprint is not submitted, then the SSH server will simply terminate the session. So in order to pivot into a high value environment all that’s needed is to locate and begin leveraging pillaged SSH private keys with the proper usernames to gain further access.

How to Pillage SSH Keys

The good news is Private keys are fairly easy to locate on users workstations and development servers. They almost always reside within the default SSH directories.

  • Linux = /home/<user>/.ssh/
  • OSX = /Users/<user>/.ssh/
  • Windows = C:\Users\<user>\.ssh\

As such they can be seamlessly picked up by an SSH client. It’s also worth digging through the home directories of Admin, Developer, and Operation users for .ppk, .key, rsa_id, dsa_id, .p12, .pem, and .pfx files, as they may be private keys.

Using Publicly Disclosed Keys

The even better news is many of the Major product vendors (F5, Cisco, Barracuda, and VMware to name a few) have been getting outted for distributing systems with static Private keys. This means if an admin doesn’t log in, remove the old keys, and manually regenerate new ones, then a shell can be established using publicly disclosed private keys.

Some good repositories to look for bad keys.

https://github.com/rapid7/ssh-badkeys

https://github.com/BenBE/kompromat

The good news is Metasploit has several modules that will make scanning discovered SSH services fairly easy. So all we need to do is feed it the proper data, run, and watch the shells rain in. Metasploit makes preforming private key authentication easy and seamless. All you need to do is give it a list of services, a username, and a private key. If authentication is successful it will even seamlessly establish a shell session for you.

Leveraging Pillaged SSH Keys

First we need a private key file, either one we’ve located from pillaging or a publicly known bad key. For example the publicly disclosed Vagrant (Vagrant preforms cross platform Virtual Machine management) Private key.

The corresponding Public key looks like the following:

ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzIw+niNltGEFHzD8+v1I2YJ6oXevct1YeS0o9HZyN1Q9qgCgzUFtdOKLv6IedplqoPkcmF0aYet2PkEDo3MlTBckFXPITAMzF8dJSIFo9D8HfdOV0IAdx4O7PtixWKn5y2hMNG0zQPyUecp4pzC6kivAIhyfHilFR61RGL+GPXQ2MWZWFYbAGjyiYJnAmCP3NOTd0jMZEnDkbUvxhMmBYSdETk1rRgm+R4LOzFUGaHqHDLKLX+FIPKcF96hrucXzcWyLbIbEgE98OHlnVYCzRdK8jlqm8tehUc9c9WhQ== vagrant insecure public key

The second thing needed is the username of the user who has the public key in their authorized key file. As stated in the note, this can normally be found in the public key note. In the case of the vagrant key, the username is also widely known to be vagrant.

With a Private key and Username combination, the auxiliary/scanner/ssh/ssh_login_pubkey module can be used to scan for systems that the private key works on. A session will be established when authentication is successful. When a session is established Metasploit will also collect basic system information for you, including hostname, kernel version, and group memberships.

Finding the Username for Pillaged SSH Keys

Public keys listed within a user authorized_keys file can have comments after the actual key data. Most SSH key generates take advantage of this comment field, to add the username and hostname when a key is generated.

It’s also worth noting that most SSH clients keep a known hosts file, for integrity purposes, which can be viewed to see which systems the key was used to access recently.

If you find just a Private key file during pillaging, the public key data can be derived form it in most cases. However the username likely won’t be associated with it. When no username is found, a common username file can be passed alongside the key in Metasploit.

Speeding Scans up with sshscan

The Metasploit SSH modules are not threaded safe and running more than one connection at a time could cause a thread to hang or exhaustion of system resources. SSH generally is not considered thread safe, because responses after the authentication process are not formally structured. However there is a SSH scanner written using the native go SSH client, which works very well. Just take care to ensure the command you run, provides a simple, small, and structured output (like id). https://github.com/CroweCybersecurity/go-sshscan

SSH Defense Strategies

  1. When generating a Public and Private key pair, a passphrase can be provided to protect the keys. When a passphrase is setup, the SSH client must prompt for the passphrase every time the private key is used. Thus if a key with a passphrase is discover by am attacker its normally not usable.
  2. Implementing an enterprise key management solution to ensure all systems have their own private keys. This would simply crush the reuse factor and stop lateral movement.
  3. Configuring the SSH Server to require both the public key and the users password for authentication. This will slow scanners to a crawl, as the password prompt would cause the session to hang, once the key authentication has completed.
  4. Have a single Private key for all hosts that provides access to a lowest privilege user. Once a connection is established legitimate users can switch to their respective user accounts. If a key was discovered during an assessment we would have to dig through all the systems hoping for a major system misconfiguration. Hopefully, a needle in a haystack.
  5. Avoid key management all together, by utilizing Certificate Authority (CA) backed system to automatically generate sign key pairs for authorized users. The biggest tech companies already do this and some have even blogged about it in the past.

Other SSH Blog Posts