Kubernetes Harvester to Gather Credentials with Limited Access

Project URL: https://github.com/sleventyeleven/Kubernetes-Harvester

Kubernetes Harvester Example Run

What is Kubernetes Harvester?

Harvester is a new python based project that attempts to leverage access in order to gather potentially sensitive information. Its designed to either leverage the access of users credentials or the default access granted to a pod via automountServiceAccountToken, which I wrote about recently. The harvester.py script currently primarily targets pod container environment variables, container manifest environment variables, and config map entries utilized as environment variables, to look for potential credentials.

Why Create A Credential Harvester

The default admission controls in many of the Kubernetes implementation apply a read/view policy to newly created users. However custom policies, admissions, and operators have become more common place. What’s more troublesome is the read permissions given to the automountServiceAccountToken by default. Without adjusting or disabling service tokens, compromised containers could effectively read all pod specs in all namespaces. With access to all pod specs, an attacker could potentially gather credentials or other sensitive information. Harvester is a tool that attempts to help automate the review process.

How Kubernetes Harvester Works

The harvester.py script utilizes the automountServiceAccountToken mounted within a given container or the standard user credentials within the Kube config file (~/.kube/config). Then the Kube API server is queried to look for sensitive information within the pod spec of each pod in the following steps.

  1. Use access to request pod specs for all namespaces within the cluster.
  2. Parse all pod specs to map and dedupe container container information
  3. Review each containers environment variables for sensitive values
  4. Review each config map entry, mapped to container environment variables for sensitive values
  5. Attempt to pull each container image and review the manifest environment variables for sensitive values
  6. Attempt to request authentication tokens from the internal metadata API for each of the major cloud provider

Other Resources:

  • Introduction to Kubernetes – A Free introduction course diving into Kubernetes as a tool for containerized infrastructure. Its a a great place to begin if your just getting started with Kubernetes.
  • The Linux Foundations Official Course – This is the most robust general knowledge based course I’ve seen. If you want to learn Kubernetes and how to do almost anything with it, get the CKA + CKAD combo package.

Valheim Dedicated Server on Kubernetes

What is Valhiem?

Valheim official Graphic

Valheim is a brand new early access games that just hit Steam. It’s a brutal exploration, survival, and crafting game inspired by Viking culture. The game world is generated based on a random or provided seed value and allows up to 10 people to play together by default.

Out of the box, the game also does support sharing your server over steam cloud, so you don’t actually need a dedicated server or forwarding to play with your friends. However, forcing one person to leave their game open and active to share the world, can caused its challenges.

So after going through the trial and tribulations from the Norse gods. Here is what I’ve learned about how to run a Valhiem Dedicated Server on Linux, Docker, and Kubernetes. I will try my best to provide updates to this post for as long as I’m able and playing the game.

Where are the world saves?

Since the world is entirely procedurally-generated, each world is saved every 30 minutes to the following folders.

  • Windows -> C:\Users\<user>\AppData\LocalLow\IronGate\Valheim\worlds
  • Linux -> ~/.config/unity3d/IronGate/Valheim/worlds

If you want to share your game saves, these files should be portable. If you want to move an existing save to a dedicated server, just move the world files over to your server and make sure the world argument is set to the same as the world file you are targeting when you start the server (example: -world “myworld” correlates to myworld.db and myworld.fwl).

A Quick note on Port Forwarding

You will need to Port Forward on your router and/or firewall for your dedicated server to work! Due to the complexity and diversity involved in port forwarding, I’m not going to include direction for it in this guide. By default these ports are 2456/UDP and 2457/UDP.

How to Run Valheim Dedicated Server Linux

Start by setting up steamcmd and downloading the game files (based on Ubuntu/Debian based systems, see official docs for other distros).

useradd -m steam # create steam user for security and isolation
cd /home/steam # move to the home directory to keep files clean
sudo apt install steamcmd # install the thing

Next we can download the game files with steamcmd.

steamcmd +login anonymous +force_install_dir ./valheim +app_update 896660 +quit

Then we can modify the server start script (start_server.sh) that comes with default server files. Changing the server name and password are a definite must, but changing the port or world name (reference to a world save) are not required. Then simply run the script to start the server.

bash ./valheim/start_server.sh

Running Valheim Dedicated Server with Docker

Without getting too far into the weeds on the details behind docker container, we can get a server up with two fairly simply commands. First we just need to download the app data to a local folder.

mkdir ${PWD}/valheim-server # make the directory if its not there
docker run -it -v ${PWD}/valheim-server:/data steamcmd/steamcmd:latest +login anonymous +force_install_dir /data +app_update 896660 +quit

Now that we have the server files, we need to modify the start script severname and password as before. But we also need to restructure the file, because the world will be saved to the ~/.config and not the /data volume we mounted.

Very Important: If you don’t capture or link your world saves to your data volume, your world could be lost because containers are ephemeral.

To make up for the lack of control of where the world data is being stored, we can just utilize symbolic links to redirect the ~/.config files to our /data volume. To do this we can use a script like the following.

export templdpath=$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/data/linux64:$LD_LIBRARY_PATH
export SteamAppId=892970

mkdir -p /root/.config/unity3d/IronGate/Valheim
ln -s /data/adminlist.txt /root/.config/unity3d/IronGate/Valheim/adminlist.txt
ln -s /data/bannedlist.txt /root/.config/unity3d/IronGate/Valheim/bannedlist.txt
ln -s /data/permittedlist.txt /root/.config/unity3d/IronGate/Valheim/permittedlist.txt
ln -s /data/prefs /root/.config/unity3d/IronGate/Valheim/prefs
ln -s /data/worlds /root/.config/unity3d/IronGate/Valheim/worlds

# Tip: Make a local copy of this script to avoid it being overwritten by steam.
# NOTE: Minimum password length is 5 characters & Password cant be in the server name.
# NOTE: You need to make sure the ports 2456-2458 is being forwarded to your server through your local router & firewall.
/data/valheim_server.x86_64 -name "Hackersvanguard" -port 2456 -world "Dedicated" -password "CHANGEME" -public 1

Finally we can just reuse the steamcmd container to run the new startup script and launch the server.

docker run -it -v ${PWD}/valheim-server:/data -p 2456:2456/udp -p 2457:2457/udp steamcmd/steamcmd:latest bash /data/start_server.sh

Running Valheim Dedicated Server On Kubernetes

To build on the ideas and method laid out in the docker section. Instead of running the docker container locally, we can create a quick deployment and service file to run on Kubernetes instead.

To start we can create simple app deployment yaml file with a volume which contains our server data and modified start script from the docker sections. Here I use a simple hostPath volume with a node selector, but a PVC would work all the same.

Very Important: If you don’t capture or link your world saves to your data volume, your world could be lost because containers are ephemeral.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: valheim-deployment
  name: valheim-deployment
  namespace: valheim
spec:
  replicas: 1
  selector:
    matchLabels:
      app: valheim-deployment-pod
  template:
    metadata:
      labels:
        app: valheim-deployment-pod
    spec:
      containers:
      - image: steamcmd/steamcmd:latest
        name: valheim-server
        ports:
        - containerPort: 2456
          protocol: UDP
        - containerPort: 2457
          protocol: UDP
        command: ["sh /data/start_server.sh"]
        volumeMounts:
        - name: valheim-data
          mountPath: /data
        lifecycle:
          preStop:
            exec:
              command: [" echo","1",">","/data/server_exit.drp"]
      volumes:
        - name: valheim-data
          hostPath:
            path: /opt/valheim-data
            type: Directory
      nodeSelector:
        kubernetes.io/hostname: kubenode1

Note: Make sure you have a copy of your server files and/or world data in your hostPath, on the node targeted by the NodeSelector. In the example I have a folder of “/opt/valheim-data” on a node with hostname kubenode1.

Next we can just create the nodPort service for the deployment so that we can portforward directly to our Valheim server.

Note: In this example we are using nodePort 32456 and and 32457. Therefore you would need to port forward to the nodes IP address and the nodePorts, not the default dedicated server ports.

apiVersion: v1
kind: Service
metadata:
  labels:
    app: valheim-deployment-svc
  name: valheim-deployment-svc
  namespace: valheim
spec:
  ports:
  - name: port-1
    nodePort: 32456
    port: 2456
    protocol: UDP
    targetPort: 2456
  - name: port-2
    nodePort: 32457
    port: 2457
    protocol: UDP
    targetPort: 2457
  selector:
    app: valheim-deployment-pod
  type: NodePort

Now all we have to do is made the resources within Kubernetes using kubectl.

kubectl create -f valheim-deployment.yaml
kubectl create -f valheim-service.yaml

Valhiem Server Access Lists

The Valhiem Sever also maintains a set of access list files to control what role users have on the server. These files can be found one directory up from your world saves, within the Valheim base directory. The files are all structured with one player ID per line. The Player ID can be found within the F2 server status menu, next to each players name.

  • adminlist.txt – list of server admin who can issue sever commands
  • bannedlist.txt – list of users who are banned form the server
  • permittedlist.txt – list of users who are allowed to join the sever when not set to public

Valhiem Server Basic Commands

Here are the basic console commands (opened by pressing F5) used to administer a Valhiem dedicated server.

  • help – Show all available commands.
  • kick [name/ip/userID] – Kick the user.
  • ban [name/ip/userID] – Ban the user.
  • unban [ip/userID] – Unban the user.
  • banned – Shows a list of banned users.
  • ping – Send a ping to the server to get your latency.
  • info – Print system info

If you run into issues and need to spawn in items or would rather play in a pseudo-creative mode. You can type “imacheater” in the console to get access to a full suite standard admin console commands.

Skal!

I wanted to get this information out as quickly as possible to help those who may be struggling. Let me know any questions, comments, or feedback on any of the socials @sleventyeleven.

CKA Exam Review, Tips, and Resources

CKA Exam Review Logo

CKA Exam Overview

The Certified Kubernetes Administrator (CKA) exam is the premiere certification to demonstrate a candidates skills with Kubernetes. The exam consists of a set of 15-20 multi-part real world tasks, which must be completed within 2 hours. The exam takes place in a live, multi-cluster Kubernetes environment. Candidates are only given command line access to the exam environment and are expected to have the efficiency to complete the majority of the tasks. Based on my CKA Exam review and experience, I believe this is one of the most rigorous exams I have encountered yet.

CKA Exam Review

The Linux Foundation (LF) and The Cloud Native Computing Foundation (CNCF) really brought their “A” game, when partnering to develop this exam. Not only is Kubernetes as a technology not very old, its still very much not mainstream. Nonetheless the packaged course work and exam itself provides a robust set of base knowledge that can probably lay the foundation for the start of a career at a modern company or startup. Or if your current company is trying to harness the cost saving power of moving to containerization in the cloud, this exam is also a good starting place.

Preparing for the CKA Exam

When it comes to preparing for the exam, you need to build a strong foundational knowledge through training and/or hands on experience. Part of the rigor of this exam is the sear fact that you will basically be typing non-stop for 2 hours straight in order to successfully complete the exam. If your not very comfortable completing tasks with kubectl, your not likely to pass the exam. When you believe your ready, I’d recommend running through some more practice questions. Just to build up your comfort and efficiency with kubectl commands.

I can’t stress enough, how much pressure is felt during these multi-part tasks as the clock ticks away. You will want to be able to complete the majority of the tasks quick enough to have time to check your work and than work through a few of the task you aren’t as sure about. Use your time wisely and use the CKA exam review feature that’s built into the exam platform. It allows you to mark items for review and add notes, so you can easily come back later.

Get used to the structure of the official kubernetes.io documentation and kubectl command line help. Even though there are tons of resources and communities for Kubernetes administration. These other resources are not available during the exam. So during exam perpetration, only use the official documentation. Limiting yourself, will only help improve familiarity and comfort as you work through more challenging exam tasks.

CKA Exam Resources

  • Kubernetes the hard way – This is a great step by step guide to, setup Kubernetes manually. Highly recommended you work through a manual setup of Kubernetes from scratch at least once. There will likely be exam tasks around installing or fixing base services.
  • The Linux Foundations Official Course – This is the most robust general knowledge based course I’ve seen. If you want to learn Kubernetes and how to do almost anything with it, get the CKA + CKAD combo package.
  • CKA with Practice Tests – The top rated Udemy course for CKA. I highly recommend this course for those who may have some work experience with Kubernetes and want a course that teaches what is directly related to the exam. Its also worth while to pick up this course for the practice tests and exercises alone.
  • CKA and CKAD Instructions – Review the exam instructions and information multiple times, before sitting for the exam. Not only do the exam instructions provide guidance as to what is required during the exam. But it also provides important information about about the exam environment and tips for using the exam platform.
  • Kubernetes Documentation Tasks – These are the administration task guides as laid out by the Kubernetes developers. Not only should you be comfortable completing the majority of these tasks, but these task guides could prove to be a valuable resource during the exam.
  • CKAD Exercises – Although not all exercises directly relate to the CKA exam, they offer a wide range of items to build comfort with the Kubernetes command line tools. What I like about these exercises is each represents an item that may be contained within an exam task. If your unsure, it provides the most direct way to complete the prompt in a hidden field.

CKA Exam Tips and Tricks

  1. Try to do everything with kubectl in order to increase your speed and accuracy. The majority (70-80%) of the exam should be completable with kubectl commands alone.
  2. ALWAYS run the context command at the top of every task, before completing any of the work. If your ever not sure, just run it again. A correct solution to an exam task, completed in the wrong context, likely wont be scored.
  3. The new exam interface is optimized to copy and paste custom strings, such as names, labels, metadata, etc. Utilize the feature heavily in order to avoid type-o’s during the exam.
  4. Read the task fully before beginning work. Things may need to be completed in a certain order or on certain nodes in order to fully complete the question.
  5. Above all priorities your time wisely. You are aloud to utilize the Kubernetes online and system documentation during the exam. But if you don’t know how to efficiently complete the prompt once you have fully read the task, then move on and come back to the task later.