Kubernetes Harvester to Gather Credentials with Limited Access

Project URL: https://github.com/sleventyeleven/Kubernetes-Harvester

Kubernetes Harvester Example Run

What is Kubernetes Harvester?

Harvester is a new python based project that attempts to leverage access in order to gather potentially sensitive information. Its designed to either leverage the access of users credentials or the default access granted to a pod via automountServiceAccountToken, which I wrote about recently. The harvester.py script currently primarily targets pod container environment variables, container manifest environment variables, and config map entries utilized as environment variables, to look for potential credentials.

Why Create A Credential Harvester

The default admission controls in many of the Kubernetes implementation apply a read/view policy to newly created users. However custom policies, admissions, and operators have become more common place. What’s more troublesome is the read permissions given to the automountServiceAccountToken by default. Without adjusting or disabling service tokens, compromised containers could effectively read all pod specs in all namespaces. With access to all pod specs, an attacker could potentially gather credentials or other sensitive information. Harvester is a tool that attempts to help automate the review process.

How Kubernetes Harvester Works

The harvester.py script utilizes the automountServiceAccountToken mounted within a given container or the standard user credentials within the Kube config file (~/.kube/config). Then the Kube API server is queried to look for sensitive information within the pod spec of each pod in the following steps.

  1. Use access to request pod specs for all namespaces within the cluster.
  2. Parse all pod specs to map and dedupe container container information
  3. Review each containers environment variables for sensitive values
  4. Review each config map entry, mapped to container environment variables for sensitive values
  5. Attempt to pull each container image and review the manifest environment variables for sensitive values
  6. Attempt to request authentication tokens from the internal metadata API for each of the major cloud provider

Other Resources:

  • Introduction to Kubernetes – A Free introduction course diving into Kubernetes as a tool for containerized infrastructure. Its a a great place to begin if your just getting started with Kubernetes.
  • The Linux Foundations Official Course – This is the most robust general knowledge based course I’ve seen. If you want to learn Kubernetes and how to do almost anything with it, get the CKA + CKAD combo package.

Valheim Dedicated Server on Kubernetes

What is Valhiem?

Valheim official Graphic

Valheim is a brand new early access games that just hit Steam. It’s a brutal exploration, survival, and crafting game inspired by Viking culture. The game world is generated based on a random or provided seed value and allows up to 10 people to play together by default.

Out of the box, the game also does support sharing your server over steam cloud, so you don’t actually need a dedicated server or forwarding to play with your friends. However, forcing one person to leave their game open and active to share the world, can caused its challenges.

So after going through the trial and tribulations from the Norse gods. Here is what I’ve learned about how to run a Valhiem Dedicated Server on Linux, Docker, and Kubernetes. I will try my best to provide updates to this post for as long as I’m able and playing the game.

Where are the world saves?

Since the world is entirely procedurally-generated, each world is saved every 30 minutes to the following folders.

  • Windows -> C:\Users\<user>\AppData\LocalLow\IronGate\Valheim\worlds
  • Linux -> ~/.config/unity3d/IronGate/Valheim/worlds

If you want to share your game saves, these files should be portable. If you want to move an existing save to a dedicated server, just move the world files over to your server and make sure the world argument is set to the same as the world file you are targeting when you start the server (example: -world “myworld” correlates to myworld.db and myworld.fwl).

A Quick note on Port Forwarding

You will need to Port Forward on your router and/or firewall for your dedicated server to work! Due to the complexity and diversity involved in port forwarding, I’m not going to include direction for it in this guide. By default these ports are 2456/UDP and 2457/UDP.

How to Run Valheim Dedicated Server Linux

Start by setting up steamcmd and downloading the game files (based on Ubuntu/Debian based systems, see official docs for other distros).

useradd -m steam # create steam user for security and isolation
cd /home/steam # move to the home directory to keep files clean
sudo apt install steamcmd # install the thing

Next we can download the game files with steamcmd.

steamcmd +login anonymous +force_install_dir ./valheim +app_update 896660 +quit

Then we can modify the server start script (start_server.sh) that comes with default server files. Changing the server name and password are a definite must, but changing the port or world name (reference to a world save) are not required. Then simply run the script to start the server.

bash ./valheim/start_server.sh

Running Valheim Dedicated Server with Docker

Without getting too far into the weeds on the details behind docker container, we can get a server up with two fairly simply commands. First we just need to download the app data to a local folder.

mkdir ${PWD}/valheim-server # make the directory if its not there
docker run -it -v ${PWD}/valheim-server:/data steamcmd/steamcmd:latest +login anonymous +force_install_dir /data +app_update 896660 +quit

Now that we have the server files, we need to modify the start script severname and password as before. But we also need to restructure the file, because the world will be saved to the ~/.config and not the /data volume we mounted.

Very Important: If you don’t capture or link your world saves to your data volume, your world could be lost because containers are ephemeral.

To make up for the lack of control of where the world data is being stored, we can just utilize symbolic links to redirect the ~/.config files to our /data volume. To do this we can use a script like the following.

export templdpath=$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/data/linux64:$LD_LIBRARY_PATH
export SteamAppId=892970

mkdir -p /root/.config/unity3d/IronGate/Valheim
ln -s /data/adminlist.txt /root/.config/unity3d/IronGate/Valheim/adminlist.txt
ln -s /data/bannedlist.txt /root/.config/unity3d/IronGate/Valheim/bannedlist.txt
ln -s /data/permittedlist.txt /root/.config/unity3d/IronGate/Valheim/permittedlist.txt
ln -s /data/prefs /root/.config/unity3d/IronGate/Valheim/prefs
ln -s /data/worlds /root/.config/unity3d/IronGate/Valheim/worlds

# Tip: Make a local copy of this script to avoid it being overwritten by steam.
# NOTE: Minimum password length is 5 characters & Password cant be in the server name.
# NOTE: You need to make sure the ports 2456-2458 is being forwarded to your server through your local router & firewall.
/data/valheim_server.x86_64 -name "Hackersvanguard" -port 2456 -world "Dedicated" -password "CHANGEME" -public 1

Finally we can just reuse the steamcmd container to run the new startup script and launch the server.

docker run -it -v ${PWD}/valheim-server:/data -p 2456:2456/udp -p 2457:2457/udp steamcmd/steamcmd:latest bash /data/start_server.sh

Running Valheim Dedicated Server On Kubernetes

To build on the ideas and method laid out in the docker section. Instead of running the docker container locally, we can create a quick deployment and service file to run on Kubernetes instead.

To start we can create simple app deployment yaml file with a volume which contains our server data and modified start script from the docker sections. Here I use a simple hostPath volume with a node selector, but a PVC would work all the same.

Very Important: If you don’t capture or link your world saves to your data volume, your world could be lost because containers are ephemeral.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: valheim-deployment
  name: valheim-deployment
  namespace: valheim
spec:
  replicas: 1
  selector:
    matchLabels:
      app: valheim-deployment-pod
  template:
    metadata:
      labels:
        app: valheim-deployment-pod
    spec:
      containers:
      - image: steamcmd/steamcmd:latest
        name: valheim-server
        ports:
        - containerPort: 2456
          protocol: UDP
        - containerPort: 2457
          protocol: UDP
        command: ["sh /data/start_server.sh"]
        volumeMounts:
        - name: valheim-data
          mountPath: /data
        lifecycle:
          preStop:
            exec:
              command: [" echo","1",">","/data/server_exit.drp"]
      volumes:
        - name: valheim-data
          hostPath:
            path: /opt/valheim-data
            type: Directory
      nodeSelector:
        kubernetes.io/hostname: kubenode1

Note: Make sure you have a copy of your server files and/or world data in your hostPath, on the node targeted by the NodeSelector. In the example I have a folder of “/opt/valheim-data” on a node with hostname kubenode1.

Next we can just create the nodPort service for the deployment so that we can portforward directly to our Valheim server.

Note: In this example we are using nodePort 32456 and and 32457. Therefore you would need to port forward to the nodes IP address and the nodePorts, not the default dedicated server ports.

apiVersion: v1
kind: Service
metadata:
  labels:
    app: valheim-deployment-svc
  name: valheim-deployment-svc
  namespace: valheim
spec:
  ports:
  - name: port-1
    nodePort: 32456
    port: 2456
    protocol: UDP
    targetPort: 2456
  - name: port-2
    nodePort: 32457
    port: 2457
    protocol: UDP
    targetPort: 2457
  selector:
    app: valheim-deployment-pod
  type: NodePort

Now all we have to do is made the resources within Kubernetes using kubectl.

kubectl create -f valheim-deployment.yaml
kubectl create -f valheim-service.yaml

Valhiem Server Access Lists

The Valhiem Sever also maintains a set of access list files to control what role users have on the server. These files can be found one directory up from your world saves, within the Valheim base directory. The files are all structured with one player ID per line. The Player ID can be found within the F2 server status menu, next to each players name.

  • adminlist.txt – list of server admin who can issue sever commands
  • bannedlist.txt – list of users who are banned form the server
  • permittedlist.txt – list of users who are allowed to join the sever when not set to public

Valhiem Server Basic Commands

Here are the basic console commands (opened by pressing F5) used to administer a Valhiem dedicated server.

  • help – Show all available commands.
  • kick [name/ip/userID] – Kick the user.
  • ban [name/ip/userID] – Ban the user.
  • unban [ip/userID] – Unban the user.
  • banned – Shows a list of banned users.
  • ping – Send a ping to the server to get your latency.
  • info – Print system info

If you run into issues and need to spawn in items or would rather play in a pseudo-creative mode. You can type “imacheater” in the console to get access to a full suite standard admin console commands.

Skal!

I wanted to get this information out as quickly as possible to help those who may be struggling. Let me know any questions, comments, or feedback on any of the socials @sleventyeleven.

LFCA Exam, Resources, and Training

Linux Foundation Certified IT Associate (LFCA)

Exam Overview

Since I announced I was part of the team of individuals who helped develop the new Linux Foundation Certified IT Associate (LFCA) exam. I have been bombarded with questions. The majority of these questions I simply will not answer. The Linux Foundation maintains a separation between exam developers and trainers to protect the integrity of the certification.

However, many have asked questions about where to find materials to prepare for the certification, since a specific training course wasn’t released along side the exam. To those questions, I would mention that the Linux Foundation offers free introduction courses that are linked right on the exam page. These same courses have topics that cover the vast majority of the listed exam domain subjects.

Nonetheless exam its self is 60 multiple choice questions with an exam time of 90 minutes. Its also proctored virtually by PSI, alongside all of the other Linux Foundation exams. To support those taking the exam, I pulled together the exam domains, voucher, and handbook links to provide them bellow in a single place. I also provided a list of the free courses listed on the LFCA training page. Additionally, I re-ordered the courses based on pervious experience with the training materials and how the topics listed in the courses map to the exam domain subjects.

LFCA Exam Domains

The following is the full list of the exam domains and subjects covered directly from the certification documentations.

  1. Linux Fundamentals – 20%
    1. Linux Operating System
    2. File Management Commands
    3. System Commands
    4. General Networking Commands
  2. System Administration Fundamentals – 20%
    1. System Administration Tasks
    2. Networking
    3. Troubleshooting
  3. Cloud Computing Fundamentals – 20%
    1. Cloud Computing Fundamentals
    2. Performance / Availability
    3. Serverless
    4. Cloud Costs and Budgeting
  4. Security Fundamentals – 16%
    1. Security Basics
    2. Data Security
    3. Network Security
    4. System Security
  5. DevOps Fundamentals – 16%
    1. DevOps Basics
    2. Containers
    3. Deployment Environments
    4. Git Concepts
  6. Supporting Applications and Developers – 8%
    1. Software Project Management
    2. Software Application Architecture
    3. Functional Analysis
    4. Open-source Software and Licensing

Free Training from the Linux Foundation

These core courses offer roughly 120 hours of free material that relate directly to the exam domains.

  • Introduction to Linux – An introduction course to help build up the foundational Linux, system administration, and security knowledge listed in the core exam domains.
  • Basics of Cloud Computing – An introduction course that covers cloud infrastructure and the technologies that drive delivery. This course relates to the Cloud Computing Fundamentals exam domain.
  • DevOps Fundamentals – An introduction course to the principles and practices of development operations (DevOps). This course relates directly to the DevOps fundamentals exam domain.

These additional recommended courses that relate to one or more exam domains and provide additional detail.

  • Introduction to Kubernetes – An more in-depth dive into Kubernetes as a tool for containerized infrastructure. Highly recommended for those looking to break into the cloud space and/or purpose the CKA Exam.
  • Open Source Licensing – Open source software is now everywhere and the licensing can be very confusing at first. This course offers a clear and concise coverage of licensing, for those who may not encounter it often.
  • Beginners Guide to Software Development – This next course provides a basic introduction into the key concepts for open source software development. This course will give those who don’t develop software often, just enough to be dangerous.

LFCA Exam and Resources

  • LFCA Exam Voucher – This is the official Linux Foundation Certified IT Associate (LFCA) training page to purchase the exam voucher. This includes a retake if you don’t pass the exam on the first attempt.
  • LFCA Handbook – Exam specific handbooks are provided for all Linux Foundation exams and LFCA is no exception. Reading through the handbook will answer common questions regarding the exam, provide an introduction to the exam environment, and help calm some of the pre-exam nerves.