Category Archives: DevOps

Kubernetes Harvester to Gather Credentials with Limited Access

Posted on by

Project URL: https://github.com/sleventyeleven/Kubernetes-Harvester

Kubernetes Harvester Example Run

What is Kubernetes Harvester?

Harvester is a new python based project that attempts to leverage access in order to gather potentially sensitive information. Its designed to either leverage the access of users credentials or the default access granted to a pod via automountServiceAccountToken, which I wrote about recently. The harvester.py script currently primarily targets pod container environment variables, container manifest environment variables, and config map entries utilized as environment variables, to look for potential credentials.

Why Create A Credential Harvester

The default admission controls in many of the Kubernetes implementation apply a read/view policy to newly created users. However custom policies, admissions, and operators have become more common place. What’s more troublesome is the read permissions given to the automountServiceAccountToken by default. Without adjusting or disabling service tokens, compromised containers could effectively read all pod specs in all namespaces. With access to all pod specs, an attacker could potentially gather credentials or other sensitive information. Harvester is a tool that attempts to help automate the review process.

How Kubernetes Harvester Works

The harvester.py script utilizes the automountServiceAccountToken mounted within a given container or the standard user credentials within the Kube config file (~/.kube/config). Then the Kube API server is queried to look for sensitive information within the pod spec of each pod in the following steps.

  1. Use access to request pod specs for all namespaces within the cluster.
  2. Parse all pod specs to map and dedupe container container information
  3. Review each containers environment variables for sensitive values
  4. Review each config map entry, mapped to container environment variables for sensitive values
  5. Attempt to pull each container image and review the manifest environment variables for sensitive values
  6. Attempt to request authentication tokens from the internal metadata API for each of the major cloud provider

Other Resources:

  • Introduction to Kubernetes – A Free introduction course diving into Kubernetes as a tool for containerized infrastructure. Its a a great place to begin if your just getting started with Kubernetes.
  • The Linux Foundations Official Course – This is the most robust general knowledge based course I’ve seen. If you want to learn Kubernetes and how to do almost anything with it, get the CKA + CKAD combo package.

LFCA Exam, Resources, and Training

Posted on by
Linux Foundation Certified IT Associate (LFCA)

Exam Overview

Since I announced I was part of the team of individuals who helped develop the new Linux Foundation Certified IT Associate (LFCA) exam. I have been bombarded with questions. The majority of these questions I simply will not answer. The Linux Foundation maintains a separation between exam developers and trainers to protect the integrity of the certification.

However, many have asked questions about where to find materials to prepare for the certification, since a specific training course wasn’t released along side the exam. To those questions, I would mention that the Linux Foundation offers free introduction courses that are linked right on the exam page. These same courses have topics that cover the vast majority of the listed exam domain subjects.

Nonetheless exam its self is 60 multiple choice questions with an exam time of 90 minutes. Its also proctored virtually by PSI, alongside all of the other Linux Foundation exams. To support those taking the exam, I pulled together the exam domains, voucher, and handbook links to provide them bellow in a single place. I also provided a list of the free courses listed on the LFCA training page. Additionally, I re-ordered the courses based on pervious experience with the training materials and how the topics listed in the courses map to the exam domain subjects.

LFCA Exam Domains

The following is the full list of the exam domains and subjects covered directly from the certification documentations.

  1. Linux Fundamentals – 20%
    1. Linux Operating System
    2. File Management Commands
    3. System Commands
    4. General Networking Commands
  2. System Administration Fundamentals – 20%
    1. System Administration Tasks
    2. Networking
    3. Troubleshooting
  3. Cloud Computing Fundamentals – 20%
    1. Cloud Computing Fundamentals
    2. Performance / Availability
    3. Serverless
    4. Cloud Costs and Budgeting
  4. Security Fundamentals – 16%
    1. Security Basics
    2. Data Security
    3. Network Security
    4. System Security
  5. DevOps Fundamentals – 16%
    1. DevOps Basics
    2. Containers
    3. Deployment Environments
    4. Git Concepts
  6. Supporting Applications and Developers – 8%
    1. Software Project Management
    2. Software Application Architecture
    3. Functional Analysis
    4. Open-source Software and Licensing

Free Training from the Linux Foundation

These core courses offer roughly 120 hours of free material that relate directly to the exam domains.

  • Introduction to Linux – An introduction course to help build up the foundational Linux, system administration, and security knowledge listed in the core exam domains.
  • Basics of Cloud Computing – An introduction course that covers cloud infrastructure and the technologies that drive delivery. This course relates to the Cloud Computing Fundamentals exam domain.
  • DevOps Fundamentals – An introduction course to the principles and practices of development operations (DevOps). This course relates directly to the DevOps fundamentals exam domain.

These additional recommended courses that relate to one or more exam domains and provide additional detail.

  • Introduction to Kubernetes – An more in-depth dive into Kubernetes as a tool for containerized infrastructure. Highly recommended for those looking to break into the cloud space and/or purpose the CKA Exam.
  • Open Source Licensing – Open source software is now everywhere and the licensing can be very confusing at first. This course offers a clear and concise coverage of licensing, for those who may not encounter it often.
  • Beginners Guide to Software Development – This next course provides a basic introduction into the key concepts for open source software development. This course will give those who don’t develop software often, just enough to be dangerous.

LFCA Exam and Resources

  • LFCA Exam Voucher – This is the official Linux Foundation Certified IT Associate (LFCA) training page to purchase the exam voucher. This includes a retake if you don’t pass the exam on the first attempt.
  • LFCA Handbook – Exam specific handbooks are provided for all Linux Foundation exams and LFCA is no exception. Reading through the handbook will answer common questions regarding the exam, provide an introduction to the exam environment, and help calm some of the pre-exam nerves.

CKA Exam Review, Tips, and Resources

Posted on by
CKA Exam Review Logo

CKA Exam Overview

The Certified Kubernetes Administrator (CKA) exam is the premiere certification to demonstrate a candidates skills with Kubernetes. The exam consists of a set of 15-20 multi-part real world tasks, which must be completed within 2 hours. The exam takes place in a live, multi-cluster Kubernetes environment. Candidates are only given command line access to the exam environment and are expected to have the efficiency to complete the majority of the tasks. Based on my CKA Exam review and experience, I believe this is one of the most rigorous exams I have encountered yet.

CKA Exam Review

The Linux Foundation (LF) and The Cloud Native Computing Foundation (CNCF) really brought their “A” game, when partnering to develop this exam. Not only is Kubernetes as a technology not very old, its still very much not mainstream. Nonetheless the packaged course work and exam itself provides a robust set of base knowledge that can probably lay the foundation for the start of a career at a modern company or startup. Or if your current company is trying to harness the cost saving power of moving to containerization in the cloud, this exam is also a good starting place.

Preparing for the CKA Exam

When it comes to preparing for the exam, you need to build a strong foundational knowledge through training and/or hands on experience. Part of the rigor of this exam is the sear fact that you will basically be typing non-stop for 2 hours straight in order to successfully complete the exam. If your not very comfortable completing tasks with kubectl, your not likely to pass the exam. When you believe your ready, I’d recommend running through some more practice questions. Just to build up your comfort and efficiency with kubectl commands.

I can’t stress enough, how much pressure is felt during these multi-part tasks as the clock ticks away. You will want to be able to complete the majority of the tasks quick enough to have time to check your work and than work through a few of the task you aren’t as sure about. Use your time wisely and use the CKA exam review feature that’s built into the exam platform. It allows you to mark items for review and add notes, so you can easily come back later.

Get used to the structure of the official kubernetes.io documentation and kubectl command line help. Even though there are tons of resources and communities for Kubernetes administration. These other resources are not available during the exam. So during exam perpetration, only use the official documentation. Limiting yourself, will only help improve familiarity and comfort as you work through more challenging exam tasks.

CKA Exam Resources

  • Kubernetes the hard way – This is a great step by step guide to, setup Kubernetes manually. Highly recommended you work through a manual setup of Kubernetes from scratch at least once. There will likely be exam tasks around installing or fixing base services.
  • The Linux Foundations Official Course – This is the most robust general knowledge based course I’ve seen. If you want to learn Kubernetes and how to do almost anything with it, get the CKA + CKAD combo package.
  • CKA with Practice Tests – The top rated Udemy course for CKA. I highly recommend this course for those who may have some work experience with Kubernetes and want a course that teaches what is directly related to the exam. Its also worth while to pick up this course for the practice tests and exercises alone.
  • CKA and CKAD Instructions – Review the exam instructions and information multiple times, before sitting for the exam. Not only do the exam instructions provide guidance as to what is required during the exam. But it also provides important information about about the exam environment and tips for using the exam platform.
  • Kubernetes Documentation Tasks – These are the administration task guides as laid out by the Kubernetes developers. Not only should you be comfortable completing the majority of these tasks, but these task guides could prove to be a valuable resource during the exam.
  • CKAD Exercises – Although not all exercises directly relate to the CKA exam, they offer a wide range of items to build comfort with the Kubernetes command line tools. What I like about these exercises is each represents an item that may be contained within an exam task. If your unsure, it provides the most direct way to complete the prompt in a hidden field.

CKA Exam Tips and Tricks

  1. Try to do everything with kubectl in order to increase your speed and accuracy. The majority (70-80%) of the exam should be completable with kubectl commands alone.
  2. ALWAYS run the context command at the top of every task, before completing any of the work. If your ever not sure, just run it again. A correct solution to an exam task, completed in the wrong context, likely wont be scored.
  3. The new exam interface is optimized to copy and paste custom strings, such as names, labels, metadata, etc. Utilize the feature heavily in order to avoid type-o’s during the exam.
  4. Read the task fully before beginning work. Things may need to be completed in a certain order or on certain nodes in order to fully complete the question.
  5. Above all priorities your time wisely. You are aloud to utilize the Kubernetes online and system documentation during the exam. But if you don’t know how to efficiently complete the prompt once you have fully read the task, then move on and come back to the task later.