Kubernetes Security 101: Fundamentals and Best Practices
SHARE:
Securing Kubernetes may seem like a mystifying task. As a highly complex system composed of an array of different components, Kubernetes is not something you can secure by simply enabling a security module or installing a security tool.
Instead, Kubernetes security requires teams to address each type of security risk that may impact the various layers and services within a Kubernetes cluster. For example, teams must understand how to secure Kubernetes nodes, networks, pods, data, and so on.
In addition, Kubernetes admins need to know which tools Kubernetes offers natively to address security concerns, and which types of third-party security tools they’ll need to integrate with their clusters to fill in the gaps. This is also a complex topic because, although Kubernetes isn’t a security platform, it does provide certain types of native security tooling, such as Role-Based Access Control (RBAC).
All of the above can feel overwhelming if you’re new to Kubernetes and still trying to wrap your head around how the whole thing works, let alone how to keep it secure. But the concepts are actually simple enough if you break them into digestible pieces. Toward that end, this article walks through the various facets of Kubernetes security and explains the fundamentals of each one, as well as best practices for Kubernetes security at every layer and service level.
Kubernetes Security 101
What you’ll learn
-
What native Kubernetes security tools are available and how to complement them
-
Securing Kubernetes - from node to pod
-
How to ensure audit logs and namespaces are safe
Kubernetes Security Piece-By-Piece
Perhaps the simplest way to approach Kubernetes security is to think about the types of risks that impact each part of the Kubernetes stack, then identify the tools and resources available to help secure them.
Node Security
Nodes are the servers that comprise Kubernetes clusters. In most cases, nodes run some version of Linux, although worker nodes may run Windows. Nodes can be virtual machines or bare-metal servers, but the difference doesn’t really matter from a security perspective.
You should adopt the same security strategies for securing Kubernetes nodes that you would use to secure any type of server. These include:
- Removing extraneous applications, libraries, and other components of the operating system in order to minimize your attack surface. Provisioning nodes with minimalist Linux distributions, such as Alpine Linux, is a best practice.
- Eliminating unnecessary user accounts.
- Ensuring that nothing runs as root unless strictly necessary.
- Where available, deploying OS-hardening frameworks, like AppArmor or SELinux.
- Collecting and analyzing OS logs to detect possible breaches.
If you have experience securing servers at the operating system level in any type of environment, then, you likely already know how to handle Kubernetes node security. At the node level, security considerations aren’t really any different when you’re dealing with nodes running Kubernetes than they are for any type of server.
There are also no fundamental differences between securing a master node and securing a worker node. Master node security is somewhat more important because a breach on a master node could cause more damage to your cluster, but the procedures for securing the operating system on a master node are the same as for a worker node.
Kubernetes API Security
The Kubernetes API is what binds the various pieces of a cluster together. As such, it’s one of the most important resources in Kubernetes to secure.
The Kubernetes API is designed to be secure by default. It will only respond to requests that it can properly authenticate and authorize.
That said, API authentication and authorization are governed by RBAC policies that you configure. Thus, the API is only as secure as your RBAC policies. Creating secure RBAC policies that enforce the principle of least privilege and assign permissions on a granular basis is thus a basic best practice for ensuring Kubernetes API security.
In addition, you can further enhance API security by taking advantage of admission controllers. Admission controllers evaluate requests after the API server has already authenticated and authorized them. In this way, admission controllers provide an optional secondary layer of defense against requests that should not be allowed. By enabling and configuring admission controllers, you can enforce various security rules related to API requests. The available rules are documented here.
Finally, API requests can be secured at the network level by configuring secure certificates and requiring the API server to serve requests on a secure port rather than on localhost.
Kubernetes Network Security
Kubernetes network security is similar to pod security in that it starts with following the best practices that you would use to secure any network.
You should make sure that, to the extent possible, you create a network architecture that isolates workloads from the public Internet unless they need to interface with it. You should deploy firewalls at the gateway level to block traffic from offending hosts. You should monitor network traffic for signs of a breach. These are all steps that you can take using tools that are external to Kubernetes, such as a service mesh.
However, Kubernetes also offers a limited amount of native tooling that can help secure networking resources. This tooling comes in the form of network policies. While network policies aren’t a security feature per se, admins can use them to control how traffic flows within a Kubernetes cluster.
Thus, you can create network policies to do things like isolate pods from each other at the network level or filter incoming traffic.
Network policies aren’t substitute for securing networking configurations outside of Kubernetes; instead, think of them as an additional resource that complements the security rules that you build into your overall network architecture.
Kubernetes Pod Security
In Kubernetes, a pod is a container or set of containers used to run an application. In order to secure your applications, then, you need to secure your pods.
Some aspects of pod security require practices that are external to Kubernetes. You should perform security tests on your application prior to deployment and scan container images before you run them. You should collect logs from pods and analyze them to detect potential breaches or abuse.
However, Kubernetes provides some native tools that can harden pod security once pods are already running. These include:
- RBAC policies, which can be used to manage access to pods by users and services within the cluster.
- Security contexts, which define the privilege level at which pods run.
- Network policies, which (as noted above) you can use to isolate pods at the network level.
- Admission controllers, which can enforce additional rules that essentially extend the rules you establish based on RBAC.
The types of pod security tools that you use and the way you configure them will depend on the nature of your workloads. There is no one-size-fits-all approach to pod security. Some pods can be entirely isolated from each other at the network level, for example, while others need to be able to communicate.
Whatever your specific requirements, however, you should evaluate the resources available to secure Kubernetes pods and make sure that you use them to their full extent.
Kubernetes Data Security
Kubernetes doesn’t store any data, apart from the non-persistent data that lives inside running pods and log data stored on nodes. Typically, the data that your clusters create and/or access will live in some kind of external storage system that interfaces with Kubernetes through a storage plugin.
To secure data associated with Kubernetes, then, you should follow the best practices that you would use to secure data inside any large-scale storage system. Encrypt data at rest wherever possible. Use access control tools to restrict who can access data. Ensure that the servers that manage your storage pools are properly locked down. Back data up to help protect yourself against data theft or ransomware attacks.
As for the relatively small amounts of data that live natively inside Kubernetes pods and nodes, Kubernetes does not offer any special tools for securing that data. However, you can protect it by protecting your pods and nodes using the best practices outlined above.
Additional Kubernetes Security Resources
Beyond the component-specific security practices described above, admins should know about additional security resources for Kubernetes.
Kubernetes can optionally keep granular records of which actions were performed in a cluster, who performed them, and what the results were. Using these audit logs, you can comprehensively audit your clusters to detect potential security issues in real time as well as research security incidents after the fact.
To use audit logs, you must first create an audit policy, which defines how Kubernetes will record events. The Kubernetes documentation includes full details on establishing audit policies.
In addition, because Kubernetes doesn’t provide tools to help you analyze audit logs at scale, you’ll likely want to stream audit logs to an external monitoring or observability platform that will help you detect anomalies and alert you to breaches. Otherwise, you can only monitor audit events manually, which is not practical at scale.
Namespaces
In Kubernetes, namespaces can be used to isolate different workloads from each other.
While you can run everything inside a single namespace if you wish, it’s a best practice from a security standpoint to create different namespaces for each team and/or type of workload in your cluster. You may want to separate your dev/test environment from production using separate namespaces, for example.
Managing multiple namespaces increases the administrative complexity of Kubernetes to a certain extent because you’ll need to create different RBAC policies for each namespace in many (but not all) cases. However, the extra effort is well worth it because it minimizes the potential impact of a breach.
Using External Security Tools with Kubernetes
Although Kubernetes provides certain types of tools to help harden resources that are running within your cluster, Kubernetes is not designed to help you detect or manage security incidents.
To manage Kubernetes security at scale, you’ll most likely need to leverage external security tools. These tools can perform several important security functions, including:
- Scanning your RBAC policies, security contexts, and other configuration data to identify misconfigurations that could create security issues.
- Provide application and container image scanning functionality, which you can use to build an automated security pipeline that feeds into your Kubernetes clusters.
- Collect, aggregate, and analyze application logs and audit logs to help you detect anomalies that may signal a breach.
There are a variety of external security tools for Kubernetes out there – including, of course, Sysdig, which was purpose-built to help DevOps teams secure all layers of Kubernetes and other cloud-native environments.