Kubernetes Security

Role-based access control (RBAC) in Kubernetes governs who can perform which actions on which resources in the API. Roles define a set of allowed verbs (get, list, create, update, delete, watch, patch) on specific API groups and resource types, scoped to a single namespace. ClusterRoles are identical but apply cluster-wide. RoleBindings attach a Role to a user, group, or service account within a namespace. ClusterRoleBindings attach a ClusterRole at the cluster level.

The principle of least privilege applies strictly to RBAC. A developer who needs to read logs and exec into Pods in their team's namespace does not need list access to Secrets or create access to deployments in other namespaces. Well-designed RBAC maps to actual job functions, what a person or system genuinely needs to do their work, rather than to org chart titles or convenience groupings.

Common RBAC mistakes include binding subjects to the cluster-admin ClusterRole (which grants unrestricted access to everything), using wildcard verbs ('*') or wildcard resources ('*') in Role definitions, and granting 'create' or 'update' on ClusterRoleBindings to workloads (which allows privilege escalation by creating new bindings). These mistakes are easy to make and difficult to detect without regular audits.

Auditing RBAC is possible with 'kubectl auth can-i --as=system:serviceaccount:namespace:name verb resource', which shows what a specific service account is allowed to do. Tools like rbac-lookup, rakkess, and kubectl-who-can make it easier to visualize what permissions exist and who has them, which is essential for periodic RBAC reviews in clusters that have grown over time.

Service accounts are the identity used by workloads to authenticate to the Kubernetes API. Every Pod runs under a service account, and by default that service account's token is automatically mounted into the Pod's filesystem at /var/run/secrets/kubernetes.io/serviceaccount. A container can use that token to authenticate to the API server and perform any action that the service account's RBAC permissions allow.

The default service account in each namespace has minimal permissions, but many operators grant broader permissions without realizing how widely those permissions are applied. If a service account with 'list secrets' permission is bound to the default service account in a namespace, every Pod in that namespace, including newly created ones that have nothing to do with the original use case, inherits that permission.

Workloads that do not need to call the Kubernetes API should disable token auto-mounting. This is done by setting automountServiceAccountToken: false either in the ServiceAccount spec (which applies to all Pods using that service account) or in individual Pod/Deployment specs. Disabling auto-mounting means a compromised container cannot use its token to probe the cluster's API.

For workloads that genuinely need to interact with cloud services, not the Kubernetes API but AWS, GCP, or Azure, the modern pattern is to use cloud provider identity integration rather than long-lived credentials. AWS IRSA (IAM Roles for Service Accounts) and GKE Workload Identity allow a Kubernetes service account to assume a cloud IAM role without storing cloud credentials in the cluster. This eliminates an entire class of credential exposure and enables fine-grained, per-workload cloud access control.

Without NetworkPolicies, all Pods in a Kubernetes cluster can communicate with all other Pods on any port. This flat network model is convenient for development but catastrophic for security in production, a compromised application can connect to any database, queue, or internal API in the cluster regardless of namespace or intended function. NetworkPolicies restrict this communication by specifying which Pods may send or receive traffic from which other Pods.

NetworkPolicies are enforced by the CNI (Container Network Interface) plugin, not by Kubernetes itself. Not all CNI plugins support NetworkPolicy enforcement. Common plugins that do include Calico, Cilium, and Antrea. If a cluster uses a CNI plugin that does not support NetworkPolicies (such as Flannel in its default configuration), NetworkPolicy objects can be created but will have no effect, a dangerous false sense of security.

The most effective pattern is default-deny. Create a NetworkPolicy that selects all Pods in a namespace and denies all ingress and all egress. Then add specific allow policies for the traffic that should be permitted. This inverts the default from 'everything is allowed unless explicitly denied' to 'everything is denied unless explicitly permitted.' A new service added to the namespace has no network access until a NetworkPolicy explicitly grants it, which prevents accidentally over-permissive configurations.

A concrete NetworkPolicy example, a frontend Pod should be able to receive HTTP traffic from the Ingress controller and send traffic to a backend API Pod. It should not be able to connect directly to the database (only the backend should). Without NetworkPolicies, all three can freely communicate. With targeted NetworkPolicies in place, a compromised frontend cannot reach the database directly. Reducing the blast radius of a frontend compromise significantly.

A security context defines operating system-level security settings for a Pod or a specific container. These settings are independent of the container image. The image defines what code runs, but the security context defines how that code is allowed to run. Key fields include runAsUser (the UID the container process runs as), runAsNonRoot (rejects the container if it would run as root), readOnlyRootFilesystem (prevents writing to the container filesystem), allowPrivilegeEscalation (prevents processes from gaining more privileges than the parent), and capabilities (add or drop specific Linux capabilities).

Pod Security Admission (PSA) is the built-in Kubernetes mechanism for enforcing security context requirements at namespace scope. It replaced the deprecated PodSecurityPolicy in Kubernetes 1.25. PSA defines three policy levels. Privileged (no restrictions), Baseline (prevents known privilege escalation), and Restricted (strongest hardening, requires non-root, read-only filesystem, dropped capabilities). Namespaces are labeled to specify which policy level applies and whether violations are enforced, audited, or warned.

The Restricted PSA level reflects what a well-hardened Pod looks like. Running as a non-root user, with an immutable root filesystem, no privilege escalation allowed, all capabilities dropped, and a seccomp profile applied. Many legitimate applications can meet these requirements with minimal changes, those that cannot should have a documented exception explaining why the relaxation is necessary and what compensating controls are in place.

A common mistake is setting privileged: true as a quick fix for a container that fails to start due to permission issues. This grants the container near-host-level capabilities and bypasses almost all isolation. The right approach is to identify the specific permission or capability the container actually needs (often CAP_NET_BIND_SERVICE for low port binding, or a specific file permission) and grant only that, not blanket privilege.

Admission controllers intercept requests to the Kubernetes API before objects are persisted to etcd. They can validate (accept or reject a request based on its content), mutate (modify a request before it is stored, for example to inject default security settings), or both. Kubernetes ships with many built-in admission controllers, the most extensible are the ValidatingAdmissionWebhook and MutatingAdmissionWebhook, which allow external policy engines to handle admission decisions.

OPA/Gatekeeper and Kyverno are the two dominant policy engines for Kubernetes admission control. Gatekeeper uses OPA's Rego language to express policies. Kyverno uses a YAML-based policy language that is closer to Kubernetes resource syntax. Both can enforce requirements like 'all containers must have resource limits', 'images must come from the approved registry', 'Pods must not run as root', and 'all objects must have required labels.

Admission control solves a category of problems that RBAC alone cannot. RBAC controls who can create or modify objects, admission control controls what those objects are allowed to contain. A developer with permission to create Deployments can create one with privileged: true unless an admission controller prevents it. This separation of concerns, RBAC for authorization, admission for content validation, is the right model for large clusters.

Mutating admission controllers are used for safe defaults enforcement. Injecting sidecar containers, setting default resource limits, adding required labels, or automatically mounting secrets. This reduces the chance of a missing required configuration causing a policy violation, because the mutation fills in the gap automatically. However, mutations should be transparent, engineers should know what the cluster is adding to their workloads, and should be documented.

Kubernetes audit logs record every request to the API server. Who made the request, what action was requested, on which resource, and what the outcome was. The audit policy defines which requests are logged and at what level of detail (None, Metadata, Request, RequestResponse). A complete audit policy balances capturing security-relevant events against the log volume that would make analysis impractical.

Security-relevant events to watch for include access to Secrets (especially from unexpected service accounts), creation or modification of ClusterRoleBindings (privilege escalation), exec into Pods (indicates hands-on access to a running container), changes to admission webhooks (could disable security controls), and API calls from the system:anonymous or system:unauthenticated subjects (should never occur in a properly configured cluster).

Audit logs are only valuable if they are collected and analyzed. The cluster should ship audit logs to an external log management system (Elasticsearch, Splunk, CloudWatch Logs) rather than storing them only on the API server nodes, where they can be lost or tampered with. Alerts on high-priority events should be configured so that security-relevant API activity triggers immediate review rather than being discovered days later during a manual log review.

A common failure mode is enabling audit logging but never defining what 'interesting' looks like. Without alert rules and a process for reviewing suspicious events, audit logs become a large archive that is only consulted after an incident has already been identified through other means. The value of audit logs is proactive detection, not just post-incident reconstruction.

Cluster hardening is the process of closing gaps between the cluster's default configuration and a secure operational baseline. The CIS Kubernetes Benchmark is the most widely used reference, covering the API server, controller manager, scheduler, etcd, kubelet, and the worker node configuration. Running the kube-bench tool against a cluster reports which CIS checks pass and fail, giving a concrete starting point for hardening.

etcd is the most sensitive component in the Kubernetes control plane. It stores all cluster state, including Secrets in base64-encoded form. Access to etcd should be restricted to the API server, direct access by humans or other systems should require strong authentication and be audited separately. etcd should use TLS for all communication and should be encrypted at rest. Backing up etcd regularly is critical. Losing the etcd data means losing the entire cluster state.

The API server exposes several options that reduce the attack surface. Disabling anonymous authentication, requiring client certificate authentication for internal component communication, enabling audit logging, restricting the set of allowed admission plugins, and, for older clusters, ensuring the insecure port (which accepted HTTP without authentication) is disabled. Many managed Kubernetes services (EKS, GKE, AKS) configure some of these correctly by default, but self-managed clusters require explicit attention.

At the node level, AppArmor and seccomp profiles add mandatory access control and system call filtering to container processes. Seccomp profiles restrict which system calls a container can make. The RuntimeDefault profile blocks many syscalls that are rarely needed by applications but are frequently used by exploits. AppArmor profiles limit file access, network access, and capability use at the kernel level. Both require explicit configuration in the Pod's security context to be applied, but the investment pays off by making many exploit techniques significantly harder to execute.