Unlocking Kubernetes Performance: 10 Key Insights on Pod-Level Resource Managers in v1.36

Kubernetes v1.36 introduces an alpha feature—Pod-Level Resource Managers—that reimagines resource allocation for performance-sensitive workloads. By shifting from a per-container to a pod-level model, this enhancement gives you finer control over CPU, memory, and NUMA alignment, eliminating long-standing trade-offs. Whether you're running ML training, high-frequency trading, or low-latency databases, understanding this feature can drastically improve your pod's efficiency and predictability. Below, we break down the ten most important things you need to know, from the problem it solves to real-world configurations.

1. What Are Pod-Level Resource Managers?

Pod-Level Resource Managers extend the kubelet's existing Topology, CPU, and Memory Managers to support resource specifications at the pod level (.spec.resources). Instead of treating each container independently, the kubelet now considers the entire pod's resource budget to make allocation decisions. This alpha feature, gated by PodLevelResourceManagers and PodLevelResources, introduces a hybrid allocation model where some containers get exclusive, NUMA-aligned resources while others share a pod-level pool. This is a paradigm shift from the strictly per-container model, offering unprecedented flexibility for mixed-container pods.

2. The Problem with Per-Container Resource Allocation

Before v1.36, Kubernetes resource managers operated on a per-container basis. To achieve NUMA alignment and guaranteed QoS, every container in a pod had to request integer CPU and memory limits. This created a binary choice: allocate exclusive resources to all containers (wasteful for lightweight sidecars) or lose guaranteed QoS entirely. For example, a metrics exporter sidecar might need only 0.5 CPU, but to keep guaranteed class, you'd have to give it 1 full core. This inefficiency plagued high-performance workloads that rely on precise resource isolation.

3. Why High-Performance Workloads Suffered

Performance-critical applications—such as machine learning training jobs, low-latency databases, or real-time analytics—demand exclusive, NUMA-aligned resources to minimize latency and maximize throughput. However, modern pods often include auxiliary containers for logging, monitoring, or service mesh proxies. Under the old model, administrators faced a dilemma: either over-allocate CPUs to sidecars (wasting cores) or settle for burstable QoS, which could degrade performance due to CPU throttling and memory contention. This trade-off was especially painful for latency-sensitive workloads where every microsecond counts.

4. How Pod-Level Resource Managers Solve the Trade-Off

Pod-Level Resource Managers eliminate the binary choice by enabling hybrid resource allocation. The kubelet performs a single NUMA alignment based on the pod's total resource budget (defined in spec.resources). Then, it carves out exclusive resources for specific containers (like the main application) and places the remaining resources into a pod shared pool. Sidecar containers can draw from this pool without needing dedicated cores, all while maintaining guaranteed QoS for the pod as a whole. This allows you to co-locate auxiliary services on the same NUMA node as your primary workload without sacrificing performance.

5. Enabling the Alpha Feature in v1.36

To use Pod-Level Resource Managers, you must enable two feature gates on your kubelet: PodLevelResourceManagers=true and PodLevelResources=true. Additionally, you'll need to configure the Topology Manager scope (--topology-manager-scope=pod) to align resources at the pod level. Once enabled, you can define spec.resources in your pod manifest to set the overall budget. Note that this is an alpha feature, so it may have limited stability and is subject to change. Always test in non-production environments first.

6. Hybrid Resource Allocation: The Pod Shared Pool

One of the most innovative aspects of this feature is the pod shared pool. After allocating exclusive CPU and memory slices to designated containers (e.g., the main database), the remaining resources from the pod's budget form a separate pool. Sidecar containers—like metrics exporters or backup agents—run from this pool, sharing resources with each other but remaining strictly isolated from the exclusive slices and the rest of the node. This model ensures that lightweight sidecars don't waste dedicated cores, yet they still benefit from NUMA locality and guaranteed QoS. It's a win for both efficiency and performance.

7. Use Case: Tightly-Coupled Database with Sidecars

Consider a latency-sensitive database pod that includes a main database container, a local metrics exporter, and a backup agent. With pod-level resource management and the Topology Manager's pod scope, the kubelet aligns all resources to a single NUMA node. The database container receives exclusive CPU and memory slices, while the metrics exporter and backup agent share the pod's remaining resources. This setup allows you to safely co-locate auxiliary services on the same NUMA node without dedicating cores to them, reducing waste while preserving low-latency performance. The database's exclusive slices are strictly isolated, ensuring no interference from sidecar activities.

8. Configuration Example: YAML Breakdown

To illustrate, here's a sample pod manifest for a tightly-coupled database:

apiVersion: v1
kind: Pod
metadata:
  name: tightly-coupled-database
spec:
  resources:
    requests:
      cpu: "8"
      memory: "16Gi"
    limits:
      cpu: "8"
      memory: "16Gi"
  initContainers:
  - name: metrics-exporter
    image: metrics-exporter:v1
  - name: backup-agent
    image: backup-agent:v1
  containers:
  - name: database
    image: mydb:v1
    resources:
      requests:
        cpu: "6"
        memory: "12Gi"
      limits:
        cpu: "6"
        memory: "12Gi"

The pod-level resources define the total budget (8 CPU, 16Gi memory). The database container claims 6 CPU/12Gi exclusively; the remaining 2 CPU/4Gi become the shared pool for the sidecars. The Topology Manager ensures all resources are NUMA-aligned from the same node.

9. Benefits for ML Training and Low-Latency Apps

For ML training workloads that often use GPU-accelerated containers alongside data preprocessors, pod-level managers allow the main training container to claim exclusive GPU and CPU resources from a NUMA node, while auxiliary containers (e.g., for logging or health checks) share the leftover pool. This eliminates GPU memory contention and reduces CPU throttling. Similarly, low-latency applications like high-frequency trading can guarantee that their primary trading engine has dedicated, aligned resources, while monitoring containers operate without interference. The result: higher throughput, reduced latency, and better resource utilization.

10. What's Next: From Alpha to Stable

As an alpha feature, Pod-Level Resource Managers are still evolving. The Kubernetes community is gathering feedback on edge cases, such as dynamic resource scaling and integration with autoscalers. Future iterations may expand the shared pool concept to support more granular control, like per-container resource shares within the pool. Additionally, expect improvements in observability, such as metrics exposing pool usage and NUMA allocation details. For now, early adopters can test the feature in staging clusters to evaluate its impact on their high-performance workloads. The move toward stable will likely refine the API and ensure backward compatibility.

Pod-Level Resource Managers represent a significant step forward in Kubernetes resource management. By allowing hybrid allocation models, they resolve the long-standing tension between performance and efficiency. As the feature matures, it promises to become an essential tool for anyone running latency-sensitive or compute-intensive applications in Kubernetes. Start experimenting with v1.36 today to unlock its full potential.