Introduction
Modern Kubernetes platforms demand elasticity, efficiency, and operational simplicity. As workloads grow and shrink dynamically, static worker node sizing often leads to one of two problems:
- Overprovisioning → Higher infrastructure cost
- Underprovisioning → Pending pods and degraded application performance
This is where the Cluster Autoscaler becomes a critical add-on for VMware Kubernetes Service (VKS) clusters.
In this blog, we’ll explore how to deploy, manage, validate, and operate the Cluster Autoscaler Add-on on VKS clusters in a production-ready way.
What is Cluster Autoscaler?
Cluster Autoscaler is an official Kubernetes component that automatically adjusts the number of worker nodes in a cluster.
It performs:
- Scale Up: When pods remain in a Pending state due to insufficient CPU / Memory.
- Scale Down: When nodes remain underutilized for a configurable duration.
Why Use Cluster Autoscaler on VKS?
When running enterprise workloads on VKS, Cluster Autoscaler provides major operational benefits:
| Feature | Benefit |
|---|---|
| Automatic Worker Scaling | No manual node intervention |
| Better Resource Utilization | Use only required capacity |
| Faster App Deployment | Pending pods trigger scale-out |
| Cost Optimization | Remove idle worker nodes |
Provisioning a VKS Cluster with Autoscaler Annotations
Bill of Materials (BOM) used in Demo
- VKS 3.6.2
- VKr 1.35.2
- VCF 9.0.1
- VKS Standard Package v3.6.0+20260211
Step 1 : Verifying the AddonRelease exists
# Run following command on Supervisor Context # kubectl get addonreleases -n vmware-system-vks-public | grep cluster-autoscaler | grep -i 1.35 NAME ADDON AGE cluster-autoscaler.kubernetes.vmware.com.1.35.0-vmware.1-vks.1 cluster-autoscaler 1d
Step 2 : Check for the built-in AddonInstall
# Run following command on Supervisor Context # kubectl get AddonInstall -n vmware-system-vks-public NAME ADDON PAUSED AGE builtin-cluster-autoscaler-addoninstall-3.5.0 cluster-autoscaler 1d
Step 3 : Provisioning a Cluster with Autoscaler Annotations
Create a file named vks_cluster_with_auto_scaler.yaml with the following content.
In this example my VKS Cluster name is vks-cluster-with-autoscaler and VKS Cluster supervisor namespace name is vks-cluster-namespace, refer yaml for referenc in Attachments sectione.
apiVersion: cluster.x-k8s.io/v1beta2
kind: Cluster
metadata:
name: <vks-cluster-name>
namespace: <vks-cluster-namespace>
spec:
clusterNetwork:
pods:
cidrBlocks:
- 192.0.2.0/16
serviceDomain: cluster.local
services:
cidrBlocks:
- 198.51.100.0/12
topology:
classRef:
name: builtin-generic-v3.6.0
namespace: vmware-system-vks-public
controlPlane:
metadata:
annotations:
run.tanzu.vmware.com/resolve-os-image: os-name=photon
replicas: 3
variables:
- name: vmClass
value: <vm_class_name>
- name: storageClass
value: <storage_class_name> # (kubectl get sc)
- name: kubernetes
value:
etcdConfiguration:
maximumDBSizeGiB: 4
version: v1.35.2---vmware.1-vkr.3
workers:
machineDeployments:
- class: node-pool
metadata:
annotations:
run.tanzu.vmware.com/resolve-os-image: os-name=photon
cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size: '1'
cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size: '2'
name: photon-worker-node
variables:
overrides:
- name: vmClass
value: <vm_class_name>
- name: volumes
value:
- name: kubelet-dir
mountPath: /var/lib/kubelet
capacity: 5Gi
storageClass: <storage_class_name>
- name: containerd-dir
mountPath: /var/lib/containerd
capacity: 20Gi
storageClass: <storage_class_name># Run following command on Supervisor Context # kubectl apply -f vks_cluster_with_auto_scaler.yaml cluster.cluster.x-k8s.io/vks-cluster-with-autoscaler created
Verifying VKS Cluster is created successfully and running fine.
# kubectl get cluster -A | grep -i vks-cluster-with-autoscaler NAMESPACE NAME CLUSTERCLASS AVAILABLE CP DESIRED CP AVAILABLE CP UP-TO-DATE W DESIRED W AVAILABLE W UP-TO-DATE PHASE AGE VERSION vks-cluster-namespace vks-cluster-with-autoscaler builtin-generic-v3.6.0 True 3 3 3 1 1 1 Provisioned 1d v1.35.2+vmware.1
Verifying the Auto Scaler Installation
Generate VKS Cluster kubeconfig for verification and run following command against VKS Cluster.
Step 1 :Verifyingy Cluster Auto Scaler Package is installed successfully
# kubectl --kubeconfig vks-cluster-with-autoscaler-kubeconfig get pkgi -A NAMESPACE NAME PACKAGE NAME PACKAGE VERSION DESCRIPTION AGE PAUSED vmware-system-tkg vks-cluster-with-autoscaler-cluster-autoscaler cluster-autoscaler.kubernetes.vmware.com 1.35.0+vmware.1-vks.1 Reconcile succeeded 1d
Step 2 :Verifyingy Cluster Auto Scaler Deployment Object is created successfully
# kubectl --kubeconfig vks-cluster-with-autoscaler-kubeconfig get deploy -A | grep -i cluster-autoscaler NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE kube-system cluster-autoscaler 1/1 1 1 1d
Step 3 :Verifyingy Cluster Auto Scaler Pod is running fine
# kubectl --kubeconfig vks-cluster-with-autoscaler-kubeconfig get pods -A | grep -i cluster-autoscaler NAMESPACE NAME READY STATUS RESTARTS AGE kube-system cluster-autoscaler-599d74b7dd-plfh2 1/1 Running 0 1d
Step 4 :Verifyingy Cluster Nodes Ready Status
# kubectl --kubeconfig vks-cluster-with-autoscaler-kubeconfig get nodes -A NAME STATUS ROLES AGE VERSION vks-cluster-with-autoscaler-4tzhk-6mbf7 Ready control-plane 1d v1.35.2+vmware.1 vks-cluster-with-autoscaler-4tzhk-bh776 Ready control-plane 1d v1.35.2+vmware.1 vks-cluster-with-autoscaler-4tzhk-bt7fp Ready control-plane 1d v1.35.2+vmware.1 vks-cluster-with-autoscaler-photon-worker-node-jkxq2-cmd9btlsqn Ready <none> 1d v1.35.2+vmware.1
Exercising Autoscale-Out & Autoscale-In workflow
Step1 : Triggering a scale-out
Deploy a workload and scale it beyond what the current nodes can accommodate. The autoscaler should detect pending pods and provision additional nodes.
Create a file named nginx.yaml with the following content.
kind: Namespace
metadata:
name: nginx-ns
labels:
pod-security.kubernetes.io/enforce: privileged
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-app
namespace: nginx-ns
spec:
replicas: 100
selector:
matchLabels:
app: nginx-app
template:
metadata:
labels:
app: nginx-app
spec:
containers:
- name: nginx
image: nginx
resources:
requests:
cpu: "100m"
memory: "100Mi"
limits:
cpu: "250m"
memory: "250Mi"# Deploy nginx workload on VKS Cluster # kubectl --kubeconfig vks-cluster-with-autoscaler-kubeconfig apply -f nginx.yaml namespace/nginx-ns created deployment.apps/nginx-app created
Verifying Pods are pending or not
# kubectl --kubeconfig vks-cluster-with-autoscaler-kubeconfig get pods -A | grep -i nginx | grep -i pending | wc -l 28
Verifying New VKS Worker is added or not.
[root@tb01-router-large tb01]# kubectl --kubeconfig vks-cluster-with-autoscaler-kubeconfig get nodes -A NAME STATUS ROLES AGE VERSION vks-cluster-with-autoscaler-4tzhk-6mbf7 Ready control-plane 1d v1.35.2+vmware.1 vks-cluster-with-autoscaler-4tzhk-bh776 Ready control-plane 1d v1.35.2+vmware.1 vks-cluster-with-autoscaler-4tzhk-bt7fp Ready control-plane 1d v1.35.2+vmware.1 vks-cluster-with-autoscaler-photon-worker-node-jkxq2-cmd9b48mh6 Ready <none> 1m v1.35.2+vmware.1 # New Node Added vks-cluster-with-autoscaler-photon-worker-node-jkxq2-cmd9btlsqn Ready <none> 1d v1.35.2+vmware.1
Step2 : Observing scale-in
Delete the nginx deployment and wait for the scaleDownUnneededTime window (10 minutes by default) to elapse. The autoscaler will cordon and drain underutilized nodes before terminating them.
Delete the nginx deployment
# kubectl --kubeconfig vks-cluster-with-autoscaler-kubeconfig delete -f nginx.yaml namespace "nginx-ns" deleted deployment.apps "nginx-app" deleted
Node vks-cluster-with-autoscaler-photon-worker-node-jkxq2-cmd9b48mh6 is removed.
# kubectl --kubeconfig vks-cluster-with-autoscaler-kubeconfig get nodes -A NAME STATUS ROLES AGE VERSION vks-cluster-with-autoscaler-4tzhk-6mbf7 Ready control-plane 1d v1.35.2+vmware.1 vks-cluster-with-autoscaler-4tzhk-bh776 Ready control-plane 1d v1.35.2+vmware.1 vks-cluster-with-autoscaler-4tzhk-bt7fp Ready control-plane 1d v1.35.2+vmware.1 vks-cluster-with-autoscaler-photon-worker-node-jkxq2-cmd9btlsqn Ready <none> 1d v1.35.2+vmware.1
Key Takeaways
-
Cluster Autoscaler is available as a built-in add-on in VKS through native Add-on Management.
-
Automatically scales worker nodes up or down based on pending pods and cluster utilization.
-
Eliminates manual node scaling operations and improves operational efficiency.
-
Helps reduce infrastructure costs by removing underutilized nodes.
-
Improves application availability during sudden traffic spikes or workload bursts.
Disclaimer
This blog is for informational and educational purposes only. The configurations, examples, and architectural guidance provided are based on general best practices and publicly available references.
Always validate configurations in a non-production environment before applying them to live systems. Features and integrations may vary depending on the versions of VKS, VKr, LCI, Supervisor and vCenter Server being used. The author is not responsible for any unintended impact caused by the use of this information in production environments.
Contents, and opinions are my own and do not reflect the opinions of my employer.
