EKS PVC Stuck Pending? Install the Amazon EBS CSI Driver Add-on: IAM Policy, Pod Identity or IRSA, gp3 StorageClass, Terraform and Every Error

If a PersistentVolumeClaim stays Pending on Amazon EKS with "waiting for a volume to be created, either by external provisioner ebs.csi.aws.com...", the Amazon EBS CSI driver is missing or cannot call EC2. Since Kubernetes 1.23, EKS no longer creates EBS volumes without it, so every cluster that uses EBS-backed storage needs the aws-ebs-csi-driver add-on installed and an IAM role with the AWS managed policy AmazonEBSCSIDriverPolicy, linked to the ebs-csi-controller-sa service account in kube-system through EKS Pod Identity or IRSA. Then create a gp3 StorageClass with provisioner: ebs.csi.aws.com and volumeBindingMode: WaitForFirstConsumer. Other common failures: UnauthorizedOperation in the controller logs (the IAM role is missing), "volume node affinity conflict" or "Multi-Attach error" (EBS volumes live in one Availability Zone and attach to one node), KMS-encrypted volumes without key permissions, and Fargate pods, which cannot use EBS at all. Below: install with the console, eksctl, the CLI and Terraform, the StorageClass, every error and its fix, resizing, snapshots and versions.

The shop moved its ticketing database to Kubernetes on EKS for a test, using a Helm chart that "just works" on other clusters. On this one, the database pod sat in Pending for an hour, and its volume claim said it was waiting for a volume to be created. Ethan had created the cluster with the console and assumed storage came with it. It did not: on EKS, the piece that creates EBS disks for Kubernetes is an add-on you install yourself, and it needs its own permission to create volumes.

New to EKS or EBS? The cards below explain both in plain English, plus the EBS and pod problems this one often sits next to.

What the EBS CSI driver does, and why EKS needs it

Kubernetes keeps your storage requests separate from the storage itself. Your pod asks for a disk through a PersistentVolumeClaim (PVC); a StorageClass you define says what kind of disk to create; and a CSI driver (Container Storage Interface) is the plugin that actually talks to the cloud and creates, attaches and deletes the disks. On AWS, the Amazon EBS CSI driver turns each PVC into an EBS volume and attaches it to the node where the pod runs.

Older Kubernetes versions had EBS support built in ("in-tree"). Since Kubernetes 1.23 on EKS, that built-in code hands everything to the CSI driver, so without the driver installed, nothing creates the volumes, even for StorageClasses that still name the old kubernetes.io/aws-ebs provisioner. That is why a chart that worked years ago now leaves its PVC Pending.

The driver runs as two parts: a controller (a Deployment in kube-system) that calls the EC2 API to create, delete and attach volumes, and a node component (a DaemonSet) on every node that formats and mounts them. The controller is the part that needs AWS permissions. If EKS itself is new to you, our plain-English EKS guide covers clusters, nodes and add-ons.

Jake: "So Kubernetes asks for a disk, but someone still has to go and get it?"

Ethan: "And that someone is the driver. We never hired it."

Install the EBS CSI driver add-on, step by step

1. Create the IAM role the driver's controller will use, with the AWS managed policy AmazonEBSCSIDriverPolicy (next section).
2. Install the add-on: in the EKS console, open the cluster, choose Add-ons, Get more add-ons, select Amazon EBS CSI Driver, choose the version, and select the IAM role (with Pod Identity, the console can create the association for you).
3. Check the pods are running: kubectl get pods -n kube-system -l app.kubernetes.io/name=aws-ebs-csi-driver should show the controller and one node pod per node, all Running.
4. Create a gp3 StorageClass and, if you want, make it the default (below).
5. Test with a small PVC and pod, then check that the PVC is Bound and a new EBS volume appears in the EC2 console.

With the CLI:

# Versions available for your Kubernetes version
aws eks describe-addon-versions --addon-name aws-ebs-csi-driver --kubernetes-version 1.31 \
  --query "addons[].addonVersions[].addonVersion" --output text

# Install the add-on (IRSA style: pass the role for the service account)
aws eks create-addon --cluster-name shop-eks --addon-name aws-ebs-csi-driver \
  --service-account-role-arn arn:aws:iam::111122223333:role/AmazonEKS_EBS_CSI_DriverRole

# Status should become ACTIVE
aws eks describe-addon --cluster-name shop-eks --addon-name aws-ebs-csi-driver \
  --query "addon.[status,addonVersion]"

With eksctl:

eksctl create addon --cluster shop-eks --name aws-ebs-csi-driver \
  --service-account-role-arn arn:aws:iam::111122223333:role/AmazonEKS_EBS_CSI_DriverRole --force

The managed add-on is the recommended route because EKS handles its updates and security patches. Installing the driver with its Helm chart also works, but then you maintain it yourself.

IAM: Pod Identity or IRSA for the controller

The controller needs to call EC2 (CreateVolume, AttachVolume, DeleteVolume, snapshots and tags). The AWS managed policy AmazonEBSCSIDriverPolicy grants exactly that. You attach it to a role and connect the role to the controller's Kubernetes service account, ebs-csi-controller-sa in kube-system, in one of two ways:

EKS Pod Identity (simpler): the role trusts the Pod Identity service, and an association links it to the service account. The cluster needs the EKS Pod Identity Agent add-on.

# Trust policy for the role
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"Service": "pods.eks.amazonaws.com"},
    "Action": ["sts:AssumeRole", "sts:TagSession"]
  }]
}

aws iam attach-role-policy --role-name AmazonEKS_EBS_CSI_DriverRole \
  --policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy

aws eks create-pod-identity-association --cluster-name shop-eks \
  --namespace kube-system --service-account ebs-csi-controller-sa \
  --role-arn arn:aws:iam::111122223333:role/AmazonEKS_EBS_CSI_DriverRole

IRSA (IAM roles for service accounts): the role trusts the cluster's OIDC provider, limited to that service account. eksctl writes it for you:

eksctl utils associate-iam-oidc-provider --cluster shop-eks --approve

eksctl create iamserviceaccount --cluster shop-eks --namespace kube-system \
  --name ebs-csi-controller-sa --role-name AmazonEKS_EBS_CSI_DriverRole \
  --attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy \
  --role-only --approve

Which should you pick?

With IRSA, the trust policy's condition must name system:serviceaccount:kube-system:ebs-csi-controller-sa exactly; a typo there produces "not authorized to perform sts:AssumeRoleWithWebIdentity" in the controller logs. A shortcut some guides suggest, attaching the policy to the node role instead, also works, but it gives every pod on the node the right to create and delete EBS volumes, so a dedicated role is the better habit.

The gp3 StorageClass to use

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: gp3
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: ebs.csi.aws.com
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
reclaimPolicy: Delete
parameters:
  type: gp3
  encrypted: "true"
  # iops: "3000"        # gp3 baseline; raise for busy databases
  # throughput: "125"   # MiB/s baseline
  # kmsKeyId: arn:aws:kms:us-east-1:111122223333:key/1a2b3c4d-...

Why each line matters:

• provisioner: ebs.csi.aws.com: the CSI driver, not the old in-tree name.
• WaitForFirstConsumer: your volume is created only once your pod is scheduled, in that node's Availability Zone. With Immediate, you can end up with a volume in a zone where your pod cannot run.
• allowVolumeExpansion: true: lets you grow your disk later by editing the PVC.
• type: gp3: the current general-purpose volume type, cheaper per GB than gp2 with a fixed baseline of performance.
• encrypted: "true": encrypts every volume, with the account's default EBS key unless you name a kmsKeyId.

Older clusters often have a gp2 StorageClass marked as default; newer EKS versions no longer mark it as default. Check with kubectl get storageclass: exactly one class should show "(default)", or PVCs without a storageClassName will pick the wrong one or none. To remove the default flag from gp2: kubectl annotate storageclass gp2 storageclass.kubernetes.io/is-default-class=false --overwrite. Our EBS vs instance store guide explains the volume types themselves.

A test claim and pod:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: test-claim
spec:
  accessModes: ["ReadWriteOnce"]
  storageClassName: gp3
  resources:
    requests:
      storage: 4Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: test-app
spec:
  containers:
    - name: app
      image: public.ecr.aws/amazonlinux/amazonlinux:2023
      command: ["/bin/sh", "-c", "date > /data/out.txt; sleep 3600"]
      volumeMounts:
        - name: data
          mountPath: /data
  volumes:
    - name: data
      persistentVolumeClaim:
        claimName: test-claim

With WaitForFirstConsumer, the claim shows Pending until the pod is scheduled; that is normal. Once the pod starts, kubectl get pvc test-claim should show Bound.

PVC Pending and other EBS CSI errors, fixed

Three commands find the cause in nearly every case:

# What the claim is waiting for
kubectl describe pvc test-claim

# What the controller tried and why AWS refused
kubectl logs -n kube-system deploy/ebs-csi-controller -c csi-provisioner --tail=50
kubectl logs -n kube-system deploy/ebs-csi-controller -c ebs-plugin --tail=50

For Ethan's cluster, the first command showed the "waiting for external provisioner" message: no driver at all. After installing the add-on, the logs showed UnauthorizedOperation, because he had skipped the IAM role. With the Pod Identity association added and the controller restarted, the PVC was Bound within a minute. If pods are Pending for reasons that have nothing to do with storage, our pods Pending guide covers the others.

Availability Zones, Multi-Attach and StatefulSets

Two EBS facts shape how you run stateful apps on EKS:

• An EBS volume lives in one Availability Zone. Your pod using it can only run on nodes in that zone. If your nodes in that zone are gone (scaled in, or the node group spans fewer zones than before), the pod stays Pending with a node affinity conflict.
• A normal EBS volume attaches to one node at a time (ReadWriteOnce). During a rolling update, a new pod on another node cannot attach the volume until the old pod releases it, which shows as a Multi-Attach error for a few minutes or until the old pod is gone.

The patterns that avoid trouble: run databases as StatefulSets (each replica gets its own volume), use the Recreate deployment strategy for single-replica apps with a volume, keep node groups in every zone where you have volumes, and for files that many pods must share at once, use Amazon EFS with its own CSI driver instead of EBS.

Jake: "So the disk can't follow the pod to another zone?"

Ethan: "No. The pod has to go where the disk is. WaitForFirstConsumer just makes sure the disk is born in the right place."

Resizing a volume

If your StorageClass has allowVolumeExpansion: true, you grow a disk with one edit to your claim:

kubectl patch pvc test-claim -p '{"spec":{"resources":{"requests":{"storage":"20Gi"}}}}'
kubectl get pvc test-claim -w

The driver enlarges the EBS volume and the file system online, usually without restarting the pod. Volumes can only grow, never shrink, and EBS allows another size change on the same volume only after a cooling-off period of several hours, so pick a sensible step rather than growing a gigabyte at a time. The same limits apply outside Kubernetes; our guide to growing an EBS volume live explains them.

Snapshots and backups

You can take Kubernetes VolumeSnapshots of your claims, and the driver turns them into EBS snapshots. For that you need the snapshot CRDs and controller, available on EKS as the CSI snapshot controller add-on, plus a VolumeSnapshotClass with driver: ebs.csi.aws.com. You can then snapshot a PVC and create new PVCs from the snapshot, which is how you clone a database for testing. For scheduled, policy-based backups across clusters, AWS Backup supports EKS too. Snapshots are incremental and billed per GB stored; our snapshot billing guide explains why the second one costs less than the first.

Terraform

resource "aws_iam_role" "ebs_csi" {
  name = "AmazonEKS_EBS_CSI_DriverRole"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect    = "Allow"
      Principal = { Service = "pods.eks.amazonaws.com" }
      Action    = ["sts:AssumeRole", "sts:TagSession"]
    }]
  })
}

resource "aws_iam_role_policy_attachment" "ebs_csi" {
  role       = aws_iam_role.ebs_csi.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy"
}

resource "aws_eks_pod_identity_association" "ebs_csi" {
  cluster_name    = aws_eks_cluster.shop.name
  namespace       = "kube-system"
  service_account = "ebs-csi-controller-sa"
  role_arn        = aws_iam_role.ebs_csi.arn
}

resource "aws_eks_addon" "ebs_csi" {
  cluster_name = aws_eks_cluster.shop.name
  addon_name   = "aws-ebs-csi-driver"
  depends_on   = [aws_eks_pod_identity_association.ebs_csi]
}

Your cluster also needs the eks-pod-identity-agent add-on for Pod Identity, and the depends_on makes sure the association exists before the driver starts, so your controller never runs without permissions. If you use IRSA instead, set service_account_role_arn on the aws_eks_addon and give the role an OIDC trust policy. The StorageClass itself can be managed with the Kubernetes provider's kubernetes_storage_class_v1 resource.

Versions and upgrades

Each add-on version supports a range of Kubernetes versions, so you check the driver every time you upgrade your cluster. A safe routine:

1. List compatible versions for your target Kubernetes version with aws eks describe-addon-versions (shown above).
2. Update the add-on first if your current version does not support the target: the console's Add-ons tab, or aws eks update-addon --cluster-name shop-eks --addon-name aws-ebs-csi-driver --addon-version VERSION.
3. Wait for ACTIVE with aws eks describe-addon, and confirm your controller and node pods are Running again.
4. Upgrade the control plane, then your node groups.
5. Run your test PVC again to prove a new volume still provisions.

Updating the driver does not touch your existing volumes; running pods keep their mounts while the driver pods restart.

Frequently asked questions

The test database ran on its gp3 volume for a month, survived two node replacements thanks to the StatefulSet, and grew once from 20 to 50 GB with a single command. Ethan now adds three things to every new cluster on day one: the EBS CSI add-on, its Pod Identity role, and a default gp3 StorageClass. Jake's summary for the team notes: "Kubernetes asks for disks. The driver fetches them. Give the driver permission."

Revision note. Written October 19, 2026. On EKS, storage is an add-on, not a given; installing it on day one saves the hour of Pending.