ECS "Task Failed ELB Health Checks" and "Task Failed Container Health Checks": Every Cause and the Fix (Fargate, ALB, curl, Grace Period)

An Amazon ECS task that keeps stopping with "Task failed ELB health checks" or "Task failed container health checks" is being killed on purpose: one of two different health checks said it wasn't working, and ECS replaced it. The stopped reason tells you which check failed, and the two have almost nothing in common. The ELB health check comes from your load balancer, from outside the container, over the network, and fails on blocked security groups, a wrong path or port, a redirect, an app listening only on 127.0.0.1, or a slow start with no grace period. The container health check is a command that runs inside the container, from your task definition, and fails when the command itself fails. Here is the surprise behind a huge share of container health check failures: the command almost every example uses, curl -f http://localhost/ || exit 1, fails in node:22-alpine, python:3.12-slim and distroless images, because none of them include curl. The check never reaches your app, the container is marked UNHEALTHY, and the task is replaced every couple of minutes while the app itself is fine. This page shows how to tell the two checks apart in one command, every cause of each in order, the exact settings and timings that matter, and a working health check command for every common base image.

Jake runs a phone repair shop, and his booking website now runs as a container on AWS Fargate behind an Application Load Balancer. His friend Ethan, a developer, set it up on a Saturday. By Monday, the site was up most of the time, but the ECS console showed a new task every few minutes and a long trail of stopped ones. The first stopped reason said "Task failed container health checks." When Ethan fixed that, the next one said "Task failed ELB health checks in (target-group ...)." Jake asked the obvious question: "If the website works when I open it, what exactly is failing?" This page is what Ethan explained over the counter: the two health checks, where to read the evidence, the missing-curl trap, the ignored Dockerfile HEALTHCHECK, the localhost-means-IPv6 trap on Alpine, why curl needs -f, the timings and the grace period, apps that listen only on 127.0.0.1, paths and success codes, security groups and ports, the replace-forever loop, and how to test a health check before you deploy it.

New to ECS? Our plain-English guide to Amazon ECS explains clusters, services and tasks in about ten minutes. You can follow this page without it; every step starts from zero.

Two health checks, two messages: which one failed?

An ECS service can watch your tasks in two independent ways, and either one can stop a task:

• The load balancer health check. If the service is attached to an Application or Network Load Balancer target group, the load balancer sends a request to each task over the network every so often, from outside. If the task stops answering correctly, the target group marks it unhealthy, and ECS stops the task and starts a replacement.
• The container health check. If the task definition includes a healthCheck for a container, the ECS agent runs that command inside the container on a schedule. A zero exit code means healthy; anything else is a failure. After enough failures in a row the container is UNHEALTHY, and if it's an essential container, the task is stopped and replaced.

Your stopped reason tells you which one fired. Here are the messages you'll see, and where each one sends you:

That last row matters, because you can easily mistake it for a health check failure. "Essential container in task exited" means your process ended (it crashed, finished, or was killed for memory), not that a check failed. If you see it, our guide to "Essential container in task exited" is the right page; exit code 137 has its own guide.

The two checks also disagree with each other a lot, and that's where most of your confusion will come from. A task can pass its container health check and fail the load balancer's, or the other way around, because they test different things from different places:

Jake: "So there are two inspectors. One walks inside the shop and checks the till works, the other stands across the street and checks the door opens."

Ethan: "Exactly. And the one inside came with a tool I forgot to pack, so he reported the till broken without ever touching it. That's the first message we got."

Read the evidence: stopped reason, service events and target health

Before you change anything, collect three pieces of evidence. Stopped tasks stay visible in the console and the API for only about an hour, so grab them while they're fresh.

1. Why did the task stop, and what did each container report?

aws ecs list-tasks --cluster shop --service-name bookings --desired-status STOPPED

aws ecs describe-tasks --cluster shop --tasks TASK_ID \
  --query "tasks[0].{reason: stoppedReason, code: stopCode, health: healthStatus, containers: containers[].{name: name, health: healthStatus, exit: exitCode, why: reason}}"

The stoppedReason is the message from the table above. Each container's healthStatus is HEALTHY, UNHEALTHY or UNKNOWN; UNKNOWN means there's no container health check, it's still being evaluated, or ECS doesn't have a result yet. The task's own health is UNHEALTHY if any essential container is UNHEALTHY.

2. What has the service been doing?

aws ecs describe-services --cluster shop --services bookings \
  --query "services[0].events[:10].[createdAt, message]" --output text

Service events give you the load balancer's reason, which the stopped reason cuts short. A line like service bookings (port 3000) is unhealthy in target-group bookings-tg due to (reason Health checks failed with these codes: [301]) tells you more than any amount of guessing.

3. What does the target group say right now?

aws elbv2 describe-target-health --target-group-arn TARGET_GROUP_ARN \
  --query "TargetHealthDescriptions[].[Target.Id, Target.Port, TargetHealth.State, TargetHealth.Reason, TargetHealth.Description]" \
  --output table

Each target's reason code points you to a section of this page:

If tasks are replaced faster than you can read them, an EventBridge rule on ECS "Task State Change" events with lastStatus STOPPED can copy every stopped reason to a log group, so the evidence survives past the hour. Our EventBridge explainer covers how rules match events.

"Task failed container health checks": the command isn't in your image

A container health check is a command you write, and it runs inside your container, using only what your image contains. If the program in the command doesn't exist there, the command fails, the exit code is non-zero, and ECS records a failed check. It never gets as far as your app. This is the single most common cause of "Task failed container health checks," because the example command nearly everyone copies is curl -f http://localhost/ || exit 1, and curl isn't in the images people build on most often:

In node:22-alpine, the copied command produces exactly this, and exit code 1 because of the || exit 1:

sh: curl: not found

Without the || exit 1, the shell's own exit code for a missing command is 127, which ECS also counts as a failure. Either way, the check fails every time. You won't find that line in your application's logs, because the check runs as a separate process next to your app, which is why this cause hides so well. The visible symptoms are the UNHEALTHY status and the stopped reason.

Health check commands that work, per image

Alpine-based images (BusyBox wget exits non-zero on connection errors and on 4xx and 5xx responses):

"healthCheck": {
  "command": ["CMD-SHELL", "wget -q --spider http://127.0.0.1:3000/health || exit 1"],
  "interval": 30, "timeout": 5, "retries": 3, "startPeriod": 30
}

Python images without curl or wget (urlopen raises an error on 4xx and 5xx, so the exit code is 1):

"command": ["CMD-SHELL", "python -c \"import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=3)\" || exit 1"]

Node.js images, any variant with a shell:

"command": ["CMD-SHELL", "node -e \"require('http').get('http://127.0.0.1:3000/health', r => process.exit(r.statusCode === 200 ? 0 : 1)).on('error', () => process.exit(1))\""]

Distroless images have no shell, so CMD-SHELL can't work at all. Use the CMD form, which runs the program directly, with the full path to the runtime:

"command": ["CMD", "/nodejs/bin/node", "-e", "require('http').get('http://127.0.0.1:3000/health', r => process.exit(r.statusCode === 200 ? 0 : 1)).on('error', () => process.exit(1))"]

If you'd rather keep curl, install it in the Dockerfile: RUN apk add --no-cache curl on Alpine, or RUN apt-get update && apt-get install -y --no-install-recommends curl && rm -rf /var/lib/apt/lists/* on Debian-based slim images. It adds a few megabytes and an extra package to patch, which is why the one-liners above are often the better choice.

Use CMD-SHELL when the command needs shell features such as ||, pipes or environment variables, and CMD when it doesn't. In the console's health check box, you type the same thing without brackets and quotes: CMD-SHELL, wget -q --spider http://127.0.0.1:3000/health || exit 1.

Jake: "So the inspector inside was told to use a tool that isn't in the shop."

Ethan: "And instead of saying 'I don't have a screwdriver,' he wrote 'till broken' on his report. Every two minutes. To a manager who replaced the whole shop each time."

ECS ignores the HEALTHCHECK in your Dockerfile

If your Dockerfile has a HEALTHCHECK line, you'll see it work under docker run and Docker Compose, and it does nothing on ECS. The ECS agent monitors and reports only health checks defined in the task definition. A HEALTHCHECK baked into the image, or inherited from a parent image, isn't monitored unless the same check is also written in the container definition, and a task definition healthCheck overrides the image's.

That causes two opposite surprises. The first: you added a careful HEALTHCHECK to the Dockerfile, deployed, and ECS shows the container health as UNKNOWN forever, because nothing in the task definition asked for a check. The second: a parent image ships a HEALTHCHECK that would fail in your setup, and it causes no trouble on ECS at all, until someone copies it into the task definition.

Here is a complete container definition with a health check, for a Node.js app on Alpine listening on port 3000:

{
  "name": "bookings",
  "image": "123456789012.dkr.ecr.us-east-1.amazonaws.com/bookings:1.4.2",
  "essential": true,
  "portMappings": [ { "containerPort": 3000, "protocol": "tcp" } ],
  "healthCheck": {
    "command": ["CMD-SHELL", "wget -q --spider http://127.0.0.1:3000/health || exit 1"],
    "interval": 30,
    "timeout": 5,
    "retries": 3,
    "startPeriod": 60
  },
  "logConfiguration": {
    "logDriver": "awslogs",
    "options": { "awslogs-group": "/ecs/bookings", "awslogs-region": "us-east-1", "awslogs-stream-prefix": "app" }
  }
}

Only essential containers with a health check count toward the task's health. A sidecar marked "essential": false can be UNHEALTHY without stopping the task, which is useful for a log shipper you don't want taking the app down with it.

localhost can mean ::1: the Alpine wget trap

Here is a trap that catches people who did everything right. In an Alpine container whose hosts file maps localhost to both 127.0.0.1 and ::1, which is how Docker and Podman set it up, the name resolves to the IPv6 address ::1 first. Node.js and many web servers listen on 0.0.0.0 by default, which is IPv4 only. BusyBox wget tries the first address and gives up:

/ # getent hosts localhost
::1               localhost  localhost
/ # wget -q --spider http://localhost:3000/health
wget: can't connect to remote host: Connection refused
/ # wget -q --spider http://127.0.0.1:3000/health ; echo $?
0

The app is running, the port is right, and the health check fails anyway. The fix is to put 127.0.0.1 in the command instead of localhost, which is what every example on this page does. The alternative is to make the app listen on ::, which accepts both IPv6 and IPv4 on most Linux systems; with that, localhost works again. Node's own http.get tries both address families, so the node -e one-liner happens to work either way, but writing 127.0.0.1 removes the question entirely.

curl without -f says yes to a broken app

The opposite failure is worse: a health check that always passes. curl's exit code reports whether it managed to talk to the server, not whether the server was happy. Without -f, a 500 Internal Server Error still exits 0. Here is the same app with three paths, one healthy, one broken, one redirecting:

Two lessons. First, always use curl -f (or --fail), so 4xx and 5xx responses fail the check. Second, -f treats a redirect as success, while the load balancer, with its default success code of 200, treats the same redirect as a failure. That's how a task ends up with a passing container health check and a failing load balancer check on the same URL. Point both at a dedicated path that returns 200 without redirecting, such as /health.

A good health endpoint answers quickly and checks only what the container itself controls: that the process is up and can serve a request. If it also pings the database, a database blip makes every task fail its check at once, and ECS replaces healthy containers that couldn't have fixed the database anyway. Keep deep dependency checks for monitoring and alarms, not for the check that decides whether to kill a task.

Timing: interval, timeout, retries, startPeriod and the grace period

Many of the "health check failing" problems you'll meet are really "health check failing too early." Both checks have timing settings, and the defaults give a slow-starting app about a minute.

Container health check settings

Load balancer target group settings

Three facts about these numbers decide most of the timing problems you'll hit:

• A newly registered target needs only one passing check to be healthy, whatever the healthy threshold says. The threshold of 5 applies only when a target recovers from unhealthy.
• With the defaults, a target that fails is unhealthy after about 60 seconds: two failed checks, 30 seconds apart. An app that needs 90 seconds to load, run migrations or warm a cache is killed before it's ready, replaced, and killed again.
• The service's healthCheckGracePeriodSeconds defaults to 0. It's the time after a task starts during which ECS ignores unhealthy load balancer, VPC Lattice and container health checks. Setting it to a bit more than your real startup time stops the startup kill loop.

aws ecs update-service --cluster shop --service bookings --health-check-grace-period-seconds 120

For the container check, startPeriod does the same job and tops out at 300 seconds. If an app needs longer than that to start, rely on the service grace period. Here are reasonable starting points:

Faster settings also speed up deployments: with an interval of 5 seconds and a healthy threshold of 2, the load balancer's part of a deployment takes about 10 seconds instead of minutes. Just keep the grace period honest, because a grace period that's far too long hides real failures for that long.

Jake: "So the inspector across the street knocked twice in a minute, nobody answered because Ethan's app was still putting the kettle on, and he had the shop torn down."

Ethan: "And rebuilt, and torn down again. The grace period is a note on the door: 'opening in two minutes, don't knock yet.'"

The app listens only on 127.0.0.1: the container check passes, the load balancer can't get in

A program that listens on 127.0.0.1 (or localhost) accepts connections only from inside its own network namespace. The container health check runs inside, so it passes. The load balancer connects from outside, to the task's IP address, so it's refused. The target shows "Health checks failed" or "Request timed out," while the container health check says HEALTHY. It's the clearest case of the two checks disagreeing:

inside the container:   wget -q --spider http://127.0.0.1:3000/health   -> exit 0
from outside:           curl http://TASK_IP:3000/health                  -> connection refused

Several popular defaults you may be relying on bind to 127.0.0.1:

• Flask: both flask run and app.run() listen on 127.0.0.1:5000 unless told otherwise. Use flask run --host=0.0.0.0, app.run(host="0.0.0.0"), or better for production, gunicorn -b 0.0.0.0:8000 app:app.
• Node.js: server.listen(3000, "localhost") or "127.0.0.1". Use server.listen(3000, "0.0.0.0"), or leave the host out.
• Development servers started with their default host, which often means loopback only. Production containers should run the production server, bound to all interfaces.

When a container works with docker run -p 8080:3000 on your laptop but fails behind a load balancer, binding is the first thing to check, because port publishing on a laptop and a load balancer on AWS both arrive from outside the container in the same way.

ELB health check path and success codes: 404, 301, 302, 401

"Health checks failed with these codes: [404]" means the load balancer reached your app, and the app answered with a status outside the target group's success codes. The connection works; the request is wrong. The load balancer sends an HTTP GET to the target group's health check path (default /) on the health check port (default: the same port that receives traffic), and only the codes in the matcher count as healthy (default 200).

Changing the health check settings takes one call:

aws elbv2 modify-target-group --target-group-arn TARGET_GROUP_ARN \
  --health-check-path /health --matcher HttpCode=200 \
  --health-check-interval-seconds 15 --healthy-threshold-count 2

Widening the matcher to 200-399 is a quick fix for redirects, but a dedicated health path is better: it tells you the app is serving, not just that something answered. If you use CloudFormation or Terraform, change the target group there too, or the next deployment of your template can put the old path back.

"Request timed out": security groups, ports, subnets and target type

"Request timed out" (reason code Target.Timeout) means the load balancer's request got no answer at all in the timeout. The app wasn't asked, or couldn't reply. Work through these in order:

1. The task's security group doesn't allow the load balancer in. With the awsvpc network mode, which Fargate always uses, each task has its own network interface and security group. That group must allow inbound traffic on the container port from the load balancer's security group. A rule for port 80 doesn't help an app listening on 3000.
2. The port doesn't match. The container port in the service's load balancer settings, the containerPort in the task definition, and the port the app actually listens on must all be the same number. ECS registers each task with that container port, and the health check uses it too unless the target group sets a different health check port; an explicit health check port must also be one the app listens on. A mismatch anywhere looks like a timeout.
3. Dynamic host ports on EC2 aren't open. In bridge mode with hostPort 0, each task gets a random port from the instance's ephemeral range, such as 32768 to 61000 on the current ECS-optimized AMI. The instance security group must allow that whole range from the load balancer's security group.
4. The target type is wrong. For awsvpc tasks the target group's target type must be ip, not instance, because the task has its own network interface rather than living behind the instance's address. The target type can't be changed later; create a new target group.
5. The load balancer isn't in the task's Availability Zone. The load balancer's subnets must cover every zone where tasks run. A task in a zone the load balancer doesn't use never passes a health check (reason Target.NotInUse), and is killed.
6. A network ACL blocks the return traffic. Security groups remember connections, but network ACLs don't, so a subnet ACL must allow the health check in and the ephemeral-port replies out. Our security group vs NACL guide shows how to tell which one is blocking.
7. The app is too slow to answer. If the health endpoint takes longer than the timeout (5 seconds by default) under load, every check fails. Make the endpoint cheaper, or raise the timeout.

Here is the rule you need on the task's security group, allowing only the load balancer's group on the container port:

aws ec2 authorize-security-group-ingress --group-id sg-TASKS \
  --protocol tcp --port 3000 --source-group sg-LOADBALANCER

One more trap: tasks in private subnets without a NAT gateway or VPC endpoints can't pull their image or reach Secrets Manager, and they fail before any health check runs, with a different error such as CannotPullContainerError or ResourceInitializationError. If your tasks never reach RUNNING, health checks aren't the problem yet.

Jake: "So 'timed out' is the inspector knocking on a door that's bricked up, and 'failed with code 404' is somebody answering and saying 'wrong address.'"

Ethan: "Perfect. One's about the road and the door; the other's about what you asked for once you got in."

The replace-forever loop, the circuit breaker, and fail-open

When a service's tasks keep failing health checks, ECS does what it's built to do: it stops each unhealthy task and starts a new one to keep the desired count. Without anything to stop it, that loop can run for hours during a deployment, with your service never reaching a steady state. Each replacement goes through the full lifecycle (deregistering from the target group, stopping, provisioning, pulling the image, starting), so the churn also costs time and image pulls.

The deployment circuit breaker ends that loop. It counts tasks that fail to reach RUNNING and, once tasks are running, tasks replaced because of failed load balancer, Cloud Map or container health checks. When the count reaches a threshold, the deployment is marked FAILED and stops launching tasks, and with rollback on, ECS returns to the last deployment that completed:

aws ecs update-service --cluster shop --service bookings \
  --deployment-configuration "deploymentCircuitBreaker={enable=true,rollback=true}"

By default the threshold is half the desired task count, rounded up, but never less than 3 or more than 200: a service of 1 to 6 tasks fails after 3 failures, a service of 25 tasks after 13. The circuit breaker works only with the rolling update (ECS) deployment controller.

There's a twist on the load balancer side you should know about. If every target in a target group is unhealthy at the same time, the load balancer fails open: it sends traffic to all of them anyway, rather than to none. That's why a site can keep answering, slowly or intermittently, while ECS is busy stopping and replacing tasks for failing their checks. "The site works when I open it" doesn't mean the health checks are fine.

Test the health check before you deploy, and inside a running task

You can try every container health check on this page on your own machine in a minute, with the same image you deploy. The routine:

1. See which tools your image actually has.
2. Run the container and the exact health check command inside it, and read the exit code.
3. Publish the port and request the health path from outside the container, the way the load balancer will.

Step one:

docker run --rm --entrypoint sh bookings:1.4.2 -c 'for t in curl wget; do printf "%s: " $t; command -v $t || echo MISSING; done'

Step two:

docker run -d --name hc bookings:1.4.2
docker exec hc sh -c 'wget -q --spider http://127.0.0.1:3000/health || exit 1'; echo "exit code: $?"
docker rm -f hc

If you see exit code 0, your check passes; anything else is exactly what ECS would record as a failure. For step three, publish the port and request it from outside the container: docker run -d -p 8080:3000 bookings:1.4.2, then curl -i http://127.0.0.1:8080/health. A refusal here, with a pass inside, is the 127.0.0.1 binding problem. The status line shows what the load balancer will get, including any redirect.

On ECS itself, ECS Exec gives you a shell in a running task, so you can run the health check where it actually runs. It has to be turned on before the task starts; it can't be added to tasks that are already running:

aws ecs update-service --cluster shop --service bookings --enable-execute-command --force-new-deployment

aws ecs execute-command --cluster shop --task TASK_ID --container bookings \
  --interactive --command "/bin/sh"

ECS Exec needs a task role with Session Manager permissions and a writable root file system, and it can't open a shell in an image that has none, such as distroless. With a short grace period, a task that fails its checks may be replaced before you can connect; raise the grace period temporarily while you investigate.

Still stuck? Troubleshooting by symptom

The container health status stays UNKNOWN forever

There's no healthCheck in the task definition, so there's nothing to evaluate. A HEALTHCHECK in the Dockerfile doesn't count on ECS. Add the check to the container definition, register a new revision, and update the service.

It fails every time, even right after deploying a fix

The service may still be running the old task definition revision, or the image tag points at an old image. Check the revision in describe-services and the image digest in describe-tasks. A new revision plus update-service, or force-new-deployment for the same revision, picks up the change.

The container health check passes but ELB health checks fail

The app is fine from inside, but unreachable or answering differently from outside. Check binding (0.0.0.0, not 127.0.0.1), the security group rule from the load balancer, the port numbers, and whether the health path redirects. Target health's reason code tells you which.

ELB health checks pass but the container health check fails

The command is the problem: missing curl or wget, localhost resolving to ::1 on Alpine, the wrong port, CMD-SHELL in an image with no shell, or a timeout shorter than the endpoint needs. Run the exact command inside the image locally and read the exit code.

Tasks pass for a while, then fail under load

A health endpoint that does real work (database queries, external calls) gets slower as traffic rises and starts timing out. Make it shallow, raise its timeout, or scale out sooner so tasks aren't saturated.

Only tasks in one Availability Zone fail

The load balancer isn't enabled in that zone, or that subnet's route table or network ACL differs. Compare the failing tasks' subnets with the load balancer's subnets, and look for Target.NotInUse in target health.

The deployment never finishes and keeps starting tasks

Tasks fail health checks and ECS keeps replacing them. Turn on the deployment circuit breaker with rollback so a failed deployment stops and returns to the last good one, then fix the check using the evidence from describe-services events.

The stopped reason is cut off

Stopped reasons are truncated. The service events carry the full load balancer reason, including the status code or "Request timed out", and describe-target-health shows the current reason code for each target.

ECS "Task failed ELB health checks" and container health checks: frequently asked questions

Jake's booking site now runs two tasks that stay up for days. The container check uses wget against 127.0.0.1, because the Alpine image never had curl, and the load balancer checks /health, which answers 200 without the HTTPS redirect that the home page does. The service has a 90-second grace period for the cold start, and the circuit breaker is on, so a bad deployment rolls itself back instead of churning all afternoon. Ethan wrote the line below on the back of a repair ticket and pinned it above the router.

Revision note. Written October 12, 2026, for every task replaced while the app was fine. "A bad workman blames his tools," the old saying goes; on ECS, check that the tools are in the box first.