ECS Task Definitions: Every Field, Common Mistakes, Best Practices

Two IAM roles, one common mistake

Every ECS task definition has two separate IAM roles. They look similar and use the same trust principal (ecs-tasks.amazonaws.com), but they serve completely different parts of the system.

The mistake: the ECS console presents executionRoleArn prominently during task definition creation. Teams add S3 or DynamoDB permissions there. The task launches fine — but every AWS SDK call from the app returns AccessDeniedException. The task is running, logs are flowing, health check passes. The only symptom is API calls failing inside the app.

The execution role needs AmazonECSTaskExecutionRolePolicy (covers ECR + CloudWatch) plus explicit permissions for any secrets you reference:

{
  "Effect": "Allow",
  "Action": [
    "ssm:GetParameters",
    "secretsmanager:GetSecretValue",
    "kms:Decrypt"
  ],
  "Resource": [
    "arn:aws:ssm:us-east-1:123456789012:parameter/prod/*",
    "arn:aws:secretsmanager:us-east-1:123456789012:secret:prod/*"
  ]
}

Source: Amazon ECS task execution IAM role · Amazon ECS task IAM role

Fargate CPU and memory — the combinations that don't exist

Fargate CPU and memory are not free-form numbers. There are 8 CPU tiers, each with a fixed set of valid memory values. Anything outside this table fails at deployment with: No Fargate configuration exists for given values.

Source: Troubleshoot Amazon ECS invalid CPU or memory errors · Fargate task definitions

Secrets: SSM vs Secrets Manager, and why rotation does nothing

Use the secrets array, not environment, for anything sensitive. The syntax difference is value vs valueFrom — and mixing them up causes a silent empty variable.

"containerDefinitions": [{
  "environment": [
    {"name": "LOG_LEVEL", "value": "warn"},
    {"name": "PORT",      "value": "8080"}
  ],
  "secrets": [
    {
      "name": "DB_PASSWORD",
      "valueFrom": "arn:aws:ssm:us-east-1:123456789012:parameter/prod/db/password"
    },
    {
      "name": "API_KEY",
      "valueFrom": "arn:aws:secretsmanager:us-east-1:123456789012:secret:prod/api-key-AbCdEf"
    },
    {
      "name": "DB_USER",
      "valueFrom": "arn:aws:secretsmanager:us-east-1:123456789012:secret:prod/db-AbCdEf:username::"
    }
  ]
}]

The last entry shows JSON key extraction from Secrets Manager — appending :username:: to the ARN injects only the username field from a JSON secret. The trailing :: (empty version-stage and version-id) are required. Requires Fargate platform 1.4.0+.

At scale, the cost difference is significant. 100 secrets: SSM is ~$0/month, Secrets Manager is ~$40/month. The common pattern: use SSM for simple key-value secrets, Secrets Manager when you need built-in rotation or JSON key extraction. Note that ssm:GetParameters (plural) is the correct permission — ssm:GetParameter (singular) will silently fail for tasks that fetch multiple parameters at once.

Source: Pass sensitive data to an Amazon ECS container · Pass Secrets Manager secrets through ECS environment variables

Health checks — three ways to loop forever

Three mistakes compound to create an endless restart loop: startPeriod too short, timeout ≥ interval, and the health check binary not installed in the image. All three are ECS defaults or copy-paste traps.

Build a proper /health endpoint

The best health check calls an endpoint your app already serves. If the app can't respond to HTTP, the container isn't healthy regardless of what curl says. Here's a minimal /health endpoint in each common language:

package main

import (
    "encoding/json"
    "net/http"
    "os"
)

func main() {
    http.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) {
        w.Header().Set("Content-Type", "application/json")
        w.WriteHeader(http.StatusOK)
        json.NewEncoder(w).Encode(map[string]string{"status": "ok"})
    })

    port := os.Getenv("PORT")
    if port == "" {
        port = "8080"
    }
    http.ListenAndServe(":"+port, nil)
}

Then reference it in the task definition:

"healthCheck": {
  "command": ["CMD-SHELL", "curl -f http://localhost:8080/health || exit 1"],
  "interval": 30,
  "timeout": 5,
  "retries": 3,
  "startPeriod": 60
}

For images without curl, use the app's runtime directly instead of CMD-SHELL:

"command": ["CMD-SHELL", "python -c \"import urllib.request; urllib.request.urlopen('http://localhost:8080/health')\""]
// or Go binary:
"command": ["CMD", "/bin/healthcheck"]

Note: for service tasks, an UNHEALTHY container is deregistered from the load balancer and replaced. For standalone tasks (run manually), health status is reported but the task is NOT stopped automatically — health checks are advisory only.

Source: Container health checks · HealthCheck API reference

Container ordering with dependsOn — and the essential trap

dependsOn controls startup order. essential controls what happens when a container exits. Misunderstanding either causes tasks that restart for no obvious reason.

The essential field defaults to true when omitted. An init container that runs a DB migration and exits with code 0 will — by default — stop the entire task. This is the correct behavior for application crashes, but wrong for intentional short-lived containers.

DB migration pattern:

"containerDefinitions": [
  {
    "name": "db-migration",
    "image": "my-app:latest",
    "command": ["python", "manage.py", "migrate"],
    "essential": false,
    "healthCheck": {
      "command": ["CMD-SHELL", "exit 0"],
      "startPeriod": 5
    }
  },
  {
    "name": "app",
    "image": "my-app:latest",
    "essential": true,
    "dependsOn": [
      {"containerName": "db-migration", "condition": "SUCCESS"}
    ],
    "startTimeout": 120
  }
]

startTimeouton the app container (max 120 s on Fargate) sets a deadline: if the migration doesn't exit with SUCCESS within 120 s, the app gives up and the task stops. Without it, a hung migration hangs the entire task forever. Shutdown order is the reverse of startup order — ECS sends SIGTERM to the app first, then the migration container.

Source: ContainerDependency API reference

Linux parameters — what works and what silently does nothing

Fargate accepts several linuxParameters fields without error but ignores them at runtime. Knowing which ones actually work prevents silent failures — especially on ML workloads.

initProcessEnabled: true runs /sbin/docker-init as PID 1. It forwards signals (so SIGTERM actually reaches your app), and reaps zombie processes. Without it, if your app spawns child processes, SIGTERM may not reach them — stopTimeout expires and SIGKILL fires instead of a graceful shutdown.

stopTimeout defaults to 30 seconds, max 120 seconds on Fargate. Set it to match your graceful shutdown time. Combined with initProcessEnabled: true, your app will actually receive SIGTERM and have time to act on it. This directly affects how cleanly environments drain during scheduled shutdowns.

Source: LinuxParameters API reference · KernelCapabilities API reference

Networking in awsvpc mode

Fargate requires networkMode: awsvpc. Each task gets its own elastic network interface with a private IP and security group. Containers within the same task share this ENI and communicate via localhost.

Source: Task networking with the awsvpc network mode

Log configuration — the one line that breaks everything

ECS defaults to the awslogs driver, creates log groups with Never Expire retention, and requires the log group to exist before task launch — or silently fail. Three traps in one feature.

"logConfiguration": {
  "logDriver": "awslogs",
  "options": {
    "awslogs-group":          "/ecs/prod-api",
    "awslogs-region":         "us-east-1",
    "awslogs-stream-prefix":  "ecs",
    "awslogs-create-group":   "true",
    "awslogs-multiline-pattern": "^(ERROR|WARN|INFO)"
  }
}

The awslogs-multiline-pattern option groups multi-line log events — useful for Java stack traces and Python tracebacks. Without it, each line in a stack trace is a separate CloudWatch event, which makes Insights queries harder and slightly increases ingestion cost. Full steps to cut CloudWatch Logs costs on ECS are in the dedicated guide.

FAQ