Set Up Amazon ECR for Edge Container Images

This article shows how to serve the Ataccama edge container images from an Amazon ECR registry in your own AWS account instead of pulling them directly from ataccama.azurecr.io.

The registry_prefix variable that drives this is described in (Optional) Use a custom registry prefix.

Two setups are available:

ECR pull-through cache (recommended): ECR pulls each Ataccama image from ataccama.azurecr.io on first use and caches it. There is no manual copying required and upgrades need no extra steps.
Manual image mirroring: You copy every image into ECR before each installation and upgrade.

Prerequisites

Before you start, make sure you have:

A self-managed edge deployment bundle. See Deploy a Self-Managed Edge.
The container registry credentials provided by Ataccama (username and password for ataccama.azurecr.io). ECR uses these to pull from the upstream registry and they are not passed to the edge.
Permissions in the edge AWS account to create AWS Secrets Manager secrets, ECR pull-through cache rules, and ECR repository creation templates.

How the edge resolves image names

The pre-populated *.auto.tfvars.json files in the bundle reference images by their Ataccama coordinates, for example:

"container_image": "ataccama.azurecr.io/saas-edge/edge-plane-manager:1.8.0-saas-edge"

At plan time, every image reference is rewritten: the ataccama.azurecr.io/saas-edge/ part is stripped and registry_prefix is prepended instead. With registry_prefix set to 111122223333.dkr.ecr.eu-west-1.amazonaws.com/saas-edge/, the preceding reference resolves to:

111122223333.dkr.ecr.eu-west-1.amazonaws.com/saas-edge/edge-plane-manager:1.8.0-saas-edge

The repository path (saas-edge/edge-plane-manager) and the tag are preserved, so your ECR repository names must follow the same structure as the upstream ones.

Set registry_prefix only.

Never edit the image URLs in the pre-populated *.auto.tfvars.json files and never modify the files inside the bundled terraform-aws-edgeinstance/ module. Both are regenerated with every release, so such changes are lost or cause conflicts on the next upgrade.

How ECR authentication works

An ECR registry host has the format <account_id>.dkr.ecr.<region>.amazonaws.com/, where <account_id> is the AWS account that hosts the registry and <region> is the Region it runs in. When registry_prefix starts with a host in this format, ECR is detected automatically and the deployment is adjusted:

No AWS Secrets Manager registry secret is created, and no registry credentials are attached to the container definitions. ECR doesn’t support this authentication method.
Image pulls are authenticated with the IAM permissions of the ECS task execution role. The standard pull permissions (ecr:GetAuthorizationToken, ecr:BatchGetImage, ecr:GetDownloadUrlForLayer, ecr:BatchCheckLayerAvailability) are granted out of the box.
ecr:BatchImportUpstreamImage is granted on the repositories of the registry referenced by registry_prefix, so that images served through a pull-through cache rule are imported on first pull.

These permissions cover container image pulls only. Driver artifacts for the Generic connector are pulled at runtime with the deploy driver job’s task role instead, independently of registry_prefix.

Don’t set registry_secret when registry_prefix points to an ECR registry. Terraform rejects the combination at plan time with registry_secret must not be set when registry_prefix points to an ECR registry.

No IAM changes inside the bundled module are needed for either setup described in this article.

ECR pull-through cache

An ECR pull-through cache rule makes ECR fetch images from ataccama.azurecr.io on demand and keep a cached copy in your account. The edge only ever talks to ECR, and new image versions are picked up automatically on the next deployment.

Together with an ECR repository creation template, the repositories are created by the ECR service-linked role rather than by the caller, so the edge doesn’t need the ecr:CreateRepository permission.

The setup has four steps:

Step 1: Store the Ataccama registry credentials.
Step 2: Create the pull-through cache rule.
Step 3: Create a repository creation template.
Step 4: Point the edge at your ECR registry.

Step 1: Store the Ataccama registry credentials

ECR reads the upstream registry credentials from AWS Secrets Manager. Create a secret in the same Region and account as the ECR registry, with a name that starts with ecr-pullthroughcache/; ECR rejects credentials stored under any other name.

Store the credentials in the following format:

{
  "username": "<USERNAME_FROM_ATACCAMA>",
  "accessToken": "<PASSWORD_FROM_ATACCAMA>"
}

Replace the placeholders with the container registry credentials Ataccama provided for your edge instance.

Step 2: Create the pull-through cache rule

Create a pull-through cache rule that maps an ECR repository prefix to the Ataccama registry:

Upstream registry URL: ataccama.azurecr.io
Upstream repository prefix: saas-edge
ECR repository prefix: saas-edge
Credentials: <secret_arn>

Replace <secret_arn> with the ARN of the secret from Step 1: Store the Ataccama registry credentials. Any ECR repository prefix works, as long as registry_prefix uses the same one in Step 4: Point the edge at your ECR registry.

Step 3: Create a repository creation template

Create a repository creation template for the same repository prefix, applied for PULL_THROUGH_CACHE.

The template defines how repositories are created on the first pull of each image: lifecycle policy, encryption, tag mutability, tags, and, if needed, a repository policy. Because repository creation is delegated to the AWSServiceRoleForECRTemplate service-linked role, the edge’s task execution role doesn’t need ecr:CreateRepository.

Skipping this step breaks pull-through caching. The first pull of each image fails, because creating the cache repository then requires ecr:CreateRepository on the calling principal, which the edge roles deliberately don’t have. Granting that permission by modifying the bundled module is not supported.

Step 4: Point the edge at your ECR registry

In the bundle root, set registry_prefix in registry.auto.tfvars.json to the ECR registry and repository prefix, and leave registry_secret unset:

{
  "registry_prefix": "111122223333.dkr.ecr.eu-west-1.amazonaws.com/saas-edge/"
}

Then continue with the deployment as described in Install the edge.

Terraform example

The following example creates the credentials secret, the pull-through cache rule, and the repository creation template using the public terraform-aws-modules/ecr/aws module. Run it separately from the edge bundle, as part of your own infrastructure code.

Do not use this example for production use.

Review and adjust it to your organization’s requirements before applying it: lifecycle policy, encryption settings, tag mutability, and resource tags. Pay particular attention to the repository policy. Pin the module and provider versions according to your standards.

variable "ataccama_registry_username" {
  type        = string
  description = "The username for the Ataccama registry."
  sensitive   = true
}

variable "ataccama_registry_access_token" {
  type        = string
  description = "The access token for the Ataccama registry."
  sensitive   = true
}

resource "aws_secretsmanager_secret" "ataccama_registry" {
  name                    = "ecr-pullthroughcache/saas-edge"
  recovery_window_in_days = 0
  kms_key_id              = "alias/aws/secretsmanager"
}

resource "aws_secretsmanager_secret_version" "ataccama_registry" {
  secret_id = aws_secretsmanager_secret.ataccama_registry.id
  secret_string = jsonencode({
    username    = var.ataccama_registry_username
    accessToken = var.ataccama_registry_access_token
  })
}

module "ecr_registry" {
  source = "terraform-aws-modules/ecr/aws"

  create_repository = false

  # Registry pull-through cache rules
  registry_pull_through_cache_rules = {
    saas_edge = {
      ecr_repository_prefix      = "saas-edge"
      upstream_registry_url      = "ataccama.azurecr.io"
      upstream_repository_prefix = "saas-edge"
      credential_arn             = aws_secretsmanager_secret.ataccama_registry.arn
    }
  }
}

module "ecr_registry_template" {
  source = "terraform-aws-modules/ecr/aws//modules/repository-template"

  prefix = "saas-edge"

  lifecycle_policy = jsonencode({
    rules = [
      {
        rulePriority = 1,
        description  = "Keep last 5 images",
        selection = {
          tagStatus   = "any",
          countType   = "imageCountMoreThan",
          countNumber = 5
        },
        action = {
          type = "expire"
        }
      }
    ]
  })

  repository_policy = jsonencode({
    Version = "2008-10-17",
    Statement = [
      {
        Sid    = "AllowPull",
        Effect = "Allow",
        Principal = {
          AWS = "arn:aws:iam::111122223333:root"
        },
        Action = [
          "ecr:BatchGetImage",
          "ecr:GetDownloadUrlForLayer"
        ]
      }
    ]
  })
}

Notes on the example:

The prefix of the repository creation template must match the ecr_repository_prefix of the cache rule. The template is applied for PULL_THROUGH_CACHE only, which is what a pull-through cache setup needs.
Repositories created from the template use immutable tags and AES256 encryption by default. If your organization requires a customer-managed KMS key, set the template’s encryption settings and provide a custom IAM role for repository creation, as required by ECR.
The lifecycle policy is an example of how you can limit the number of cached versions per repository.

Ataccama image tags are version-based (for example 1.8.0-saas-edge), so rules that filter by tag prefix don’t match them. Expired images are re-imported from the upstream registry on the next pull.
The repository policy in the example allows pulls by principals in the account whose ID is given in the policy.

Verify the setup

After terraform apply on the edge bundle completes, go to Amazon ECR > Private registry > Repositories and confirm that the repositories contain the tags referenced by the bundle. Repositories such as saas-edge/edge-plane-manager appear only after the first pull of the corresponding image.

Also check CloudWatch > Log groups > /aws/ecs/ata-edge-* for tasks that failed to start.

If a repository is missing, check CloudTrail for the CreateRepository event. With a repository creation template in place, the event is performed by ecr.amazonaws.com through the AWSServiceRoleForECRTemplate role.

Manual image mirroring

Instead of caching on demand, you can copy the images into ECR before each deployment. Choose this option only if pull-through cache rules aren’t permitted in your environment as every installation and every upgrade requires the images to be copied again before Terraform runs.

To mirror the images:

Create one ECR repository per edge image, preserving the upstream repository path under a prefix of your choice, for example saas-edge/edge-plane-manager.
Copy each image and tag referenced in the bundle’s *.auto.tfvars.json files from ataccama.azurecr.io/saas-edge/ into the matching ECR repository.

Any registry-to-registry copy works, for example docker pull, docker tag, and docker push, or a tool such as crane or skopeo.
Set registry_prefix to your ECR registry and prefix, exactly as in Step 4: Point the edge at your ECR registry, and leave registry_secret unset.

Every image and tag referenced by the bundle must exist in ECR before terraform apply, otherwise the ECS tasks fail to start with an image pull error.

Troubleshooting Amazon ECR setup

Tasks fail with CannotPullContainerError

Check the ECS task’s Stopped reason and the CloudWatch logs of the affected task definition.

Repository does not exist: The pull-through cache rule prefix and the registry_prefix path don’t match, or no repository creation template exists. See Step 3: Create a repository creation template.
Image not found: With a mirrored setup, the exact tag referenced by the bundle hasn’t been copied into ECR. See Manual image mirroring.
Authentication or authorization errors on the upstream registry: The credentials in the pull-through cache secret are invalid, or the secret name doesn’t start with ecr-pullthroughcache/. See Step 1: Store the Ataccama registry credentials.

Repository creation fails with AccessDenied on ecr:CreateRepository

The pull-through cache rule has no matching repository creation template, so ECR attempts to create the repository with the permissions of the calling principal — the edge’s ECS task execution role — which doesn’t have ecr:CreateRepository.

Create a repository creation template for the repository prefix, as described in Step 3: Create a repository creation template. Don’t add the permission by editing the bundled terraform-aws-edgeinstance/ module: such changes are lost on the next upgrade.