Deploy a Self-Managed Edge

This guide describes how to deploy a self-managed Ataccama edge instance in your own AWS account using Terraform.

See Edge Processing for background on edge processing and how responsibilities are split between Ataccama and your organization.

The deployment splits into four phases:

You set up AWS infrastructure (VPC, S3 bucket, IAM role) and register the edge instance in the Cloud Portal.
Ataccama provides a Terraform deployment package customized for your environment.
You run Terraform from your workstation or CI/CD runner to deploy the edge into your account.
Ataccama confirms control plane connectivity, and the edge is registered.

Prerequisites

Complete Prepare AWS Infrastructure before starting.

After completing AWS infrastructure preparation, use those values to register the edge instance directly in the Cloud Portal.

In the Cloud Portal, open your environment and select the Settings tab. In the Edge section, switch to the Self-managed tab.

Select Register Edge instance and fill in the values from your AWS infrastructure preparation.

Registering the edge instance generates a deployment package. Your Ataccama Customer Success Manager then provides you with the edge deployment ZIP to run in your AWS account.

Tooling

Install the following on the machine where you will run Terraform — your workstation or a CI/CD runner:

Terraform CLI version 1.14 or later. See Install Terraform
AWS CLI v2 latest version. See Install the AWS CLI.

Confirm the AWS CLI is authenticated before proceeding:

aws sts get-caller-identity

Workstation network access

The machine running Terraform needs outbound HTTPS access to:

registry.terraform.io: For downloading Terraform providers (for example, the AWS provider).
ataccama.azurecr.io: For pulling container images and OCI artifacts.
AWS service endpoints (ECS, IAM, S3, SQS, Lambda, CloudWatch): For Terraform AWS provider API calls.

IAM permissions for Terraform

The IAM principal used to run Terraform needs administrative access to the target AWS account, or at minimum permissions to create and manage the following resources:

ECS
IAM
S3
SQS
Lambda
EFS
KMS
CloudWatch

Ataccama doesn’t receive an IAM role in your account. Cross-account communication is initiated outbound by the edge using an IAM role created by Terraform and scoped to the SQS operations required for control plane messaging.

Deployment artifacts

Before starting installation, confirm you have received the following materials from Ataccama:

Edge deployment ZIP via secure download. Contains Terraform manifests, component configurations, Lambda artifacts, and the bundled terraform-aws-edgeinstance module.

Bucket name, IAM role ARN, and Region are pre-populated from values you supplied during onboarding.
Container registry credentials via secure credential sharing. Username and password for ataccama.azurecr.io. Unique to your edge instance.

Store credentials in a secrets manager or a .tfvars file that is never committed to source control.

Each credential set is scoped to a single edge instance; do not reuse across deployments.

Bundle contents

The edge deployment ZIP contains all Terraform configuration and component definitions.

Do not modify the bundled module or the preconfigured variable files.

<edge_name>-<version>-<timestamp>-bundle.zip
├── main.tf                          # Root Terraform configuration
├── terraform.tf                     # Provider version constraints
├── variables.tf                     # Input variable declarations
├── outputs.tf                       # Output definitions
├── terraform.tfvars.json            # Edge + cluster configuration (pre-populated)
├── *.auto.tfvars.json               # Component configurations (pre-populated, ~10 files)
├── registry.auto.tfvars.json        # Container registry credentials — you fill this in
├── vpc-endpoint-policy.json         # VPC endpoint policy — apply to customer-provisioned VPC endpoints (see VPC endpoints in the Prepare AWS Infrastructure guide)
├── artifacts/                       # Lambda deployment packages
│   ├── edgeinteractivejobs/*.zip
│   └── dqresultsreader/*.zip
└── terraform-aws-edgeinstance/      # Bundled Ataccama module — do not modify

Install the edge

Step 1: Fill in container registry credentials

Extract the edge deployment ZIP, open registry.auto.tfvars.json, and fill in the credentials provided by Ataccama:

{
  "registry_secret": {
    "username": "<username-from-ataccama>",
    "password": "<password-from-ataccama>"
  }
}

To avoid writing secrets to disk, export them as an environment variable:

TF_VAR_registry_secret='{"username":"...","password":"..."}'

If you pull images from Amazon ECR (see (Optional) Use a custom registry prefix), do not provide registry_secret — leave it unset.

ECR does not support registry credentials; authentication uses the ECS task execution role’s IAM permissions instead, which the module grants automatically. Setting registry_secret together with an ECR registry_prefix fails validation.

(Optional) Use a custom registry prefix

By default, container images are pulled from ataccama.azurecr.io/saas-edge/. To pull from your own registry or pull-through cache, set registry_prefix in registry.auto.tfvars.json:

{
  "registry_prefix": "internal-registry.corp.local/saas-edge/",
  "registry_secret": {
    "username": "",
    "password": ""
  }
}

With this example, image references resolve as follows:

ataccama.azurecr.io/saas-edge/dqc-runtime-job:16.6.5-saas-edge
→ internal-registry.corp.local/saas-edge/dqc-runtime-job:16.6.5-saas-edge

Amazon ECR

When registry_prefix points to an Amazon ECR registry (host format <account-id>.dkr.ecr.<region>.amazonaws.com/), omit registry_secret entirely. ECR authentication is handled through IAM on the ECS task execution role, no username and password are used:

{
  "registry_prefix": "111122223333.dkr.ecr.eu-west-1.amazonaws.com/saas-edge/"
}

Step 2: Configure optional parameters

The deployment runs without any changes for most environments. Some AWS accounts have organizational requirements that need an additional variable to be set.

Set optional parameters by creating a new .auto.tfvars.json file in the bundle root. Terraform automatically loads any file matching .auto.tfvars.json, so you don’t need to edit the pre-populated variable files.

If none of the following applies to your environment, continue to the next step.

(Optional) Attach an IAM permissions boundary

If your organization requires an IAM permissions boundary on every IAM role, set iam_permissions_boundary_arn to the ARN of your boundary policy. Terraform then attaches it to every IAM role the edge module creates: ECS task and task execution roles, Lambda execution roles, the cleanup scheduler role, and the EventBridge target role.

Create a file named, for example, permissions-boundary.auto.tfvars.json in the bundle root:

{
  "iam_permissions_boundary_arn": "arn:aws:iam::<your-account-id>:policy/<boundary-policy-name>"
}

When the variable is left unset, no permissions boundary is attached and role creation is unchanged.

The permissions boundary must allow the actions the edge roles need (ECS, SQS, S3, KMS, Secrets Manager, CloudWatch Logs, STS, and Lambda operations). A boundary that is too restrictive can let terraform apply succeed while the edge fails at runtime.

(Optional) Disable the S3 public access block

By default, the edge creates an S3 public access block (all four block settings enabled) on its S3 buckets. If your organization denies the s3:PutBucketPublicAccessBlock action through a Service Control Policy (SCP), creating this resource fails and the deployment stops with an AccessDenied error.

To opt out, set s3_block_public_access to false by creating a new file, for example s3.auto.tfvars.json, in the bundle root:

{
  "s3_block_public_access": false
}

This is safe: AWS blocks public access on all new buckets by default (since April 2023), so the buckets stay non-public without the explicit resource.

(Optional) Restrict security group egress

By default, the edge security groups allow broad outbound access: HTTPS (443) to 0.0.0.0/0 on every component, in addition to — on the processing job components — HTTP (80, for Snowflake JDBC OCSP checks) and TCP 1024–65535 to 0.0.0.0/0 for reaching customer data sources on any port. If your organization requires tightly scoped outbound access, two variables let you replace these rules with your own narrowed set:

vpc_external_egress_rules: General egress (container registry, platform and AWS APIs), applied to every edge component.
vpc_external_datasource_egress_rules: Data source egress (Snowflake OCSP, JDBC), applied only to the processing job components that connect to your data sources.

Each rule targets exactly one of cidr_ipv4, cidr_ipv6, prefix_list_id, or referenced_security_group_id (for example, a VPC endpoint prefix list instead of the open internet), and ip_protocol defaults to tcp.

To override, create a new file, for example egress.auto.tfvars.json, in the bundle root:

{
  "vpc_external_egress_rules": {
    "aws_and_platform_apis": {
      "description": "AWS and platform APIs via VPC endpoints",
      "from_port": 443,
      "to_port": 443,
      "prefix_list_id": "pl-0123456789abcdef0"
    },
    "container_registry": {
      "description": "Ataccama container registry",
      "from_port": 443,
      "to_port": 443,
      "cidr_ipv4": "0.0.0.0/0"
    }
  },
  "vpc_external_datasource_egress_rules": {
    "corporate_postgres": {
      "description": "Corporate PostgreSQL",
      "from_port": 5432,
      "to_port": 5432,
      "cidr_ipv4": "10.20.0.0/24"
    }
  }
}

When both variables are left unset, these built-in rules are used and behavior is unchanged.

Setting a variable fully replaces its default rules — the values are not merged with the built-ins. Include every destination the edge still needs, or it will fail at runtime: HTTPS (443) to the container registry and the platform and AWS APIs, HTTP (80) if you use public Snowflake as a data source, and one rule per data source. See Edge runtime network access for the full list of outbound destinations.

A map key used in vpc_external_datasource_egress_rules must not also appear in vpc_external_egress_rules; the two are merged on the job components and a shared key would drop the general rule. Terraform rejects this at plan time.

Step 3: Configure a Terraform backend

Configure a Terraform backend according to your organization’s standards — for example, an S3 backend with DynamoDB state locking. Add a backend block to terraform.tf or create a separate backend configuration file.

This stores your Terraform state remotely and enables collaboration and state locking.

Step 4: Initialize Terraform

From the root of the extracted bundle directory:

cd <extracted-bundle-directory>
terraform init

The Ataccama edge module is bundled locally; no external module registry access is required. However, access to the public Terraform Registry is required to download Terraform providers (such as the AWS provider).

Expected output

Initializing the backend...
Initializing provider plugins...
- Installing hashicorp/aws ...
Terraform has been successfully initialized!

Step 5: Review and apply

terraform plan -out=edge.tfplan

Review the planned resources. Terraform creates resources in your AWS account only, including:

ECS cluster and Fargate task definitions.
IAM roles and policies (scoped to your account; no Ataccama access).
SQS queues for control plane communication.
Lambda functions for auxiliary processing jobs.
CloudWatch log groups.
VPC security groups.

When satisfied with the plan, apply it:

terraform apply edge.tfplan

Deployment typically completes in 10–20 minutes. Don’t interrupt the process once it has started.

Terraform waits for ECS tasks to start successfully before completing.

Step 6: Verify and register

After terraform apply completes:

Navigate to CloudWatch > Log groups > /ataccama/edge/ and confirm log streams are being written without repeated errors.
Email your Ataccama contact with:
- The edge name (from Terraform outputs).
- Your AWS account ID and Region.
- Confirmation that terraform apply completed without errors.

Ataccama will verify control plane connectivity and confirm the edge is registered.

Terraform waits for ECS tasks to start successfully during the apply process. If it completed without errors, the main workloads are running.

Configure data sources to use the edge

When creating or editing your data source connection, select the edge instance you want to use. All edge instances available for your environment appear in this list.

Select Allow edge export if you want to load data processed on this connection’s edge to Reference Data. If it isn’t selected, edge-processed data remains within the edge boundary. For details, see Allow edge export.

Select edge instance in data source connection

Edge runtime network access

All edge traffic is outbound from your VPC. No inbound firewall rules, VPN tunnels, or peering connections are required.

The Terraform module does not create VPC endpoints for self-managed edges. By default, AWS API traffic and traffic to ataccama.azurecr.io leave your VPC through the NAT gateway.

We recommend provisioning your own VPC endpoints to keep AWS API traffic on the AWS network:

See VPC endpoints for setup details. NAT-only egress is supported but highly discouraged. Although AWS API calls are TLS-encrypted in transit, NAT routing exposes them to public-internet paths and incurs NAT data-transfer charges. Only use it if you cannot provision VPC endpoints in your environment.

Allow the following outbound destinations:

Destination Protocol Port Purpose

Destination	Protocol	Port	Purpose
`ataccama.azurecr.io`	HTTPS	443	Pull container images and OCI artifacts.
AWS SQS (your Region)	HTTPS	443	Control plane task exchange (cross-account IAM).
AWS S3 (your Ataccama environment’s Region)	HTTPS	443	Data exchange with the Ataccama-managed cloud bucket via presigned URLs.
AWS service APIs	HTTPS	443	S3, KMS, Secrets Manager, SQS, CloudWatch Logs, STS, CloudWatch, ECS, IAM, and Lambda API calls. Routed through the NAT gateway or, if provisioned, through your VPC endpoints.

ataccama.azurecr.io

HTTPS

443

Pull container images and OCI artifacts.

AWS SQS (your Region)

HTTPS

443

Control plane task exchange (cross-account IAM).

AWS S3 (your Ataccama environment’s Region)

HTTPS

443

Data exchange with the Ataccama-managed cloud bucket via presigned URLs.

AWS service APIs

HTTPS

443

S3, KMS, Secrets Manager, SQS, CloudWatch Logs, STS, CloudWatch, ECS, IAM, and Lambda API calls.

Routed through the NAT gateway or, if provisioned, through your VPC endpoints.

For provisioning details — including the endpoint policy to apply and how to keep cross-region traffic private when the edge runs in a different AWS region than your Ataccama control plane — see VPC endpoints.

Data source connectivity

Data sources must be reachable from the edge’s ECS security group on the appropriate port.

Terraform doesn’t configure this connectivity — you’re responsible for routing and firewall rules between the edge VPC and your data source endpoints.

Upgrade the edge

Your edge version remains supported for 90 days after a new release is available. When your version is approaching the end of this window, a warning is displayed in the Ataccama Cloud Portal.

Upgrades are not applied automatically; you control the timing.

Exceeding the 90-day supported window might result in degraded functionality or loss of control plane connectivity.

Don’t skip versions: apply each release in sequence. If you have missed multiple versions, contact Ataccama Support before proceeding.

To upgrade:

Download the new edge deployment ZIP from the Cloud Portal.
Extract it into a new directory. Keep the previous directory as a backup.
Open registry.auto.tfvars.json and fill in your container registry credentials (same as for the initial installation).
Configure the Terraform backend. Must match the backend used for the initial installation.

Run the following sequence:

terraform init
terraform plan -out=edge.tfplan
terraform apply edge.tfplan

Terraform applies only the changes between the previous and new versions. Resources that haven’t changed are not touched and no data is lost during an upgrade.

Observability

Observability ships telemetry to Ataccama’s monitoring stack using the configuration included in the edge deployment ZIP.

Troubleshooting edge deployment

terraform init fails: module not found

Run terraform init from the root of the extracted directory, where main.tf is located. The terraform-aws-edgeinstance/ subdirectory must be present alongside main.tf.

terraform apply fails with UnauthorizedAccess

Run aws sts get-caller-identity to confirm which IAM principal is active. Verify it has the permissions listed in IAM permissions for Terraform.

Deployment fails with AccessDenied when creating the S3 public access block

Running terraform apply fails with an AccessDenied error while creating aws_s3_bucket_public_access_block when your organization denies the s3:PutBucketPublicAccessBlock action through a Service Control Policy (SCP).

Opt out by setting s3_block_public_access to false, then run terraform apply again. For instructions, see (Optional) Disable the S3 public access block.

ECS tasks stuck in PENDING or immediately STOPPED

Check CloudWatch Logs for the affected task. Common causes include:

Incorrect container registry credentials: Verify username and password in registry.auto.tfvars.json.
No internet egress: Confirm the subnet’s route table points to a NAT gateway and that outbound HTTPS (port 443) is permitted by security groups and network ACLs.
SCP or firewall blocking egress: If your organization enforces AWS Service Control Policies, ensure ECS task roles aren’t blocked from calling SQS or pulling from ataccama.azurecr.io.

Edge not showing as connected in Cloud Portal after 15 minutes

Check CloudWatch Logs for SQS connectivity errors.

Contact Ataccama Support with your edge name and the relevant log output.

Contacting Ataccama Support

When contacting Ataccama Support, provide:

Edge name (visible in the Cloud Portal and in Terraform outputs).
AWS Region and account ID.
CloudWatch log excerpts.
Edge version (from the ZIP filename).

Destroy the edge

terraform destroy permanently deletes all edge AWS resources.

The resources Terraform creates hold only technical, re-provisionable content (artifact and configuration storage and the EFS cache), which destroy completely removes. The DQ-encryption KMS key is scheduled for deletion with a 7-day recovery window; if you need to restore DQ results encrypted under it, cancel the scheduled deletion in your AWS account within that window.

Before proceeding, save any other data you need from the edge environment. Notify Ataccama afterwards so the edge registration can be removed from the control plane.

terraform destroy

Appendix: AWS resources deployed by Terraform

The following AWS resources are created by Terraform in your account.

Foundation (always deployed)

KMS: Two customer-managed keys (general and DQ-encryption), plus a KMS alias. Rotation every 90 days, 7-day deletion window.

For how these keys are used and why, see Encryption and key management.
Secrets Manager: One secret holding the Ataccama container registry credentials (KMS-encrypted), used when the Ataccama registry is accessed directly.

The credential is issued per edge instance and can be re-issued by Ataccama at any time. The secret is deleted immediately on uninstall (no recovery window) to keep terraform destroy clean. Keep your own backup if your policy requires one.
S3: One bucket for Lambda artifacts. Versioned, KMS-encrypted; lifecycle rule expires old versions after seven days.

Holds only Lambda deployment artifacts shipped with each release and no customer data. Contents are fully recreated on the next deployment, so the bucket is removed together with the installation when the edge is destroyed.

An explicit per-bucket public access block (all four block settings enabled) is created by default on the edge’s S3 buckets. If your organization denies the s3:PutBucketPublicAccessBlock action through an SCP, the deployment fails; opt out with s3_block_public_access = false, see Deployment fails with AccessDenied when creating the S3 public access block. AWS blocks public access on all new buckets by default (since April 2023), so the buckets stay non-public either way.
EFS: One file system, KMS-encrypted. Two access points (drivers, otel). Mount target per private subnet. Dedicated security group with NFS ingress rules from each workload.

Mounts Ataccama connectors to processing jobs and the metadata-browsing Lambda, and collects observability data before shipping (when enabled).

The file system is a distribution cache and temporary workspace only. Content is re-downloaded automatically, which is why AWS Backup is not enabled for it.
VPC endpoints: Not created by the Terraform module for self-managed edges. Customers provision their own (recommended) or rely on NAT egress; see VPC endpoints.

ECS management cluster

Plane Manager: Permanent ECS service, two replicas. Owns its security group and IAM task role.

Consumes the local SQS job-status queue and uses sts:AssumeRole to reach both the Ataccama edge-access role and the customer result-S3 role.
Connectors Rollout: Task definition only; no permanent service. Run on demand by EventBridge when the container image changes. Mounted to the EFS drivers access point.
Observability service An ECS service running an OpenTelemetry collector, CloudWatch Exporter, and config-init container. S3 bucket stores configuration; EFS provides supporting storage.

ECS job cluster

Processing jobs: Six task definitions (DQC, anomaly detection, anomaly detection auxiliary, metadata import, Snowflake pushdown, create-table). Ephemeral — started on demand by the Plane Manager, stopped when work completes.
One shared security group for all job tasks, egress any.
Per task definition: Task execution role, task role, CloudWatch log group.

Lambda functions

Metadata Browsing / Connection Testing: Java 21, 1.3 GB, x86_64, VPC-attached. Mounts the EFS drivers access point.

Alias current plus provisioned concurrency. Event-source mapping on the Ataccama-managed SQS request queue.
DQ Results Reader: Java 21, 1 GB, x86_64, VPC-attached. Alias current plus provisioned concurrency.

Event-source mapping on the Ataccama-managed result-reader SQS queue.
Cleanup: Python 3.12, non-VPC, no EFS. Invoked daily by EventBridge Scheduler. Trims old Lambda versions (keeps three).

Messaging and events (local)

SQS: One main queue and one DLQ for ECS job-status-changed events. KMS-encrypted. Max-receive 3 → DLQ.
EventBridge rule: ECS Task State Change on the job cluster forwards to the local SQS queue.
EventBridge rule: Custom event com.ataccama.edge.connectorsrollout / image_update triggers ecs:RunTask on Connectors Rollout.
EventBridge Scheduler: Daily invocation of the Cleanup Lambda via a dedicated scheduler IAM role.

CloudWatch

Log group per ECS service, per job task definition, and per Lambda.
Container Insights on both ECS clusters.

IAM

Terraform creates the IAM roles needed for services to function and communicate. No IAM role is granted to Ataccama inside the customer account.

Was this page useful?