If you manage AWS infrastructure with code, the European Sovereign Cloud adds a new partition to think about. Different endpoints, separate IAM, its own console. This guide covers what works out of the box, what needs changes, and the patterns that help when you deploy across both ESC and commercial AWS.

Why This Exists

AWS has had European regions since 2007. Ireland came first, then Frankfurt, London, Paris, Stockholm, Milan, Zurich, Spain. Eight regions across Europe. Data stays in Europe. GDPR compliant. Problem solved, right?

Not quite.

Here's the thing about eu-central-1 (Frankfurt) — your data sits in Germany, sure. But AWS operations? Support tickets? Billing metadata? That stuff flows through global systems. American employees can access it. The control plane lives in the US. When you call support at 3am, someone in Seattle might answer.

For plenty of companies, that's fine. You're running a SaaS product, your customers don't care where the ops team sits. But for German government agencies processing citizen data? French hospitals handling patient records? Banks under BaFin scrutiny? They've been asking harder questions.

The US Cloud Act made it worse. Passed in 2018, it lets American authorities compel US companies to hand over data, even if that data sits on servers in Frankfurt. Doesn't matter where the bits are physically stored — if an American company controls them, American courts can demand them. AWS has always pushed back on these requests, but "trust us, we'll fight it" isn't the same as "technically impossible."

Then came Schrems II in 2020, when the EU Court of Justice invalidated Privacy Shield. Suddenly every European company using American cloud providers had to justify why their data transfers were legal. Standard contractual clauses helped, but the legal uncertainty never fully went away.

That's the gap ESC fills. Not just "data in Europe" but "everything in Europe" — operations, support, billing, leadership, legal jurisdiction.

What's Actually Different

The European Sovereign Cloud is a separate partition entirely. Not a region — a partition. Like how AWS GovCloud is separate from commercial AWS, or how China regions are isolated. Different domain (amazonaws.eu instead of amazonaws.com), different IAM system, different control plane.

The region code is eusc-de-east-1, sitting in Brandenburg, Germany. The partition identifier is aws-eusc. When you construct ARNs, it's arn:aws-eusc: not arn:aws:.

AWS set up a new German parent company to run it — AWS European Sovereign Cloud GmbH — with three subsidiaries handling infrastructure, certificates, and employment. The managing directors are Stéphane Israël (former CEO of Arianespace) and Stefan Hoechbauer (VP of AWS Germany), both EU citizens based in the EU. The board includes independent third-party representatives specifically for sovereignty oversight. Not Amazon employees — actual independent oversight.

Only EU residents work there. Not just "based in Europe" — actually residing in the EU with EU contracts. And going forward, they're only hiring EU citizens. The transition is gradual, but the end state is clear: EU citizens only, no exceptions. No "follow-the-sun" support routing your ticket to Virginia at 3am.

When AWS says the infrastructure has "no critical dependencies on non-EU infrastructure", they mean it literally. The system can keep running even if someone cuts the transatlantic cables. Billing systems, metering engines, security operations center — all contained within the EU. Metadata created in ESC stays in ESC. Your usage data doesn't flow to a US billing system.

The Security Foundation

This matters more than the org chart stuff, honestly. Legal structures can change. Technical architecture is harder to undo.

ESC runs on the Nitro System, same as regular AWS. But the Nitro architecture is what makes the sovereignty claims credible. It's not just policy — it's hardware design.

The Nitro System was built with zero operator access as a design goal. There's no SSH into the hypervisor. No console access. No mechanism for AWS employees — or anyone — to access EC2 instance memory or customer data on encrypted storage. When they say "no backdoors", it's not a policy promise, it's a constraint enforced by the silicon.

Administrative access happens through authenticated, authorized, and logged APIs that provide no path to customer data. You can audit operations without giving operators data access. These restrictions are built into the Nitro firmware itself. Not a software toggle someone can flip during an emergency or under legal pressure.

NCC Group, an independent security firm, validated these claims in an audit published May 2023. They specifically looked for gaps that would let someone access customer data or memory. Found none. That audit applies to Nitro everywhere, including ESC.

For ESC specifically, AWS added the Sovereignty Reference Framework (ESC-SRF). It's an independently validated framework with third-party auditor reports documenting the sovereignty controls. Your compliance team can hand these reports to regulators instead of trying to explain AWS architecture themselves.

The Catch (There's Always a Catch)

You can't just add ESC to your existing AWS Organization and call it a day. This is a separate cloud, and that separation creates friction.

Separate console, separate login. ESC has its own management console on the amazonaws.eu domain, separate from console.aws.amazon.com. Different URL, different accounts, different credentials. You can't switch between ESC and commercial AWS with the account dropdown — they're completely separate consoles. Bookmark both if you work in both.

No cross-partition IAM. Can't assume roles from your regular AWS account into ESC. If you have workloads in both places, you need separate identity management. Set up federation through a third-party IdP like Okta or Azure AD, maintain separate credentials, design your CI/CD to handle both partitions. Your developers need two sets of AWS credentials.

No VPC peering. Want to connect eu-central-1 to ESC? Treat it like connecting to on-premises infrastructure. VPN, Direct Connect, or application-level APIs. You're bridging two clouds, not two regions. Network architects used to multi-region deployments need to reset their mental model.

Separate accounts entirely. Different accounts, different Organizations, different invoices, different cost allocation tags. If your finance team tracks cloud spend by AWS account ID, they need new processes. Your existing FinOps dashboards won't see ESC spend.

ECR isolation. You can't pull container images from your existing ECR repos in eu-central-1. ESC's isolation means no cross-partition image pulls. Push your images to ECR in eusc-de-east-1, use a public registry, or set up replication through your CI/CD pipeline.

Terraform works, but check your version. Terraform 1.14+ and AWS provider 6.x support ESC natively — endpoints resolve correctly without manual configuration. Just set the region:

provider "aws" {
  region = "eusc-de-east-1"
}

If you're on an older version, you'll need to upgrade or configure endpoints manually. The S3 backend for state storage also requires Terraform 1.14+.

What Services Are Available

AWS didn't launch this with five services and a "coming soon" page. You get 90+ services from day one. That matters because previous sovereign cloud offerings often meant accepting a skeleton service catalog.

Containers: ECS, EKS, ECR. Full Fargate support. If you're running containers anywhere on AWS today, same capabilities.

Compute: EC2 with multiple instance families, Lambda for serverless. Enough instance types for most workloads.

AI/ML: Bedrock, SageMaker, Amazon Q. All available from day one.

Database: Aurora (MySQL and PostgreSQL compatible), DynamoDB, RDS for managed databases. All the usual engines.

Storage: S3 with full feature parity, EBS for block storage.

Networking: VPC, Direct Connect, Route 53 for private hosted zones. Transit Gateway for complex topologies.

Security: KMS for encryption keys, Secrets Manager, Private CA, IAM with all the normal features.

If you're running containers on Fargate in Frankfurt today, you can run the same workloads on ESC. Same task definitions, same service configs, just different region and endpoints.

What's Missing

90 services sounds good until you remember AWS has 240+. Some gaps matter more than others:

CloudFront — No CDN at launch. If your architecture relies on edge caching, you'll need alternatives. Expected end of 2026.

IAM Identity Center — The modern way to manage SSO across an Organization isn't there yet. You can still use IAM with external identity providers, but you'll configure it per-account instead of centrally. Expected Q1 2026.

Shield Advanced & Firewall Manager — DDoS protection and centralized firewall rules aren't available. Basic Shield is included, but advanced protections aren't.

Amazon Inspector — No automated vulnerability scanning for workloads yet.

GuardDuty — Available but limited. No Organization-level management, missing some newer detection capabilities.

IoT Services — IoT Core, Greengrass, and related services aren't included. If you're running IoT workloads, ESC isn't ready for them.

Organizations features — You get AWS Organizations, but delegated administration isn't supported. StackSets and other governance tools must run from the Management Account.

Also worth noting: S3 Block Public Access isn't enabled by default like it is in commercial AWS. Enable it manually.

Pricing: Expect 10-15% premium over Frankfurt (eu-central-1) for comparable services.

Deploying Containers — The Practical Bits

The patterns are identical to regular AWS. I'm not going to paste hundreds of lines of Terraform — you know how to deploy ECS. The differences are configuration, not architecture:

1. Region: eusc-de-east-1

2. ARNs use aws-eusc partition: arn:aws-eusc:iam::aws:policy/...

3. ECR images must come from ESC or public registries

4. Tag resources with compliance markers for your auditors

A minimal ECS task definition:

resource "aws_ecs_task_definition" "app" {
  family                   = "my-app"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  cpu                      = "256"
  memory                   = "512"
  execution_role_arn       = aws_iam_role.execution.arn

  container_definitions = jsonencode([{
    name  = "app"
    image = "your-ecr.eusc-de-east-1.amazonaws.eu/app:latest"
    portMappings = [{ containerPort = 80 }]
    logConfiguration = {
      logDriver = "awslogs"
      options = {
        "awslogs-group"  = "/ecs/my-app"
        "awslogs-region" = "eusc-de-east-1"
      }
    }
  }])
}

VPC setup is standard — public subnets for load balancers, private subnets for tasks, NAT gateways for outbound traffic. Security groups, ALB config, service definitions — all identical to what you'd write for Frankfurt.

Infrastructure as Code: The Real Story

If you're managing infrastructure with code (and you should be), here's what actually works with ESC right now.

Terraform and OpenTofu

As mentioned, Terraform 1.14+ handles ESC out of the box. But there's more to it than just setting the region. The aws_partition data source correctly returns aws-eusc, which is useful when you're building partition-aware modules:

data "aws_partition" "current" {}

# Returns "aws-eusc" in ESC, "aws" in commercial
output "partition" {
  value = data.aws_partition.current.partition
}

For multi-partition deployments, use provider aliases:

provider "aws" {
  alias  = "esc"
  region = "eusc-de-east-1"
}

provider "aws" {
  alias  = "commercial"
  region = "eu-central-1"
}

# Deploy to ESC
resource "aws_s3_bucket" "sovereign_data" {
  provider = aws.esc
  bucket   = "my-sovereign-bucket"
}

# Deploy to commercial
resource "aws_s3_bucket" "public_assets" {
  provider = aws.commercial
  bucket   = "my-public-bucket"
}

OpenTofu 1.11+ also supports ESC natively, including the S3 backend in eusc-de-east-1. Confirmed working by community testing in December 2025. If you've switched to OpenTofu, same patterns apply.

AWS CDK

CDK supports ESC since August 2025. Region registration for eusc-de-east-1 and VPC endpoint handling were added in PR #34860. No workarounds needed — just set the region:

const app = new cdk.App();
const stack = new cdk.Stack(app, 'EscStack', {
  env: {
    account: '123456789012',
    region: 'eusc-de-east-1',
  },
});

ARNs, service endpoints, and partition references resolve correctly out of the box.

CloudFormation

Works as expected. CloudFormation is partition-aware by design, so templates deploy without modification. The AWS::Partition pseudo parameter returns aws-eusc automatically.

Resources:
  MyRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service: ecs-tasks.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        # Automatically uses aws-eusc partition
        - !Sub "arn:${AWS::Partition}:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"

One exception: Landing Zone Accelerator doesn't work. LZA maps to a single AWS Organization and can't span partitions. You'll need separate LZA deployments for ESC and commercial, with duplicated configurations.

Multi-Partition Patterns

Running workloads in both ESC and commercial AWS? Here are patterns that work:

Shared modules with partition-aware variables:

variable "partition" {
  description = "AWS partition (aws or aws-eusc)"
  type        = string
}

variable "region" {
  description = "AWS region"
  type        = string
}

locals {
  is_sovereign = var.partition == "aws-eusc"

  # Adjust for service availability
  enable_cloudfront    = !local.is_sovereign  # Not available in ESC yet
  enable_guardduty_org = !local.is_sovereign  # Limited in ESC
}

Separate state files per partition:

# ESC backend
terraform {
  backend "s3" {
    bucket = "my-tfstate-esc"
    key    = "infrastructure/terraform.tfstate"
    region = "eusc-de-east-1"
  }
}

Don't try to share state across partitions. The isolation is the point.

CI/CD branching strategy:

Some teams run completely separate pipelines per partition. Others use a single pipeline with partition as a variable. The right choice depends on how different your ESC and commercial configurations are. If they're mostly identical, one pipeline with environment variables works. If they diverge significantly, separate pipelines prevent accidents.

Planning Your Architecture

If you're considering ESC, think about workload segmentation early. Not everything needs sovereignty guarantees, and putting everything in ESC when it doesn't need to be there adds cost and complexity.

Tier 0 — Sovereign (ESC): Sensitive data requiring sovereignty guarantees. Patient health records, citizen personal data, financial records under regulatory requirements, classified government workloads. This is your ESC tier.

Tier 1 — Standard (Commercial AWS or ESC): Business data without special regulatory requirements. Internal tools, development environments, public-facing websites, marketing systems.

The hard part is the boundary. Your sovereign tier probably needs data from the standard tier sometimes. Options:

API gateways at the boundary. ESC workloads call commercial AWS through a controlled API layer. Strict authentication, audit logging, minimal data exposure. The API becomes your compliance checkpoint.

Data diodes for one-way flow. ESC can pull data from commercial AWS on a schedule. Commercial can't push to ESC. Useful for reference data that needs to be in ESC but originates elsewhere.

Message queues with encryption. Async communication through something like SQS or external message brokers. Decouples the systems while maintaining the boundary.

Don't try to architect this like multi-region. It's multi-cloud, practically speaking. Your eu-central-1 workloads can't directly call your ESC workloads over private networking. Plan for that from day one, not as an afterthought.

Migration Path

If you're moving existing workloads to ESC, here's a rough sequence:

Phase 1: Assessment. Which workloads actually need sovereignty? Many teams discover only 20-30% of their infrastructure handles truly sensitive data. Don't move everything just because you can.

Phase 2: Identity setup. Get your IAM structure in ESC before anything else. Set up federation, create roles, establish your permission model. Test authentication flows.

Phase 3: Network foundation. VPC, subnets, NAT gateways, security groups. If you need connectivity back to commercial AWS, set up the VPN or Direct Connect tunnel.

Phase 4: Container registry. Push your images to ECR in ESC. Update your CI/CD to build and push to both registries if you're running in both partitions.

Phase 5: Workload deployment. Start with non-critical workloads to validate your Terraform and deployment pipelines. Work through the endpoint configuration issues before touching production.

Phase 6: Data migration. This is usually the hardest part. How do you move data without downtime? Often involves running parallel systems temporarily, with replication from source.

Phase 7: Cutover. Switch traffic to ESC workloads. Keep the old deployment running until you're confident, then decommission.

Cost Reality

ESC pricing follows standard AWS models — you pay for what you use. But the isolation adds costs:

NAT Gateways: ~€0.045/hour each plus data processing. High availability means two gateways, roughly €65/month before data charges. You're paying this in Frankfurt too, but now you're paying it twice if you have workloads in both partitions.

Data transfer between partitions: Not free internal transfer. Treat it like cross-region or internet egress. If your architecture involves heavy data movement between ESC and commercial AWS, model those costs.

Operational overhead: Managing two partitions means duplicated effort. Two sets of IAM policies, two CI/CD pipelines, two monitoring dashboards, two on-call rotations if you have partition-specific issues. That's engineering time.

Compliance tooling: You'll probably want separate security scanning, compliance monitoring, and audit tooling for ESC. Or tools that understand both partitions. Either way, cost.

AWS has confirmed a 10-15% pricing premium over Frankfurt for comparable services — what they call the "sovereignty premium." Combined with the hidden costs above, budget accordingly.

Who Should Actually Use This

Move to ESC if:

• You handle data under strict EU sovereignty requirements — not just GDPR, but sector-specific rules that mandate operational control

• Regulators or auditors have specifically asked about US Cloud Act exposure

• You're in public sector, healthcare (especially in Germany with patient data), or finance with explicit data residency mandates

• Your contracts require EU-only operations and personnel — government contracts often do

• You need to demonstrate sovereignty compliance with third-party validated reports

Stick with regular EU regions if:

• Standard GDPR compliance is sufficient for your use case

• You need services that haven't launched in ESC yet

• Cost optimization is priority over sovereignty guarantees

• You're already running multi-region and partition complexity doesn't fit your operating model

• Your compliance requirements don't specifically call out operational sovereignty or personnel location

ESC isn't "better" than Frankfurt. It solves a specific problem. If you don't have that problem, you're adding complexity and cost for no benefit. Frankfurt with proper encryption and access controls is fine for most workloads.

The Competitive Landscape

AWS isn't alone here. Microsoft announced sovereign cloud offerings for EU customers. Google has Sovereign Controls for GCP. But the approaches differ.

Microsoft's approach involves partnerships with local operators — like T-Systems in Germany running Azure infrastructure. Google focuses on software controls and key management.

AWS went further with complete partition isolation. New legal entities, new domain, separate IAM, the whole stack. Whether that matters depends on what your regulators care about.

The 90+ service catalog at launch also sets AWS apart. Competitors often launch sovereign offerings with limited services and catch up over time. ESC starts nearly feature-complete.

What's Coming

AWS announced expansion plans. Local Zones in Belgium, Netherlands, and Portugal — same sovereignty model, lower latency for users in those countries. These extend ESC's footprint without requiring new full regions.

The workforce transition continues. Current staff are EU residents; future hires will be EU citizens only. Over time, the entire operation shifts to citizen-only. That's a commitment you can point to in RFPs.

More regions within ESC are likely but not announced. If demand justifies it, a second ESC region (France? Italy?) would add redundancy options.

The €7.8 billion investment through 2040 signals this isn't an experiment. Amazon is building parallel infrastructure for the next fifteen years.

Bottom Line

The European Sovereign Cloud answers three questions that every regulated European organization has been asking. Where exactly is my data? Who can access it? What happens when a foreign government asks for it?

For workloads where those questions have regulatory or contractual weight, ESC provides answers backed by legal structure, organizational isolation, and hardware-level security design. The ESC-SRF gives you auditor reports to prove it.

For everything else, eu-central-1 works fine and doesn't require rethinking your account structure, identity model, and network architecture.

Just remember: ESC is a different cloud, not a different region. The isolation that provides sovereignty guarantees also creates operational boundaries. That's the point — but it's also the cost.

References

• tecRacer: AWS ESC Launch, Pricing, and What's Next

• AWS Blog: Opening the European Sovereign Cloud

• AWS ESC Whitepaper

• AWS Security Blog: ESC Services

• AWS Security Blog: Sovereignty Reference Framework

• Terraform ESC Support (resolved)

• DEV: Why ESC Changes Everything

• DEV: AWS ESC Essential Guide

• Amazon Press Release

You know what? I see teams spinning up Kubernetes clusters for three microservices all the time. Then they spend two months figuring out pods, ingress controllers, and all that magic. And then they pay $70 per month just for three clusters in different regions, not counting the actual servers.

Here's the honest truth: Kubernetes is a powerful tool but you don't always need it. Amazon ECS is a simpler alternative that handles most tasks faster and cheaper.

In this article I'll show you:

• When ECS beats EKS (and saves you tons of money)

• Real scenarios with numbers and examples

• Ready-to-use code snippets for deploying to both platforms

• How to make the decision without headaches

Let's dive in!

Quick Comparison: ECS vs EKS

First let's look at the main differences in a simple table:

Architecture: How It Works

ECS Architecture:

Your Application
    ↓
Docker Image (you need this!)
    ↓
Task Definition (container description)
    ↓
ECS Service (manages launch)
    ↓
EC2 or Fargate (where it runs)
    ↓
Container running

EKS Architecture:

Your Application
    ↓
Docker Image
    ↓
Kubernetes Pod specification
    ↓
Deployment/StatefulSet
    ↓
Kubernetes Control Plane ($$$)
    ↓
Worker Nodes
    ↓
Container in Pod

See the difference? ECS has two fewer steps and each one is easier to understand.

When ECS is Your Best Choice

This is where it gets interesting. Many people think Kubernetes is always needed but that's not true. Let's break down real situations where ECS wins.

Scenario 1: Multi-Regional Deployment (3-5 Services)

Imagine: you have a simple API and a couple supporting services. You need to deploy them in three regions - Europe, Asia, USA. For redundancy, you know.

With EKS you pay:

• Europe cluster: $70/month

• Asia cluster: $70/month

• USA cluster: $70/month

• Total: $210/month just for the right to run containers

With ECS you pay:

• Cluster is free: $0

• Total: $0 for management

In other words, save $2,520 per year just on the control plane! And you still gotta pay for the actual servers.

Real Example

I had a project - e-commerce backend. Five services:

1. API Gateway (Node.js)

2. Order Service (Python)

3. Payment Service (Go)

4. Notification Service (Node.js)

5. Analytics Worker (Python)

Each service needed a Docker image. Here's a simple Dockerfile example for the Node.js API:

# Dockerfile for API Gateway
FROM node:18-alpine

WORKDIR /app

COPY package*.json ./
RUN npm install --production

COPY . .

EXPOSE 3000
CMD ["node", "server.js"]

We deployed across three regions using ECS Fargate. Setup time: 4 hours including Terraform code. If we'd done it with EKS - that's minimum a week with Helm charts, ingress controllers and all that kitchen.

Here's how we defined the task in ECS (simplified):

# ECS Task Definition - just the container part
resource "aws_ecs_task_definition" "api_gateway" {
  family                   = "api-gateway"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  cpu                      = "256"
  memory                   = "512"

  container_definitions = jsonencode([{
    name      = "api"
    image     = "123456789.dkr.ecr.us-east-1.amazonaws.com/api-gateway:latest"
    essential = true

    portMappings = [{
      containerPort = 3000
      protocol      = "tcp"
    }]

    environment = [
      { name = "NODE_ENV", value = "production" },
      { name = "PORT", value = "3000" }
    ]
  }])
}

Compare this to Kubernetes - you'd need Deployment YAML, Service YAML, maybe Ingress, ConfigMaps... it adds up.

Scenario 2: Quick Start and Simplicity

You're a startup. You have an MVP that needs to ship yesterday. Team of three people nobody knows Kubernetes deeply.

ECS gives you:

• Launch in couple hours (not days!)

• AWS integration out of the box

• No need to hire Kubernetes expert

• Less moving parts = less things break

Look I'm not saying Kubernetes is bad. It's awesome! But do you need it when you just wanna run a container? It's like buying a truck to get bread from the store.

Time to learn:

• ECS: 2-3 days to work comfortably

• EKS: 2-3 weeks minimum (or even a month)

Here's a complete minimal ECS setup with Terraform:

# Minimal ECS cluster
resource "aws_ecs_cluster" "main" {
  name = "my-app-cluster"
}

# ECS Service - runs 2 copies of your container
resource "aws_ecs_service" "app" {
  name            = "my-app"
  cluster         = aws_ecs_cluster.main.id
  task_definition = aws_ecs_task_definition.api_gateway.arn
  desired_count   = 2
  launch_type     = "FARGATE"

  network_configuration {
    subnets          = ["subnet-xxx", "subnet-yyy"]
    security_groups  = ["sg-xxx"]
    assign_public_ip = true
  }
}

That's it! No Helm, no kubectl, no YAML soup.

Scenario 3: AWS-Native Project

Your project is fully in AWS:

• Database - RDS

• Files - S3

• Queues - SQS

• Cache - ElastiCache

• Logs - CloudWatch

Why Kubernetes here? ECS integrates with these services natively and simpler.

Example - S3 access:

ECS Task Role (simple):

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": "s3:*",
    "Resource": "arn:aws:s3:::my-bucket/*"
  }]
}

Attach the role to Task Definition - done.

In EKS you do the same through IRSA (IAM Roles for Service Accounts):

• Setup OIDC provider

• Create ServiceAccount in Kubernetes

• Link with IAM role

• Annotate the pod

More steps = more places to mess up.

When EKS Becomes Necessary

Alright enough praising ECS. Let's be honest - there are situations where EKS is really better.

Scenario 1: Large Microservices Architecture (20+ Services)

When you have 20, 30, 50 microservices - that's different math.

Why EKS wins:

• $70 per cluster is fixed price (whether 5 services or 50)

• Kubernetes scales complexity better

• Ecosystem: Helm, Operators, service mesh (Istio, Linkerd)

• Centralized management of all services

Cost example:

With 30 services in one region:

• ECS: 30 separate ECS Services = lots of config hard to manage

• EKS: One cluster all services in namespaces manage through GitOps

Here $70/month pays for convenience.

A typical Kubernetes deployment:

# Kubernetes Deployment - simpler at scale
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-gateway
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: api-gateway
  template:
    metadata:
      labels:
        app: api-gateway
    spec:
      containers:
      - name: api
        image: my-registry/api-gateway:v1.2.3
        ports:
        - containerPort: 3000
        resources:
          requests:
            memory: "512Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "250m"

With Kubernetes you get built-in health checks, rolling updates, easy rollbacks.

Scenario 2: Multi-Cloud or Hybrid Infrastructure

Your company wants:

• Work in AWS and GCP simultaneously

• Keep some workloads on-premise

• Have ability to migrate between clouds

EKS (Kubernetes) gives portability:

• Same YAML manifests work everywhere

• Can move applications between clouds

• Standardization across all infra

ECS is AWS only. Can't move it anywhere. (ECS anywhere?! :) )

Scenario 3: Advanced Features

GPU workloads for ML/AI: EKS supports GPU nodes out of the box + all tooling like Kubeflow.

Complex networking policies: Network Policies in Kubernetes give precise traffic control between pods.

# Network Policy example
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: api-policy
spec:
  podSelector:
    matchLabels:
      app: api-gateway
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: frontend
    ports:
    - protocol: TCP
      port: 3000

Stateful applications: StatefulSets Persistent Volumes - all this works better in Kubernetes.

Practical Deployment Examples

Enough theory let's get hands dirty. I'll show how to deploy a simple application to both ECS and EKS. Same application to compare.

Our application: Nginx + simple Node.js API (both need Docker images)

Building Docker Images First

Before deploying anywhere you need Docker images. Here's our setup:

# Dockerfile for our Node.js app
FROM node:18-alpine

WORKDIR /app
COPY package*.json ./
RUN npm ci --production
COPY . .

EXPOSE 3000
CMD ["node", "index.js"]

Build and push:

# Build image
docker build -t my-app:latest .

# Tag for ECR
docker tag my-app:latest 123456789.dkr.ecr.us-east-1.amazonaws.com/my-app:latest

# Push to ECR
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 123456789.dkr.ecr.us-east-1.amazonaws.com
docker push 123456789.dkr.ecr.us-east-1.amazonaws.com/my-app:latest

ECS Deployment with Terraform

Let's start with the simpler one - ECS.

Step 1: VPC Setup

# Create VPC for containers
resource "aws_vpc" "main" {
  cidr_block           = "10.0.0.0/16"
  enable_dns_hostnames = true
  enable_dns_support   = true
}

# Public subnets
resource "aws_subnet" "public" {
  count                   = 2
  vpc_id                  = aws_vpc.main.id
  cidr_block              = "10.0.${count.index}.0/24"
  availability_zone       = data.aws_availability_zones.available.names[count.index]
  map_public_ip_on_launch = true
}

# Internet Gateway
resource "aws_internet_gateway" "main" {
  vpc_id = aws_vpc.main.id
}

Step 2: ECS Cluster and Service

# Create ECS cluster 
resource "aws_ecs_cluster" "main" {
  name = "my-app-cluster"
}

# Task Definition - describes your Docker container
resource "aws_ecs_task_definition" "app" {
  family                   = "my-app"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  cpu                      = "256"
  memory                   = "512"

  execution_role_arn = aws_iam_role.ecs_execution.arn
  task_role_arn      = aws_iam_role.ecs_task.arn

  container_definitions = jsonencode([{
    name      = "app"
    image     = "123456789.dkr.ecr.us-east-1.amazonaws.com/my-app:latest"
    essential = true

    portMappings = [{
      containerPort = 3000
      protocol      = "tcp"
    }]

    logConfiguration = {
      logDriver = "awslogs"
      options = {
        "awslogs-group"         = "/ecs/my-app"
        "awslogs-region"        = "us-east-1"
        "awslogs-stream-prefix" = "app"
      }
    }
  }])
}

# ECS Service - runs and maintains containers
resource "aws_ecs_service" "app" {
  name            = "my-app-service"
  cluster         = aws_ecs_cluster.main.id
  task_definition = aws_ecs_task_definition.app.arn
  desired_count   = 2
  launch_type     = "FARGATE"

  network_configuration {
    subnets          = aws_subnet.public[*].id
    security_groups  = [aws_security_group.ecs_tasks.id]
    assign_public_ip = true
  }
}

Step 3: IAM Roles

# Role for ECS to pull Docker images and write logs
resource "aws_iam_role" "ecs_execution" {
  name = "ecs-execution-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "ecs-tasks.amazonaws.com"
      }
    }]
  })
}

resource "aws_iam_role_policy_attachment" "ecs_execution_policy" {
  role       = aws_iam_role.ecs_execution.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
}

# Role for your application (e.g., S3 access)
resource "aws_iam_role" "ecs_task" {
  name = "ecs-task-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "ecs-tasks.amazonaws.com"
      }
    }]
  })
}

Deploy It

terraform init
terraform plan
terraform apply

Done! Container is running.

EKS Deployment with Terraform

Now the same thing but in EKS.

Step 1: EKS Cluster

# EKS cluster 
resource "aws_eks_cluster" "main" {
  name     = "my-eks-cluster"
  role_arn = aws_iam_role.eks_cluster.arn
  version  = "1.28"

  vpc_config {
    subnet_ids = concat(aws_subnet.public[*].id, aws_subnet.private[*].id)
  }
}

# Worker nodes
resource "aws_eks_node_group" "main" {
  cluster_name    = aws_eks_cluster.main.name
  node_group_name = "main-nodes"
  node_role_arn   = aws_iam_role.eks_node.arn
  subnet_ids      = aws_subnet.private[*].id

  scaling_config {
    desired_size = 2
    max_size     = 4
    min_size     = 1
  }

  instance_types = ["t3.medium"]
}

Step 2: IAM for EKS

# Cluster role
resource "aws_iam_role" "eks_cluster" {
  name = "eks-cluster-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "eks.amazonaws.com"
      }
    }]
  })
}

resource "aws_iam_role_policy_attachment" "eks_cluster_policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
  role       = aws_iam_role.eks_cluster.name
}

# Node role
resource "aws_iam_role" "eks_node" {
  name = "eks-node-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "ec2.amazonaws.com"
      }
    }]
  })
}

resource "aws_iam_role_policy_attachment" "eks_worker_node" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"
  role       = aws_iam_role.eks_node.name
}

resource "aws_iam_role_policy_attachment" "eks_cni" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
  role       = aws_iam_role.eks_node.name
}

Step 3: Kubernetes Manifests

After cluster is created deploy application with kubectl:

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: app
        image: 123456789.dkr.ecr.us-east-1.amazonaws.com/my-app:latest
        ports:
        - containerPort: 3000
        resources:
          requests:
            memory: "512Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "250m"
---
apiVersion: v1
kind: Service
metadata:
  name: my-app-service
spec:
  type: LoadBalancer
  selector:
    app: my-app
  ports:
  - port: 80
    targetPort: 3000

Deploy It

# 1. Apply Terraform 
terraform init
terraform apply

# 2. Configure kubectl
aws eks update-kubeconfig --name my-eks-cluster --region us-east-1

# 3. Check nodes
kubectl get nodes

# 4. Deploy application
kubectl apply -f deployment.yaml

# 5. Check status
kubectl get pods
kubectl get svc

Difference:

• ECS: one terraform apply and done

• EKS: terraform apply + kubectl commands + wait for everything to come up

Complexity Comparison

Real Cases and Economics

Let's calculate concrete numbers for typical scenarios.

Case 1: Startup with 5 Microservices in 3 Regions

Requirements:

• 5 services (API Workers Background Jobs)

• 3 regions: US EU Asia

• 2 instances each service

• All need Docker images built and stored in ECR

ECS Fargate:

Cluster cost: $0
ECR storage: ~$5/month (for Docker images)
Compute (Fargate):
  - 5 services × 2 instances × 3 regions = 30 tasks
  - Each task: 0.25 vCPU 512 MB
  - $0.04048/hour per vCPU $0.004445/hour per GB
  - (~0.25 × $0.04048 + 0.5 × $0.004445) × 730 hours = ~$9/task/month
  - 30 tasks × $9 = $270/month

Total: ~$275/month

EKS:

Cluster cost: $70 × 3 regions = $210/month
ECR storage: ~$5/month (same Docker images)
Compute (EC2 nodes):
  - Minimum 2× t3.medium per region = 6 instances
  - t3.medium = $0.0416/hour × 730 = ~$30/month
  - 6 × $30 = $180/month

Total: $210 + $5 + $180 = $395/month

Savings with ECS: $120/month or $1440/year

Plus with ECS you don't pay DevOps engineer to manage Kubernetes :)

Case 2: Large Project with 30 Services in 1 Region

ECS:

Cluster: $0
Management: 30 separate ECS Services (hard to manage!)
Compute: depends on load

EKS:

Cluster: $70/month
Management: One namespace GitOps Helm (easier!)
Compute: same + better resource utilization

Here EKS wins on management convenience. $70 pays for itself.

Time for Setup and Maintenance

Real numbers from my experience:

Initial setup:

• ECS: 4 hours (Terraform + tests + Docker builds)

• EKS: 2 days (cluster + addons + monitoring setup + Docker builds)

Weekly maintenance:

• ECS: ~30 minutes (check logs updates)

• EKS: ~2 hours (updates cluster checks monitoring)

Platform updates:

• ECS: automatic

• EKS: need to update control plane once a year (takes half a day with tests)

Decision Checklist: What to Choose?

So here's a simple flowchart for decision making:

Choose ECS if:

✅ You have less than 10-15 microservices ✅ Project is AWS only (no multi-cloud plans) ✅ Team doesn't know Kubernetes (and doesn't want to learn) ✅ Need to launch quickly (MVP startup) ✅ Budget is limited ✅ Simple application without complex dependencies ✅ Multi-regional deploy (save on clusters) ✅ Comfortable with Docker basics

Choose EKS if:

✅ More than 20+ microservices ✅ Need portability (multi-cloud hybrid) ✅ Team knows Kubernetes ✅ Need advanced features (service mesh operators) ✅ GPU workloads for ML/AI ✅ Already using Kubernetes elsewhere ✅ Complex microservices architecture ✅ Want access to Kubernetes ecosystem

Middle Ground

You can start with ECS and migrate later!

Many companies do this:

1. Start on ECS (fast and cheap)

2. Grow to 15-20 services

3. Kubernetes developers join the team

4. Gradually migrate to EKS

This is normal evolution. Don't go Kubernetes just "because it's cool".

Conclusions

Here's what's important to remember:

ECS is not a "second-rate" option. It's a full-fledged solution that handles many tasks excellently. Yes EKS is more powerful in capabilities but most projects simply don't need those capabilities.

Main ECS advantages:

• Free control plane (save $70-210+ per month)

• Simplicity and launch speed

• Less operational overhead

• Native AWS integration

• Perfect for multi-regional deployment of small services

When EKS is really needed:

• Large scale (20+ services)

• Code portability

• Advanced Kubernetes features

• Already have expertise in team

My advice: Don't chase the hype. Start with ECS if the task allows. Save time money and nerves. And when you really grow into Kubernetes - then migrate.

Kubernetes is like a Ferrari - cool car but for a trip to the store a regular Toyota works fine. And uses less gas 😄

Final Thoughts

The choice between ECS and EKS isn't about "better" or "worse" - it's about right tool for the job.

Start simple. ECS lets you ship fast without the Kubernetes learning curve. Your Docker skills transfer directly. AWS handles the orchestration.

As you grow, reassess. When you hit 15-20 services or need multi-cloud, EKS makes sense. But many successful companies run production on ECS for years.

Remember: Complexity is a cost. Every abstraction layer you add costs time money and mental overhead. Sometimes the best architecture is the simplest one that works.

Both platforms use Docker. Both run containers. Both scale. The question is: how much complexity do you actually need?

Choose wisely!

Sources

• AWS Official: ECS vs EKS

• Amazon ECS Documentation

• Amazon EKS Documentation

• Terraform AWS ECS Module

• Terraform AWS EKS Module

• ECS vs EKS 2025 Comparison

• AWS Container Services Guide

ECS Managed Instances: Bridging the Gap Between Control and Simplicity

Amazon ECS Managed Instances represents a new compute option that aims to combine the operational simplicity of managed infrastructure with the flexibility of EC2. This offering positions itself between AWS Fargate and self-managed EC2 instances in the ECS ecosystem.

What Makes It Different?

The key differentiator lies in how it handles infrastructure management. Unlike Fargate, which abstracts away the underlying compute entirely, ECS Managed Instances gives you visibility and control over instance types while AWS handles the operational burden. Unlike traditional EC2-backed ECS clusters, you don't need to manage instance provisioning, scaling, or patching.

Key capabilities include:

• Instance Selection Flexibility: By default, AWS automatically selects cost-optimized instance types based on your workload requirements. However, you can specify particular instance attributes when needed, including GPU acceleration, specific CPU architectures (ARM/x86), or enhanced networking capabilities.

• Task Bin-Packing: Unlike Fargate's one-task-per-instance model, Managed Instances supports multiple tasks per instance, optimizing resource utilization and potentially reducing costs through better instance consolidation.

• Automated Maintenance: The service implements security patches every 14 days and handles instance lifecycle management. You can schedule maintenance windows using EC2 event windows to minimize application disruption during critical business hours.

• Bottlerocket OS: Instances run on Bottlerocket, AWS's purpose-built container operating system, which provides a minimal attack surface and improved security posture.

Understanding the Cost Model

It's important to note that ECS Managed Instances adds a management fee on top of EC2 instance costs. This charge varies by instance class and size and is billed at on-demand pricing (per second with a one-minute minimum), even if you're using EC2 Savings Plans for the underlying instances. Teams should evaluate whether the operational savings justify the additional cost for their specific workloads.

When to Choose ECS Managed Instances

This option makes sense when you need:

• Access to specific instance types (bare metal, GPU instances, or specialized compute)

• Better cost optimization through task bin-packing

• EC2-level control without operational overhead

• Integration with existing EC2 pricing commitments

Advanced Deployment Strategies: Linear and Canary Deployments

Alongside Managed Instances, AWS introduced native support for Linear and Canary deployment strategies in ECS, expanding beyond the existing blue/green deployment option. These strategies are available for services using Application Load Balancer (ALB) or ECS Service Connect.

Canary Deployments: Controlled Risk Exposure

Canary deployments allow you to validate new service revisions with minimal risk by routing a small percentage of production traffic to the new version first.

The deployment process follows a two-step traffic shift:

1. Initially shift a configured percentage (e.g., 10%) to the new revision

2. After the canary bake time completes successfully, shift 100% of remaining traffic

During the canary bake time, both versions run simultaneously, allowing you to monitor metrics, health checks, and application behavior. If issues are detected, you can quickly roll back by shifting traffic back to the original version.

Linear Deployments: Gradual Traffic Migration

Linear deployments provide a more gradual approach, shifting traffic in equal percentage increments over a specified time period. You configure:

• Step percentage: How much traffic shifts at each increment (e.g., 10%)

• Step bake time: The wait period between each increment for monitoring

This strategy validates your application at multiple stages with progressively increasing production traffic, providing more data points for validation compared to canary deployments.

Deployment Lifecycle and Monitoring

Both strategies support several critical features:

• Deployment Bake Time: After all traffic has shifted to the new revision, AWS waits a configurable period before terminating the old revision, enabling quick rollback without downtime if issues emerge.

• Lifecycle Hooks: You can configure Lambda functions to execute at specific deployment stages for automated validation, custom health checks, or integration with external monitoring systems.

• CloudWatch Alarm Integration: Configure automatic rollback triggers based on CloudWatch alarms, enabling automated failure detection and recovery.

• Lifecycle Stages: Each deployment progresses through distinct stages (SCALE_UP, TEST_TRAFFIC_SHIFT, PRODUCTION_TRAFFIC_SHIFT, BAKE_TIME, CLEAN_UP), with each stage lasting up to 24 hours. For CloudFormation deployments, the entire process must complete within 36 hours.

Best Practices for Production Use

When implementing these deployment strategies, consider:

1. Start Conservative: Begin with smaller percentages (5-10% for canary) to minimize impact if issues occur

2. Sufficient Monitoring: Ensure your canary percentage generates enough traffic for meaningful validation

3. Appropriate Bake Times: Set evaluation periods long enough to capture meaningful performance data (typically 10-30 minutes)

4. Comprehensive Metrics: Monitor response time, error rates, throughput, and business-specific metrics

5. Automated Rollback: Configure CloudWatch alarms to automatically trigger rollback when metrics exceed thresholds

Regional Availability

ECS Managed Instances launched in six AWS Regions: US East (North Virginia), US West (Oregon), Europe (Ireland), Africa (Cape Town), Asia Pacific (Singapore), and Asia Pacific (Tokyo).

Linear and Canary deployment strategies are available in all commercial AWS Regions where Amazon ECS is available and can be configured through the Console, SDK, CLI, CloudFormation, CDK, and Terraform.

Conclusion

These enhancements demonstrate AWS's continued investment in making ECS more flexible and operationally efficient. ECS Managed Instances provides a middle ground between Fargate's simplicity and EC2's control, while the new deployment strategies offer production-grade deployment patterns that many organizations previously had to build themselves.

For teams running containerized workloads on AWS, these features warrant evaluation against existing deployment patterns and infrastructure management practices. The key is understanding your specific requirements around control, cost optimization, and operational complexity to determine which combination of ECS features best serves your needs.

I've been working with Infrastructure as Code for the better part of eight years—starting with CloudFormation, migrating teams to Terraform, and lately exploring AWS CDK. Over that time, I've seen platforms grow from a handful of templates to hundreds of modules scattered across dozens of repositories. I've also watched technical debt accumulate: legacy EC2 instance types chosen three years ago, untagged resources, container images piling up in ECR, and NAT gateways draining budgets while sitting mostly idle.

The traditional FinOps workflow—reactive hunting for idle resources using cost optimization hubs and billing alerts—works, but it's exhausting and slow. I wanted to shift left: catch inefficiencies before they hit production, bake cost and security best practices into the templates themselves, and help platform engineers understand inherited code without spending days spelunking through thousands of lines.

This article documents how I've integrated Amazon Q Developer into my IaC workflow—not as a replacement for human judgment, but as a force multiplier. I'll walk through real scenarios, concrete examples, workflow integration, limitations I've encountered, and a practical framework for measuring impact.

The Pain Points of Traditional IaC Management

Before introducing AI assistance, my team faced several recurring bottlenecks:

Legacy comprehension: Inheriting a 2,000-line Terraform module written by someone who left the company two years ago. No README. Cryptic variable names. Comments? Optional, apparently. Understanding what it does, how components interact, and where optimization opportunities exist consumed days of calendar time.

Migration friction: Translating a CloudFormation template to CDK or Terraform—or vice versa—is tedious and error-prone. Even straightforward resources involve syntax mapping, API differences, and validation loops. Multiply that by dozens of modules, and migration projects drag on for quarters.

Review latency: Pull requests with IaC changes sat in queues waiting for someone with enough context to spot that the new RDS instance lacks encryption, or that the NAT gateway could be replaced with VPC endpoints, or that the instance type is three generations old.

Standardization gaps: Every engineer writes modules slightly differently. Some include lifecycle policies; others don't. Tagging strategies diverge. IAM policies are either too permissive or so locked down they break deployments.

Security and cost blind spots: Static analysis tools (tfsec, Checkov) catch obvious mistakes, but they don't suggest improvements. They tell you what's wrong, not what could be better. Cost estimation tools (Infracost) show projected spend, but they don't recommend Graviton instances or Spot for batch workloads.

Onboarding friction: New hires need weeks to become productive with our IaC codebase. The learning curve is steep, and tribal knowledge is poorly documented.

How Amazon Q Developer Fits In

Amazon Q Developer is an AI-powered coding assistant built on over 17 years of AWS cloud experience. It integrates directly into VS Code, JetBrains IDEs, and provides CLI capabilities for automated transformations. It generates deployment-ready infrastructure code for Terraform, AWS CDK, and CloudFormation.

I use it for:

• Code comprehension: Summarizing what a template does, mapping resource dependencies, identifying entry points.

• Optimization discovery: Scanning templates for cost, security, and performance improvements aligned with AWS Well-Architected Framework.

• IaC transformation: Automated translation between IaC frameworks (Terraform ↔ CDK ↔ CloudFormation) using the four-step process: assess, translate, test and refine, deploy.

• Module generation: Creating deployment-ready modules from natural language requirements with built-in AWS best practices.

• Pull request reviews: Analyzing diffs, flagging risks, suggesting improvements based on AWS standards.

• Custom rule enforcement: Using rule-based automation to encode team standards and ensure consistent, repeatable suggestions.

According to AWS internal testing, Amazon Q's agentic capabilities deliver 10x-50x time savings for legacy IaC remediation compared to manual processes. For VMware network migrations, AWS teams translated configurations for 500 VMs in 1 hour—80 times faster than the traditional 2-week manual approach.

I treat Q as a highly skilled junior engineer: fast, knowledgeable, but requiring validation and context.

End-to-End Workflow Integration

Here's how Amazon Q fits into my current IaC lifecycle:

1. Local Development (VS Code + Amazon Q)

• Open a CDK stack or Terraform module

• Prompt Q: "Review this file and identify opportunities to optimize for cost efficiency"

• Q returns recommendations: instance type downsizing, ECR lifecycle policies, Graviton migration paths, NAT gateway elimination, subnet configuration changes

• I validate recommendations against workload requirements, commitments, and architectural constraints

• Implement approved changes with Q's assistance (it can write the code inline)

2. Static Analysis

• Run Checkov, tfsec, or cfn-lint locally

• If violations appear, I prompt Q: "Fix the security issues flagged by Checkov in this file"

• Q suggests remediation (e.g., enable encryption, add bucket policies, restrict ingress rules)

3. Policy-as-Code Validation

• Apply OPA/Conftest or CloudFormation Guard policies

• For failures, I ask Q to explain the policy intent and adjust the template accordingly

• Example policy (Rego):

    package terraform.tags
    deny[msg] {
      input.resource_type == "aws_instance"
      not input.tags.Environment
      msg = "Missing required Environment tag"
    }
  

4. Cost Estimation

• Run Infracost to project monthly spend

• If costs are higher than expected, I prompt Q: "Suggest ways to reduce cost for this infrastructure while maintaining performance"

• Q might recommend reserved capacity, Savings Plans eligibility, or Graviton alternatives

5. CI/CD Pipeline Gates

• Pre-commit hooks run formatters (terraform fmt, prettier)

• GitHub Actions execute tests, static analysis, policy checks, Infracost diff

• If checks fail, the pipeline surfaces Q-generated suggestions in PR comments (I've scripted this using Q's API)

6. Pull Request Review

• Reviewers use Q to summarize changes: "Explain what this PR changes and flag any cost, security, or operational risks"

• Q highlights: "This PR introduces a new RDS instance without backup retention configured. Estimated monthly cost increase: $150. Consider enabling automated backups and Multi-AZ for production."

7. Post-Deployment

• Monitor drift using AWS Config or Terraform state comparisons

• Track cost variance via Cost Explorer

• Feed learnings back into Q prompts for future iterations

Practical Example 1: Generating a Terraform Module from Requirements

Scenario: I need a reusable Terraform module for an S3 bucket with versioning, lifecycle rules, encryption, and least-privilege IAM access.

Prompt:

Create a Terraform module for an S3 bucket with:
- Versioning enabled
- Lifecycle policy: expire non-current versions after 90 days
- AES-256 encryption at rest
- Block public access
- IAM policy allowing read-only access for a Lambda execution role

Q Output (trimmed):

resource "aws_s3_bucket" "main" {
  bucket = var.bucket_name
}

resource "aws_s3_bucket_versioning" "main" {
  bucket = aws_s3_bucket.main.id
  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_lifecycle_configuration" "main" {
  bucket = aws_s3_bucket.main.id
  rule {
    id     = "expire-noncurrent"
    status = "Enabled"
    noncurrent_version_expiration {
      noncurrent_days = 90
    }
  }
}

resource "aws_s3_bucket_server_side_encryption_configuration" "main" {
  bucket = aws_s3_bucket.main.id
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

resource "aws_s3_bucket_public_access_block" "main" {
  bucket                  = aws_s3_bucket.main.id
  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

Outcome: In 30 seconds, I have a production-ready module skeleton. I add variables, outputs, and documentation, then commit. Time saved: 45 minutes.

Practical Example 2: Refactoring Terraform to CDK

Scenario: A team wants to migrate a Terraform module to CDK (TypeScript) for better IDE support and type safety.

Original Terraform (simplified):

resource "aws_lambda_function" "processor" {
  function_name = "data-processor"
  runtime       = "python3.9"
  handler       = "index.handler"
  role          = aws_iam_role.lambda_role.arn
  filename      = "function.zip"
}

resource "aws_iam_role" "lambda_role" {
  name = "lambda-exec-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "lambda.amazonaws.com"
      }
    }]
  })
}

Prompt:

Translate this Terraform configuration to AWS CDK TypeScript.
Use best practices: managed policies, environment variables, and proper construct hierarchy.

Q Output (trimmed):

import * as cdk from 'aws-cdk-lib';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as iam from 'aws-cdk-lib/aws-iam';

export class DataProcessorStack extends cdk.Stack {
  constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const lambdaRole = new iam.Role(this, 'LambdaRole', {
      assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'),
      managedPolicies: [
        iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole')
      ]
    });

    const processor = new lambda.Function(this, 'Processor', {
      runtime: lambda.Runtime.PYTHON_3_9,
      handler: 'index.handler',
      code: lambda.Code.fromAsset('function.zip'),
      role: lambdaRole,
      functionName: 'data-processor'
    });
  }
}

Outcome: Migration skeleton in under a minute. I adjust asset paths, add CloudWatch alarms, refine IAM permissions. Time saved per module: 1-2 hours.

Practical Example 3: Pull Request Review with Amazon Q

Scenario: A PR introduces a new ECS service with Fargate tasks. Reviewer uses Q to assess impact.

Prompt:

Review this CDK stack diff. Summarize changes, identify cost implications,
flag security or operational risks.

Q Analysis:

This PR creates:

• ECS Fargate service with 2 tasks (2 vCPU, 4GB RAM each)

• Application Load Balancer

• CloudWatch log group (no retention configured)

Cost estimate: ~$95/month (Fargate) + ~$25/month (ALB) = $120/month

Risks:

• Log group has no retention policy → unbounded storage costs

• Tasks use x86_64 architecture; Graviton would reduce cost ~20%

• No autoscaling configured; manual intervention required under load

• ALB security group allows 0.0.0.0/0 ingress on port 443 (verify if intentional)

Recommendations:

• Set log retention to 30 days

• Evaluate ARM64/Graviton compatibility

• Add autoscaling with target tracking

• Restrict ALB ingress to known CIDR ranges or CloudFront

Outcome: Reviewer approves with conditions. Author updates the stack. Review cycle time: 15 minutes instead of 2 hours.

Security, Compliance, and Quality Integration

Amazon Q doesn't replace security tooling—it augments it.

IAM Least Privilege

I prompt Q: "Review this IAM policy and restrict to least privilege for a Lambda reading from S3 and writing to DynamoDB."

Q tightens wildcards, removes unnecessary actions, adds conditions for resource tagging.

Secrets Hygiene

Q flags hardcoded credentials or API keys during reviews. I pair this with git-secrets and AWS Secrets Manager integration.

Drift Detection

After deployments, I compare actual infrastructure (via AWS Config or Terraform state) against source templates. If drift occurs, I ask Q: "Why might this resource configuration differ from the template?" It helps hypothesize causes (manual changes, out-of-band automation, CloudFormation stack updates).

Policy-as-Code

I maintain Conftest policies (OPA/Rego) for tagging, encryption, and network segmentation. When policies fail, Q explains the rule intent and suggests compliant configurations.

Cost Guardrails

I integrate Infracost in CI and set thresholds (e.g., no PR increasing monthly cost by >$500 without approval). Q helps identify cost drivers and alternatives.

Repository Improvement Plan (Prioritized)

If I were assessing a typical IaC codebase today, here's what I'd prioritize:

1. Add (High Priority):

   • Pre-commit hooks: terraform fmt, tflint, Checkov

   • Infracost integration in CI

   • Basic Conftest policies (tagging, encryption)

   • ECR lifecycle policies across all container builds

   • Automated README generation (Q can draft from code)

2. Refactor (Medium Priority):

   • Consolidate duplicate modules

   • Standardize naming conventions (use Q to generate renaming scripts)

   • Migrate legacy instance types to Graviton where compatible

   • Replace NAT gateways with VPC endpoints for AWS services

3. Harden (Medium Priority):

   • IAM policy reviews (Q-assisted least privilege tightening)

   • Enable Terraform state locking (DynamoDB + S3)

   • Add drift detection automation (aws-config or Terraform Cloud)

   • Implement environment-specific configurations (dev/stage/prod variants)

4. Automate (Lower Priority, High Impact):

   • Q-generated PR comment summaries (cost/security/drift)

   • Automated documentation updates on merge

   • Ephemeral preview environments for PRs (using Terraform workspaces or CDK context)

5. Measure (Ongoing):

   • Track PR review time and test coverage improvements

   • Iterate on Q prompts based on false positives/negatives

   • Refine policy-as-code rules based on team feedback

Skepticism and Limitations

I've hit real limitations with Q:

Hallucinations: Q occasionally invents AWS resource properties that don't exist (e.g., fictional CloudFormation parameters). Always validate against official docs.

Context window: For massive monorepo structures, Q loses context. I work around this by targeting specific files or summarizing first.

Organizational standards: Q doesn't know your company's naming conventions, approved instance families, or compliance requirements unless you explicitly provide them in prompts or customization files.

Noisy recommendations: Q sometimes suggests optimizations that conflict with architectural decisions (e.g., recommending smaller instances when you've standardized on m5.large for operational simplicity). Filtering signal from noise requires domain knowledge.

Overfitting to public examples: Q trained on public repos. If your IaC patterns are highly proprietary or unconventional, its suggestions may miss the mark.

Human validation is non-negotiable: I never merge Q-generated code without review, testing, and static analysis. Treat Q as a draft generator, not a replacement for engineering judgment.

Conclusion

Amazon Q Developer has changed how I work with infrastructure code. It doesn't replace engineering judgment, but it handles the tedious parts—reading legacy code, translating between IaC languages, spotting optimization opportunities, and catching security issues early.

The biggest wins for me have been:

• Understanding inherited codebases in minutes instead of days

• Generating module skeletons from plain English requirements

• Cutting PR review time by helping reviewers quickly understand changes and impacts

• Catching cost and security issues before they reach production

The key is treating Q as a tool, not a magic solution. I always validate its suggestions, test changes thoroughly, and integrate it with existing tooling like static analysis and policy checks.

If you're considering trying it, start small: pick one messy legacy file, ask Q to explain it, and see what optimization opportunities it finds. Install the VS Code extension (there's a free tier), experiment with prompts, and adjust based on what works for your workflow.

The goal isn't perfection—it's making IaC work less frustrating and more efficient, one template at a time.

The Production Problem

AI coding tools excel at quick prototyping—what some call "vibe coding." However, getting these prototypes production-ready presents challenges:

• Undocumented assumptions: The AI made decisions during development, but those choices aren't captured anywhere

• Missing requirements clarity: You guided the agent throughout, but fuzzy requirements mean you can't verify if the application truly meets needs

• Architecture blindspots: Understanding how the system design affects performance, scalability, and your infrastructure isn't immediately clear

• Maintenance difficulties: Without proper documentation and structure, future changes become increasingly complex

Enter Kiro: Spec-Driven Development Meets AI

Kiro is a new agentic IDE :) that tackles these challenges through spec-driven development. Rather than jumping straight to code, Kiro helps you think through decisions systematically while maintaining the speed of AI-assisted development.

Key Features

1. Requirements Specification

Kiro transforms a simple prompt like "Add a review system for products" into detailed user stories with EARS (Easy Approach to Requirements Syntax) acceptance criteria. This makes implicit assumptions explicit, ensuring the AI builds what you actually need—not what it thinks you need.

2. Technical Design Documentation

After requirements approval, Kiro analyzes your codebase and generates comprehensive design documents including:

• Data flow diagrams

• TypeScript interfaces and type definitions

• Database schemas

• API endpoint specifications

This eliminates the typical back-and-forth on requirements clarity that slows down development cycles.

3. Task Decomposition and Sequencing

Kiro automatically generates implementation tasks with proper dependency ordering. Each task includes considerations often missed in quick prototypes:

• Unit and integration tests

• Loading states and error handling

• Mobile responsiveness

• Accessibility requirements (WCAG compliance)

4. Agent Hooks for Automation

Hooks are event-driven automations that act like an experienced team member catching issues in the background:

• Update tests automatically when components change

• Refresh API documentation when endpoints are modified

• Scan for security issues before commits

• Enforce coding standards across the entire team

These hooks commit to Git, ensuring consistent quality checks across all developers.

Why This Matters for DevOps and Cloud Architecture

For those of us managing infrastructure and deployment pipelines, Kiro addresses several pain points:

Infrastructure as Code Compatibility: Spec-driven development aligns naturally with IaC practices. Design documents provide the clarity needed for proper resource planning and cost optimization.

CI/CD Integration: Automated test generation and security scanning hooks integrate seamlessly into existing pipelines, reducing manual review overhead.

Documentation Drift Prevention: Kiro keeps specs synchronized with code changes—solving the eternal problem of outdated documentation that complicates infrastructure modifications.

Team Consistency: When managing multiple services or microservices architectures, enforcing standards through hooks ensures uniform code quality across repositories.

Technical Details

• Built on Code OSS (VS Code compatible)

• Supports Model Context Protocol (MCP) for specialized tool integration

• Works with Open VSX plugins

• Available for Mac, Windows, and Linux

• Supports most popular programming languages

• Free during preview period

The Bigger Picture

While Kiro isn't specifically an AWS or cloud tool, its approach to structured development, automated quality checks, and documentation maintenance addresses fundamental challenges in modern software delivery—challenges that become amplified when deploying to cloud environments where misconfigurations can have immediate cost and security implications.

For DevOps practitioners and cloud architects, Kiro represents a shift from treating AI coding assistants as simple code generators to treating them as collaborative partners in the entire development lifecycle—from requirements gathering through production deployment.

The rise of artificial intelligence (AI) has brought about an unprecedented demand for infrastructure that can handle large-scale computations, support GPU acceleration, and provide scalable, flexible management of workloads. Kubernetes has emerged as a leading platform for orchestrating these workloads, and Amazon Elastic Kubernetes Service (EKS) extends Kubernetes’ capabilities by simplifying deployment and scaling in the cloud.

NVIDIA Infrastructure Manager (NIM) complements Kubernetes by optimizing GPU workloads, a critical need for training large language models (LLMs), computer vision, and other computationally intensive AI tasks. Additionally, OpenAI models can be integrated into this ecosystem to unlock cutting-edge AI capabilities, such as text generation, image recognition, and decision-making systems.

This article provides an in-depth guide to building a complete AI platform using EKS, NVIDIA NIM, and OpenAI models, with Terraform automating the deployment. Whether you are an AI researcher or a business looking to adopt AI, this guide outlines how to build a robust and scalable platform. Complete code for this setup is available on GitHub https://github.com/timurgaleev/eks-nim-llm-openai.

Why Choose NVIDIA NIM and EKS for AI Workloads?

Challenges of AI Workloads

AI applications, especially those involving LLMs, have unique challenges:

• GPU Resource Management: Training and inference rely on GPUs, which are scarce and expensive resources. Efficient allocation and monitoring are crucial.

• Scalability: AI workloads often need to scale dynamically based on user demand or data processing requirements.

• Storage for Large Datasets: AI models and datasets can require hundreds of gigabytes, necessitating persistent, shared, and scalable storage.

• Observability: Monitoring system performance, especially GPU utilization and latency, is essential for optimizing workloads.

NVIDIA NIM: A Solution for GPU Workloads

NVIDIA NIM addresses these challenges by providing:

1. GPU Scheduling: Maximizes GPU usage across workloads.

2. Integration with Kubernetes: Leverages Kubernetes to manage pods, jobs, and resources efficiently.

3. AI Model Management: Simplifies deployment and scaling of AI models with Helm charts and Kubernetes CRDs (Custom Resource Definitions).

4. Support for Persistent Storage: Integrates with shared storage solutions like AWS EFS for storing datasets and models.

Amazon EKS: A Scalable Kubernetes Solution

Amazon EKS adds value by:

1. Managed Kubernetes: Reduces operational overhead by handling Kubernetes cluster setup, updates, and management.

2. Elastic Compute Integration: Dynamically provisions GPU-enabled instances, such as g4dn and p4d, to handle AI workloads. Ensure that your AWS account has sufficient quotas and availability for these instance types to avoid provisioning issues.

3. Built-in Security: Integrates with AWS IAM and VPC for secure access and network segmentation.

Together, NVIDIA NIM and Amazon EKS create a powerful platform for AI model training, inference, and experimentation.

Architecture Overview

The platform architecture integrates NVIDIA NIM and OpenAI models into an EKS cluster, combining compute, storage, and monitoring components.

Key Components

1. EKS Cluster: Manages Kubernetes workloads and scales GPU-enabled nodes.

2. Karpenter: Dynamically provisions and scales nodes (CPU and GPU) based on workload demands, optimizing resource utilization and cost.

3. GPU Node Groups: Nodes equipped with NVIDIA GPUs for ML and AI inference tasks.

4. NVIDIA NIM: Deploys GPU workloads, manages AI pipelines, and integrates with Kubernetes.

5. OpenAI Web UI: Provides a user-friendly interface for interacting with AI models.

6. Persistent Storage: AWS EFS supports shared storage for datasets and models.

7. Observability Tools: Prometheus and Grafana offer real-time monitoring of system metrics, including GPU utilization and pod performance.

Deployment Guide

This guide provides step-by-step instructions to deploy the architecture using Terraform. While the focus is on essential components like EKS, GPU workloads, and observability, we skip detailed VPC configuration to allow flexibility based on your specific requirements.

For a VPC example that fits this deployment, refer to the repository: https://github.com/timurgaleev/eks-nim-llm-openai.

Step 1: Provisioning the EKS Cluster

Provisioning an Amazon EKS cluster is the foundation for Kubernetes workloads. Below is the EKS Cluster Configuration with key highlights to focus on scalability, system add-ons, and Karpenter integration.

EKS Cluster Configuration

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 19.15"

  cluster_name                   = local.name
  cluster_version                = var.eks_cluster_version
  cluster_endpoint_public_access = true

  vpc_id     = module.vpc.vpc_id
  subnet_ids = compact([
    for subnet_id, cidr_block in zipmap(module.vpc.private_subnets, module.vpc.private_subnets_cidr_blocks) :
    substr(cidr_block, 0, 4) == "100." ? subnet_id : null
  ])

  manage_aws_auth_configmap = true
  aws_auth_roles = [
    {
      rolearn  = module.eks_blueprints_addons.karpenter.node_iam_role_arn
      username = "system:node:{{EC2PrivateDNSName}}"
      groups = [
        "system:bootstrappers",
        "system:nodes"
      ]
    }
  ]

  eks_managed_node_group_defaults = {
    iam_role_additional_policies = {
      AmazonSSMManagedInstanceCore = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
    }
    ebs_optimized = true
    block_device_mappings = {
      xvda = {
        device_name = "/dev/xvda"
        ebs = {
          volume_size = 100
          volume_type = "gp3"
        }
      }
    }
  }

  eks_managed_node_groups = {
    core_node_group = {
      name            = "core-node-group"
      description     = "EKS Core node group for hosting system add-ons"
      subnet_ids      = compact([
        for subnet_id, cidr_block in zipmap(module.vpc.private_subnets, module.vpc.private_subnets_cidr_blocks) :
        substr(cidr_block, 0, 4) == "100." ? subnet_id : null
      ])
      ami_type        = "AL2_x86_64"
      instance_types  = ["m5.xlarge"]
      capacity_type   = "SPOT"
      desired_size    = 2
      min_size        = 2
      max_size        = 4
      labels = {
        WorkerType    = "SPOT"
        NodeGroupType = "core"
      }
      tags = merge(local.tags, { Name = "core-node-grp" })
    }
  }
}

Key Highlights

1. Networking:

   • Subnets are filtered to include only CIDR blocks starting with 100. to ensure specific subnet assignment for nodes.

2. IAM and Auth:

   • Integration with Karpenter is configured via the aws_auth_roles block, allowing Karpenter to dynamically provision nodes.

3. Managed Node Groups:

   • Core Node Group:

     • Optimized for system-level workloads.

     • Configured with m5.xlarge spot instances for cost efficiency.

     • Labels such as NodeGroupType: core and taints can be used to restrict workloads to this node group.

4. Storage:

   • Nodes are configured with gp3 root volumes (100 GiB) for system usage. Additional storage for workloads should be configured separately.

5. Scaling:

   • Use Karpenter for workload-based scaling instead of additional managed node groups. The eks_managed_node_groups block here is only for critical system workloads.

Step 2: Deploying NVIDIA NIM for AI Workloads

Deploying NVIDIA NIM (NVIDIA Inference Manager) requires configuring persistent storage for large datasets and allocating GPU resources for optimal performance. Here's an expanded guide breaking down the essential steps.

1. Persistent Storage with AWS EFS

AI workloads often require storage that exceeds local node capacity. AWS EFS (Elastic File System) provides a shared and scalable storage solution across multiple pods. Below is the configuration for creating a Persistent Volume Claim (PVC) backed by EFS:

Code: Persistent Volume Claim (PVC)

kubernetes_persistent_volume_claim_v1 "efs_pvc" {
  metadata {
    name      = "efs-storage"
    namespace = "nim"
  }
  spec {
    access_modes       = ["ReadWriteMany"] # Enables sharing storage across multiple pods.
    storage_class_name = "efs"             # Links the PVC to an EFS storage class.
    resources {
      requests = {
        storage = "200Gi" # Reserves 200 GiB of scalable storage.
      }
    }
  }
}

Key Points:

• Access Mode: "ReadWriteMany" allows simultaneous access by multiple pods, critical for parallel workloads.

• Storage Class: Must correspond to an EFS provisioner configured in the Kubernetes cluster.

• Capacity: Start with 200 GiB and scale as per your dataset requirements.

2. Deploying NVIDIA NIM Helm Chart

After configuring storage, deploy NVIDIA NIM using Helm. The Helm chart simplifies GPU allocation and links the persistent storage to NIM-managed workloads.

Configure the NGC API Key

Before deploying NVIDIA NIM, you need to retrieve your NGC API Key from NVIDIA’s cloud platform and set it as an environment variable. This key enables secure authentication with NVIDIA’s container registry and services.

Steps to Retrieve the NGC API Key:

1. Log in to your NGC account.

2. Navigate to Setup > API Keys.

3. Click Generate API Key if you don’t already have one.

4. Copy the generated key to use in your deployment process.

Set the NGC API Key as an Environment Variable:

Run the following command in your terminal to make the key accessible to Terraform during deployment:

export TF_VAR_ngc_api_key=<replace-me>

Replace <replace-me> with your actual API key. This key will be passed to NVIDIA NIM to enable seamless model deployment.

Code: Helm Release for NVIDIA NIM

helm_release "nim_llm" {
  name      = "nim-llm"
  chart     = "./nim-llm"                # Points to the NIM Helm chart location.
  namespace = "nim"
  values = [
    templatefile("nim-llm-values.yaml", {
      model_id    = var.model_id            # Specifies the LLM model (e.g., GPT-like models).
      num_gpu     = var.num_gpu             # Allocates GPU resources for inference tasks.
      ngc_api_key = var.ngc_api_key
      pvc_name    = kubernetes_persistent_volume_claim_v1.efs_pvc.metadata[0].name
    })
  ]
}

Key Points:

• model_id: The identifier of the model being deployed (e.g., GPT-3, BERT).

• num_gpu: Configures GPU resources for inference tasks. The value should align with the instance type used in your cluster (e.g., g4dn.xlarge for one GPU).

• pvc_name: Links the EFS-backed PVC to the workload for storing large datasets or models.

3. Configuration Highlights

Why Persistent Storage?

• AI models and datasets are often larger than the node's local storage. Using EFS ensures:

  • Scalability: Adjust storage as required without downtime.

  • High Availability: Accessible across multiple Availability Zones.

GPU Allocation

• NVIDIA NIM optimizes GPU usage for inference. Use the num_gpu variable to specify the number of GPUs for your workload, ensuring efficient resource utilization.

Summary

1. Storage Configuration: Use AWS EFS with Kubernetes PVC for shared, scalable storage across pods.

2. GPU Allocation: NVIDIA NIM enables efficient GPU resource management for AI inference tasks.

3. Helm Chart Deployment: Leverage Helm for streamlined deployment, linking GPU resources and persistent storage.

Step 3: Adding OpenAI Web UI

The OpenAI Web UI provides an interface for users to interact with deployed AI models.

"helm_release" "openai_webui" {
  name       = "openai-webui"
  chart      = "open-webui"
  repository = "https://helm.openwebui.com/"
  namespace  = "openai-webui"
  values = [
    jsonencode({
      replicaCount = 1,
      image = {
        repository = "ghcr.io/open-webui/open-webui"
        tag        = "main"
      }
    })
  ]
}

Step 4: Observability with Prometheus, Grafana, and Custom Metrics

Prometheus and Grafana are essential tools for monitoring AI workloads. Prometheus collects resource metrics, including GPU-specific data, while Grafana visualizes these metrics through tailored dashboards. These tools help ensure that AI operations are running smoothly and efficiently.

To extend observability, the Prometheus Adapter is configured with custom rules for tracking AI-specific metrics. Key configurations include:

• Tracking Active Requests: Using the num_requests_running metric, Prometheus monitors the number of ongoing requests, providing insights into workload intensity.

• Inference Queue Monitoring: The nv_inference_queue_duration_us metric tracks NVIDIA inference queue times, converted into milliseconds for enhanced readability.

Sample Configuration for Prometheus Adapter:

prometheus:
  url: http://kube-prometheus-stack-prometheus.${prometheus_namespace}
  port: 9090
rules:
  default: false
  custom:
  - seriesQuery: '{__name__=~"num_requests_running"}'
    resources:
      template: <<.Resource>>
    name:
      matches: "num_requests_running"
      as: ""
    metricsQuery: sum(<<.Series>>{<<.LabelMatchers>>}) by (<<.GroupBy>>)
  - seriesQuery: 'nv_inference_queue_duration_us{namespace!="", pod!=""}'
    resources:
      overrides:
        namespace:
          resource: "namespace"
        pod:
          resource: "pod"
    name:
      matches: "nv_inference_queue_duration_us"
      as: "nv_inference_queue_duration_ms"
    metricsQuery: 'avg(rate(nv_inference_queue_duration_us{<<.LabelMatchers>>}[1m])/1000) by (<<.GroupBy>>)'

These configurations enable Prometheus to expose meaningful custom metrics that are critical for scaling and optimizing AI workloads. By integrating these metrics into Grafana dashboards, users gain actionable insights into system performance and bottlenecks.

Step 5: Scaling and Optimization with Karpenter

In large-scale AI deployments, workload demands fluctuate significantly. Dynamic scaling is essential for managing these workloads effectively while minimizing costs. Karpenter, a Kubernetes-native cluster autoscaler, provides powerful mechanisms for optimizing resource utilization. It dynamically provisions nodes tailored to the specific demands of applications, including GPU-heavy AI workloads.

This section integrates Karpenter into the EKS Blueprint framework, highlighting its configuration for both CPU and GPU workloads. The full implementation and configurations are available in the https://github.com/timurgaleev/eks-nim-llm-openai.

Deploying Karpenter with EKS Blueprints

Karpenter is added to the EKS cluster as a Blueprint add-on. Below is an example of the configuration block for enabling Karpenter, focusing on both CPU and GPU workload optimization:

module "eks_blueprints_addons" {
  source  = "aws-ia/eks-blueprints-addons/aws"
  version = "~> 1.2"

  enable_karpenter                  = true
  karpenter_enable_spot_termination = true
  karpenter_node = {
    iam_role_additional_policies = {
      AmazonSSMManagedInstanceCore = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
    }
  }
  karpenter = {
    chart_version = "0.37.0"
  }
}

This configuration enables Karpenter with support for Spot instance termination handling and assigns additional IAM policies for managing nodes.

Configuring Karpenter for CPU and GPU Workloads

For effective scaling, Karpenter relies on Provisioner configurations tailored to workload requirements. The following examples showcase how Karpenter dynamically provisions CPU and GPU nodes.

CPU Workloads

name: cpu-karpenter
clusterName: ${module.eks.cluster_name}
ec2NodeClass:
  karpenterRole: ${split("/", module.eks_blueprints_addons.karpenter.node_iam_role_arn)[1]}
  subnetSelectorTerms:
    id: ${module.vpc.private_subnets[2]}
  securityGroupSelectorTerms:
    tags:
      Name: ${module.eks.cluster_name}-node
  instanceStorePolicy: RAID0

nodePool:
  labels:
    - type: karpenter
    - NodeGroupType: cpu-karpenter
  requirements:
    - key: "karpenter.k8s.aws/instance-family"
      operator: In
      values: ["m5"]
    - key: "karpenter.k8s.aws/instance-size"
      operator: In
      values: ["xlarge", "2xlarge", "4xlarge"]
    - key: "kubernetes.io/arch"
      operator: In
      values: ["amd64"]
    - key: "karpenter.sh/capacity-type"
      operator: In
      values: ["spot", "on-demand"]
  limits:
    cpu: 1000
  disruption:
    consolidationPolicy: WhenEmpty
    consolidateAfter: 180s
    expireAfter: 720h
  weight: 100

GPU Workloads

name: gpu-workloads
clusterName: ${module.eks.cluster_name}
ec2NodeClass:
  karpenterRole: ${split("/", module.eks_blueprints_addons.karpenter.node_iam_role_arn)[1]}
  subnetSelectorTerms:
    id: ${module.vpc.private_subnets[1]}
  securityGroupSelectorTerms:
    tags:
      Name: ${module.eks.cluster_name}-node
  instanceStorePolicy: RAID0

nodePool:
  labels:
    - type: karpenter
    - NodeGroupType: gpu-workloads
  requirements:
    - key: "karpenter.k8s.aws/instance-family"
      operator: In
      values: ["g5", "p4", "p5"]  # GPU instances
    - key: "karpenter.k8s.aws/instance-size"
      operator: In
      values: ["2xlarge", "4xlarge", "8xlarge", "12xlarge"]
    - key: "kubernetes.io/arch"
      operator: In
      values: ["amd64"]
    - key: "karpenter.sh/capacity-type"
      operator: In
      values: ["spot", "on-demand"]
  limits:
    cpu: 1000
  disruption:
    consolidationPolicy: WhenEmpty
    consolidateAfter: 180s
    expireAfter: 720h
  weight: 100

Terraform Automation Scripts

To streamline the deployment and teardown of resources, the project includes two utility scripts: install.sh and cleanup.sh.

• install.sh: Automates the deployment process. It initializes Terraform, applies modules sequentially (e.g., VPC and EKS), and ensures all resources are provisioned successfully. A final Terraform apply captures any remaining dependencies.

• cleanup.sh: Safely destroys the deployed infrastructure. It handles dependencies like Kubernetes services, Load Balancers, and Security Groups, ensuring proper teardown order. Each module is destroyed sequentially, with a final pass to catch residual resources.

These scripts enhance operational efficiency and minimize errors during deployment and cleanup phases, making the workflow more robust and reproducible.

Key Features of Karpenter in AI Ecosystems

1. Dynamic Node Provisioning: Automatically provisions CPU or GPU nodes based on real-time workload needs.

2. Cost Optimization: Leverages Spot instances while ensuring reliable on-demand scaling for critical workloads.

3. Enhanced Resource Utilization: Consolidates underutilized nodes and removes idle resources with disruption policies.

4. Tailored Scaling Policies: Supports node pools for diverse workload types, such as inference tasks or data preprocessing.

Karpenter’s integration with GPU-optimized workloads ensures that demanding AI models benefit from high-performance compute nodes while maintaining cost efficiency.

Use Cases

1. AI Model Training

NVIDIA NIM’s GPU optimizations allow for efficient training of models like BERT or GPT, reducing runtime and costs.

2. Real-Time Inference

Deploy models for real-time applications such as fraud detection, image recognition, or natural language understanding.

3. Experimentation and Research

With the OpenAI Web UI, data scientists can quickly test and iterate on models.

Conclusion

This platform enables the scalable and efficient deployment of AI workloads by integrating NVIDIA NIM with Amazon EKS. Terraform automates the process, ensuring repeatable and reliable setups. With GPU optimization, persistent storage, and observability tools, the platform is well-suited for businesses and researchers alike.

By following this guide, you can build a scalable and efficient AI platform. For detailed code and further exploration, visit the GitHub repository https://github.com/timurgaleev/eks-nim-llm-openai.

In this article, I'll guide you through using Terraform to deploy EKS with essential add-ons, which streamline the configuration and management of your Kubernetes clusters. With these modular add-ons, you can quickly incorporate features like CoreDNS, the AWS Load Balancer Controller, and other powerful tools to customize and enhance your setup. Whether you’re new to container orchestration or just seeking an efficient AWS solution, this guide will help you build a resilient EKS environment in a few straightforward steps.

So let’s start.

Setting Up the VPC for EKS

The VPC configuration is foundational for your EKS cluster, establishing a secure, isolated environment with both public and private subnets. Private subnets are typically used to host your Kubernetes nodes, keeping them inaccessible from the internet. Here’s the configuration provided in the vpc.tf file, which sets up both public and private subnets along with NAT and Internet Gateway options for flexible networking.

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  name                 = local.name
  cidr                 = var.vpc_cidr
  azs                  = local.azs
  secondary_cidr_blocks = var.secondary_cidr_blocks
  private_subnets      = concat(local.private_subnets, local.secondary_ip_range_private_subnets)
  public_subnets       = local.public_subnets
  enable_nat_gateway   = true
  single_nat_gateway   = true
  public_subnet_tags   = {"kubernetes.io/role/elb" = 1}
  private_subnet_tags  = {
    "kubernetes.io/role/internal-elb" = 1
    "karpenter.sh/discovery" = local.name
  }
  tags = local.tags
}

This setup:

• Creates private and public subnets across multiple availability zones.

• Configures a secondary CIDR block for the EKS data plane, which is crucial for large-scale deployments.

• Enables a NAT gateway for private subnets, ensuring secure internet access for internal resources.

• Tags subnets for Kubernetes service and discovery, essential for integration with other AWS services like load balancers and Karpenter.

Deploying EKS with Managed Node Groups

Now that the VPC is configured, let’s move on to deploying the EKS cluster with the eks.tf file configuration. This setup includes defining managed node groups within the EKS cluster, specifying node configurations, security rules, and IAM roles.

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 19.15"

  cluster_name                   = local.name
  cluster_version                = var.eks_cluster_version
  cluster_endpoint_public_access = true
  vpc_id                         = module.vpc.vpc_id
  subnet_ids                     = compact([for subnet_id, cidr_block in zipmap(module.vpc.private_subnets, module.vpc.private_subnets_cidr_blocks) : substr(cidr_block, 0, 4) == "100." ? subnet_id : null])

  aws_auth_roles = [
    {
      rolearn  = module.eks_blueprints_addons.karpenter.node_iam_role_arn
      username = "system:node:{{EC2PrivateDNSName}}"
      groups   = ["system:bootstrappers", "system:nodes"]
    }
  ]

  eks_managed_node_groups = {
    core_node_group = {
      name             = "core-node-group"
      ami_type         = "AL2_x86_64"
      min_size         = 2
      max_size         = 8
      desired_size     = 2
      instance_types   = ["m5.xlarge"]
      capacity_type    = "SPOT"
      labels           = { WorkerType = "SPOT", NodeGroupType = "core" }
      tags             = merge(local.tags, { Name = "core-node-grp" })
    }
  }
}

Key components:

• VPC and Subnets: The vpc_id and subnet_ids reference the private subnets, providing a secure foundation for EKS nodes.

• Managed Node Groups: This setup defines a core node group with spot instances (capacity_type = "SPOT") to optimize cost, with configurable instance types, sizes, and labels for workload placement.

• Security Rules and IAM Roles: Configures additional security rules to manage access between nodes and clusters, along with IAM roles to control permissions for Karpenter and node management.

This configuration gives you a scalable and cost-effective EKS environment that is ready for production workloads, with flexibility to adjust nodes and subnets as needed

Configuring EKS Add-ons

Add-ons enhance your EKS cluster by integrating additional AWS services and open-source tools. With the EKS Blueprints, you can easily set up these add-ons, which range from storage solutions to observability and monitoring tools.

Setting Up the EBS CSI Driver for Persistent Storage

The Amazon EBS CSI Driver is essential for persistent storage on EKS. This module configures the necessary IAM roles for the driver, enabling it to provision and manage EBS volumes.

module "ebs_csi_driver_irsa" {
  source                = "terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks"
  version               = "~> 5.20"
  role_name_prefix      = format("%s-%s-", local.name, "ebs-csi-driver")
  attach_ebs_csi_policy = true
  oidc_providers = {
    main = {
      provider_arn               = module.eks.oidc_provider_arn
      namespace_service_accounts = ["kube-system:ebs-csi-controller-sa"]
    }
  }
  tags = local.tags
}

This configuration creates an IAM role for the EBS CSI Driver using IAM Roles for Service Accounts (IRSA), which allows the driver to interact with EBS securely.

Enabling Amazon CloudWatch Observability

The amazon-cloudwatch-observability add-on integrates CloudWatch for monitoring and logging, providing insights into your cluster’s performance.

eks_addons = {
  amazon-cloudwatch-observability = {
    preserve                 = true
    service_account_role_arn = aws_iam_role.cloudwatch_observability_role.arn
  }
}

This snippet specifies the IAM role required for CloudWatch, enabling detailed observability for your workloads.

Integrating AWS Load Balancer Controller

The AWS Load Balancer Controller allows you to provision and manage Application Load Balancers (ALBs) for Kubernetes services. Here’s how it’s configured:

enable_aws_load_balancer_controller = true
aws_load_balancer_controller = {
  set = [{
    name  = "enableServiceMutatorWebhook"
    value = "false"
  }]
}

The enableServiceMutatorWebhook setting is disabled to avoid automatic modification of service annotations, making it ideal for custom configurations.

Adding Karpenter for Autoscaling

Karpenter is an open-source autoscaler designed for Kubernetes, enabling efficient and dynamic scaling of EC2 instances based on workload requirements. This configuration sets up Karpenter with support for spot instances, reducing costs for non-critical workloads.

enable_karpenter                  = true
karpenter_enable_spot_termination = true
karpenter_node = {
  iam_role_additional_policies = {
    AmazonSSMManagedInstanceCore = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
  }
}
karpenter = {
  chart_version       = "0.37.0"
  repository_username = data.aws_ecrpublic_authorization_token.token.user_name
  repository_password = data.aws_ecrpublic_authorization_token.token.password
}

This configuration includes additional IAM policies for Karpenter nodes, making it easier to integrate with AWS services like EC2 for flexible scaling.

These add-ons, configured through the AWS EKS Blueprints and Terraform, help streamline Kubernetes management on AWS while offering enhanced storage, observability, and autoscaling. ​

To explore the complete configuration, you can find the full code in the GitHub repository https://github.com/timurgaleev/aws-eks-terraform-addons. The repository includes install.sh to deploy the EKS cluster and configure the add-ons seamlessly, along with cleanup.sh to tear down the environment when it’s no longer needed.

Conclusion

This Terraform setup provides a powerful framework for deploying EKS with essential add-ons, such as storage, observability, and autoscaling, to support scalable applications. Specifically, this configuration is designed to enable deployment of applications like OpenAI Chat, showcasing Kubernetes' flexibility for real-time, interactive workloads. With this setup, you’re ready to deploy and manage robust, production-grade EKS clusters in AWS.

As businesses increasingly move towards hybrid and multi-cloud environments, managing infrastructure across multiple platforms has become more complex. However, Amazon Web Services (AWS) has introduced a game-changer for organizations that want the power and flexibility of Kubernetes on their on-premise infrastructure. This is where Amazon EKS Anywhere comes into play. In this article, we’ll explore what EKS Anywhere is, its benefits, and how you can set up and manage Kubernetes clusters on your own on-prem servers using VMware vSphere.

Having recently tested EKS Anywhere with my on-prem servers, I can confidently say that it streamlines the process of deploying and managing Kubernetes clusters without the need for complicated third-party tools. Let's walk through the process, from setup to deployment, with some real-world examples.

What is Amazon EKS Anywhere?

Amazon Elastic Kubernetes Service (EKS) is a managed Kubernetes service provided by AWS for running containerized applications. While it simplifies the Kubernetes management process, it traditionally required cloud infrastructure like AWS EC2 instances. However, with EKS Anywhere, AWS now offers a deployment option for customers to create and manage Kubernetes clusters on their on-premise hardware.

The key benefits of EKS Anywhere are:

1. Consistent Management Experience: It offers the same management tools and experience as Amazon EKS in the AWS Cloud.

2. Open Source: Built on Amazon EKS Distro, an open-source Kubernetes distribution, it allows users to deploy Kubernetes clusters with minimal effort.

3. Integration with AWS Tools: Seamlessly integrates with AWS services like AWS Systems Manager (SSM) for monitoring and operations.

In essence, EKS Anywhere allows you to run Kubernetes on your existing infrastructure, ensuring you still benefit from the rich ecosystem and features AWS provides.

Key Features of EKS Anywhere

• Hardware Support: It runs on your own hardware or on VMware vSphere, making it ideal for on-premise deployments.

• Control Plane: Unlike EKS, where the control plane is managed by AWS, with EKS Anywhere, you manage the control plane yourself.

• Cluster Lifecycle Management: EKS Anywhere includes tooling for automating cluster creation, scaling, updates, and even the destruction of Kubernetes clusters.

• AWS Integration: Easily view and manage your on-prem Kubernetes clusters using the EKS console, integrating seamlessly with AWS Cloud services.

• Support for Third-party Tools: EKS Anywhere supports integrations with tools like Flux for GitOps, eksctl for cluster management, and Cilium for networking.

Setting Up EKS Anywhere on VMware vSphere

For this guide, I’ll walk you through setting up an EKS Anywhere cluster on your on-prem VMware vSphere infrastructure. While you can set up a test cluster on your desktop, here we focus on a more realistic production setup.

Prerequisites:

• VMware vSphere version 7.0 or higher.

• EKS Anywhere tools installed on your machine.

• At least three control plane nodes and three worker nodes for high availability.

Step 1: Install EKS Anywhere CLI Tools

Start by installing the necessary CLI tools. On a Mac, you can do this via Homebrew.

$ brew install aws/tap/eks-anywhere
$ eksctl anywhere version
v0.5.0

Step 2: Generate Cluster Config and Create a Cluster

Let’s create a Kubernetes cluster using eksctl. First, you need to generate a cluster configuration file.

$ CLUSTER_NAME=my-eks-cluster
$ eksctl anywhere generate clusterconfig $CLUSTER_NAME --provider vsphere > $CLUSTER_NAME.yaml

Now that we have the configuration, we can create the cluster on vSphere.

$ eksctl anywhere create cluster -f $CLUSTER_NAME.yaml

The CLI will handle the setup of the control plane, the worker nodes, and the networking components for your cluster. Once the cluster is created, it will be fully operational, and you can use kubectl to interact with it.

Step 3: Export Kubeconfig and Deploy a Test App

Once the cluster is created, you'll have a kubeconfig file to connect to your Kubernetes cluster:

$ export KUBECONFIG=${PWD}/${CLUSTER_NAME}/${CLUSTER_NAME}-eks-a-cluster.kubeconfig
$ kubectl get ns

You can now deploy a simple test application to verify everything is working:

$ kubectl apply -f "https://anywhere.eks.amazonaws.com/manifests/hello-eks-a.yaml"
$ kubectl get pods -l app=hello-eks-a

This will deploy a basic pod that you can access locally:

$ kubectl port-forward deploy/hello-eks-a 8000:80
$ curl localhost:8000

You should see a simple “Hello from EKS Anywhere” message, confirming that the cluster is up and running.

Managing Your Cluster: High Availability and Updates

In a production environment, you’ll want to ensure high availability and smooth updates for your clusters. EKS Anywhere allows you to scale your cluster as needed and manage rolling updates.

For high availability, it's recommended to have at least three control plane nodes and three worker nodes. You can scale the cluster using:

$ eksctl anywhere scale cluster --control-plane-nodes 3 --worker-nodes 3

To update the cluster, use the built-in update tools provided by EKS Anywhere, which work much like the updates on AWS-managed EKS clusters. The update process ensures that your cluster remains stable during the upgrade, even with multiple nodes.

Using EKS Connector for Centralized Management

One of the standout features of EKS Anywhere is EKS Connector, which allows you to manage your on-prem clusters directly from the EKS console. This makes it easy to view and monitor all your Kubernetes clusters, whether they’re running on AWS or on-prem.

To connect your EKS Anywhere cluster to the EKS console:

1. Register the cluster through the EKS console.

2. Download and apply the necessary eks-connector.yaml configuration to your cluster.

3. Once applied, your cluster will be available in the AWS Management Console for monitoring and management.

$ kubectl apply -f eks-connector.yaml

This allows you to manage your on-prem clusters alongside your AWS-based clusters in a single interface.

Conclusion

Amazon EKS Anywhere has made managing on-prem Kubernetes clusters much simpler by bringing AWS-level tools and support to local infrastructures. Whether you're running on VMware vSphere or other compatible environments, EKS Anywhere allows you to benefit from a consistent, simplified management experience, without the need for complex, third-party tools. It also integrates seamlessly with AWS services, making it easy to monitor and scale your infrastructure.

If you're looking to bring Kubernetes to your on-prem servers, EKS Anywhere is an excellent choice that I would highly recommend based on my recent hands-on testing.

1. What is AWS ECS?

AWS Elastic Container Service (ECS) is a fully managed container service built by Amazon to deploy and manage containers. ECS is optimized for simplicity and efficiency, especially for AWS environments, making it a great choice if you want a straightforward, managed solution for running containers without the complexity of Kubernetes.

Example Code for ECS (Using AWS CLI):

bashCopy code# Create an ECS cluster
aws ecs create-cluster --cluster-name my-ecs-cluster

# Register a task definition
aws ecs register-task-definition --family my-task \
  --container-definitions '[{"name":"my-container","image":"nginx","memory":512,"cpu":256}]'

# Run a task in the ECS cluster
aws ecs run-task --cluster my-ecs-cluster --task-definition my-task

2. What is AWS EKS?

AWS Elastic Kubernetes Service (EKS) is a managed Kubernetes service. It allows you to deploy, scale, and operate Kubernetes on AWS, fully aligned with Kubernetes standards. EKS provides flexibility and compatibility with the Kubernetes ecosystem, ideal for teams with experience in Kubernetes or those wanting more control over container orchestration.

Example Code for EKS (Using kubectl and AWS CLI):

bashCopy code# Create a Kubernetes deployment
kubectl create deployment nginx-deployment --image=nginx

# Expose the deployment as a service
kubectl expose deployment nginx-deployment --port=80 --target-port=80 --type=LoadBalancer

3. Key Differences Between ECS and EKS

The primary difference between ECS and EKS lies in the underlying orchestration. ECS is AWS-native, offering tight integration and simplified operations but is exclusive to AWS. EKS, being Kubernetes-based, is portable and lets you run the same configurations across different cloud providers or on-premises if you use Kubernetes elsewhere.

4. Advantages of AWS ECS

• Simplicity: ECS is straightforward to set up, especially if you’re already working with other AWS services.

• Tight AWS Integration: ECS has deep integration with AWS IAM, CloudWatch, and other services, making security and monitoring seamless.

• Lower Management Overhead: ECS manages most of the infrastructure, so you don’t have to worry about the underlying components like control planes or etcd clusters.

5. Advantages of AWS EKS

• Kubernetes Compatibility: EKS is compatible with Kubernetes, making it ideal for teams familiar with Kubernetes and tools like Helm, kubectl, and Prometheus.

• Hybrid and Multi-Cloud Flexibility: Since it’s based on Kubernetes, EKS allows applications to be portable, ideal for multi-cloud or hybrid environments.

• Extensibility: EKS enables integration with a wide array of Kubernetes plugins and tools, giving developers more control and customization options.

6. When to Choose ECS Over EKS

If your team values simplicity and deep AWS integration, ECS can be an excellent choice. ECS is also ideal when running smaller applications or when your team prefers a managed service that takes care of infrastructure details. ECS may require less management and works well when you need to deploy on AWS alone without multi-cloud portability.

7. When to Choose EKS Over ECS

EKS is a powerful choice if your team has Kubernetes experience or needs hybrid cloud deployment. EKS enables portability, so if there’s a need to run parts of your app on other clouds or on-premises, EKS is better. Kubernetes allows more control over networking, storage, and plugins—ideal for complex applications.

8. Pros and Cons Summary

9. Which to Use: Practical Scenarios

For example, if you’re a small team running microservices exclusively on AWS, ECS will likely meet your needs with less management overhead. However, if you’re developing a complex, multi-tiered application that may need to scale across multiple clouds, EKS could be more suitable.

10. Conclusion

While both AWS ECS and EKS are strong options, the choice depends on your team’s needs, skill level, and deployment goals. ECS is straightforward and integrates deeply into AWS, making it perfect for teams focused on AWS-native applications. EKS, on the other hand, is ideal for those who want flexibility, Kubernetes compatibility, and multi-cloud options. For most straightforward applications, ECS is often the preferred choice, but EKS brings value for larger and more complex architectures. Choose wisely based on your priorities, but remember that both services are backed by AWS, ensuring scalability and reliability.

AWS CodeBuild is a fully managed, scalable, and flexible build service that compiles your source code, runs tests, and produces software artifacts. With the addition of GitLab and GitLab Self-Managed as source providers, developers can now seamlessly connect their projects to AWS CodeBuild and automate build processes.

To set up a connection between your GitLab repository and AWS CodeBuild, follow these steps:

1. Navigate to the AWS CodeBuild console in your AWS Management Console.

2. Create a new build project or select an existing one.

3. In the "Source" section, choose "GitLab" as your source provider

   

4. Provide the necessary details about your GitLab repository, such as the project URL and branch name.

5. Create or select an existing IAM role with appropriate permissions for AWS CodeBuild to interact with your GitLab repository, AWS resources, and other required services.

6. Set up any necessary buildspec files or configurations for your project.

    version: 0.2
   
    phases:
      install:
        runtime-versions:
          nodejs: 14
      build:
        commands:
          - npm install
          - npm run build
   

7. Complete the setup process and start or schedule a new build.

The integration between GitLab and AWS CodeBuild enables developers to take advantage of the following benefits:

• Streamlined CI/CD processes: With direct access to your GitLab repositories, AWS CodeBuild can automatically initiate builds when changes are detected in the source code. This automation reduces manual intervention and accelerates development cycles.

• Enhanced security: By establishing a secure connection using an access token, AWS CodeBuild can interact with your GitLab repository and other related resources while maintaining the necessary security measures.

• Scalability: AWS CodeBuild offers a highly scalable build service, allowing you to handle multiple builds concurrently and efficiently. This capability is particularly valuable for large projects or teams that require parallel processing.

• Flexibility: The integration supports both GitLab and self-managed GitLab instances, providing developers with the flexibility to choose their preferred source code management solution.

In conclusion, the integration of GitLab and GitLab Self-Managed as source providers in AWS CodeBuild is a significant step forward for streamlined CI/CD processes. By enabling builds to be initiated directly from changes in GitLab repositories, developers can now enjoy an even more efficient and secure development experience when working with AWS services.

At the last AWS re:Invent, there were numerous updates related to ML/AI and everything associated with these technologies. I also decided to delve into these topics and immerse myself in this field.

I won't delve into explaining the meanings of ML, AI, DL(Deep Learning), and GenAI. However, I'd like to touch upon FMs and LLM as we will focus our attention there. I found myself losing the same question when I came across this topic in my reading or listening. :)

Foundational Models (FMs) within the AWS ecosystem represent fundamental structures and algorithms essential for diverse AI applications. These models, often created by industry-leading AI companies, are integral to the development and functionality of AWS services, shaping the landscape of artificial intelligence on the platform. In the context of Amazon Bedrock, Language Models (LMs) play a pivotal role. These LMs contribute to the service's linguistic capabilities, facilitating advanced language understanding and content generation within the AWS environment.

AWS provides various services for Machine Learning and Artificial Intelligence, including Amazon SageMaker, AWS DeepLens, AWS DeepComposer, Amazon Forecast and more. Familiarize yourself with the services available to determine which ones suit your specific needs.

Generative Artificial Intelligence (GenAI) is a type of artificial intelligence that can generate text, images, or other media using generative models. AWS offers a range of services for building and scaling generative AI applications, including Amazon SageMaker, Amazon Rekognition, AWS DeepRacer, and Amazon Forecast. AWS has also invested in developing foundation models (FMs) for generative AI, which are ultra-large machine learning models that generative AI relies on. AWS has also launched the Generative AI Innovation Center, which connects AWS AI and ML experts with customers around the world to help them envision, design, and launch new generative AI products and services. Generative AI has the potential to revolutionize the way we create and consume media, but it is important to use it responsibly and ethically.

Some examples GenAI: One of the most well-known examples of GenAI is ChatGPT, launched by OpenAI, which became wildly popular overnight and galvanized public attention. Another model from OpenAI, called text-embedding-ada-002, is specifically designed to work with embeddings a type of database specifically designed to feed data into large language models (LLM). However, it’s important to note that generative AI creates artifacts that can be inaccurate or biased, making human validation essential and potentially limiting the time it saves workers. Therefore, end users should be realistic about the value they are looking to achieve, especially when using a service as is.

I've also delved a bit deeper into Broad AI when learning GenAI and I'd like to show this in the form of the following picture as it explains a lot.

Broad AI includes task-specific algorithms, Machine Learning (ML), and Deep Learning. These layers enable AI to perform tasks like image recognition, natural language processing, and complex pattern modeling.

The transition to GenAI involves Transfer Learning, Reinforcement Learning, and Autonomous Learning. These layers allow AI to apply knowledge across contexts, learn from interactions, and independently gather and learn from information.

So, the journey from Broad AI to GenAI represents significant leaps in AI capabilities, moving towards AI systems that can truly understand, learn, and adapt like a human brain.

Let's explore a couple of AWS services that, from my perspective, are among the more popular today.

Amazon SageMaker:

Amazon SageMaker is a comprehensive platform that simplifies the machine learning workflow. It covers everything from data labeling and preparation to model training and deployment. Take advantage of SageMaker's Jupyter notebook integration for interactive data exploration and model development. The platform also supports popular ML frameworks like TensorFlow and PyTorch.

Amazon Q is a groundbreaking Generative AI assistant crafted with a focus on security and privacy. Its purpose is to unleash the transformative capabilities of this technology for employees within organizations of varying sizes and across diverse industries.

Introduces robust enhancements to the generative AI service, Amazon Bedrock.

Amazon Bedrock, an entirely managed service on AWS, provides access to extensive language models and other foundational models (FMs) from prominent artificial intelligence (AI) companies such as AI21, Anthropic, Cohere, Meta, and Stability AI, all consolidated through a unified API.

I would also like to share more information about Amazon Bedrock here about the innovations that were announced at the latest AWS re:Invent.

Fine-tuning for Amazon Bedrock:
Now, there are increased opportunities for model customization in Amazon Bedrock, featuring fine-tuning support for Cohere Command Lite, Meta Llama 2, and Amazon Titan Text models, with Anthropic Claude's support expected soon.

These recent enhancements to Amazon Bedrock significantly reshape how organizations, regardless of their size or industry, can leverage generative AI to drive innovation and redefine customer experiences.

AWS is compatible with all the leading deep-learning frameworks, facilitating their deployment. The deep-learning Amazon Machine Image, accessible on both Amazon Linux and Ubuntu, allows for the creation of managed, auto-scalable GPU clusters. This enables training and inference processes to be conducted at any scale. Also, AWS offers a range of AI services that allow you to integrate pre-trained models into your applications without the need for deep expertise in machine learning. Services like Amazon Rekognition for image and video analysis, Amazon Comprehend for natural language processing, and Amazon Polly for text-to-speech can enhance your applications with AI capabilities.

The best way to solidify your understanding of ML, AI, and GenAI on AWS is through hands-on projects. Start with simple projects and gradually increase complexity as you gain confidence. Use datasets available on platforms like Kaggle or create your own to train and test models.

Conclusion:

Embarking on a journey into Machine Learning, Artificial Intelligence, and Generative Artificial Intelligence on AWS is an exciting endeavor. By following these steps, you can lay a solid foundation for your understanding and proficiency in leveraging AWS services for ML and AI applications. Remember, the key to success is a combination of hands-on experience, continuous learning, and active engagement with the AWS community. Happy training!

Infrastructure Modeling

CloudFormation uses YAML/JSON templates that define resources sequentially.

CloudFormation uses JSON/YAML templates to define AWS resources and their properties sequentially. Resources are created in the order defined in the template.

Terraform uses declarative configuration files and references between resources.

Terraform uses declarative configuration files written in HCL to define resources. Resources can reference attributes of other resources to establish dependencies between them in a flexible way.

Example

# CloudFormation
Resources:
  VPC:
    Type: AWS::EC2::VPC

  Subnet:
    Type: AWS::EC2::Subnet 
    Properties: 
      VpcId: !Ref VPC

# Terraform
resource "aws_vpc" "main" {}

resource "aws_subnet" "example" {
  vpc_id = aws_vpc.main.id
}

State Management

CloudFormation relies on the template to implicitly define the desired state. It does not maintain an explicit real-time state of deployed resources.

Terraform explicitly tracks the real-time state of all resources in a state file, usually stored locally or in remote storage like S3. This allows checking differences between the configuration and current state to maintain consistency.

Programming Interface

CloudFormation provides CLI and APIs.

CloudFormation provides a CLI and AWS APIs for managing templates and deployments. Custom logic can be added through custom resources.

Terraform offers rich plugins and SDK for custom providers.

In addition to the CLI and APIs, Terraform has a rich plugin ecosystem and supports programming infrastructure with its own API and SDK. This allows writing custom providers, provisioners and other automation tools.

Use Cases

• Simple single AWS account deployments use CloudFormation

• Complex multi-account infrastructure uses Terraform

• Automating tasks beyond IaC requires Terraform

For example, a multi-tier app could use:

• CloudFormation for per-account VPCs and load balancers

• Terraform for cross-account databases/queues

• Custom Terraform provider to deploy containers

Other Considerations

• Version control

• Stack policies

• Change sets

• Target types

• Modules

• Automation

• IDE integration

In summary, while both serve IaC purposes, Terraform provides more flexibility, portability and automation capabilities - especially for multi-account, hybrid infrastructure deployments at scale.

Deploying Lambda functions to EKS

The main way to integrate EKS and Lambda is by deploying Lambda functions as Kubernetes deployments and services. This allows Kubernetes to manage and orchestrate the execution of Lambda code.

The steps to deploy a Lambda function to EKS are:

1. Create a Lambda function using the AWS CLI or SDK. This deploys the code and configuration to Lambda.

2. Create a Kubernetes Deployment and Service that points to the Lambda function ARN (Amazon Resource Name). The service exposes the function through a ClusterIP.

3. Kubernetes will trigger invocations of the Lambda function through the service endpoint. It handles load balancing, auto-scaling and orchestration of the function.

Benefits of using EKS and Lambda together

• Leverage Kubernetes APIs and tools to deploy, manage and scale Lambda functions

• Build serverless applications as Kubernetes workloads for portability across environments

• Take advantage of Kubernetes features like auto-scaling, rolling updates, blue-green deployments etc. for Lambda code

• Integrate Lambda functions into existing container-based applications on EKS

This allows building fully serverless applications that leverage the power of Kubernetes for orchestration along with AWS Lambda's ease of use.

Here is a sample Python code for deploying a Lambda function as a Kubernetes workload:

# Create Lambda function
lambda_client.create_function(
   FunctionName='myfunction',
   Runtime='python3.8', 
   Handler='index.handler',
   Code={
      'ZipFile': bytecode
   }
)

# Create Kubernetes Deployment
api.create_namespaced_deployment(
   namespace='default',
   body={
      'apiVersion': 'apps/v1',
      'kind': 'Deployment',
      'metadata': {
         'name': 'lambda-deploy'
      },
      'spec': {
         'replicas': 1,
         'selector': {
            'matchLabels': {
               'app': 'lambda'
            }
         },
         'template': {
            'metadata': {
               'labels': {
                  'app': 'lambda'
               }
            },
            'spec': {
               'containers': [
                  {
                     'name': 'lambda-container',
                     'image': 'public.ecr.aws/lambda/python:3.8',
                     'env': [
                        {
                           'name': 'AWS_LAMBDA_FUNCTION_NAME', 
                           'value': 'myfunction'
                        },
                        {
                           'name': 'AWS_REGION',
                           'value': 'us-east-1' 
                        }
                     ]
                  }
               ]
            }
         }
      }
   }
)

# Expose Deployment as Kubernetes Service
api.create_namespaced_service(
   namespace='default',
   body={
      'apiVersion': 'v1', 
      'kind': 'Service',
      'metadata': {
         'name': 'lambda-service' 
      },
      'spec': {
         'ports': [
            {
               'port': 8080, 
               'targetPort': 8080
            }
         ],
         'selector': {
            'app': 'lambda'
         }
      }
   }
)

This demonstrates how to deploy a Lambda function as a Kubernetes workload and expose it through a service for invocation. EKS and Lambda provide a powerful way to build serverless applications on Kubernetes.

With this update, you can configure rules to ensure that your NLB only accepts traffic from trusted IP addresses, and centrally enforce access control policies

If you are using EKS just update your LB controller to 2.6.0 version and configure it🫡

Please check out more information here:

https://aws.amazon.com/about-aws/whats-new/2023/08/network-load-balancer-supports-security-groups/

https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html

Let's jump deep

A crucial aspect of configuring an NLB is setting up security groups to control inbound and outbound traffic to and from the load balancer. A security group acts as a virtual firewall, allowing only specific traffic to reach the NLB based on predefined rules. This article will discuss how to configure security groups for an AWS NLB.

Creating a Security Group for NLB

To create a security group for an NLB, follow these steps:

1. Log in to the AWS Management Console and navigate to the VPC dashboard.

2. Click on "Security Groups" in the left-hand menu, then click the "Create security group" button.

3. Enter a name and description for the security group, select the VPC in which to create it, and click "Create."

Configuring Inbound Rules

Once the security group is created, you need to configure inbound rules to allow traffic to reach the NLB. To do this, follow these steps:

1. Click on the security group you just created.

2. Click the "Edit inbound rules" button.

3. Add a rule for each type of traffic you want to allow, specifying the protocol, port range, and source. For example, if you want to allow HTTP traffic from anywhere, add a rule with the following settings:

   • Type: Custom TCP Rule

   • Protocol: TCP

   • Port range: 80

   • Source: 0.0.0.0/0 (or a specific IP address or range)

4. Click "Save rules" to apply the changes.

Configuring Outbound Rules

By default, outbound traffic is allowed from an NLB. However, you can configure outbound rules to restrict the types of traffic that can leave the NLB. To do this, follow these steps:

1. Click on the security group you just created.

2. Click the "Edit outbound rules" button.

3. Add a rule for each type of traffic you want to allow, specifying the protocol, port range, and destination. For example, if you want to allow all outbound traffic, add a rule with the following settings:

   • Type: Allow All Traffic

   • Protocol: All

   • Port range: All

   • Destination: 0.0.0.0/0 (or a specific IP address or range)

4. Click "Save rules" to apply the changes.

Best Practices for Configuring Security Groups

When configuring security groups for an AWS NLB, follow these best practices:

• Allow only the minimum necessary traffic: Only allow the specific types of traffic that your application requires. This reduces the attack surface and helps prevent unauthorized access.

• Use specific sources and destinations: Instead of allowing all traffic from anywhere, specify a specific IP address or range. This provides an additional layer of security.

• Use security groups in combination with network ACLs: Security groups and network access control lists (ACLs) work together to provide an additional layer of security. While security groups are stateful, meaning that they track the state of connections and allow return traffic, network ACLs are stateless and do not track the state of connections.

• Regularly review security group rules: Regularly review your security group rules to ensure that they still meet your needs and are up-to-date with any changes in your application requirements.

Conclusion

Configuring security groups for an AWS NLB is a crucial aspect of setting up and securing your load balancer. By following the best practices outlined in this article, you can ensure that only the necessary traffic is allowed to reach your NLB and that your application remains secure.

With this update you can create a mount point and mount AWS S3 bucket (or a path within a bucket) at the mount point, and then access the bucket using shell commands (ls, cat, dd, find, and so forth), library functions (open, close, read, write, creat, opendir, and so forth) or equivalent commands and functions as supported in the tools and languages that you already use.

Before many AWS users use the S3 APIs and the AWS SDKs to build applications that can list, access, and process the contents of an S3 bucket and now you can more 🫡🥳

Some information about this update:

Pricing – you pay only for the underlying S3 operations.
Performance – Mountpoint is able to take advantage of the elastic throughput offered by S3, including data transfer at up to 100 Gb/second between each EC2 instance and S3.
Credentials – Mountpoint accesses your S3 buckets using the AWS credentials that are in effect when you mount the bucket.
Storage Classes – You can use Mountpoint to access S3 objects in all storage classes except S3 Glacier Flexible Retrieval, S3 Glacier Deep Archive, S3 Intelligent-Tiering Archive Access Tier, and S3 Intelligent-Tiering Deep Archive Access Tier.
Open Source – Mountpoint is open source and has a public roadmap. Your contributions are welcome; be sure to read our Contributing Guidelines and our Code of Conduct first.

Some links:

https://aws.amazon.com/about-aws/whats-new/2023/08/mountpoint-amazon-s3-generally-available/

Let's jump deep

Mountpoint for AWS S3 is an open-source tool that enables you to mount an S3 bucket as a file system in your Linux environment, effectively bridging the gap between object storage and traditional file systems. Developed by MinIO, Mountpoint for AWS S3 brings several benefits to the table, making it easier for businesses to manage their cloud storage and streamline operations.

Ease of Integration

Mountpoint for AWS S3 allows you to mount an S3 bucket as a local file system, enabling seamless integration with existing applications and tools that rely on traditional file I/O operations. This eliminates the need for additional programming or customization efforts when working with object storage, saving both time and resources.

Moreover, Mountpoint for AWS S3 supports various Linux file systems, including XFS, ext4, and Btrfs, providing flexibility in choosing the right file system for your specific use case.

Improved Performance

By mounting an S3 bucket as a file system, Mountpoint for AWS S3 enables you to take advantage of native Linux caching mechanisms, such as the page cache and the dentry cache. These caches help reduce latency and improve throughput by storing frequently accessed data in memory, resulting in faster access times and more efficient data transfers.

Additionally, Mountpoint for AWS S3 supports multi-threaded operations, allowing it to leverage multiple CPU cores for parallel data processing. This further enhances performance and enables you to handle large datasets more efficiently.

Data Durability and Security

Amazon S3 is designed for 99.999999999% durability and provides a range of security features, such as access control policies, encryption, and data integrity checks. Mountpoint for AWS S3 ensures that these benefits are passed on to the file system level, allowing you to maintain the same level of data protection and durability without additional configuration.

Furthermore, Mountpoint for AWS S3 supports object locking, a feature that provides an additional layer of protection against accidental or malicious data modifications. Object locking can be used to create write-once-read-many (WORM) workflows, ensuring that critical data remains immutable and cannot be altered or deleted for a specified retention period.

Cost-Effective Scalability

As your storage needs grow, so does the cost of managing and maintaining on-premises infrastructure. AWS S3 offers a pay-as-you-go pricing model, allowing you to scale your storage capacity without the need for upfront investments or complex capacity planning.

Mountpoint for AWS S3 enables you to tap into this scalability while maintaining a familiar file system interface, making it an attractive option for businesses looking to optimize their cloud storage costs.

Conclusion

Mountpoint for AWS S3 is a powerful tool that simplifies the integration of Amazon S3 with your Linux environment, improves performance, and maintains data durability and security. By bridging the gap between object storage and traditional file systems, Mountpoint for AWS S3 offers a cost-effective and scalable solution for managing your cloud storage needs. Whether you're working with big data, media assets, or backup and archival data, Mountpoint for AWS S3 can help streamline your operations and improve overall efficiency.

So let's start to create:

To get started, install the k8s-cdk library:

npm install --save @kubernetes-cdk/cdk-core

Then create a new CDK app:

cdk init app --language=typescript

This will set up a basic CDK app structure with TypeScript support.

Defining a Cluster

To define a Kubernetes cluster, import the Cluster construct and provide configuration:

import { Cluster } from '@kubernetes-cdk/cdk-core';

//...

new Cluster(this, 'MyCluster', {
  version: k8s.KubernetesVersion.V1_21,
  subnets: [subnet1, subnet2] 
});

This will provision a managed EKS cluster running Kubernetes across the specified subnets.

Deploying Resources

Additional Kubernetes resources like Pods, Services, etc. can then be defined and added to the cluster:

const nginxDeployment = new k8s.Deployment(this, 'NginxDeployment', {
  cluster: cluster,
  spec: {
    selector: {
      matchLabels: {
        app: 'nginx',
      },
    },
    //...
  },
});

Deploying

Finally, synthesize and deploy the CDK app to provision the Kubernetes infrastructure and deploy the resources:

cdk deploy

The k8s-cdk makes it simple to define Kubernetes clusters and applications using familiar AWS CDK patterns. This allows for infrastructure as code deployments of Kubernetes on AWS.

Is Domain-Driven Design usefull for me?

The purpose of Domain-Driven Design is to free the domain code from technical details to have more room to work with its complexity. It is well suited to work with very complex domains and projects that are starting to dive into legacy.

Domain-Driven Design requires an understanding of the business idea or understanding of the final 'business product'. It requires time and commitment from both business experts and technical implementers. Domain-Driven Design should not be used in situations where you need 'quick solutions'. Instead, use Domain-Driven Design for software that supports the core business area rather than supporting areas. Running Domain-Driven Design can be achieved through an event-storming session. However, as mentioned above, this is a commitment worth making. It will allow you to develop software that is more tailored to the needs of your end clients. It will also help create decoupled services that are more scalable and maintainable. The combination will result in greater business agility.

Event Storming

Event Storming helps teams of business and technical people come to a consensus on what the solution should be. This happens without being distracted by the specific implementation details of how it will be implemented. This means, that it may take longer for the teams to start providing source code. However, all teams will be better aligned as to what each microservice should be responsible for. The event storming workshop is a brainstorming session. In this session all stakeholders in the solution work together to define the business events that correspond to the domains. Suppose, we have a commerce development task where the business event might be a customer who applies for a new product. During the workshop, the group will begin to identify the object that triggered the event, the processes that should occur as a result, and any subsequent event triggered by the original event.

To do this, a team brainstorming session takes place where the event groups can identify areas of their business and then the contexts in which they operate. These can be used to define the usage and the relationship that occurs between each microservice and it’s context. Once the domains have been defined with the help of the business experts, the technical implementers can start designing the solution.

The result of the event-storming session is a domain model for development. The domain model can be used to define a number of bounded contexts.

Bounded Contexts

A bounded context is the boundary where each domain applies. The order contract opening example can be thought of as the 'Order Contract Opening Context' in a shop. In a complete system, there may be other contexts such as the product context, the description context and the manufacturing context. Identifying the business events that cause interactions between the different constrained contexts helps to determine how your microservices will interact with each other in the new architecture.

The example context map is just a sample of the core domains. There are also a number of supporting and subsequent domains. Although it is necessary to have a service that manages these, this is not part of the core application domain for sending products.

Core Domains and refactoring

When you start defining "Core Domains and Subsequent Domains" the question that usually arises is how to manage requests between domains.To do this, we'll look at options for using AWS services for how domains can be implemented. Containerisation or serverless diagram of such solutions would rather be a 'modern architecture' than a good old-fashioned network and virtual machine deployment diagram. The advantage of these solutions is that the diagram itself helps to actually outline, what the logical functionality is, since we can be more expressive and fine-grained with resource usage. The undisputed king of serverless computing platforms has been AWS LambdaAWS Lambda for several years now. It satisfies all the aforementioned conditions and can be used in a number of languages/implementations, including TypeScript. Other viable options might include some of the more well-known container services such as AWS ECS or AWS EKS wrapped Fargate. However, they require considerably more setup and configuration, and also require that containerization actually takes place. It doesn't mean that containerisation is bad, in general containerisation can be good, it all depends on your development idea whether it's refactoring into microservices or starting a new application. If you need Eventing then here it is Simple Notification Service (SNS). It is a push-based service, i.e. it automatically handles the distribution of the event to the recipients. SNS uses a pay-per-use model and it is essentially serverless as the only infrastructure you need is the SNS subject. The modern cloud is about using its own API products to expose its applications, rather than building something of its own with Fastify, Kong or the like. The API gateway acts as the only public interface connected to any other infrastructure, in our case primarily our Lambda compute functions, which will respond to paths defined in the gateway. In the case of AWS the service of interest, unsurprisingly, is called the AWS API Gateway.

The following diagram shows how monolith receives some of the traffic during the gradual addition of new microservices in the example application.

There is also AWS Migration Hub, it will help you in finding your domains and even offer AWS services that you can implement. This will help you to plan a refactoring or plan for migrating from your old OnPrem solutions to AWS with all modern solutions.

Conclusion

To summarise, people just don't tend to talk about 'domains' all day. Most employees do not pay attention to the implementation of domains in the organisation. It is also worth noting that dividing systems into domains after they have been fully designed is also useless. DDD should be done, at least approximately, at the initial design stage. But in any case, DDD is the place to be. In the example of using DDD in AWS it looks simple but when you start to go deeper here you find a lot of services where they all have to be interconnected and here comes the methods and dependencies. That's why it's very important to create a structure at the beginning of the work.

Infrastructure management has changed a lot over the years. So much so that the traditional system administrator managed a rack full of servers. In many cases, the initial setup required manual intervention at the console.

Why AWS for WordPress might be a great choice To begin with, AWS is a big deal. It's the cloud hosting provider with the largest market share. AWS is such a successful platform that it accounts for half of Amazon's operating income, which is in the billions of dollars.

AWS offers high scalability, making it ideal for sites with thousands of daily visitors. The platform also allows for any server configuration. This is ideal for high-performance sites such as online stores.

In this article, I will show you how to use Terraform to install Wordpress on AWS.

Our objectives

• Create a ‘tf’ files which will hold all of our relevant configuration information (main.tf, …)

• Define which provider we will be using in the Terraform config. (aws, cloudflare)

• Certificate handling (AWS Certificate Manager)

• Define security group rules and names.

• Define the EC2 instances we want to create. (AWS Auto Scaling Group)

• Run Terraform to plan and apply our configuration.

Before you get started, you’ll need to sign up for AWS. During the process, you’ll need to verify your account using a credit card – onto which they’ll charge $1 – and receive a verification code via SMS. When you’re ready, select the Free support plan and you’ll get access to your console, which is where the magic happens

So, let's start creating our wordpress in AWS

I will assume that you have already configured all the necessary tools to run such as Terraform, aws-cli

The first thing you need to do is clone the repository: $ git clone https://github.com/timurgaleev/wordpress-ec2-rds-alb-vpc.git

Once you've taken the templates from the repository, you now need to configure the 'tf' files, which will hold all of the relevant Terraform configuration. You can encapsulate it in one file, but for simplicity and convenience we will work within multiple files.

The structure of the repository is as follows:

acm.tf - AWS Certificate Manager Terraform module alb.tf - AWS Application and Network Load Balancer Terraform module asg.tf - AWS Auto Scaling Group (ASG) Terraform module cloudflare.tf - Cloudflare Provider efs.tf - Provides an Elastic File System (EFS) File System resource output.tf - Terraform Output Values rds.tf - AWS RDS Terraform module security_group.tf - AWS EC2-VPC Security Group Terraform module vpc.tf - AWS VPC Terraform module variables.tf - variables used in Terraform.

Prerequisites needed for creating a envirenment provider.tf you will need a change.

backend "s3" {
    bucket         = "ecs-terraform-examplecom-state"
    key            = "example/com.tfstate"
    region         = "eu-west-1"
    encrypt        = "true"
    dynamodb_table = "ecs-terraform-remote-state-dynamodb"
  }

It will also use an S3 bucket that will be used as a remote store for our Terraform state. This allows multiple users to work with one set of Infrastructure as Code without causing conflicts.

Then change the variables according to your needs. variables.tf like environment, Domain, Cloudflare API Token

variable "asg_min_size" {
  description = "AutoScaling Group Min Size "
  default     = 1
}

variable "asg_max_size" {
  description = "AutoScaling Group Max Size "
  default     = 2
}

variable "asg_desired_capacity" {
  description = "AutoScaling Group Desired Capacity"
  default     = 1
}

variable "rds_engine" {
  description = "RDS engine"
  default     = "mariadb"
}

variable "rds_engine_version" {
  description = "RDS engine version"
  default     = "10.6.7"
}

variable "rds_instance_class" {
  description = "RDS instance class"
  default     = "db.t3.micro"
}

variable "site_domain" {
  description = "Domain"
  default     = "example.com"
}

variable "cloudflare_zone" {
  description = "cloudflare Zone Id"
}

variable "dns_ttl" {
  description = "cloudflare for dns = 1 is automatic."
  default     = 1
}

variable "dns_allow_overwrite_records" {
  description = "cloudflare allow overwrite records."
  default     = true
}

variable "cloudflare_api_token" {
  description = "cloudflare api token"
}

variable "ssh_key_name" {
  description = "SSH Key"
}

Now we can run our WordPress.

• Run terraform init

• Run terraform plan and review

• Run terraform apply

Now enter your server domain, and you'll see the following WordPress installation screen. If that's the case, great job! If you don't see that screen, you may need to double-check the steps you followed.

You can destroy this WordPress by running:

terraform plan -destroy
terraform destroy  --force

Conclusion We've only had a superficial look at how Terraform can be used in AWS, but I think a simple introduction is the best part! We created a database in RDS and plugged it into Wordpress as well as created and wired the necessary data into Cloudflare.

In this video, we explained AWS organization's best practices.

Benefits of using AWS Control Tower and AWS Config.

How to connect Azure AD to AWS IAM Identity Center (AWS SSO) and how you can manage users via AD.

Some examples of open source Service Meshes like Linkerd, Istio and Kuma are ways to control how different parts of an application share data with each other. Service Meshes provide an overarching view of your service and aid with complex activities like tests, roll-outs, access restrictions and end-to-end authentication.

Service Mesh helps you push operational issues into the infrastructure so the application code is easier to understand, maintain and adapt. Service Mesh integration, explains how to manage traffic management, security and policy to manage a microservice architecture.

So, what does it mean? Service Mesh in the real case and what will it solve. Let’s look at some examples:

Applications are monolithic

Typically applications are monolithic, meaning it is one program, built as one binary and ran as one process.

Example of a Monolithic application

You got a mesh of services and the following questions that you will have to solve:

• Scale and Release. Scaling a monolithic application is difficult and difficult to maintain some autoscaling. Lots of teams trying to make changes at the same time, and get your own version of many bugs.
• Reduced flexibility of technology. Trying new technologies (e.g., a new programming language) without changes the entire code base to them is often problematic.
• Sprints into team dynamics. Last but not least, it’s harder to draw boundaries of responsibility, assign roles and develop a team when you only have one big result.

Many of these issues are only a problem when a particular scale is reached. In most cases a monolithic application is a reasonable choice, again, this depends on requirements. However, you have started to scale and this is no longer a choice for you.

Microservices implementation

Microservices architecture helps developers modify application services without redeployment. The difference in the different architectures of the applications being developed is that individual microservices are created by small teams with a choice of their own tools and coding languages. In general, microservices are created independently of each other, communicate with each other, and can individually fail without causing disruption to the entire application.

A typical application is like a monolith, it consists of several logical modules: such as frontend, backend, database and so on. Communication between services is what makes microservices unique. The logic that controls communication can be encoded in each service without a service mesh layer, but as communication becomes more complex, the service mesh becomes more valuable. For cloud applications built in a microservices architecture, service meshing is a way to combine a large number of individual services into a functional application.

Example microservices architecture

Moving to microservices, you will get the following benefits:

• You can develop each service independently of the other.
• You can scale each service independently.
• You are free to write each service using the technology of your choice, as long as they use the same communication interface between them.

Although the Service Mesh architecture is not limited to microservice-based systems, they provide a good example of how the service mesh is used in action.

You could shift the mentioned problems to the microservices by giving them additional, infrastructure-specific logic. This would then mean the following:

• You will be forced to recreate this logic for every technology stack you use.
• The application would expand considerably beyond its business needs. So do developers want to work on the application or on the infrastructure?

So you need some sort of software stack that modernizes your deployment and allows your services to discover each other, control traffic and policies, and provide observability, ideally without modifying the services themselves. This infrastructure solution is called the Service Mesh.

How can the integration of the service mesh optimize communication?

Each service that is added to an application, or a new instance of an existing service running in a container, complicates the communication environment and creates new points of potential failure. In a complex microservices architecture, it can become nearly impossible to identify where problems are occurring without the Service Mesh.

This happens because the Service Mesh also captures every aspect of communication between services in the form of performance metrics. For example, if a particular service fails, the Service Mesh can collect data on how long it took before a successful retry. As failure time data is accumulated for a given service, rules can be written to determine the optimal wait time before re-requesting that service, so that the system is not overwhelmed by unnecessary re-requests.

Conclusion

Implementing a Service Mesh to manage system communications can accelerate microservice deployment by providing a consistent approach that manages key network functions. The Service Mesh architecture is ideal for systems with a large number of services because it allows network issues to be separated from the application code and policies to be applied from a central source of truth, either universally or selectively based on specified criteria.

I hope this explanation can help you to choose the right mental model for understanding the source of the problems that Service Mesh solves. In the next article, I would like to explain how the AWS App Mesh can be used, as well as the pros and cons.