AWS EKS Configuration¶

Overview¶

This guide covers Amazon EKS-specific configurations and best practices for deploying Traceable Platform.

Networking/VPC¶

1.1 Private Subnets with VPC Endpoints¶

If EKS nodes are deployed in fully private subnets (without NAT gateway rules), then ensure that VPC endpoints are set up for pods to access AWS services like EFS, ECR, EC2, etc.

Reference: AWS EKS Private Clusters

Required VPC Endpoints for Private Clusters¶

When running in fully private subnets, configure VPC endpoints for:

Amazon EFS - For ReadWriteMany (RWX) persistent volumes
Amazon ECR - For pulling container images
Amazon EC2 - For node management
Amazon S3 - For ECR image layers
Amazon STS - For IAM authentication

1.2 Subnet IP Address Availability¶

Ensure that each subnet associated with the EKS cluster has at least 256 Available IPv4 addresses.

Why 256 IPs?¶

Each pod gets its own IP address in EKS
Nodes need IPs for system pods, application pods, and overhead
Insufficient IPs can prevent pod scheduling and cluster scaling

Verification¶

Check available IPs in your subnets:

aws ec2 describe-subnets --subnet-ids <subnet-id> \
  --query 'Subnets[*].[SubnetId,AvailableIpAddressCount]' \
  --output table

Nodes per Zone¶

The Challenge¶

EBS volumes in EKS are tied to a specific availability zone. If an EBS volume is created in a particular zone, it can only be accessed by nodes within the same zone. This setup presents challenges if a node in that availability zone goes down.

The Solution¶

The number of nodes in each availability zone should always be greater than 1.

Why Multiple Nodes per Zone?¶

If one of the nodes in a zone fails, then pods that rely on EBS volumes in that zone will be scheduled on the second node in the zone. This ensures:

High availability for stateful workloads
Minimal disruption during node failures
Proper pod rescheduling without cross-zone volume attachment issues

Best Practices¶

Minimum 2 nodes per AZ: Always maintain at least 2 nodes in each availability zone
Use Node Groups: Configure node groups to distribute nodes evenly across AZs
Enable Auto-Scaling: Configure cluster autoscaler to maintain minimum node count per AZ
Monitor Node Health: Set up alerts for node failures to ensure quick response

Example Node Group Configuration¶

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
  name: traceable-cluster
  region: us-west-2

nodeGroups:
    - name: traceable-nodes
    instanceType: m5.4xlarge
    desiredCapacity: 6
    minSize: 6
    maxSize: 12
    availabilityZones:
      - us-west-2a
      - us-west-2b
      - us-west-2c
    # This ensures 2 nodes per AZ (6 nodes / 3 AZs)

Storage Classes¶

EBS (ReadWriteOnce)¶

For RWO volumes, use EBS-backed storage class:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ebs-sc
provisioner: ebs.csi.aws.com
volumeBindingMode: WaitForFirstConsumer
parameters:
  type: gp3
  iops: "3000"
  throughput: "125"

EFS (ReadWriteMany)¶

For RWX volumes, use EFS-backed storage class:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: efs-sc
provisioner: efs.csi.aws.com
parameters:
  provisioningMode: efs-ap
  fileSystemId: fs-XXXXXXX # The EFS File System ID from Step #3
  directoryPerms: "775"
  basePath: "/traceable"
  gid: "65532"
  uid: "65532"

IAM Roles and Policies¶

Ensure the following IAM roles and policies are configured:

EBS CSI Driver: IAM role for EBS volume management
EFS CSI Driver: IAM role for EFS access
Cluster Autoscaler: IAM role for auto-scaling
Load Balancer Controller: IAM role for ALB/NLB management

Exposing the Platform¶

On EKS, the recommended way to reach the platform is a Service of type LoadBalancer on the traceable-router, which provisions an AWS NLB. The detailed NLB scenarios (TLS passthrough, ACM termination, and re-encryption) are documented in Connecting to the Traceable Platform.

Support¶

For EKS-specific deployment assistance, contact support@harness.io.