Skip to content

Kubernetes Cluster Requirements

K8s Cluster

Depending on the cluster, the K8s cluster must be available either with the node auto-scaling or with a fixed node size. A PVC Provisioner must be available for the cluster.

Cluster Sizing

Profile Nodes CPU (per node) Memory (per node) Storage (shared)
Xsmall (POV Only) 1 16 vCPUs 96 GB 500 GB
Medium (Minimum Profile) 6 16 vCPUs 64 GB 6 TB

Important Notes:

  1. Entire shared storage needs to be allocated at the time of install. Resizing the storage post install is not supported.
  2. For call volume beyond 3 billion/month, the sizing depends on various factors. Reach out to support@harness.io for the assessment.

Persistent Volumes

Traceable needs both RWO and RWX types of Persistent Volumes.

Provider RWO (ReadWriteOnce) RWX (ReadWriteMany)
EKS EBS EFS
AKS Azure Disk Azure Files
GKE Persistent Disk File Store
Local nfs-client nfs-client
OpenShift Ceph RBD CephFS

Additional Resources

After completion of these steps, note down the name of the storage classes for each since it is needed before the Traceable Platform Installation.

NOTE: If you use a multizone cluster, ensure that the StorageClasses you choose are Zone Redundant Storage.

Loadbalancer / Traffic Routing

Traceable uses the Ingress-Nginx Controller for traffic routing.

Important Configuration Notes

  • Ensure that allow-snippet-annotations: "true" is present if the ingress-nginx controller version is 1.9 or greater.
  • If you are using 1.12.1 version of ingress-nginx controller, then we need to add below entries:
data:
  allow-snippet-annotations: "true"    
  annotations-risk-level: Critical
  strict-validate-path-type: "false"
  • Make sure HTTP2 is enabled if you are using cloud or self-managed LB

DNS

A dedicated DNS name must be available for the Traceable Platform. This DNS is used for:

  • Agent (TPA) traffic to the platform
  • UI access
  • Internal service communication (e.g., iam-v2 connecting to the platform)

DNS Resolution Requirements

The DNS must be resolvable from:

  1. User machines - For accessing the platform UI
  2. Kubernetes cluster - Services running inside the cluster must be able to connect to the platform using the DNS name

DNS Target by Deployment Type

Deployment Type DNS Should Point To Notes
VM-based (Single/Multi Node) Master node IP(s) DNS record can be created before installation
Cloud-based (EKS/AKS/GKE) Load Balancer LB is created during platform installation; update DNS after TI install
OpenShift OpenShift Route canonical hostname Route is created during installation; update DNS after TI install

DNS Configuration Timing

VM-based installations:

  • Create the DNS record pointing to master node(s) before starting the installation

Cloud-based and OpenShift installations:

  • The ingress (nginx for cloud, OpenShift Route for OCP) is deployed as part of the platform installation
  • Create a placeholder DNS or update the DNS record after installing TI and obtaining the Load Balancer IP or Route hostname

Verify DNS Resolution from Cluster

After DNS is configured, verify that pods can resolve the DNS:

kubectl run -it --rm dns-test --image=busybox --restart=Never -- nslookup <your-platform-dns>

Network Requirements

There should not be any firewalls between the traceable nodes. The network must support 10Gbps.

Permissions

The user must have admin permissions either on the cluster or on a dedicated namespace in the cluster.

Time Sync

Both the Kubernetes nodes and the TPA machine must have their NTP time synchronised. Since spans and heartbeats rely on timestamps, any time discrepancy between the TPA and the platform could lead to potential issues.