Kubernetes Cluster Requirements¶
K8s Cluster¶
Depending on the cluster, the K8s cluster must be available either with the node auto-scaling or with a fixed node size. A PVC Provisioner must be available for the cluster.
Cluster Sizing¶
| Profile | Nodes | CPU (per node) | Memory (per node) | Storage (shared) |
|---|---|---|---|---|
| Xsmall (POV Only) | 1 | 16 vCPUs | 96 GB | 500 GB |
| Medium (Minimum Profile) | 6 | 16 vCPUs | 64 GB | 6 TB |
Important Notes:
- Entire shared storage needs to be allocated at the time of install. Resizing the storage post install is not supported.
- For call volume beyond 3 billion/month, the sizing depends on various factors. Reach out to support@harness.io for the assessment.
Persistent Volumes¶
Traceable needs both RWO and RWX types of Persistent Volumes.
| Provider | RWO (ReadWriteOnce) | RWX (ReadWriteMany) |
|---|---|---|
| EKS | EBS | EFS |
| AKS | Azure Disk | Azure Files |
| GKE | Persistent Disk | File Store |
| Local | nfs-client | nfs-client |
| OpenShift | Ceph RBD | CephFS |
Additional Resources¶
After completion of these steps, note down the name of the storage classes for each since it is needed before the Traceable Platform Installation.
NOTE: If you use a multizone cluster, ensure that the StorageClasses you choose are Zone Redundant Storage.
Loadbalancer / Traffic Routing¶
Traceable uses the Ingress-Nginx Controller for traffic routing.
Important Configuration Notes¶
- Ensure that
allow-snippet-annotations: "true"is present if the ingress-nginx controller version is 1.9 or greater. - If you are using 1.12.1 version of ingress-nginx controller, then we need to add below entries:
data:
allow-snippet-annotations: "true"
annotations-risk-level: Critical
strict-validate-path-type: "false"
- Make sure HTTP2 is enabled if you are using cloud or self-managed LB
DNS¶
A dedicated DNS name must be available for the Traceable Platform. This DNS is used for:
- Agent (TPA) traffic to the platform
- UI access
- Internal service communication (e.g.,
iam-v2connecting to the platform)
DNS Resolution Requirements¶
The DNS must be resolvable from:
- User machines - For accessing the platform UI
- Kubernetes cluster - Services running inside the cluster must be able to connect to the platform using the DNS name
DNS Target by Deployment Type¶
| Deployment Type | DNS Should Point To | Notes |
|---|---|---|
| VM-based (Single/Multi Node) | Master node IP(s) | DNS record can be created before installation |
| Cloud-based (EKS/AKS/GKE) | Load Balancer | LB is created during platform installation; update DNS after TI install |
| OpenShift | OpenShift Route canonical hostname | Route is created during installation; update DNS after TI install |
DNS Configuration Timing¶
VM-based installations:
- Create the DNS record pointing to master node(s) before starting the installation
Cloud-based and OpenShift installations:
- The ingress (nginx for cloud, OpenShift Route for OCP) is deployed as part of the platform installation
- Create a placeholder DNS or update the DNS record after installing TI and obtaining the Load Balancer IP or Route hostname
Verify DNS Resolution from Cluster¶
After DNS is configured, verify that pods can resolve the DNS:
Network Requirements¶
There should not be any firewalls between the traceable nodes. The network must support 10Gbps.
Permissions¶
The user must have admin permissions either on the cluster or on a dedicated namespace in the cluster.
Time Sync¶
Both the Kubernetes nodes and the TPA machine must have their NTP time synchronised. Since spans and heartbeats rely on timestamps, any time discrepancy between the TPA and the platform could lead to potential issues.