Traceable Platform - Multi-Node Installation¶

This guide provides step-by-step instructions for installing the Traceable Platform on a multi-node Kubernetes cluster.

Overview¶

Multi-node installation is suitable for:

Production environments
High availability deployments
Large-scale deployments with significant traffic

For development or testing environments, see Single Node Installation.

Prerequisites¶

Before proceeding, ensure you have:

Completed the Platform Checklist
An existing Kubernetes cluster (cloud-based or VM-based)
Storage classes configured (block storage and shared storage)
NFS server (for VM-based clusters without shared storage)
Registry credentials
kubectl access to the cluster

Step 1: Download ClusterMgr¶

Export Environment Variables¶

export CLUSTERMGR_VERSION=<version>

Download ClusterMgr¶

curl -L https://downloads.traceable.ai/platform/installer/clustermgr/linux-amd64/traceable-clustermgr-${CLUSTERMGR_VERSION}-linux-amd64.tgz | tar -zxv
cd traceable-clustermgr-${CLUSTERMGR_VERSION}

Step 2: Create Override Helm Values¶

Choose the appropriate configuration based on your cluster type.

Option A: Cloud-Based K8s with Nginx Ingress Controller¶

For existing cloud-based Kubernetes clusters with Nginx Ingress controller already installed.

Export Variables:

export STORAGE_CLASS=<your-storage-class>
export SHARED_STORAGE_CLASS=<your-shared-storage-class>
export PROFILE=<deployment-profile>

Create Values File:

cat << EOF > traceable-installer-values.yaml
cluster:
  profile: ${PROFILE}
  tfi:
    storageClass: ${STORAGE_CLASS}
    sharedPvStorageClass: ${SHARED_STORAGE_CLASS}
storageClassName: ${STORAGE_CLASS}
nfs-provisioner:
  enabled: false
traceable-nginx:
  enabled: false
EOF

Option B: VM-Based K8s with NFS (No Nginx Ingress)¶

For existing VM-based Kubernetes clusters with NFS server but without Nginx Ingress controller.

Export Variables:

export STORAGE_CLASS=<your-storage-class>
export SHARED_STORAGE_CLASS=<your-shared-storage-class>
export PROFILE=<deployment-profile>
export NFS_SERVER=<nfs-server-ip>
export NFS_PATH=<nfs-export-path>

Create Values File:

cat << EOF > traceable-installer-values.yaml
cluster:
  profile: ${PROFILE}
  tfi:
    storageClass: ${STORAGE_CLASS}
    sharedPvStorageClass: ${SHARED_STORAGE_CLASS}
storageClassName: ${STORAGE_CLASS}
nfs-provisioner:
  enabled: true
  nfs:
    server: ${NFS_SERVER}
    path: ${NFS_PATH}
traceable-nginx:
  enabled: true
EOF

Option C: VM-Based K8s with NFS and Nginx Ingress¶

For existing VM-based Kubernetes clusters with NFS server and Nginx Ingress controller already installed.

Export Variables:

export STORAGE_CLASS=<your-storage-class>
export SHARED_STORAGE_CLASS=<your-shared-storage-class>
export PROFILE=<deployment-profile>
export NFS_SERVER=<nfs-server-ip>
export NFS_PATH=<nfs-export-path>

Create Values File:

cat << EOF > traceable-installer-values.yaml
cluster:
  profile: ${PROFILE}
  tfi:
    storageClass: ${STORAGE_CLASS}
    sharedPvStorageClass: ${SHARED_STORAGE_CLASS}
storageClassName: ${STORAGE_CLASS}
nfs-provisioner:
  enabled: true
  nfs:
    server: ${NFS_SERVER}
    path: ${NFS_PATH}
traceable-nginx:
  enabled: false
EOF

Step 3: Additional Configuration Options¶

Pod Security Context¶

Enable pod security context for environments requiring specific user/group IDs:

cluster:
  tfi:
    podSecurityContextEnabled: true

Custom User and Group IDs¶

For environments with specific UID/GID range requirements (e.g., OpenShift):

cluster:
  tfi:
    podSecurityContextEnabled: true
    runAsUser: 1000710000    # kubectl describe namespace NS [uid-range]
    fsGroup: 1000710000      # kubectl describe namespace NS [supplemental-groups]

Disable Privilege Escalation¶

For environments requiring containers to run without privilege escalation (e.g., Azure Policy compliance):

cluster:  
  helmValues:
    global:
      postgresql:
        primary:
          containerSecurityContext:
            enabled: true
            allowPrivilegeEscalation: false
        metrics:
          containerSecurityContext:
            enabled: true
            allowPrivilegeEscalation: false
      newContainerSecurityContext:
        allowPrivilegeEscalation: false
      kube-prometheus-stack:
        grafana:
          newContainerSecurityContext:
            allowPrivilegeEscalation: false
        prometheus-node-exporter:
          containerSecurityContext:
            allowPrivilegeEscalation: true
      config-bootstrapper:
        newContainerSecurityContext:
          allowPrivilegeEscalation: false
        waitContainers:
          securityContext:
            allowPrivilegeEscalation: false
      kafka-topic-creator:
        newContainerSecurityContext:
          allowPrivilegeEscalation: false
      hypertrace-pinot:
        newContainerSecurityContext:
          allowPrivilegeEscalation: false
      db-schema-manager:
        newContainerSecurityContext:
          allowPrivilegeEscalation: false
      apiNamingCleanup:
        securityContext:
          allowPrivilegeEscalation: false
      apiNamingStreamTrainer:
        securityContext:
          allowPrivilegeEscalation: false
      waitContainers:
        securityContext:
          allowPrivilegeEscalation: false
      metrics:
        securityContext:
          allowPrivilegeEscalation: false
      selfSignedCertificates:
        securityContext:
          allowPrivilegeEscalation: false
      changeManagement:
        securityContext:
          allowPrivilegeEscalation: false
      exporter:
        securityContext:
          allowPrivilegeEscalation: false
      kafkaTopicPartitionChecker:
        securityContext:
          allowPrivilegeEscalation: false
      containerSecurityContext:
        allowPrivilegeEscalation: false
      prometheusOperator:
        containerSecurityContext:
          allowPrivilegeEscalation: false
        admissionWebhooks:
          patchWebhookJob:
            securityContext:
              allowPrivilegeEscalation: false
          createSecretJob:
            securityContext:
              allowPrivilegeEscalation: false
      kubeRBACProxy:
        containerSecurityContext:
          allowPrivilegeEscalation: false
      imageRenderer:
        containerSecurityContext:
          allowPrivilegeEscalation: false
      defaultBackend:
        image:
          allowPrivilegeEscalation: false
      volumePermissions:
        containerSecurityContext:
          allowPrivilegeEscalation: false

Reference: Azure Policy - Container Privilege Escalation

Step 4: Run Installation¶

Export Installation Variables¶

export REPO_USER=<your-registry-username>
export REPO_PWD=<your-registry-password>
export DNS_FOR_PLATFORM=<platform-dns-name>

Execute Installation¶

./clustermgr install-ti --dns=${DNS_FOR_PLATFORM} --multi-node \
    --registry-host="pkg.harness.io/ieq3j_9otlwbpfz-uryoda/docker-external" \
    --registry-prefix="" \
    --registry-user=${REPO_USER} --registry-password=${REPO_PWD} \
    --ti-values traceable-installer-values.yaml

Note: If using an internal registry, update --registry-host and --registry-prefix accordingly.

Installer Options Reference¶

Common Flags¶

Flag	Description
`--airgap`	Enable airgap install mode
`--dns <hostname>`	DNS name for the platform (defaults to `$(hostname -f)`)
`--dry-run`	Execute in dry run mode (pre-flight checks only)
`--email <email>`	Admin user email (defaults to `admin@<domain>`)
`--password <password>`	Admin password (auto-generated if not set)
`--version <version>`	Platform version to install

Multi-node only: Pass --multi-node to enable multi-node installation mode.

Registry Flags¶

Flag	Description
`--registry-host <host>`	Docker registry hostname
`--registry-prefix <prefix>`	Path prefix for images
`--registry-user <user>`	Registry username
`--registry-password <password>`	Registry password

Helm Configuration Flags¶

Flag	Description
`--ti-values <file>`	Helm values override file
`--ti-args <args>`	Additional arguments for installer
`--ti-set <key=value>`	Helm set arguments
`--ti-set-file <file>`	Helm set-file arguments
`--helm-values <file>`	Helm values override file

Advanced Flags¶

Flag	Description
`--disable-installer-auth`	Disable authentication for installer UI
`--skip-ti`	Skip Traceable Installer helm install
`--skip-image-load`	Skip loading images
`--mount-to <path>`	Mount `/var/**` directories to different location
`--mount-data-to <path>`	Mount only data directories to different location

Verification¶

After installation completes:

Check Pod Status:

kubectl get pods -n traceable

All pods should be in Running state.

Access Platform UI: Navigate to https://<DNS_FOR_PLATFORM> and login with admin credentials.
Verify Installation: See Post-Installation Checks for comprehensive validation.

Troubleshooting¶

Storage Class Issues¶

If pods fail to start due to PVC issues:

Verify storage class exists: kubectl get storageclass
Check PVC status: kubectl get pvc -n traceable
Verify NFS server connectivity (for NFS-based storage)

Ingress Issues¶

If platform is not accessible:

Check ingress status: kubectl get ingress -n traceable
Verify Nginx ingress controller is running
Check DNS resolution for platform hostname

Node Resource Issues¶

If pods are pending due to resources:

Check node resources: kubectl describe nodes
Verify sufficient CPU and memory across nodes
Consider adding more nodes or increasing node sizes

Next Steps¶

After successful installation:

Complete Post-Installation Checks
Configure SMTP Setup for email notifications
Configure SAML Setup for SSO authentication
Deploy Traceable Platform Agent (TPA)

Support¶

For installation assistance, contact: support@harness.io

Provide the following information:

ClusterMgr version
Kubernetes version and distribution
Storage class configuration
Helm values file used
Error messages and logs