Validate Images in Internal Registry¶

This guide explains how to validate that all required Traceable Platform images are present in your internal Docker registry, particularly important for airgap installations.

Overview¶

When deploying Traceable Platform in an airgap environment or using an internal Docker registry, it's critical to ensure all required container images have been successfully transferred. This validation process helps identify missing images before attempting installation or upgrade, preventing deployment failures.

Prerequisites¶

Before validating images, ensure you have:

ClusterMgr binary
Workflow files extracted
Skopeo installed on your system
Access credentials to your internal registry
Image mapping YAML file from the workflow

Required Tools¶

Install Skopeo¶

Skopeo is required for inspecting container images in registries.

RHEL/CentOS:

sudo yum install skopeo -y

Ubuntu/Debian:

sudo apt-get update
sudo apt-get install skopeo -y

macOS:

brew install skopeo

Verify Installation:

skopeo --version

Step 1: Generate Image List¶

Understanding the Original Sync Command¶

If you previously generated a sync script using ClusterMgr:

./clustermgr image generate \
  -f ../workflow/airgap/image-mapping/custom/D3002.custom.ingress-install.yaml \
  --src-host pkg.harness.io/ieq3j_9otlwbpfz-uryoda/docker-external \
  --dest-host=ip-172-168-0-14:5000 \
  --skopeo > imageSyncSkopeo.sh

Generate Image List for Validation¶

To create a list of all images in your internal registry, modify the command:

Changes:

Remove --src-host parameter
Remove --skopeo parameter
Add --list-dest parameter
Keep all other parameters the same

Command:

./clustermgr image generate \
  -f ../workflow/airgap/image-mapping/custom/D3002.custom.ingress-install.yaml \
  --dest-host ip-172-168-0-14:5000 \
  --list-dest > imageList.txt

Parameters Explained:

-f: Path to the image mapping YAML file from your workflow
--dest-host: Your internal registry hostname and port
--list-dest: Generate list of destination images
> imageList.txt: Redirect output to a file

Verify Image List¶

Check the generated file:

cat imageList.txt

Expected Output:

ip-172-168-0-14:5000/traceable/api-server:2.5.0
ip-172-168-0-14:5000/traceable/platform-agent:2.5.0
ip-172-168-0-14:5000/traceable/hypertrace-ui:2.5.0
...

Step 2: Run Validation Script¶

Create Validation Script¶

Create a file named validate-images.sh:

#!/bin/bash

# Check if skopeo is installed
which skopeo || exit -1

# Login to registry if credentials are provided
if [[ ! -z $repoPassword ]]; then
  skopeo login $registryHost -u $repoUser -p $repoPassword
fi

# Check if imageList.txt exists
if [[ ! -f imageList.txt ]]; then
  echo "imageList.txt file doesn't exist."
  exit -1
fi

# Read image list into array
imageList=($(cat imageList.txt))

# Array to store failed images
failedImage=()

# Validate each image
for image in "${imageList[@]}"; do
  skopeo inspect docker://$image --tls-verify=false > /dev/null 2>&1
  if [[ $? -ne 0 ]]; then
    failedImage+=($image)
  fi
done

# Logout from registry
if [[ ! -z $repoPassword ]]; then
  skopeo logout $registryHost
fi

# Report results
if [[ ${#failedImage[@]} -ne 0 ]]; then
  echo "Following images are not present in the Docker repo"
  for image in "${failedImage[@]}"; do
    echo $image
  done
  exit -1
else
  echo "All the images are present"
fi

Make Script Executable¶

chmod +x validate-images.sh

Run Validation¶

Without Authentication:

./validate-images.sh

With Authentication:

export registryHost="ip-172-168-0-14:5000"
export repoUser="your-username"
export repoPassword="your-password"
./validate-images.sh

Understanding the Validation Process¶

What the Script Does¶

Checks Prerequisites: Verifies skopeo is installed
Authenticates: Logs into the registry if credentials are provided
Reads Image List: Loads all images from imageList.txt
Validates Each Image: Uses skopeo to inspect each image in the registry
Collects Failures: Tracks any images that cannot be found
Reports Results: Lists missing images or confirms all are present

Validation Logic¶

For each image, the script:

Runs skopeo inspect docker://<image> to check if the image exists
Uses --tls-verify=false to skip TLS verification (useful for self-signed certificates)
Redirects output to /dev/null to suppress verbose output
Checks the exit code ($?) to determine success or failure

Expected Results¶

Success Output¶

If all images are present:

All the images are present

Exit code: 0

Failure Output¶

If images are missing:

Following images are not present in the Docker repo
ip-172-168-0-14:5000/traceable/api-server:2.5.0
ip-172-168-0-14:5000/traceable/platform-agent:2.5.0

Exit code: -1 (or 255)

Troubleshooting¶

Issue 1: Skopeo Not Found¶

Symptoms:

bash: skopeo: command not found

Solution: Install skopeo using the commands in the Prerequisites section.

Issue 2: imageList.txt Not Found¶

Symptoms:

imageList.txt file doesn't exist.

Solutions:

Verify you ran the image list generation command
Check you're in the correct directory
Verify the output file was created:

ls -lh imageList.txt

Issue 3: Authentication Failed¶

Symptoms:

Error: authentication required

Solutions:

Verify credentials are correct
Set environment variables properly:

export registryHost="your-registry:5000"
export repoUser="your-username"
export repoPassword="your-password"

Test manual login:

skopeo login $registryHost -u $repoUser -p $repoPassword

Issue 4: TLS Certificate Errors¶

Symptoms:

Error: x509: certificate signed by unknown authority

Solutions:

The script already uses --tls-verify=false to skip verification
If you want to verify certificates, remove --tls-verify=false and ensure:
- CA certificates are installed on the system
- Registry certificate is trusted

Issue 5: Connection Timeout¶

Symptoms:

Error: connection timeout

Solutions:

Verify network connectivity to the registry:

nc -zv ip-172-168-0-14 5000

Check firewall rules
Verify registry is running:

curl -k https://ip-172-168-0-14:5000/v2/_catalog

Issue 6: Many Images Missing¶

Symptoms: Large number of images reported as missing

Possible Causes:

Image sync process didn't complete successfully
Wrong registry hostname specified
Images were pushed to a different registry or namespace
Registry was reset or cleared

Solutions:

Re-run the image sync process
Verify the --dest-host matches your actual registry
Check registry contents:

skopeo list-tags docker://ip-172-168-0-14:5000/traceable/api-server

Advanced Usage¶

Validate Specific Images Only¶

Create a custom image list with only specific images:

cat > custom-imageList.txt <<EOF
ip-172-168-0-14:5000/traceable/api-server:2.5.0
ip-172-168-0-14:5000/traceable/platform-agent:2.5.0
EOF

./validate-images.sh

Validate with TLS Verification¶

Modify the script to enable TLS verification:

# Change this line:
skopeo inspect docker://$image --tls-verify=false > /dev/null 2>&1

# To:
skopeo inspect docker://$image --tls-verify=true > /dev/null 2>&1

Generate Detailed Report¶

Modify the script to show more details:

# Add verbose output for each image
for image in "${imageList[@]}"; do
  echo "Checking: $image"
  skopeo inspect docker://$image --tls-verify=false > /dev/null 2>&1
  if [[ $? -ne 0 ]]; then
    echo "  ❌ MISSING"
    failedImage+=($image)
  else
    echo "  ✓ PRESENT"
  fi
done

Parallel Validation¶

For faster validation with many images, use GNU parallel:

#!/bin/bash
which parallel || { echo "GNU parallel not installed"; exit -1; }

validate_image() {
  image=$1
  skopeo inspect docker://$image --tls-verify=false > /dev/null 2>&1
  if [[ $? -ne 0 ]]; then
    echo "$image"
  fi
}

export -f validate_image

cat imageList.txt | parallel -j 10 validate_image > missing-images.txt

if [[ -s missing-images.txt ]]; then
  echo "Following images are not present:"
  cat missing-images.txt
  exit -1
else
  echo "All images are present"
fi

Best Practices¶

Validate Before Installation: Always validate images before attempting platform installation
Automate Validation: Include validation in your deployment pipeline
Keep Image Lists: Store imageList.txt for documentation and troubleshooting
Version Control: Track which images were validated for each deployment
Regular Checks: Periodically validate images haven't been accidentally deleted
Document Failures: Keep logs of validation failures for audit purposes
Test Credentials: Verify registry credentials before running large validations

Security Considerations¶

Credential Management: Never hardcode credentials in scripts
Use Environment Variables: Store sensitive data in environment variables
Secure Storage: Use secret management tools (Vault, AWS Secrets Manager)
TLS Verification: Enable TLS verification in production environments
Audit Logs: Monitor who validates images and when
Access Control: Limit who can access the internal registry

Next Steps¶

After successful validation:

For Installation: Proceed with single-node-install.md / multi-node-install.md
For Upgrade: Continue with Upgrading the Platform
If Validation Fails: Re-run the image sync process and validate again

Support¶

For issues with image validation or missing images, contact: support@harness.io

Provide the following information when requesting support:

Output of imageList.txt
List of missing images
Registry hostname and configuration
Skopeo version (skopeo --version)
Error messages from validation script
Image sync logs (if available)

Validate Images in Internal Registry¶

Overview¶

Prerequisites¶

Required Tools¶

Install Skopeo¶

Step 1: Generate Image List¶

Understanding the Original Sync Command¶

Generate Image List for Validation¶

Verify Image List¶

Step 2: Run Validation Script¶

Create Validation Script¶

Make Script Executable¶

Run Validation¶

Understanding the Validation Process¶

What the Script Does¶

Validation Logic¶

Expected Results¶

Success Output¶

Failure Output¶

Troubleshooting¶

Issue 1: Skopeo Not Found¶

Issue 2: imageList.txt Not Found¶

Issue 3: Authentication Failed¶

Issue 4: TLS Certificate Errors¶

Issue 5: Connection Timeout¶

Issue 6: Many Images Missing¶

Advanced Usage¶

Validate Specific Images Only¶

Validate with TLS Verification¶

Generate Detailed Report¶

Parallel Validation¶

Best Practices¶

Security Considerations¶

Next Steps¶

Related Documentation¶

Support¶