Analytiks Logo
Back to Tech Corner
DevOps

How to Automate Post-Upgrade Validation in SAS Viya 4 Using Kubernetes, REST APIs, and Bash

After completing this guide, you will know how to verify Kubernetes workloads after an upgrade, validate CAS availability, test critical SAS Viya REST APIs, verify SAS Viya CLI connectivity, automate post-upgrade validation using a Bash script, and generate a repeatable PASS/FAIL validation report.

A successful deployment is not determined by whether every pod starts successfully. An upgrade should only be considered successful after confirming that Kubernetes workloads are healthy, SAS Logon is operational, CAS accepts client connections, REST APIs respond correctly, SAS Viya CLI can authenticate, and end users can access the platform.

The following workflow can be executed manually or integrated into GitHub Actions, GitLab CI, Jenkins, or Azure DevOps.

Applies To

  • SAS Viya 4 (Latest LTS)
  • Kubernetes
  • OpenShift (where applicable)

Please adjust commands to suit your Viya deployment.

What You'll Learn

After completing this guide, you will know how to:

  • Verify Kubernetes workloads after an upgrade
  • Validate CAS availability
  • Test critical SAS Viya REST APIs
  • Verify SAS Viya CLI connectivity
  • Automate post-upgrade validation using a Bash script
  • Generate a repeatable PASS/FAIL validation report

Why Automate Post-Upgrade Validation?

A successful deployment is not determined by whether every pod starts successfully.

An upgrade should only be considered successful after confirming:

  • Kubernetes workloads are healthy
  • SAS Logon is operational
  • CAS accepts client connections
  • REST APIs respond correctly
  • SAS Viya CLI can authenticate
  • End users can access the platform

The following workflow can be executed manually or integrated into GitHub Actions, GitLab CI, Jenkins, or Azure DevOps.

Step 1 – Verify Kubernetes Workloads

Begin by confirming the cluster is healthy. Note, we are using sas-viya as the namespace.

Check pods:

kubectl get pods -n sas-viya

Expected result:

Long-running services should report:
STATUS READY
Running 1/1

Initialization jobs should report:
Completed

If you see CrashLoopBackOff, collect additional information with these commands.

kubectl logs <pod-name> -n sas-viya
kubectl describe pod <pod-name> -n sas-viya

These commands typically identify failed configuration, certificate problems, missing secrets, storage failures, and image pull issues.

Check deployments:

kubectl get deployments -n sas-viya

Verify that READY = AVAILABLE.

READY UP-TO-DATE AVAILABLE
1/1 1 1

Check recent events:

kubectl get events \
  -n sas-viya \
  --sort-by=.lastTimestamp

Look for FailedMount, FailedScheduling, FailedCreate, and Unhealthy. A healthy deployment produces very few Warning events.

Step 2 – Validate CAS

Pods may be healthy while CAS is unavailable. Always validate CAS independently.

If using the Viya CLI:

viya auth login
viya cas servers list

Expected output:

Default CAS Server
State: Running

If your environment uses another administrative method, verify that CAS starts successfully, sessions can be created, memory is allocated, and users can connect.

Failure here usually indicates CAS configuration changes, storage issues, CAS Controller startup problems, or licensing problems.

Step 3 – Validate SAS Logon

Authentication should be validated before any functional testing.

Example:

curl -k \
  https://viya.company.com/SASLogon/oauth/token

Expected:

HTTP 200
or
HTTP 401
Unauthorized

Both indicate the service is operational. Do not expect a token without valid credentials.

If the endpoint returns HTTP 503 or HTTP 500, review the sas-logon pod, ingress, certificates, and identity provider configuration.

Step 4 – Validate REST APIs

A simple API call quickly verifies that core services are functioning.

Example:

curl -sk \
  -H "Authorization: Bearer <ACCESS_TOKEN>" \
  https://viya.company.com/identities/users

Expected:

{
  "items":[
  ...
  ]
}

If the request returns 401, authentication failed. If the request returns 404, verify the endpoint URL. If the request returns 503, investigate backend services.

Another useful endpoint:

curl -sk \
  -H "Authorization: Bearer <ACCESS_TOKEN>" \
  https://viya.company.com/casManagement/servers

Successful output confirms REST API, Authentication, and CAS Management Service are functioning together.

Step 5 – Validate the SAS Viya CLI

Confirm administrative commands execute correctly.

Authenticate:

viya auth login

Verify folders:

viya folders list

Verify transfer service:

viya transfer list

Successful execution confirms authentication, authorization, Content Service, and Transfer Service are functioning.

Step 6 – Automate Everything with Bash

The following script performs the essential validation checks.

#!/bin/bash
NAMESPACE="sas-viya"
HOST="https://viya.company.com"

echo "======================================"
echo "SAS Viya Post Upgrade Validation"
echo "======================================"
echo
echo "[1] Checking Pods"
kubectl get pods -n $NAMESPACE
echo
echo "[2] Checking Deployments"
kubectl get deployments -n $NAMESPACE
echo
echo "[3] Checking Events"
kubectl get events -n $NAMESPACE \
  --sort-by=.lastTimestamp \
  | tail
echo
echo "[4] Checking SAS Logon"
curl -sk \
  -o /dev/null \
  -w "%{http_code}\n" \
  $HOST/SASLogon/oauth/token
echo
echo "[5] Checking CAS Management"
curl -sk \
  -H "Authorization: Bearer $TOKEN" \
  $HOST/casManagement/servers
echo
echo "[6] Checking Identities API"
curl -sk \
  -H "Authorization: Bearer $TOKEN" \
  $HOST/identities/users
echo
echo "Validation Complete"

Save as validate_viya.sh, make executable, and run it.

chmod +x validate_viya.sh
./validate_viya.sh

Example Kubernetes Job (Optional)

If you'd like Kubernetes to execute the validation automatically after an upgrade, the following Job provides a simple starting point.

apiVersion: batch/v1
kind: Job
metadata:
  name: viya-post-upgrade-validation
spec:
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: validator
          image: registry.access.redhat.com/ubi9/ubi
          command:
            - /bin/bash
            - -c
            - |
              chmod +x /scripts/validate_viya.sh
              /scripts/validate_viya.sh
          volumeMounts:
            - name: scripts
              mountPath: /scripts
      volumes:
        - name: scripts
          configMap:
            name: validation-script

YAML Highlights

  • Job executes once after deployment.
  • restartPolicy: Never prevents automatic retries that could mask failures.
  • ConfigMap stores the validation script separately from the container image, making updates easier.
  • The script can be replaced without rebuilding the image.

Common Troubleshooting

Pods Running but Users Cannot Login

Check the sas-logon-app deployment logs and review Identity Provider, LDAP, SCIM, OIDC, and Certificates.

kubectl logs deployment/sas-logon-app -n sas-viya

CAS Not Available

Verify CAS pods and review the CAS controller logs.

kubectl get pods | grep cas
kubectl logs <cas-controller-pod>

REST API Returns HTTP 503

Usually indicates an ingress issue, backend service unavailable, or a startup failure.

kubectl describe ingress
kubectl get svc

Common Pitfalls

Avoid these common mistakes:

  • Assuming Running pods mean the platform is healthy.
  • Skipping CAS validation.
  • Validating only the Kubernetes layer.
  • Ignoring REST API failures.
  • Not checking Warning events after an upgrade.
  • Using an expired OAuth access token during API validation.
  • Forgetting to test representative business functionality.

Best Practices

  • Run the validation script after every upgrade.
  • Integrate the script into your CI/CD pipeline.
  • Store validation results as pipeline artifacts.
  • Fail the deployment pipeline if any critical validation step fails.
  • Keep endpoint URLs and namespaces in variables for portability.
  • Extend the script to include organization-specific checks, such as scheduled jobs or report execution.

Next Steps

Consider enhancing the script to generate HTML or JSON reports, send email or Slack notifications, validate Compute sessions, test report execution, verify scheduled jobs, and publish results to your CI/CD dashboard.

References

  • SAS Viya Administration Documentation
  • SAS Viya CLI Documentation
  • Kubernetes Documentation
  • Red Hat OpenShift Documentation

Automating post-upgrade validation provides a consistent and repeatable method to verify platform health, reduce deployment risk, and identify issues before they impact users. A small investment in automation can significantly improve operational reliability and confidence in every SAS Viya upgrade.

Visit https://github.com/sassoftware/viya4-ark to review the SAS Viya ARK GitHub repository for feature-rich Viya administrator tools and utilities.

View All Articles