Customer Deployment Troubleshooting
Start with redacted state:
pretorin --json deployment flux status
pretorin --json deployment license status
kubectl --context <customer-context> --namespace pretorin get pods,jobs,helmreleases
kubectl --context <customer-context> --namespace flux-system \
get ocirepositories,kustomizations
Update failures
| Symptom | Meaning | Action |
|---|---|---|
SourceVerified=False | Release or chart signature did not verify | Stop. Confirm the out-of-band public-key fingerprint and registry referrers. Do not bypass verification. |
| OCI source authentication failure | Read-only registry Secret is missing, expired, or lacks repository access | Refresh flux-system/pretorin-registry-credentials; do not put credentials in Helm values. |
| Release apply is suspended | Approval mode is waiting | Inspect the resolved candidate digest, approve the change, then run pretorin deployment flux resume. |
| Auth/API migration Job fails | Schema change did not complete | Inspect the Job log and database health. Correct the cause and roll forward; do not force runtime Pods past the hook. |
| API waits for auth, or AI/web waits for API | Dependency is not ready at the same release ID | Fix the first failed component. Later components are intentionally blocked. |
| Registry contains new images but nothing rolls | Images are data, not a deployment trigger | Publish and promote the signed release artifact. Never restart Pods merely to chase a mutable tag. |
| Old and new versions appear together | A rollout is in progress or blocked | Check HelmRelease conditions, Deployment rollout status, and Pod image IDs. Do not retag images. |
Useful controller logs:
kubectl --context <customer-context> --namespace flux-system \
logs deployment/source-controller --since=30m
kubectl --context <customer-context> --namespace flux-system \
logs deployment/kustomize-controller --since=30m
kubectl --context <customer-context> --namespace flux-system \
logs deployment/helm-controller --since=30m
Review logs before sharing them outside the customer environment.
License errors
| Status or denial code | Action |
|---|---|
license_missing / missing | Install license.jwt and the trust bundle in the configured namespace. |
license_not_yet_valid / not_yet_valid | Verify UTC time and the issued validity start. |
license_expired / expired | Obtain a renewal and run pretorin deployment license install; no rollout is needed. |
license_deployment_mismatch / wrong_deployment | Compare pretorin deployment identity show with the request used for issuance. Restore the original identity or request reissuance. |
license_invalid, invalid_signature, or unknown_key | Confirm the token and current public trust bundle came from the approved handoff. Do not edit the token. |
license_configuration_error / configuration_error | Confirm the projected paths and trust-bundle JSON are present and readable; use only redacted status and local Auth diagnostics. |
systems_limit_exceeded | Existing systems remain available. Contact Pretorin for a larger signed limit. Archived systems count. |
Confirm resources exist without decoding them:
kubectl --context <customer-context> --namespace pretorin \
get secret pretorin-license pretorin-deployment-identity
kubectl --context <customer-context> --namespace pretorin \
get configmap pretorin-license-trust
Kubernetes projected volumes update eventually. Wait for the status endpoint to observe a renewal. Do not restart auth unless normal diagnostics prove the mounted file is not updating.
Platform and provider failures
| Symptom | First check |
|---|---|
| API/AI provider validation fails | Confirm the OpenAI-compatible base URL ends in /v1, the configured model IDs exist, and NetworkPolicy permits both API and AI egress. |
| Embedding dimension mismatch | Use the deployment’s approved embedding model/dimension; a schema change requires a planned migration. |
| Framework or source data is missing | Inspect the API seed/sync hook Jobs for the current release. |
| CLI hits hosted Pretorin | Run pretorin config list and set the private /api/v1/public endpoint. |
| CLI returns 401/403 | Authenticate against that endpoint with an API token carrying read scope for status commands. |
If escalation is required, use the safe support inputs in Operations. Never attach decoded Secret data or a license token.