Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Customer Deployment Operations

Use these checks for routine monitoring, change records, support handoff, and disaster recovery. They return metadata, not credentials or license contents.

Current release and reconciliation

pretorin --json deployment flux status

Record these fields in a change ticket:

  • release tag, revision, and OCI artifact digest;
  • whether the source is suspended;
  • each component’s release ID and readiness; and
  • observed generation.

For controller diagnostics:

kubectl --context <customer-context> --namespace flux-system \
  describe ocirepository pretorin-release
kubectl --context <customer-context> --namespace flux-system \
  describe kustomization pretorin-release
kubectl --context <customer-context> --namespace flux-system get kustomizations \
  pretorin-release-auth pretorin-release-api pretorin-release-ai pretorin-release-web
kubectl --context <customer-context> --namespace pretorin get helmreleases

Current license

pretorin --json deployment license status

The application sends owners/admins local inbox notifications at 30, 14, 7, 1, and 0 days and on missing or failure-state transitions. Existing notification preferences and configured customer email, Slack, and Teams channels apply. Never retrieve or decode license.jwt for routine monitoring.

Prometheus and Alertmanager

The customer Helm profile includes local rules based on the bounded Auth metrics. If the Prometheus Operator CRDs are installed, enable their resources in the customer values file:

auth:
  serviceMonitor:
    enabled: true
  prometheusRule:
    enabled: true

The resulting rules are:

AlertSeverityCondition
PretorinCustomerLicenseExpiresWithin30DayswarningValid, more than 7 and at most 30 days remaining
PretorinCustomerLicenseExpiresWithin7DayscriticalValid, more than 0 and at most 7 days remaining
PretorinCustomerLicenseUnavailablecriticalMissing, expired, not-yet-valid, invalid, wrong-deployment, or configuration-error state

Route component="customer-license" through the customer’s Alertmanager. For example, merge this route into the locally managed Alertmanager configuration:

route:
  routes:
    - receiver: customer-license-operations
      matchers:
        - component="customer-license"
receivers:
  - name: customer-license-operations
    # Configure only customer-owned local receivers here.

Use the customer’s approved local receiver—such as an enclave SMTP relay or on-premises incident manager. The metrics, rules, and evaluation make no callback to Pretorin and remain useful with no internet access.

For a renewal or system-ceiling change, verify the replacement token through the approved handoff and install it independently of an application release:

pretorin deployment license install \
  --context <customer-context> --namespace pretorin \
  --license-file ./license.jwt \
  --trust-bundle ./license-trust.json

pretorin --json deployment license status

The install command updates the license Secret and public trust ConfigMap; it does not request a Helm reconciliation or restart. The active banner and inbox condition clear after the normal state refresh, and the replacement license’s future milestones are re-armed. Existing systems stay available if a license expires, but new system creation remains blocked until a valid token for the persistent deployment ID is observed.

Approval window

pretorin deployment flux suspend --context <customer-context>
pretorin --json deployment flux status

# After approval
pretorin deployment flux resume --context <customer-context>

Suspension controls application of the release channel. Signed candidates continue to resolve so their digest can be reviewed. It does not stop running workloads.

Registry outage

Running Pods continue using their resolved images. Flux reports source fetch failures and retries. Do not change image pull policy or tags. Restore registry availability and request reconciliation with flux resume; a reinstall is not required.

Ensure retention policies preserve:

  • all supported release digests and their signature referrers;
  • chart 1.0.0 artifacts and their signatures;
  • exact image manifests referenced by retained releases; and
  • foundation images for the installed maintenance release.

Backups and disaster recovery

Back up PostgreSQL, Garage data, customer values ConfigMaps, platform Secrets, registry credentials, release trust, license trust, and deployment identity.

Restore in this order:

  1. Cluster foundation and storage.
  2. pretorin-deployment-identity and other customer-owned Secrets.
  3. License Secret and trust ConfigMap.
  4. Flux registry credentials and release public key.
  5. Customer values ConfigMaps and bootstrap objects.
  6. The same immutable release digest recorded before the incident.
  7. Validate database and object storage, then resume application updates.

Confirm the restored identity before auth starts:

pretorin deployment identity show \
  --context <recovery-context> --namespace pretorin

Post-update smoke test

kubectl --context <customer-context> --namespace pretorin get pods,jobs
pretorin whoami
pretorin frameworks list
pretorin --json deployment license status
NAMESPACE=pretorin scripts/customer/validate-airgap-install.sh

Use validate-airgap-install.sh --skip-chat only to isolate provider or data setup before the full chat test.

Safe support bundle

Include:

  • pretorin --json deployment flux status;
  • pretorin --json deployment license status;
  • names, phases, restart counts, and recent Events;
  • failed Job and controller logs after review for customer data; and
  • chart contract, Kubernetes version, and CLI version.

Exclude decoded Secrets, .dockerconfigjson, license tokens, API/model keys, database URLs with passwords, private certificates, and full environment dumps.