Skip to content

Troubleshooting

Central hub for diagnosing Kollect export, collection, and sink issues. Start with the condition catalog below, then follow links to symptom-specific guides.

First checks

Run kubectl describe on the sink and inventory when export stalls — ConnectionVerified, SinkReachable, and Synced conditions usually pinpoint credential, namespace, or selector issues before diving into controller logs.

Related guides

Symptom Q&A: FAQ. Step-by-step install and upgrade: Operator manual. Per-scenario walkthroughs: Examples.

Condition catalog

Kollect follows Kubernetes condition conventions (Ready, Synced, Degraded) with sink-specific types on static kinds. See ADR-0602 for reconcile behavior.

By kind

Kind Conditions When to inspect
Family sinks (KollectSnapshotSink, KollectDatabaseSink, KollectEventSink) ConnectionVerified, TLSInsecure, Degraded Before export; credential or endpoint problems
KollectTarget Ready, Synced, Degraded, SinkReachable Collection stalled or scope denied
KollectInventory Ready, Synced, Degraded, SinkReachable Export not running or payload errors
KollectConnectionTest ConnectionVerified, Ready Audited composite probes

Static kinds (KollectProfile, KollectScope) do not set Ready — admission webhooks and events surface validation errors instead.

Cross-object conditions

Condition Object Meaning
ConnectionVerified KollectSink Last connectivity probe succeeded (credentials, TLS, network)
SinkReachable KollectInventory, KollectTarget Export pipeline resolved and can reach the referenced sink
Synced KollectInventory, KollectTarget Last export or collection cycle completed successfully
Degraded Reconciled kinds Hard block — fix reason before expecting progress

A sink can show ConnectionVerified=True while inventory shows SinkReachable=False if the name or namespace in sinkRefs is wrong — fix the reference, not just credentials.

Common Degraded reasons

Reason Typical object Cause Fix
SinkNotFound Inventory, Target Typo or wrong namespace in sinkRefs Match exact sink name in same namespace
SinkUnreachable Inventory, Target ConnectionVerified=False on sink Fix Secret, DSN, network; re-probe sink
ScopeSinkDenied Inventory Sink not in KollectScope allow-list Add sink to spec.sinkRefs on scope
ScopeGVKDenied Target GVK blocked by scope Update KollectScope.spec.allowedGVKs
ScopeNamespaceDenied Target Workload namespace blocked Add to allowedNamespaces
ProfileNotFound Target Missing KollectProfile Apply profile in same namespace as target
PayloadTooLarge Inventory Exceeds maxExportBytes Split targets or trim attributes
ExportTerminal Inventory Non-retryable sink error Fix sink config; check operator logs
Suspended Target, Inventory spec.suspend: true Set suspend: false
Progressing Inventory Transient network or 429 Usually self-heals; inspect metrics

Full per-kind tables: KollectInventory, KollectTarget, KollectSnapshotSink.

Symptom → cause quick reference

Symptom Likely cause Next step
Export never runs SinkReachable=False (SinkNotFound / SinkUnreachable) FAQ — export never runs
ConnectionVerified=False Missing Secret, bad DSN, TLS failure Connection test
Empty status.itemCount Selector mismatch, suspended target, scope denied Deployment inventory — Troubleshooting
Namespace skipped Watch label or OptIn without enabled Annotations and labels
Postgres rows stale Upsert-only drift or export error Postgres state store
Hub spoke not merging Transport or registration misconfig Hub mode
CR stopped working after upgrade Pre-beta schema change FAQ — pre-beta

Diagnostic commands

# Pipeline status (short names)
kubectl get kprof,ksink,ktgt,kinv -n <namespace>
kubectl describe kollectsink <name> -n <namespace>
kubectl describe kollectinventory <name> -n <namespace>

# Wait for sink probe
kubectl wait --for=condition=ConnectionVerified kollectsink/<name> \
  -n <namespace> --timeout=60s

# Re-probe without editing spec
kubectl annotate kollectsink <name> -n <namespace> \
  kollect.dev/test-connection=true --overwrite

# Operator logs
kubectl -n kollect-system logs deployment/kollect-controller-manager -f --tail=200

More shortcuts: Command reference.

Example troubleshooting guides

Scenario Guide
First inventory pipeline on kind Deployment inventory
Postgres DSN and delete reconciliation Postgres state store
Helm / Argo Application attributes Helm release inventory
Sink connectivity probes Connection test
Multi-tenant watch scope Multi-tenant watch namespaces
Spoke → shared sink Spoke cluster inventory
Hub aggregation Hub mode

When to escalate

Pre-beta API

v1alpha1 fields may change without conversion webhook. Check ROADMAP before production use.

  1. Collect kubectl describe output for sink, target, and inventory.
  2. Capture operator logs (sanitize Secrets before sharing).
  3. Note Helm mode, tenantMode, and watchNamespaces values.
  4. Open a GitHub issue with repro steps and condition JSON from status.conditions.