Skip to content

ADR-0407: Git and object-store export layout and workflow

Where exported inventory lands in a repo or bucket, and how the Git/GitLab sinks commit, branch, and open merge requests.

Theme: 04 · Export & sinks · Status: Current

Context

The Git, GitLab, S3, and GCS backends are snapshot stores (ADR-0401): each export writes the whole inventory document to a path. The path layout is a contract (consumers and GitOps tooling depend on it) and the Git write workflow (commit vs branch vs MR) is an ergonomics decision. Both were implemented (internal/sink/git, internal/sink/gitlab) without an ADR.

Decision

Object path layout

The inventory controller passes a canonical object path; backends map it to their store:

inventory/<inventory-namespace>/<inventory-name>.json
  • Git/GitLab/S3/GCS write the JSON payload (ADR-0405) at that path.
  • Multi-cluster fan-in disambiguates by cluster in the path/key (clusters/<cluster>/… or a cluster column/header — ADR-0501), so concurrent spokes never collide on the same object.

Git write workflow

internal/sink/git/export_file.go clones a single branch into a temp dir, writes the file, and:

  • Commit identity defaults to kollect <kollect@kollect.dev>; message templates and rich defaults are defined in ADR-0415.
  • No-op guard: if git add produces no change, the export errors rather than pushing an empty commit (idempotent exports stay quiet — ADR-0406).
  • Push policy : spec.git.pushPolicy defaults to Commit (append-only pushes). Optional ForcePush restores snapshot semantics (push --force) for demo repos.
  • Empty-remote bootstrap: a fresh/empty repo is git init'd and the branch created, so the first export works against a bare repository.
  • Custom CA: trusted via resolved caPEM from BuildContext (ADR-0104); no insecureSkipVerify for Git.

GitLab merge-request mode

internal/sink/gitlab adds a review workflow on top of the Git backend:

  • Direct mode: commit straight to the target branch (same as Git).
  • Branch+MR mode (MergeRequestModeBranchMR): push to a per-inventory feature branch (BranchNameForExport(prefix, ns, name)) off the target branch, then EnsureMergeRequest opens or updates one MR per inventory via the GitLab API. One stable branch/MR per inventory keeps churn reviewable instead of spawning an MR per export.

Consequences

  • Predictable, diffable layout for GitOps and audit (ADR-0103).
  • Force-push means Git holds current state with audit trail, not a queryable history — relational history is Postgres' job (ADR-0401).
  • One branch/MR per inventory bounds review noise but means concurrent exports to the same inventory serialize on the remote.

Open questions

  • DECIDED : Make the object path a spec.pathTemplate (e.g. {cluster}/{namespace}/{name}.json, default inventory/{namespace}/{name}.json) so layout is configurable per sink.
  • DECIDED : spec.git adds pushPolicy, branch, auth, commitMessage, commitBody, commitTrailers, author, cloneDepth, and prune on snapshot sinks — see ADR-0415.
  • OPEN: GCS Parquet export — S3 ships format: parquet in v0.4; GCS JSON default until parity lands.