Skip to content

Add infrahubctl marketplace CLI for fetching schemas and collections#952

Open
minitriga wants to merge 7 commits intostablefrom
knotty-dibble
Open

Add infrahubctl marketplace CLI for fetching schemas and collections#952
minitriga wants to merge 7 commits intostablefrom
knotty-dibble

Conversation

@minitriga
Copy link
Copy Markdown
Contributor

@minitriga minitriga commented Apr 21, 2026

Summary

Adds a new infrahubctl marketplace sub-app with a get command that pulls schemas and collections from the public Infrahub Marketplace (marketplace.infrahub.app) into a local directory or to stdout.

infrahubctl marketplace get acme/network-base               # any item, auto-detected
infrahubctl marketplace get acme/network-base -v 1.2.0      # pin a schema version
infrahubctl marketplace get acme/starter-pack               # collection resolved automatically
infrahubctl marketplace get acme/network-base --stdout      # print to stdout, status to stderr
infrahubctl marketplace get acme/foo -o ./infra/schemas     # custom destination

What the command does

  • REST-based — talks to the public marketplace REST API (/api/v1/schemas/..., /api/v1/collections/...); no GraphQL involved.
  • Auto-detect schema vs collection — probes both endpoints in parallel. One command handles either item type without the caller needing to know which it is. On a rare namespace/name collision, schema wins with a warning; --collection/-c forces the other path.
  • Version pinning--version <semver> pins the download to a specific published schema version; default is whatever the marketplace resolves as latest (echoed back on success).
  • Stdout mode--stdout/-s writes content to real stdout and routes status, warnings, and errors to stderr, so the output can be piped into other tools (e.g. infrahubctl schema check once it grows stdin support). Collections are emitted as multi-doc YAML, with --- separators only inserted when a schema doesn't already start with one. No files are written when --stdout is set.
  • Custom destination--output-dir/-o (default ./schemas) controls where files land; missing parent directories are created.
  • Marketplace override--marketplace-url, infrahubctl.toml marketplace_url, or INFRAHUB_MARKETPLACE_URL env var let you point at staging or a local instance.
  • Clean error taxonomy — distinct failure classes (invalid input, not found, network) with exit code 1 for deterministic failures and 2 for transient/network so CI can branch on it. No tracebacks on an unwritable output directory, a missing schema, or a connection refused.

Options

Option Default Purpose
IDENTIFIER (positional) required namespace/name — a schema or collection
-v, --version TEXT latest Pin to a specific published schema version (schemas only)
-c, --collection auto-detect Force the collection code path
-s, --stdout off Print to stdout, status to stderr; skips disk writes
-o, --output-dir PATH ./schemas Where to write files
--marketplace-url TEXT https://marketplace.infrahub.app Override the marketplace host

Behaviour notes

  • Output lines state the resolved item type (Downloaded schema … / Collection …) so the user can detect an unintended match in a collision case.
  • Passing --version alongside a collection identifier prints a warning and proceeds with the collection download, rather than failing.
  • A missing version on an existing schema reports "no published version ''" with a hint to drop --version, distinct from "schema not found".

Out of scope

No --load convenience flag. get is fetch-only — chain with infrahubctl schema load <dir> (or pipe via --stdout once stdin support lands) to push the result into a running Infrahub.

Test plan

  • uv run pytest tests/unit/ctl/test_marketplace_app.py — 20 tests covering auto-detect, collision, error classes, --version, --output-dir, --marketplace-url, --collection override, and --stdout mode (single schema, collection, separator injection).
  • uv run invoke lint — ruff, ty, mypy, vale, markdownlint all clean.
  • uv run invoke generate-infrahubctldocs/docs/infrahubctl/infrahubctl-marketplace.mdx regenerated.
  • End-to-end smoke test against the live marketplace: infrahubctl marketplace get nonexistent/nonexistent returns the expected not-found message with exit 1.

Artefacts

Full spec, plan, research, contracts, quickstart, and task breakdown under specs/001-marketplace-api-update/ for reviewers who want the design rationale (auto-detect strategy evaluation, error taxonomy decisions, collision precedence).

🤖 Generated with Claude Code

minitriga and others added 2 commits April 21, 2026 11:46
- Probe schema and collection endpoints in parallel so
  `infrahubctl marketplace download <ns>/<name>` no longer requires
  the user to pass `--collection` up front. Collision between the two
  resolves to schema with a warning (pass `--collection` to force the
  other path). `_detect_item_type` returns the winning 200 response so
  the download helper reuses it instead of re-fetching.
- Remove the `--load` convenience flag; `download` is write-only.
  Users who want to push into Infrahub chain with `infrahubctl schema
  load`.
- Introduce a four-class error taxonomy (invalid-input, not-found,
  network) with exit codes 1 vs 2, distinguishing "version not found"
  from "schema not found" when `--version` is passed.
- Surface filesystem failures (unwritable `--output-dir`) cleanly
  rather than as a traceback.
- Regenerate `infrahubctl-marketplace.mdx`; no longer advertises
  `--load`.
- Add spec, plan, research, contract, quickstart, tasks, and
  checklist artifacts under `specs/001-marketplace-api-update/`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@minitriga minitriga requested a review from a team as a code owner April 21, 2026 12:29
@github-actions github-actions Bot added the type/documentation Improvements or additions to documentation label Apr 21, 2026
@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages Bot commented Apr 21, 2026

Deploying infrahub-sdk-python with  Cloudflare Pages  Cloudflare Pages

Latest commit: 76f1948
Status: ✅  Deploy successful!
Preview URL: https://0a853e8e.infrahub-sdk-python.pages.dev
Branch Preview URL: https://knotty-dibble.infrahub-sdk-python.pages.dev

View logs

Spell out 'semver' and replace 'config/env' with the full words so
vale's spelling and word-swap rules pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 21, 2026

Codecov Report

❌ Patch coverage is 87.50000% with 18 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
infrahub_sdk/ctl/marketplace.py 87.23% 12 Missing and 6 partials ⚠️
@@            Coverage Diff             @@
##           stable     #952      +/-   ##
==========================================
+ Coverage   81.41%   81.48%   +0.07%     
==========================================
  Files         134      135       +1     
  Lines       11347    11491     +144     
  Branches     1703     1727      +24     
==========================================
+ Hits         9238     9364     +126     
- Misses       1566     1578      +12     
- Partials      543      549       +6     
Flag Coverage Δ
integration-tests 41.63% <25.00%> (-0.22%) ⬇️
python-3.10 54.76% <87.50%> (+0.39%) ⬆️
python-3.11 54.76% <87.50%> (+0.41%) ⬆️
python-3.12 54.78% <87.50%> (+0.43%) ⬆️
python-3.13 54.76% <87.50%> (+0.39%) ⬆️
python-3.14 54.77% <87.50%> (+0.41%) ⬆️
python-filler-3.12 22.46% <0.69%> (-0.28%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
infrahub_sdk/ctl/cli_commands.py 72.54% <100.00%> (+0.21%) ⬆️
infrahub_sdk/ctl/config.py 67.85% <100.00%> (+0.58%) ⬆️
infrahub_sdk/ctl/marketplace.py 87.23% <87.23%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@minitriga minitriga changed the title Auto-detect marketplace items; drop --load from download Add infrahubctl marketplace CLI for downloading schemas and collections Apr 21, 2026
Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 issues found across 12 files

You’re at about 95% of the monthly review limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="specs/001-marketplace-api-update/quickstart.md">

<violation number="1" location="specs/001-marketplace-api-update/quickstart.md:69">
P2: Update the schema success example to include the resolved type (`schema`) so it matches the documented requirement and new CLI behavior.</violation>
</file>

<file name="infrahub_sdk/ctl/marketplace.py">

<violation number="1" location="infrahub_sdk/ctl/marketplace.py:99">
P1: Partial probe failures are misclassified as not-found. A transport failure on either endpoint (when no 200 winner exists) should return a network error, not deterministic not-found.</violation>

<violation number="2" location="infrahub_sdk/ctl/marketplace.py:243">
P1: Network/HTTP errors during the actual download path can still exit with code 1 because uncaught `httpx` exceptions are handled by the default decorator exit code.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Comment thread infrahub_sdk/ctl/marketplace.py Outdated
else:
item_type = "schema"

if item_type == "collection":
Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: Network/HTTP errors during the actual download path can still exit with code 1 because uncaught httpx exceptions are handled by the default decorator exit code.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At infrahub_sdk/ctl/marketplace.py, line 243:

<comment>Network/HTTP errors during the actual download path can still exit with code 1 because uncaught `httpx` exceptions are handled by the default decorator exit code.</comment>

<file context>
@@ -0,0 +1,248 @@
+        else:
+            item_type = "schema"
+
+        if item_type == "collection":
+            if version:
+                console.print("[yellow]Warning: --version is ignored when downloading a collection.")
</file context>
Fix with Cubic

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 76f1948 — wrapped the post-detect download dispatch in try/except httpx.HTTPError, re-raising via _fail("network", ...) so transport failures and raise_for_status 5xxs during the actual fetch exit 2 instead of bubbling up to the default @catch_exception (exit 1). Added test_versioned_download_network_error and test_collection_flag_network_error to cover the two paths that make a fresh request after auto-detect (versioned schema fetch and explicit --collection).

Comment thread infrahub_sdk/ctl/marketplace.py Outdated
return True
return isinstance(r, httpx.Response) and r.status_code >= 500

if is_transport_failure(schema_resp) and is_transport_failure(collection_resp):
Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: Partial probe failures are misclassified as not-found. A transport failure on either endpoint (when no 200 winner exists) should return a network error, not deterministic not-found.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At infrahub_sdk/ctl/marketplace.py, line 99:

<comment>Partial probe failures are misclassified as not-found. A transport failure on either endpoint (when no 200 winner exists) should return a network error, not deterministic not-found.</comment>

<file context>
@@ -0,0 +1,248 @@
+            return True
+        return isinstance(r, httpx.Response) and r.status_code >= 500
+
+    if is_transport_failure(schema_resp) and is_transport_failure(collection_resp):
+        raise _fail(
+            "network",
</file context>
Fix with Cubic

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 76f1948 — changed the guard to or so a transport failure on either probe (when no 200 winner exists) raises network rather than falling through to not-found. Added test_autodetect_partial_probe_failure_is_network to cover the schema-404 + collection-transport-failure case.

For a schema:

```text
Downloaded acme/network-base v1.2.0 -> schemas/network-base.yml
Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Update the schema success example to include the resolved type (schema) so it matches the documented requirement and new CLI behavior.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At specs/001-marketplace-api-update/quickstart.md, line 69:

<comment>Update the schema success example to include the resolved type (`schema`) so it matches the documented requirement and new CLI behavior.</comment>

<file context>
@@ -0,0 +1,103 @@
+For a schema:
+
+```text
+Downloaded acme/network-base v1.2.0 -> schemas/network-base.yml
+```
+
</file context>
Suggested change
Downloaded acme/network-base v1.2.0 -> schemas/network-base.yml
Downloaded schema acme/network-base v1.2.0 -> schemas/network-base.yml
Fix with Cubic

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 76f1948 — updated the schema success example to Downloaded schema acme/network-base v1.2.0 -> schemas/network-base.yml to match the actual CLI output.

CI runs markdownlint v0.40.0 which flags MD060 (table-column-style)
when the separator row uses compact pipes while the data rows are
padded. Match the repo's existing style (docs/python-sdk/introduction.mdx)
by padding the separator rows too.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@minitriga minitriga marked this pull request as draft April 21, 2026 12:50
@minitriga minitriga changed the title Add infrahubctl marketplace CLI for downloading schemas and collections Add infrahubctl marketplace CLI for fetching schemas and collections Apr 27, 2026
minitriga and others added 3 commits April 27, 2026 14:46
- Rename `infrahubctl marketplace download` to `infrahubctl marketplace get` to match the convention used by similar tools (kubectl, gh, etc.) and stay consistent with future `marketplace list` / `marketplace search` commands.
- Add `--stdout/-s` flag that streams content to stdout (status, warnings, and errors routed to stderr) so output can be piped into commands like `infrahubctl schema check` once stdin support lands. Skips disk writes entirely. For collections, schemas are concatenated as multi-doc YAML, only injecting `---` separators when missing.
- Route `_fail` errors through stderr unconditionally so failures don't pollute stdout when piping.
- Regenerate `infrahubctl-marketplace.mdx` and add `stdout`/`stderr` to the vale spelling exceptions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
# Conflicts:
#	infrahub_sdk/ctl/cli_commands.py
#	specs~origin_stable
…rors as network

- `_detect_item_type`: a transport failure on either probe (when no 200 winner exists) now raises `network`, not `not-found`. Mixing a 404 on one endpoint with a 5xx or transport failure on the other previously fell through to `not-found`, which masked real connectivity problems. Identified by cubic.
- `get`: wrap the post-detect download calls so `httpx.HTTPError` (transport failures and 5xx via `raise_for_status`) raises `_fail("network", ...)` (exit 2) instead of being swallowed by the default `@catch_exception` exit 1. Identified by cubic.
- Update quickstart success example to match the actual `Downloaded schema …` output.
- Add three tests: partial probe failure -> network, versioned download network error, and `--collection` flag network error.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@minitriga minitriga marked this pull request as ready for review April 27, 2026 14:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type/documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant