feat: multi-profile support and project discovery

[rahim-bhojani]

Multi-Profile Support for Dremio CLI

Problem

The Dremio CLI currently supports a single configured context — one PAT, one project ID, one region — stored in ~/.config/dremioai/config.yaml. Users who work across multiple organizations or projects must either:

• Re-run dremio setup each time they switch, which overwrites their existing config
• Manually juggle environment variables (DREMIO_TOKEN, DREMIO_PROJECT_ID)
• Maintain separate config files and pass --config /path/to/other.yaml on every invocation

The setup experience is also unnecessarily manual — users must copy-paste a project ID from the UI when the CLI could discover available projects automatically.

Proposed Solution

Mirror the Dremio Cloud UI flow: authenticate, discover projects, and let the user choose — then persist the result as a named profile.

Setup flow (interactive):

$ dremio setup

  Step 1: Choose your region
    1) US  (api.dremio.cloud)
    2) EU  (api.eu.dremio.cloud)

  Step 2: Enter your Personal Access Token
    [instructions for creating a PAT]

  Step 3: Choose a project          ← NEW: discovered from the API
    Org: Acme Corp
    1) Production Analytics  (a1b2c3d4-...)
    2) Staging               (e5f6g7h8-...)
    3) Dev Sandbox           (i9j0k1l2-...)

  Step 4: Name this profile         ← NEW
    Profile name [acme-production]: 

  ✓ Saved profile "acme-production" (set as default)

Running dremio setup again adds another profile without overwriting existing ones. Users with multiple orgs run setup once per org (each org has its own PAT).

Config format:

default_profile: acme-production

profiles:
  acme-production:
    uri: https://api.dremio.cloud
    pat: dremio_pat_xxx
    project_id: a1b2c3d4-...
  acme-staging:
    uri: https://api.dremio.cloud
    pat: dremio_pat_xxx          # same PAT, different project
    project_id: e5f6g7h8-...
  eu-analytics:
    uri: https://api.eu.dremio.cloud
    pat: dremio_pat_yyy          # different org/region
    project_id: m3n4o5p6-...

Profile switching:

# Global flag (per-command)
dremio --profile acme-staging query run "SELECT 1"

# Switch default
dremio context use acme-staging

# List all profiles
dremio context list
  acme-production   US   Production Analytics
* acme-staging      US   Staging               ← active
  eu-analytics      EU   EU Analytics

# Environment variable
export DREMIO_PROFILE=eu-analytics

Resolution order (unchanged in spirit): CLI flags (--token, etc.) > env vars > named profile > default profile.

Backwards compatible: Existing flat config files (no profiles key) are treated as an implicit default profile. No migration required.

What Changed

File	Change
auth.py	load_config() gains profile param, reads profiles dict, DREMIO_PROFILE env var, helper functions for listing/switching profiles
setup.py	Discovery flow (calls list_projects after PAT auth), additive profile writes, profile naming with slug suggestion
cli.py	--profile / -p global option, registers context command group
commands/context.py	New — list, use, current subcommands for profile management
client.py	No changes
All other commands	No changes — they consume DrsConfig as before

Open Questions

1. Profile naming — auto-suggest from project name (current approach), or always ask?
2. Org name discovery — the list_projects response may include org metadata we can display. Need to verify the response shape.
3. PAT reuse across projects — a single PAT works for all projects in one org. Should setup detect an existing PAT and skip re-prompting when adding another project from the same org?

Test plan

• All 186 existing tests pass
• 21 new tests for auth profiles, setup discovery flow, and context commands
• Backwards compatibility: legacy flat configs load correctly
• Profile resolution: --profile > DREMIO_PROFILE env > default_profile in file
• Manual test: dremio setup against a real Dremio Cloud account
• Manual test: dremio context list/use/current workflow

🤖 Generated with Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cb49fe0871

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Handle implicit default profile in set_default_profile

set_default_profile only checks the explicit profiles map, so legacy flat configs (which the same module treats as an implicit default profile) always raise “Profile not found”. This makes dremio context use default fail for valid legacy configs even though context list/context current expose default, which breaks the advertised backward-compatibility path for profile management.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Include legacy default in setup overwrite checks

Setup computes existing_profiles only from the raw profiles key, so legacy flat configs are treated as having no existing profile names during the prompt. If the user chooses default (or accepts it when slug fallback yields that), write_profile first migrates legacy credentials into profiles.default and then silently overwrites them with the new profile without any overwrite confirmation, causing unintended credential loss.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Escape dataset-name filter before SQL interpolation

The --dataset-name value is interpolated directly into a quoted SQL literal without escaping. Inputs containing a single quote (for example O'Reilly) generate invalid SQL and cause reflection list to fail, even though this flag is intended for free-form substring matching.

Useful? React with 👍 / 👎.