feat(pam): add KCM database import command with auto-detect and group filtering by jlima8900 · Pull Request #1942 · Keeper-Security/Commander

jlima8900 · 2026-04-08T14:17:11Z

Summary

Adds pam project kcm-import — a new command that imports connections directly from a KCM/Guacamole database into Keeper PAM, and pam project kcm-cleanup to reverse imports.

Key features

Docker auto-detect: discovers KCM DB container, detects MySQL/PostgreSQL, resolves container IP, extracts credentials
Group filtering: --groups, --exclude-groups, --list-groups with wildcard support
Interactive pickers: gateway selection, group selection, vault password search
Adaptive throttling: probes server RTT, adjusts batch size/delay to avoid HTTP 403
2-phase batched import: external users first, then resources with nested users
Gateway token capture: includes in report and vault record custom fields
Per-record tracking: pass/fail/skip breakdown by record type
Import report: vault record with copyable fields + MD file attachment
150+ KCM parameter mappings: SSH, RDP, VNC, Telnet, HTTP, MySQL, PostgreSQL, Oracle, SQL Server, LDAP, K8s
Cleanup command: pam project kcm-cleanup

Usage

```bash

Auto-detect everything from Docker (simplest)

pam project kcm-import --docker-detect --name "My KCM Migration"

Preview without vault changes

pam project kcm-import --docker-detect --dry-run

List available connection groups

pam project kcm-import --docker-detect --list-groups

Import only specific groups (wildcard support)

pam project kcm-import --docker-detect --groups "Production*,Staging*" --name "Prod"

Exclude groups from import

pam project kcm-import --docker-detect --exclude-groups "Test*,Incomplete*"

Manual database connection with vault-stored password

pam project kcm-import --db-host 10.0.0.5 --db-type postgresql --db-password-record "KCM DB Creds" --db-ssl --name "Production KCM"

Extend an existing PAM config with new groups

pam project kcm-import --docker-detect --config "Existing Config" --groups "NewDept*"

Save JSON for review before importing

pam project kcm-import --docker-detect --output ~/kcm-review.json

Non-interactive batch mode

pam project kcm-import --docker-detect --name "Auto Import" --gateway "My GW" --yes

Size estimate without importing

pam project kcm-import --docker-detect --estimate

Clean up an import (reverse everything)

pam project kcm-cleanup --name "KCM Migration" --dry-run
pam project kcm-cleanup --name "KCM Migration" --yes
```

Security: DB passwords are never accepted as CLI arguments. Use `--db-password-record` (vault record) or respond to the interactive prompt. Dry-run output redacts all credentials.

Gateway

If no existing gateway is specified with `--gateway`, the command creates a new one
`--max-instances N` sets the gateway pool size (0 = skip pool creation)
In interactive mode, an online gateway picker is shown
The deploy command (`docker run`) is included in the report record for easy copy/paste

Import Report

After a successful import, a vault record is created at the project's top-level folder containing:

Copyable custom fields: `Deploy Gateway (copy & paste)` with the full docker command, Gateway Token, Config UID, Gateway UID, KSM App UID, folder names
KCM-Import-Report.md file attachment with the full report
Per-record pass/fail/skip breakdown by type (including nested users)
Failed/skipped records with reasons
Throttle statistics
Redacted CLI command for reproducibility

Files changed

File	Change
`kcm_import.py`	New — 4340 lines
`kcm_mappings.json`	New — parameter mappings
`commands.py`	Register kcm-import + kcm-cleanup
`base.py` / `edit.py`	Minor pamUser validation
`README.md`	Updated with new flags/examples
`KCM_IMPORT.md`	New — quick-start guide
`test_kcm_import.py`	New — 206 tests, 3779 lines

Test plan

206 unit tests passing
E2E live import: 129 records, 0 throttles, report saved
E2E cleanup: records + folders + gateway + app deleted
Security audit: 0 critical/high
Code quality audit: clean
Docker auto-detect tested against live KCM PostgreSQL
Group filtering tested (216→65 resources)
Batch mode tested (`--yes`)

jlima8900 · 2026-04-10T08:09:45Z

PR #1904 — KCM Database Import Command Test Guide

PR: Keeper-Security/Commander#1904
Branch: feat/pam-kcm-import-pr1889 on jlima8900/Commander
Target: release

1. Prerequisites

Requirement	Details
Python	3.7+ (tested on 3.7, 3.9, 3.12)
Docker	Running a KCM/Guacamole stack (PostgreSQL or MySQL)
Keeper Vault	PAM module enabled
DB Driver	`psycopg2-binary` (PostgreSQL) or `pymysql` (MySQL)

2. Setup

# Clone and checkout the PR branch
git clone https://github.com/Keeper-Security/Commander.git
cd Commander
git fetch origin pull/1904/head:test-pr-1904
git checkout test-pr-1904

# Create venv and install
python3 -m venv venv
source venv/bin/activate
pip install -e .
pip install psycopg2-binary   # for PostgreSQL
# pip install pymysql          # for MySQL (if testing against MySQL)

3. Unit Tests

No vault or database required. Tests use mocks.

python -m pytest unit-tests/pam/test_kcm_import.py -v

Expected: 179 passed, 27 skipped (skipped = live DB tests, require KCM_TEST_DB_HOST env var).

Key test areas:

Parameter mapping (130+ Guacamole params → Keeper fields)
Default port assignment (SSH=22, RDP=3389, etc.)
Group filtering (--groups, --exclude-groups, ROOT item handling)
TOTP base32 validation
Disabled connection filtering
SSL enforcement logic
Docker auto-detection
Temp file cleanup on error
Credential cleanup from memory

4. Vault Tests

Log into Commander first:

source venv/bin/activate
python -m keepercommander shell
# (authenticate to your vault)

All commands below are run from the Keeper shell prompt (My Vault>).

4a. Dry Run

Preview what would be imported — no vault changes.

pam project kcm-import --docker-detect --dry-run

Or with a remote database:

pam project kcm-import --db-host 10.0.0.5 --db-type postgresql --dry-run

Verify:

Connection count matches your KCM instance
No Python tracebacks
Resource types are correct (pamMachine, pamDatabase, pamRemoteBrowser)
Folder paths make sense
Passwords show as [REDACTED] in console output

4b. List Groups

pam project kcm-import --docker-detect --list-groups

Verify:

All KCM connection groups are listed
Resource and user counts per group are correct
Nested groups show full path

4c. Full Import

pam project kcm-import --docker-detect --name "PR1904-Test" --yes

Verify:

Import completes with 0 errors
Record counts match the dry-run preview
Records created in vault with correct titles (KCM Resource - ..., KCM User - ...)
Default ports applied (SSH=22, RDP=3389, VNC=5900, etc.) for connections without explicit port in KCM
Import report record created with:
- Copyable command fields (cleanup, extend, verify, deploy)
- Gateway token in a password field (hidden)
- Deploy command with embedded token in a password field
- DB source info
- Results summary
- Attached .md report file (token redacted)

4d. Group Filtering

# Include only specific groups
pam project kcm-import --docker-detect --groups "Production*" --name "PR1904-Filter" --dry-run

# Exclude specific groups
pam project kcm-import --docker-detect --exclude-groups "Test*" --dry-run

# Both together
pam project kcm-import --docker-detect --groups "Staging*" --exclude-groups "Staging/Old*" --dry-run

Verify:

--groups imports only matching groups, excludes ungrouped (ROOT) items
--exclude-groups excludes matching groups, keeps ungrouped (ROOT) items
Wildcards work (*, ?)
--list-groups shows group names that match the filter patterns

4e. Extend

After a full import, add new connections to the same project:

pam project kcm-import --docker-detect --config "PR1904-Test" --yes

Verify:

Existing records are skipped (not duplicated)
Only new connections are created
Shared folders and gateway are reused

4f. Cleanup

# Preview what will be deleted
pam project kcm-cleanup --name "PR1904-Test" --dry-run

# Delete
pam project kcm-cleanup --name "PR1904-Test" --yes

Verify:

Dry run shows correct record count
Only records with KCM Resource - or KCM User - title prefix are deleted
Manually added records in the same shared folders survive cleanup
The KCM Import Report record is not deleted (by design)
Shared folders and gateway are removed

5. Security Tests

5a. SSL Enforcement

# Remote host without SSL — should be REJECTED
pam project kcm-import --db-host remote.example.com --dry-run

# Remote host with SSL — should work
pam project kcm-import --db-host remote.example.com --db-ssl --dry-run

# Remote host with cleartext override — should warn but work
pam project kcm-import --db-host remote.example.com --allow-cleartext --dry-run

# Localhost — should work without SSL (exempt)
pam project kcm-import --db-host 127.0.0.1 --dry-run

# Private network (RFC 1918) — should work without SSL (exempt)
pam project kcm-import --db-host 10.0.0.5 --dry-run

5b. Gateway Token Handling

After a full import, inspect the report record:

get <REPORT_RECORD_UID>

Verify:

Gateway token is in a password field (displayed as ********)
Deploy Gateway command is in a password field (contains token)
Notes text shows (see "Gateway Access Token" field on this record) — NOT the raw token
Attached .md file does NOT contain the raw token
Console output during import DOES show the token (intentional — needed for deployment)

5c. Output File Permissions

pam project kcm-import --docker-detect --output /tmp/kcm-review.json --dry-run

Then from a system shell:

ls -la /tmp/kcm-review.json
# Should show: -rw------- (0600, owner read/write only)

With credentials:

pam project kcm-import --docker-detect --output /tmp/kcm-creds.json --include-credentials

Verify:

Default output has passwords as [REDACTED]
--include-credentials shows actual passwords
File permissions are 0600 in both cases

6. Edge Cases

Scenario	Command	Expected
Empty KCM database	`--dry-run` against empty DB	Clean exit, 0 connections message
Disabled connections	`--dry-run`	Disabled connections excluded by default
Include disabled	`--include-disabled --dry-run`	Disabled connections included
Skip users	`--skip-users --dry-run`	Only resources, no `pamUser` records
Size estimate	`--estimate`	Shows count and exits, no import
Cancel prompt	Run without `--yes`, answer `n`	Clean abort, no changes
Wrong DB type	`--db-type mysql` against PostgreSQL	Clear error message
Bad credentials	Wrong `--db-user`	Clear error, no traceback

7. Key Features to Test

Feature	What to look for
Gateway token	Hidden password field in report, redacted in notes and `.md` attachment
Default ports	SSH=22, RDP=3389, VNC=5900 auto-assigned when KCM has no explicit port
Group filtering	`--exclude-groups` keeps ungrouped items, `--groups` excludes them
Cleanup safety	Only deletes `KCM Resource -` / `KCM User -` records, not manual ones
Import report	Copyable fields for deploy, extend, cleanup, verify, re-import
SSL enforcement	Remote hosts require `--db-ssl` or `--allow-cleartext`

… filtering Adds pam project kcm-import and pam project kcm-cleanup commands. Imports connections directly from a KCM/Guacamole database into Keeper PAM with Docker auto-detect, group filtering, adaptive throttling, and 150+ parameter mappings across SSH, RDP, VNC, Telnet, HTTP, MySQL, PostgreSQL, Oracle, SQL Server, LDAP, and Kubernetes protocols.

Bug 1: Reuse existing gateway on re-import instead of creating Keeper-Security#2, Keeper-Security#3 Bug 2: Reuse existing project folder + shared subfolders on re-import Bug 3: Cleanup only matches exact "Name - Resources" / "Name - Users" folders (removed overly broad startswith match) Bug 4: Windows file permissions test accepts 0o666/0o644 on Windows (os.open 0o600 is ignored by Windows OS)

validate_schema() now checks for guacamole_connection_attribute and guacamole_connection_group_attribute as optional tables. If missing (e.g. vanilla Guacamole without KCM extensions), logs a warning that connection attributes won't be available instead of failing silently when the attribute query runs later.

Bug D: --name retry now detects existing config by title and extends instead of creating duplicate project skeleton (also prevents Bug A duplicate KSM Applications on failed retries) Bug C: Report record creation adds Strategy 4 fallback — find project folder via config record's subfolder location when res_sf_name is empty (happens with --groups filter) Bug E: Empty environment_type defaults to "local" from settings without triggering the warning message Bug F: Handle bytes config values in extend.py — decode to str before casefold() to prevent AttributeError

jlima8900 · 2026-04-11T00:37:21Z

Ready to merge — all fixes applied

Fixes from Ivan's review feedback (commit `a8e6c4a`)

Duplicate gateways: Re-import now reuses existing gateway by name instead of creating KC-13 Clean up #2, Streamlining the record object #3
Duplicate folders: Re-import reuses existing project folder + shared subfolders instead of creating empty numbered copies
Cleanup deleting everything: Changed folder matching to exact match on "Name - Resources" / "Name - Users" only — removed overly broad startswith() that caught unrelated folders
Windows test failure: test_output_file_owner_only now accepts 0o666/0o644 on Windows (OS ignores Unix permission bits)
Unix-only docs: Added Windows PowerShell equivalents for source, ls -la, /tmp/ paths

Additional fixes found during testing (commits `d615e43`, `6d00985`)

--name retry fails: 2nd --name "AAA" --yes now detects existing config by title and extends instead of failing — also prevents orphaned KSM Applications KC-13 Clean up #2, Streamlining the record object #3 on failed attempts
Report not created with --groups: Added fallback folder lookup via config record's subfolder location when resources folder name is empty
Unrecognized environment type "": Empty environment type now defaults to "local" from settings silently (no warning)
'bytes' object has no attribute 'casefold': Decode bytes to str before casefold in extend.py
Schema validation: Upgraded from checking 3 tables to validating 5 tables + 13 columns. Warns on missing optional attribute tables instead of failing silently

Testing

14 command syntaxes tested against live vault (dry-run, import, re-import, groups, exclude-groups, combined filters, list-groups, output, estimate, cleanup dry-run, cleanup, skip-users, include-disabled)
Re-import with --name confirmed: "Found existing project — extending", 0 duplicates
Cleanup confirmed: exact folder match, only KCM-prefixed records deleted
179 unit tests passing (pytest 3.7 + 3.12)
Security audit: 0 critical, 0 high

No longer in progress — ready to merge.

Rename PEDM to EPM

6f035f6

sk-keeper force-pushed the release branch from 1a6c40b to 6f035f6 Compare April 9, 2026 23:20

jlima8900 force-pushed the feat/pam-kcm-import-pr1889 branch from af25290 to ff340a8 Compare April 10, 2026 08:12

jlima8900 added 2 commits April 10, 2026 15:29

idimov-keeper approved these changes Apr 10, 2026

View reviewed changes

sk-keeper force-pushed the release branch 2 times, most recently from 442d7af to 9ec9b7a Compare April 15, 2026 03:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(pam): add KCM database import command with auto-detect and group filtering#1942

feat(pam): add KCM database import command with auto-detect and group filtering#1942
jlima8900 wants to merge 5 commits intoKeeper-Security:releasefrom
jlima8900:feat/pam-kcm-import-pr1889

jlima8900 commented Apr 8, 2026 •

edited by pvagare-ks

Loading

Uh oh!

jlima8900 commented Apr 10, 2026

Uh oh!

jlima8900 commented Apr 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

jlima8900 commented Apr 8, 2026 • edited by pvagare-ks Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Key features

Usage

Auto-detect everything from Docker (simplest)

Preview without vault changes

List available connection groups

Import only specific groups (wildcard support)

Exclude groups from import

Manual database connection with vault-stored password

Extend an existing PAM config with new groups

Save JSON for review before importing

Non-interactive batch mode

Size estimate without importing

Clean up an import (reverse everything)

Gateway

Import Report

Files changed

Test plan

Uh oh!

jlima8900 commented Apr 10, 2026

PR #1904 — KCM Database Import Command Test Guide

Table of Contents

1. Prerequisites

2. Setup

3. Unit Tests

4. Vault Tests

4a. Dry Run

4b. List Groups

4c. Full Import

4d. Group Filtering

4e. Extend

4f. Cleanup

5. Security Tests

5a. SSL Enforcement

5b. Gateway Token Handling

5c. Output File Permissions

6. Edge Cases

7. Key Features to Test

Uh oh!

jlima8900 commented Apr 11, 2026

Ready to merge — all fixes applied

Fixes from Ivan's review feedback (commit a8e6c4a)

Additional fixes found during testing (commits d615e43, 6d00985)

Testing

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jlima8900 commented Apr 8, 2026 •

edited by pvagare-ks

Loading

Fixes from Ivan's review feedback (commit `a8e6c4a`)

Additional fixes found during testing (commits `d615e43`, `6d00985`)