Add JunOS pre-transfer free-space check (NAPPS-1085)#373
Merged
jeffkala merged 4 commits intonetworktocode:developfrom Apr 22, 2026
Merged
Add JunOS pre-transfer free-space check (NAPPS-1085)#373jeffkala merged 4 commits intonetworktocode:developfrom
jeffkala merged 4 commits intonetworktocode:developfrom
Conversation
Implements the fail-open pre-transfer free-space check on Juniper Junos using the seam added in NAPPS-1091 (PR networktocode#370). Image transfers now fail fast when the target filesystem lacks room instead of half-writing flash. Lab-validated against an SRX340 (Junos 24.x). Driver (pyntc/devices/jnpr_device.py): - JunosDevice._get_free_space uses PyEZ FS.storage_usage() and parses the human-readable avail format (e.g., "126M", "1.0G"). PyEZ does not expose a native block size and Junos block semantics vary by release, so parsing the normalised human-readable field avoids the ambiguity. - Mount resolution uses longest-prefix match (same logic df uses) via _mount_encloses_path. SRX hardware does not expose /var/tmp as its own mount — it lives inside /var — so strict equality would raise; / becomes the universal fallback. Directory-boundary semantics prevent /vari from matching /var/tmp. - file_copy calls _check_free_space(os.path.getsize(src)) before any SCP put. - remote_file_copy gains optional file_system + **kwargs for BaseDevice parity, and calls _pre_transfer_space_check before fs.cp so the check fires only when a transfer would actually happen; still fail-open when src.file_size_bytes is None. - remote_file_copy appends src.file_name to the URL when the URL carries no path (mirroring ASA), so callers can point at a bare host like ftp://server. Tests: - Unit tests cover avail parse across K/M/G/T/P units, longest-prefix match (exact hit, most-specific wins, root fallback, /vari boundary rejected, empty storage raises), file_copy raising before SCP.put, remote_file_copy raising before fs.cp, fail-open path, and URL append / keep-intact. - Existing test_file_copy and test_remote_file_copy prime fs.storage_usage so the new probe succeeds. - New integration suite (tests/integration/test_jnpr_device.py) mirrors the EOS / ASA patterns for manual lab runs. TFTP is excluded because PyEZ fs.cp does not accept TFTP URLs. Shared integration helper: - integration_hash_algo() reads FILE_HASH_ALGO (default "sha512") so lab runs can pick the algorithm their devices support. Junos does not implement sha512 — SRX/MX runs set FILE_HASH_ALGO=sha256. Backward-compatible for EOS / ASA runs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
0f7b828 to
5b6b634
Compare
Contributor
Author
|
jeffkala
reviewed
Apr 22, 2026
mattmiller87
requested changes
Apr 22, 2026
Contributor
mattmiller87
left a comment
There was a problem hiding this comment.
I think I would want to add a JUNOS_SUPPORTED_HASHING_ALGORITHMS so that we can make any integration or testing simpler.
…ller87
- Add JUNOS_SUPPORTED_HASHING_ALGORITHMS = {"md5", "sha1", "sha256"}
module constant mirroring EOS_SUPPORTED_HASHING_ALGORITHMS and
NXOS_SUPPORTED_HASHING_ALGORITHMS, plus a matching guard in
get_remote_checksum that raises ValueError before reaching PyEZ.
Callers asking for sha512 now fail fast with a clear driver-level
message instead of tripping over PyEZ's raw ValueError.
(mattmiller87)
- Clarify _get_free_space and remote_file_copy docstrings so the
``/var/tmp`` default is attributed to the underlying
``_JUNOS_DEFAULT_FILE_SYSTEM`` constant rather than implying the
public signature carries the literal default. (jeffkala)
- Rename single-letter ``k, v`` loop variables in
JUNOS_PROTOCOL_URL_VARS to ``scheme, env_var``. NTC style prefers
descriptive names. (jeffkala)
- New unit test covers the ValueError path on an unsupported algo.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per @mattmiller87's review: each device should carry its own default algorithm rather than routing through a shared env-var helper. - Delete ``integration_hash_algo()`` from ``tests/integration/_helpers.py``. - ``build_file_copy_model`` now takes ``hashing_algorithm`` as a kwarg (default ``"sha512"`` preserves EOS / ASA behaviour without any caller changes). - ``any_file_copy_model`` fixture in ``conftest.py`` reverts to the literal ``"sha512"`` default. - ``test_jnpr_device.py`` exposes ``JUNOS_INTEGRATION_HASH_ALGO = "sha256"`` at module level and passes it explicitly to every site that builds or verifies a checksum; driver-level validation (added in the prior commit) ensures Junos never sees sha512. - Docstring updated: no more ``FILE_HASH_ALGO`` env var; labs that ship md5 / sha1 binaries edit the module constant. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Supports the ``run every device's integration suite in one pytest invocation'' workflow: each platform test module maps to the hashing algorithm its device family implements, and an autouse module-scoped fixture copies the matching ``FILE_CHECKSUM_<SUFFIX>`` env var into ``FILE_CHECKSUM`` (and ``FILE_HASH_ALGO`` to the algo name) before that module's tests run. The env is restored on teardown. - ``tests/integration/conftest.py`` adds ``_PLATFORM_HASH_ALGOS`` (module name → algo) and ``_HASH_ALGO_ENV_SUFFIXES`` (algo → env suffix), plus the ``_configure_integration_env`` autouse fixture that applies the pair for each module. When the suffix env var is missing the fixture leaves both vars untouched so the user's own shell-level pair wins (or ``build_file_copy_model`` skips cleanly on no env) — guards against the silent algo/checksum mismatch that would otherwise fail later at ``verify_file``. - ``tests/integration/_helpers.py`` simplifies ``build_file_copy_model``: it now reads ``FILE_HASH_ALGO`` and ``FILE_CHECKSUM`` from the environment (the autouse fixture populates both), dropping the ``hashing_algorithm`` kwarg it grew in the previous commit. - ``tests/integration/test_jnpr_device.py`` drops the ``JUNOS_INTEGRATION_HASH_ALGO`` module constant — the autouse fixture now sets ``FILE_HASH_ALGO="sha256"`` for this module. Direct ``FileCopyModel(...)`` constructions read the algo from env; ``test_verify_file_after_copy`` uses ``any_file_copy_model.hashing_algorithm`` instead of a hardcode. Docstring updated to reference ``FILE_CHECKSUM_256``. User setup: export ``FILE_CHECKSUM_512`` / ``FILE_CHECKSUM_256`` / ``FILE_CHECKSUM_MD5`` once; ``FILE_HASH_ALGO`` and ``FILE_CHECKSUM`` no longer need to live in .env at all. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
mattmiller87
approved these changes
Apr 22, 2026
Contributor
mattmiller87
left a comment
There was a problem hiding this comment.
Junos looks good to me.
jeffkala
approved these changes
Apr 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/var/tmp) lacks room instead of half-writing flash.JunosDevice._get_free_spaceuses PyEZ'sFS.storage_usage()and parses the human-readableavailformat (e.g.,"126M","1.0G"). PyEZ does not expose a native block size and Junos block semantics vary by release, so parsing the normalised human-readable field avoids the ambiguity. Defaults to/var/tmpand raisesFileSystemNotFoundErrorwhen the mount is absent.file_copycalls_check_free_space(os.path.getsize(src))before SCP put;remote_file_copygains an optionalfile_systemkwarg and calls_pre_transfer_space_checkbeforefs.cpso the check fires only when an actual transfer would happen.file_copyraising before SCP put,remote_file_copyraising beforefs.cp, and the fail-open path. Integration tests mirror the EOS / ASA patterns for manual lab runs. TFTP is excluded because PyEZfs.cpdoes not accept TFTP URLs.NAPPS-1085
Test plan
poetry run pytest tests/unit/test_devices/test_jnpr_device.py— all 45 pass (7 new).poetry run pytest tests/unit— all 661 pass.poetry run ruff check pyntc tests— clean.poetry run pytest tests/integration/test_jnpr_device.py -vwithJUNOS_HOST/JUNOS_USER/JUNOS_PASS+ protocol URLs +FILE_CHECKSUM/FILE_SIZE/FILE_SIZE_UNITset — verifies the probe, the oversized reject, and the fail-open path end-to-end.🤖 Generated with Claude Code