Consolidate multisession assignment and ROI similarity utilities

[FlorianPfaff]

Summary

This PR consolidates the reusable core across the recent tracking-related branches into one proposal for main.

Included here:

• the generic multi-session assignment solver and observation-cost extensions from Codex/multisess convenience functions #1804
• the overlapping multisession pieces from Codex/multisess track2p convenience v2 #1806, treated as subsumed by the more complete Codex/multisess convenience functions #1804 implementation
• the reusable ROI-similarity / ROI-cost-matrix utilities from Add Suite2p-aware weighted ROI similarity utilities #1803

What this keeps in PyRecEst

• generic global multi-session assignment
• heterogeneous start/end costs per observation
• similarity-input wrapper and dense track-by-session export
• reusable weighted ROI similarity metrics
• centroid distances and fused ROI association-cost construction

Notes

• This intentionally keeps the utility-level ROI core, but not any Track2p-specific I/O or pipeline glue.
• The ROI utility layer was integrated on top of the consolidated multisession branch and exported through pyrecest.utils.
• The ROI helper module was lightly refactored while integrating to keep it cleaner as a library utility.

Supersedes

This PR is intended to supersede:

• Add Suite2p-aware weighted ROI similarity utilities #1803
• Codex/multisess convenience functions #1804
• Codex/multisess track2p convenience v2 #1806

[github-actions]

❌MegaLinter analysis: Error

Descriptor	Linter	Files	Fixed	Errors	Warnings	Elapsed time
✅ COPYPASTE	jscpd	yes		no	no	14.3s
✅ JSON	prettier	2	0	0	0	0.44s
✅ JSON	v8r	2		0	0	2.67s
✅ MARKDOWN	markdownlint	2	0	0	0	0.71s
✅ MARKDOWN	markdown-table-formatter	2	0	0	0	0.26s
✅ PYTHON	bandit	357		0	0	5.44s
✅ PYTHON	black	357	13	0	0	9.25s
❌ PYTHON	flake8	357		1	0	3.02s
✅ PYTHON	isort	357	18	0	0	0.71s
❌ PYTHON	mypy	357		3	0	5.12s
❌ PYTHON	pylint	357		20	0	111.41s
✅ PYTHON	ruff	357	18	0	0	0.07s
✅ REPOSITORY	checkov	yes		no	no	22.91s
✅ REPOSITORY	gitleaks	yes		no	no	6.1s
✅ REPOSITORY	git_diff	yes		no	no	0.07s
✅ REPOSITORY	secretlint	yes		no	no	6.58s
✅ REPOSITORY	syft	yes		no	no	6.34s
✅ REPOSITORY	trivy-sbom	yes		no	no	2.68s
✅ REPOSITORY	trufflehog	yes		no	no	19.23s
✅ YAML	prettier	4	0	0	0	0.48s
✅ YAML	v8r	4		0	0	4.8s
✅ YAML	yamllint	4		0	0	0.43s

Detailed Issues

❌ PYTHON / flake8 - 1 error

pyrecest/utils/multisession_assignment.py:49:5: F811 redefinition of unused 'np' from line 29

❌ PYTHON / mypy - 3 errors

pyrecest/utils/multisession_assignment_observation_costs.py:20: error: Variable "pyrecest.utils.multisession_assignment_observation_costs.ObservationCostValue" is not valid as a type  [valid-type]
pyrecest/utils/multisession_assignment_observation_costs.py:20: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
pyrecest/utils/multisession_assignment_observation_costs.py:214: error: Variable "pyrecest.utils.multisession_assignment_observation_costs.ObservationCostValue" is not valid as a type  [valid-type]
pyrecest/utils/multisession_assignment_observation_costs.py:214: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
pyrecest/utils/multisession_assignment_observation_costs.py:252: error: Variable "pyrecest.utils.multisession_assignment_observation_costs.ObservationCostValue" is not valid as a type  [valid-type]
pyrecest/utils/multisession_assignment_observation_costs.py:252: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
pyrecest/smoothers/unscented_rauch_tung_striebel_smoother.py:51: note: By default the bodies of untyped functions are not checked, consider using --check-untyped-defs  [annotation-unchecked]
Found 3 errors in 1 file (checked 357 source files)

❌ PYTHON / pylint - 20 errors

************* Module pyrecest.tests.test_multisession_assignment
pyrecest/tests/test_multisession_assignment.py:246:22: W0212: Access to a protected member _solve_max_weight_matching of a client class (protected-access)
pyrecest/tests/test_multisession_assignment.py:252:23: W0212: Access to a protected member _solve_max_weight_matching_via_linprog of a client class (protected-access)
************* Module pyrecest.utils.multisession_assignment
pyrecest/utils/multisession_assignment.py:49:4: W0404: Reimport 'numpy' (imported line 29) (reimported)
pyrecest/utils/multisession_assignment.py:662:0: R0914: Too many local variables (17/15) (too-many-locals)
************* Module pyrecest.utils.multisession_assignment_observation_costs
pyrecest/utils/multisession_assignment_observation_costs.py:22:0: R0913: Too many arguments (8/6) (too-many-arguments)
pyrecest/utils/multisession_assignment_observation_costs.py:22:0: R0914: Too many local variables (23/15) (too-many-locals)
pyrecest/utils/multisession_assignment_observation_costs.py:206:22: R1721: Unnecessary use of a comprehension, use dict(enumerate(observation_costs)) instead. (unnecessary-comprehension)
pyrecest/utils/multisession_assignment_observation_costs.py:254:0: R0913: Too many arguments (8/6) (too-many-arguments)
pyrecest/utils/multisession_assignment_observation_costs.py:254:0: R0914: Too many local variables (19/15) (too-many-locals)
************* Module pyrecest.utils.roi_similarity
pyrecest/utils/roi_similarity.py:420:0: R0914: Too many local variables (17/15) (too-many-locals)
************* Module update_init_helper
update_init_helper.py:1:0: R0801: Similar lines in 2 files
==pyrecest.utils.multisession_assignment:[503:554]
==pyrecest.utils.multisession_assignment_observation_costs:[145:200]
        if matrix.ndim != 2:
            raise ValueError("Each pairwise cost matrix must be two-dimensional.")
        normalized[(session_idx, session_idx + 1)] = matrix
    return normalized


def _normalize_session_sizes(
    session_sizes: SessionSizesInput | None,
) -> dict[int, int]:
    if session_sizes is None:
        return {}
    if isinstance(session_sizes, Mapping):
        normalized = {int(session_idx): int(size) for session_idx, size in session_sizes.items()}
    else:
        normalized = {session_idx: int(size) for session_idx, size in enumerate(session_sizes)}
    for session_idx, size in normalized.items():
        if size < 0:
            raise ValueError(f"Session {session_idx} has a negative detection count.")
    return normalized


def _infer_and_validate_session_sizes(
    pairwise_costs: Mapping[tuple[int, int], np.ndarray],
    session_sizes: Mapping[int, int],
) -> dict[int, int]:
    inferred_sizes = dict(session_sizes)
    for (source_session, target_session), cost_matrix in pairwise_costs.items():
        source_size, target_size = cost_matrix.shape
        _check_or_set_session_size(inferred_sizes, source_session, source_size)
        _check_or_set_session_size(inferred_sizes, target_session, target_size)

    if not inferred_sizes and not pairwise_costs:
        raise ValueError("No observations were provided. Supply pairwise_costs or session_sizes.")

    return dict(sorted(inferred_sizes.items()))


def _check_or_set_session_size(
    inferred_sizes: dict[int, int],
    session_idx: int,
    candidate_size: int,
) -> None:
    if session_idx in inferred_sizes and inferred_sizes[session_idx] != candidate_size:
        raise ValueError(
            f"Inconsistent detection count for session {session_idx}: "
            f"expected {inferred_sizes[session_idx]}, got {candidate_size}."
        )
    inferred_sizes[session_idx] = int(candidate_size)


def _build_session_offsets(session_sizes: Mapping[int, int]) -> dict[int, int]: (duplicate-code)
update_init_helper.py:1:0: R0801: Similar lines in 2 files
==pyrecest.utils.nonrigid_point_set_registration:[211:221]
==pyrecest.utils.point_set_registration:[241:251]
    costs = asarray(cost_matrix)
    if costs.ndim != 2:
        raise ValueError("cost_matrix must be two-dimensional.")
    if costs.shape[0] == 0:
        return zeros((0,), dtype=int64)
    if costs.shape[1] == 0:
        return zeros((costs.shape[0],), dtype=int64) - 1

    finite_mask = isfinite(costs)
    finite_costs = costs[finite_mask] (duplicate-code)
update_init_helper.py:1:0: R0801: Similar lines in 2 files
==pyrecest.utils.multisession_assignment:[486:494]
==pyrecest.utils.multisession_assignment_observation_costs:[128:136]
    if isinstance(pairwise_costs, Mapping):
        normalized: dict[tuple[int, int], np.ndarray] = {}
        for key, value in pairwise_costs.items():
            if len(key) != 2:
                raise ValueError("Each pairwise-cost key must contain two session indices.")
            source_session, target_session = int(key[0]), int(key[1])
            if source_session >= target_session:
                raise ValueError("Pairwise-cost keys must satisfy source_session < target_session.") (duplicate-code)
update_init_helper.py:1:0: R0801: Similar lines in 2 files
==pyrecest.utils.nonrigid_point_set_registration:[124:135]
==pyrecest.utils.point_set_registration:[107:118]
    assignment: Any
    matched_reference_indices: Any
    matched_moving_indices: Any
    transformed_reference_points: Any
    matched_costs: Any
    rmse: float
    n_iterations: int
    converged: bool


def _as_point_array(points): (duplicate-code)
update_init_helper.py:1:0: R0801: Similar lines in 2 files
==pyrecest.utils.nonrigid_point_set_registration:[319:329]
==pyrecest.utils.point_set_registration:[390:400]
    association_cost = _default_cost if cost_function is None else cost_function

    for iteration in range(1, max_iterations + 1):
        transformed_reference = transform.apply(reference)
        current_costs = asarray(association_cost(transformed_reference, moving))
        if current_costs.shape != (reference.shape[0], moving.shape[0]):
            raise ValueError(
                "cost_function must return an array of shape (n_reference, n_moving)."
            )
 (duplicate-code)
update_init_helper.py:1:0: R0801: Similar lines in 2 files
==pyrecest.utils.multisession_assignment:[495:502]
==pyrecest.utils.multisession_assignment_observation_costs:[137:144]
            if matrix.ndim != 2:
                raise ValueError("Each pairwise cost matrix must be two-dimensional.")
            normalized[(source_session, target_session)] = matrix
        return normalized

    normalized = {}
    for session_idx, value in enumerate(pairwise_costs): (duplicate-code)
update_init_helper.py:1:0: R0801: Similar lines in 2 files
==pyrecest.utils.__init__:[27:32]
==pyrecest.utils.multisession_assignment:[816:821]
__all__ = [
    "MultiSessionAssignmentResult",
    "solve_multisession_assignment",
    "solve_multisession_assignment_from_similarity",
    "tracks_to_index_matrix", (duplicate-code)
update_init_helper.py:1:0: R0801: Similar lines in 2 files
==pyrecest.utils.multisession_assignment:[601:607]
==pyrecest.utils.multisession_assignment_observation_costs:[266:272]
        source_position = session_positions[source_session]
        target_position = session_positions[target_session]
        gap = target_position - source_position - 1
        if gap < 0:
            raise ValueError("Session indices must define a forward-in-time edge ordering.")
 (duplicate-code)
update_init_helper.py:1:0: R0801: Similar lines in 2 files
==pyrecest.utils.nonrigid_point_set_registration:[146:154]
==pyrecest.utils.point_set_registration:[129:137]
    source = _as_point_array(source_points)
    target = _as_point_array(target_points)
    if source.shape != target.shape:
        raise ValueError("source_points and target_points must have the same shape.")
    return source, target


def _normalize_weights(weights, n_points): (duplicate-code)
update_init_helper.py:1:0: R0801: Similar lines in 2 files
==pyrecest.utils.nonrigid_point_set_registration:[230:236]
==pyrecest.utils.point_set_registration:[262:268]
    row_indices, col_indices = linear_sum_assignment(padded_costs)
    assignment = zeros((costs.shape[0],), dtype=int64) - 1

    for row_index, col_index in zip(row_indices, col_indices):
        if row_index >= costs.shape[0] or col_index >= costs.shape[1]:
            continue (duplicate-code)

-----------------------------------
Your code has been rated at 9.99/10

See detailed reports in MegaLinter artifacts

Your project could benefit from a custom flavor, which would allow you to run only the linters you need, and thus improve runtime performances. (Skip this info by defining FLAVOR_SUGGESTIONS: false)

• Documentation: Custom Flavors
• Command: npx mega-linter-runner@9.4.0 --custom-flavor-setup --custom-flavor-linters PYTHON_PYLINT,PYTHON_BLACK,PYTHON_FLAKE8,PYTHON_ISORT,PYTHON_BANDIT,PYTHON_MYPY,PYTHON_RUFF,COPYPASTE_JSCPD,JSON_V8R,JSON_PRETTIER,MARKDOWN_MARKDOWNLINT,MARKDOWN_MARKDOWN_TABLE_FORMATTER,REPOSITORY_CHECKOV,REPOSITORY_GIT_DIFF,REPOSITORY_GITLEAKS,REPOSITORY_SECRETLINT,REPOSITORY_SYFT,REPOSITORY_TRIVY_SBOM,REPOSITORY_TRUFFLEHOG,YAML_PRETTIER,YAML_YAMLLINT,YAML_V8R

Show us your support by starring ⭐ the repository