Add build → score pipeline example

[jammastergirish]

Summary

Adds a runnable two-step pipeline example demonstrating the no-compression scoring path:

bergson pipeline examples/pipelines/build_then_score.yaml

Step 1 (build) creates a one-gradient on-disk query index with projection_dim: 0 and aggregation: mean. Step 2 (score) loads that query into memory and dot-products it against each item in a small slice of NeelNanda/pile-10k, producing per-item influence scores. Until now, this required two CLI invocations — this single YAML now drives the whole flow through bergson pipeline.

Why

1. Showcase the YAML pipeline functionality introduced in ELE-11: Add YAML-mediated search pipeline to Bergson #246 with a real, useful example.
2. Demonstrate a feature that previously required multiple commands (bergson build + bergson score) — now reducible to a single bergson pipeline invocation.
3. Lay groundwork for EK-FAC support in score — having an end-to-end no-compression scoring pipeline in YAML form is a prerequisite for then layering Hessian application onto the uncompressed gradients (Lewis's worker has the existing reference for that), without entangling Hessian work with the rest of the codebase.

Files

• examples/pipelines/build_then_score.yaml — the example pipeline (small model: gpt2, small dataset: NeelNanda/pile-10k with train[:20] for the query and train[:100] for scoring, chunk_length: 1024).
• README.md — extends the "Run a Multi-Step Pipeline" section introduced in ELE-11: Add YAML-mediated search pipeline to Bergson #246 with a one-sentence reference to build_then_score.yaml as a second example covering the no-compression scoring path. (The section itself, the description of bergson pipeline, and the link to hessian_then_build.yaml all live in ELE-11: Add YAML-mediated search pipeline to Bergson #246.)
• tests/test_yaml_pipeline.py — test_build_then_score_example_parses asserts the shipped YAML hydrates into the right Build/Score commands with the expected configs (parse-only, no GPU needed).

Configuration notes

• skip_preconditioners: true is set on both build and score steps. With projection_dim: 0 the per-token gradient P retains the full flattened module-gradient dimension; the default per-module Adafactor preconditioner update P.mT @ P would otherwise materialize a (full_grad_dim × full_grad_dim) matrix — multi-TB even for GPT-2's c_attn — and OOM immediately. The no-compression scoring path uses raw dot products of full gradients, so preconditioners aren't needed here. See issue Default skip_preconditioners=True for score (or have projection_dim=0 imply it) #255 for a suggested follow-up to make this implicit.
• unit_normalize and --preconditioner_path from the CLI command in the Linear task aren't replicated here. Those belong to the compressed/preconditioned path. This example is deliberately the no-compression demo.

Dependencies

• Stacked on ELE-11: Add YAML-mediated search pipeline to Bergson #246 (add_yaml-mediated_search_pipeline) — this branch sits on top of the YAML pipeline runner introduced there. The PR base reflects that.
• Requires Add length column to tokenize_and_chunk output #252 (Add length column to tokenize_and_chunk output) to run successfully. Without it the build step crashes at ValueError: Column 'length' doesn't exist. because tokenize_and_chunk (triggered by chunk_length: 1024) doesn't add the column expected by the build/score workers. Please merge Add length column to tokenize_and_chunk output #252 before this.
• Requires Require torch>=2.4 for FSDP2 fully_shard #251 (torch>=2.4) — the CLI fails to import on torch < 2.4 due to FSDP2 fully_shard. Not blocking for review but blocks running the example on stale environments.