feat: consolidate probing; rename @lde/sparql-monitor → @lde/distribution-monitor#358
Merged
feat: consolidate probing; rename @lde/sparql-monitor → @lde/distribution-monitor#358
Conversation
…tion-monitor
Extend @lde/distribution-probe:
- probe() now accepts a ProbeOptions object (timeoutMs, headers, sparqlQuery)
with a deprecated back-compat shim for the old number-as-second-arg form
- SparqlProbeResult, DataDumpProbeResult and NetworkError all expose
responseTimeMs measured from just before fetch to just after the response
is fully consumed (or the error thrown)
- URL-embedded Basic auth (user:pass@host) is extracted into an
Authorization header for both SPARQL and data-dump probes; caller-supplied
Authorization takes precedence
- SPARQL probing dispatches on query type (ASK/SELECT/CONSTRUCT/DESCRIBE)
with a query-type-aware Accept header and body validation. Query type
detection uses an inline regex with comment stripping; using
fetch-sparql-endpoint would obscure the underlying Response needed for
SparqlProbeResult metadata
Rename @lde/sparql-monitor → @lde/distribution-monitor:
- New config shape targets any DCAT Distribution: { accessUrl, mediaType?,
conformsTo? } + optional sparqlQuery, replacing the SPARQL-only
endpointUrl + query shape
- SparqlMonitor class removed; MonitorService now calls probe() directly
and maps ProbeResultType onto the existing CheckResult shape
- PostgreSQL schema unchanged – CheckResult has the same minimal shape
- Bin name renamed to distribution-monitor with sparql-monitor kept as an
alias for one minor version
- @lde/sparql-monitor is reduced to a deprecated re-export shim (v0.6.0)
that forwards to @lde/distribution-monitor with a runtime warning; after
that final publish the maintainer should run
'npm deprecate @lde/sparql-monitor "Renamed to @lde/distribution-monitor"'
and remove packages/sparql-monitor/ in a follow-up
Update pipeline test fixtures for the new result-class constructor
signatures (responseTimeMs; SparqlProbeResult also takes acceptedContentType).
- Delete packages/sparql-monitor/ entirely; @lde/sparql-monitor stops
receiving updates on npm rather than shipping a final re-export shim.
Consumers migrate to @lde/distribution-monitor directly.
- Drop the sparql-monitor bin alias from @lde/distribution-monitor; the CLI
is invoked as distribution-monitor only.
- probe() no longer accepts a bare number as the second argument; only the
ProbeOptions object form is supported. Migrate pipeline's call site in
SparqlDistributionResolver.resolve() to pass { timeoutMs }.
Main released @lde/distribution-probe 0.1.1 while this branch was open. Point @lde/distribution-monitor at the released version and regenerate the lockfile so 'npm ci' can resolve both the workspace entry and the transitive n3@2.0.3 install.
CI failed because the old sparql-monitor thresholds don't match the rewritten service.ts + new config.ts + types.ts. Drop to the numbers vitest actually reports on CI (lines 94.04, functions 96.29, branches 77.27, statements 92.47) and fix the cacheDir rename that got missed during the package rename.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes
@lde/distribution-probethe single source of truth for probing (including SPARQL endpoints), and renames@lde/sparql-monitorto@lde/distribution-monitorto reflect its broader scope – it now monitors any DCAT distribution, SPARQL endpoint or data dump.This is a breaking-change PR. Pre-release status per
CLAUDE.md, so no shims.Part A – Extend
@lde/distribution-probeprobe()now takesProbeOptions(timeoutMs,headers,sparqlQuery). Breaking: the oldprobe(distribution, 5000)number form is gone; pass{ timeoutMs: 5000 }.@lde/pipeline’s one call site is migrated in this PR.SparqlProbeResult,DataDumpProbeResult, andNetworkErrorall carry a newresponseTimeMs: numberfield, measured from just before the fetch to just after the response is fully consumed (or the error is thrown). Breaking: the result-class constructors take this as a required parameter;SparqlProbeResultadditionally takesacceptedContentType. Pipeline test fixtures that construct these directly are updated.user:pass@host) is extracted into anAuthorization: Basic …header. Applied to both SPARQL and data-dump probes; caller-suppliedAuthorizationwins on conflict.booleanfor ASK, JSONresultsfor SELECT, non-empty RDF body for CONSTRUCT / DESCRIBE). Query-type detection uses an inline regex with#-comment stripping.fetch-sparql-endpoint: the task suggested delegating toSparqlEndpointFetcher.fetchAsk/fetchBindings/fetchTriplesfor query dispatch. I kept rawfetchinstead soSparqlProbeResultretains the underlyingResponse(statusCode, statusText, contentType, lastModified).SparqlEndpointFetcherthrows on HTTP errors and hides the response, which is awkward when the probe’s job is to describe failures. The inline regex is the task’s explicitly-allowed fallback.Part B – Rename
@lde/sparql-monitor→@lde/distribution-monitor{ identifier, distribution: { accessUrl, mediaType?, conformsTo? }, sparqlQuery? }. Any DCAT distribution is now supported;sparqlQueryonly applies to SPARQL endpoints.SparqlMonitorclass removed;MonitorServicenow callsprobe()directly and maps the returnedProbeResultTypeonto the existingCheckResultshape (failing probes becomesuccess: falsewithfailureReason/joined warnings/HTTP-status as the error message).CheckResult’s shape is already minimal enough to cover both cases. Adding aprobe_typecolumn is noted as a possible follow-up.distribution-monitor. Breaking: nosparql-monitoralias.MonitorServiceOptionsgainstimeoutMs,headers, and an injectableprobe(for tests).@lde/sparql-monitordeprecationpackages/sparql-monitor/is deleted from the workspace. No final re-export shim is published.@lde/sparql-monitoron npm stops receiving updates; consumers must migrate to@lde/distribution-monitordirectly.Post-merge action for the maintainer (outside this PR): run
npm deprecate @lde/sparql-monitor "Renamed to @lde/distribution-monitor"sonpm installof the old name surfaces a warning, and enable npm Trusted Publisher on@lde/distribution-monitorafter its first publish.Out of scope
probeResultsToQuadsoutput in@lde/pipeline/src/distribution/report.ts.