Technical reference¶
Implementation language¶
Python >=3.12.
Interfaces¶
- CLI help, experiment selection, component listing and cache commands through
interop-traceorpython -m interop_trace. - Optional local NiceGUI interface through
interop-trace ui, using the same experiment catalogue and CLI runner. - Package version through
interop_trace.__version__. - Component protocols and records, exported by the four domain subpackages.
Python API¶
Command-line entry point for Interop Trace.
clear_downloads(directory=DEFAULT_CACHE_DIR, yes=False)
¶
Remove cached downloads and their registry. Run between workflow runs.
launch_interface(directory=Path('.'), port=8080)
¶
Open a local experiment dashboard (requires the ui extra).
list_components()
¶
List available implementations without downloads or optional scoring imports.
list_downloads(directory=DEFAULT_CACHE_DIR)
¶
Show names, component types, sources, sizes, releases and download times.
list_experiments(directory=Path('experiments'))
¶
List experiment IDs and their source files without setup or downloads.
main()
¶
Run the command-line application.
run(experiment_ids=None, all_experiments=False, directory=Path('experiments'), output=Path('results/run-001'), cache_directory=DEFAULT_CACHE_DIR, check=False)
¶
Select experiment IDs across YAML files and execute them sequentially.
Setup and registration¶
Shared setup contract and explicit, decorator-populated component registries.
ComponentEntry
dataclass
¶
An implementation's constructor and public catalogue information.
paths identifies constructor parameters resolved relative to their
component definition file. Other parameters retain their native values.
Registry
¶
Bases: dict[str, ComponentEntry[T]]
One layer's explicitly imported implementations, keyed by stable name.
RequiresSetup
¶
Bases: Protocol
A component whose resources must be resolved before execution.
Setup may download through the supplied cache and must be safe to repeat. Construction and registration must not perform setup or network requests.
setup(*, cache)
¶
Resolve resources or raise, without producing experiment reports.
Experiments¶
Load separate component definitions and construct selected experiments.
Experiment
dataclass
¶
One named combination of configured components, not yet set up.
RunConfiguration
dataclass
¶
Constructed experiments and a snapshot of all referenced YAML contents.
load_configuration(path)
¶
Validate references and construct every experiment before any setup.
Each layer references one component file. Paths resolve relative to the file containing them. Constructors accept implementation-specific parameters. The loader never downloads resources or starts workflow processes.
Discover root-level experiment files and select a run by experiment ID.
load_experiments(directory, ids=None)
¶
Validate the catalogue without setup; select IDs in the requested order.
Only immediate YAML files are discovered. None selects every experiment;
an explicit selection must be nonempty. Source and component YAML contents
are retained for the selected experiments, with globally unique IDs.
Execute explicit experiments sequentially through the component contracts.
RunResult
dataclass
¶
Successful evaluations and errors, keyed by experiment ID.
run_experiments(configuration, *, output_dir, cache_dir=DEFAULT_CACHE_DIR)
¶
Run each experiment in a new directory, continuing after ordinary errors.
Preserve the loaded YAML contents and selected settings before setup. A failed experiment retains its reports, logs and failed-stage record. Its remaining stages are skipped; the next experiment starts. Interruptions still propagate. Downloads share a cache, but workflow results are not reused.
Local run browser¶
Launch the existing CLI and inspect its files without a UI dependency.
RunBrowser
¶
SavedRun
dataclass
¶
One output directory, its recorded experiments and execution status.
file_preview(path, limit=32768)
¶
Read a bounded text preview without rendering binary outputs.
log_tail(path, limit=32768)
¶
Read a bounded UTF-8 preview even while a process appends to the file.
read_record(path)
¶
Tolerate missing or partially written records; the next poll retries.
Datasets¶
Contracts for obtaining experimental measurements and sample information.
Dataset
dataclass
¶
Prepared measurements and optional sample metadata, at absolute paths.
files can contain raw measurements or converted files such as mzML.
sample_metadata describes the samples and their relationship to files;
its format must be understood by the consuming workflow or benchmark.
This record does not read or validate the files.
DatasetProvider
¶
Bases: Protocol
A dataset source configured with its implementation-specific parameters.
prepare(*, output_dir)
¶
Return local inputs, using output_dir for any generated files.
Implementations may retrieve remote data or reuse existing local data. They must return existing, absolute paths or raise an exception.
Retrieve explicitly selected files from a public PRIDE project.
PrideProvider
dataclass
¶
A public project accession and exact filenames to use as measurements.
setup resolves files in the shared cache with their native names. prepare
writes a provenance record; it does not convert files or infer sample metadata.
prepare(*, output_dir)
¶
Return resolved measurements and write provenance in a new directory.
setup(*, cache)
¶
Resolve the catalogue and all selections, then fetch or reuse files.
Resolve every filename and URL before measurement downloads. Cache hits preserve original source records. Failed setup leaves the component unready.
Databases¶
Contracts for preparing protein sequence databases.
Database
dataclass
¶
A prepared protein database at an absolute path, with an explicit format.
format identifies the representation, such as fasta or uniprot_xml.
Consumers decide how to handle this representation.
The sequences are search candidates, not benchmark ground truth.
This record does not parse, convert or validate the contents.
DatabaseProvider
¶
Bases: Protocol
A sequence source configured with its implementation-specific parameters.
prepare(*, output_dir)
¶
Return a local database, using output_dir for any generated files.
Implementations select or retrieve sequences according to their own settings and declare the resulting format. They must return an existing, absolute path or raise an exception.
Retrieve working database files from the UniProtKB REST API.
UniProtProvider
dataclass
¶
A UniProtKB query and native export format, interpreted by UniProt.
setup resolves cached content. prepare writes its original release
record and returns the cached database without duplicating it.
prepare(*, output_dir)
¶
Return the resolved database and record its original source release.
Setup must have completed. output_dir must be new and holds metadata
only; this method does not request data. UniProt interprets query/export
settings during setup. Clear the cache between runs and set up again to
fetch a fresh response for the same query.
setup(*, cache)
¶
Fetch or reuse the query response and its original release metadata.
Workflows¶
Contracts for running analyses and describing their native outputs.
Workflow
¶
Bases: Protocol
An analysis configured with its own software version and parameters.
run(dataset, database, *, output_dir)
¶
Analyse the supplied inputs and write outputs under output_dir.
Input parsing and validation may be delegated to the analysis tool. Implementations must raise on execution failure. Successful execution does not establish scientific accuracy.
WorkflowResult
dataclass
¶
Named native output files, including settings or logs when applicable.
format identifies the output dialect understood by a benchmark reader.
The workflow defines the file names/roles and returns absolute paths in
files. This record does not normalize tables or validate their contents.
Run the Sage command-line tool with its native JSON configuration.
SageWorkflow
dataclass
¶
A pinned Sage release or managed executable, with native search settings.
setup resolves executable or downloads version through the cache.
configuration is a native Sage JSON file, interpreted by Sage itself.
run(dataset, database, *, output_dir)
¶
Run Sage in a new directory and return its files, log and version.
The explicit inputs override paths in the configuration. An empty dataset
is rejected to prevent Sage from using configuration inputs instead.
Execute a saved copy of the settings, keeping the original configuration
directory as the working directory for relative references.
Sage errors propagate as subprocess.CalledProcessError; files already
written remain available for inspection. Existing output directories are
never reused. Database format labels and file extensions are not checked.
setup(*, cache)
¶
Resolve a local executable or fetch and extract the pinned release.
download_sage(version='0.14.7', *, cache_dir=DEFAULT_CACHE_DIR)
¶
Download a pinned Sage release once and return its cached executable.
Select the published build for this operating system and CPU. Pooch reuses extracted files and restores missing ones from the cached archive. The shared cache records the release URL, version and original download time.
Evaluation¶
Contracts for evaluating a workflow result against a defined benchmark.
Benchmark
¶
Bases: Protocol
An evaluator configured with its own expectations and scoring parameters.
evaluate(result, dataset, database, *, output_dir)
¶
Score one run, writing any reports under output_dir.
The supplied dataset and database must be the inputs used for this run. Implementations must reject incompatible result formats or experimental designs and raise on evaluation failure. The database is not ground truth.
EvaluationResult
dataclass
¶
Named numeric metrics and optional report files at absolute paths.
Each benchmark defines its metric names, units and interpretation. Metrics from different benchmarks are not automatically comparable. Reports may contain richer tables or explanations; no universal score is required.
Check an explicit peptide reference against native search-engine output.
ExpectedPeptides
dataclass
¶
Count exact peptide strings among rank-one target PSMs.
Expectations must come from an independent reference, not the search database. Recovery is diagnostic: it does not estimate precision, FDR or quantification accuracy. Exact Sage notation remains the default. Sequence-only matching ignores modifications and also supports Comet and MS Amanda native tables.
evaluate(result, dataset, database, *, output_dir)
¶
Report recovery, applying a confidence threshold only when requested.
Local ProteoBench evaluation of Sage LFQ results for the Q Exactive design.
ProteoBenchDDA
dataclass
¶
ProteoBench's six-run Q Exactive DDA ion benchmark, using Sage outputs.
min_observations is the minimum number of the six runs in which an ion
must be quantified. The reference mixture and run names come from ProteoBench.
The search database is recorded as provenance, rather than used as ground truth.
evaluate(result, dataset, database, *, output_dir)
¶
Score locally and preserve the metrics, per-ion table and settings.
Require Sage's lfq.tsv and results.json and the six reference run
names. Reports use a new directory. Undefined metrics are listed in the
JSON report and omitted from the numeric result. No data are submitted.
Download cache¶
A local registry and reusable storage for completed downloads.
Download
dataclass
¶
A registered file and the source metadata recorded when it was fetched.
available
property
¶
Whether the local file still exists with its recorded size.
DownloadCache
¶
Store downloads by exact request URL in a dedicated, owned directory.
clear()
¶
Delete stored content and its registry, preserving the parent folder.
Only this cache's managed objects and pending directories are
removed. Call between workflow runs, when cached files are no longer in use.
entries()
¶
List registered downloads, including files that are now missing.
fetch(url, *, name=None, kind=None, filename='content', metadata=None, metadata_headers=None)
¶
Reuse a completed file or download and register it after success.
Pooch handles HTTP transfers and temporary files. metadata supplies
known source information; metadata_headers maps record fields to
response headers captured during that same transfer. Missing headers are
recorded as None. Cache hits preserve the original metadata. name
and kind label the component; older entries gain missing labels on reuse.
filename preserves native extensions when first storing a file.