bCLEARer PDK

Pipeline orchestration

The orchestration services package handles execution, control, and management of bCLEARer data pipelines — from scaffolding a new pipeline to running, logging, and versioning production workloads.

Services

What the orchestration layer provides.

Application runner

Manages the execution lifecycle of bCLEARer applications and processes — startup, shutdown, and error handling for pipeline runs.

Pipeline builder CLI

Generates complete pipeline skeletons from JSON configuration or interactive prompts: domains, pipelines, thin slices, stages, sub-stages, orchestrators, and b-units.

BIE infrastructure

The BORO Identity Ecosystem (BIE) provides deterministic identity creation, registries, universes, and relation registers. The BIE ID creation facade hashes consumable objects into stable identifiers, while registries and identity spaces track types, instances, and relationships across pipeline stages.

Logging and reporting

Standardised log environments and reporting services capture the full execution history of every pipeline run for audit and debugging.

Data loading

The bclearer load service handles the ingestion and transformation of data into the framework, bridging raw sources to pipeline-ready formats.

Version control

Manages versioning of data, processes, and code within the orchestration layer so pipeline evolution is tracked and reproducible.

Static analysis

Validates pipeline code against best practices and structural rules before execution, catching issues early in the development cycle.

Utilities

Date/time handling, string and Unicode manipulation, unit-of-measure management, and file-system snapshots round out the services available to every pipeline stage.