1. Validation Framework
1.1 Validation Levels
| Level |
Code |
Scope |
Requirement |
| Unit |
VL-1 |
Single function/class |
Isolated tests pass, no external deps |
| Integration |
VL-2 |
Component interaction |
Multi-module tests pass with real data |
| Conformance |
VL-3 |
Contract compliance |
All CT- tests for the layer contract pass |
| E2E |
VL-4 |
Full flow execution |
Pipeline runs end-to-end with production-like data |
| Load |
VL-5 |
Performance under stress |
Meets SLA at specified concurrency |
1.2 Validation Bundle Structure
Each component's validation is packaged as a bundle:
| Field |
Type |
Description |
bundle_id |
string |
Unique identifier: VB-{component}-{version} |
component |
string |
Component name from Component Status Matrix |
version |
string |
Component version being validated |
level |
VL-1..VL-5 |
Highest validation level achieved |
tests_total |
integer |
Number of tests in the bundle |
tests_passed |
integer |
Number of tests that passed |
tests_failed |
integer |
Number of tests that failed |
coverage_pct |
float |
Code coverage percentage |
validated_date |
string |
ISO date of last validation run |
validator |
string |
Who/what executed the validation |
evidence |
string |
Path to test output / CI run ID |
2. Current Validation Status
2.1 Core Engine Bundles
| Bundle ID |
Component |
Version |
Level |
Tests |
Pass |
Fail |
Coverage |
Date |
| VB-formula-parser-0.2.0 |
Formula Parser |
0.2.0 |
VL-3 |
142 |
142 |
0 |
98% |
2026-05-02 |
| VB-rule-engine-0.2.0 |
Rule Engine |
0.2.0 |
VL-3 |
87 |
87 |
0 |
95% |
2026-05-02 |
| VB-rule-loader-0.2.0 |
Rule Loader |
0.2.0 |
VL-2 |
45 |
45 |
0 |
92% |
2026-05-02 |
| VB-bdt-validator-0.2.0 |
BDT Validator |
0.2.0 |
VL-3 |
63 |
63 |
0 |
97% |
2026-05-02 |
| VB-consolidation-0.2.0 |
Consolidation Engine |
0.2.0 |
VL-2 |
28 |
28 |
0 |
88% |
2026-05-02 |
| VB-orchestration-0.1.0 |
Orchestration Engine |
0.1.0 |
VL-2 |
34 |
34 |
0 |
85% |
2026-05-02 |
| VB-run-tracer-0.2.0 |
Run Tracer |
0.2.0 |
VL-2 |
22 |
22 |
0 |
90% |
2026-05-02 |
2.2 Pipeline Bundles
| Bundle ID |
Component |
Version |
Level |
Tests |
Pass |
Fail |
Coverage |
Date |
| VB-edr-pipeline-0.2.0 |
EDR Pipeline |
0.2.0 |
VL-4 |
56 |
56 |
0 |
93% |
2026-05-02 |
| VB-pfp-pipeline-0.2.0 |
PFP Pipeline |
0.2.0 |
VL-4 |
48 |
48 |
0 |
91% |
2026-05-02 |
| VB-freelance-pipeline-0.1.0 |
Freelance Pipeline |
0.1.0 |
VL-2 |
32 |
32 |
0 |
87% |
2026-05-02 |
2.3 Node Executor Bundles
| Bundle ID |
Component |
Version |
Level |
Tests |
Pass |
Fail |
Date |
| VB-nodes-bl06-bl12 |
IngestNode, StoreNode, EgressNode, AiTransform, AtlasOcr, GmailIngest, BankCsvIngest, EmailForward |
0.1.0 |
VL-2 |
89 |
89 |
0 |
2026-05-02 |
| VB-nodes-bl13-bl15 |
FormNode, ValidateNode, HumanGate, ConditionBranch, LeadScoreNode, MatchNode |
0.1.0 |
VL-2 |
76 |
76 |
0 |
2026-05-02 |
| VB-nodes-bl16 |
VoiceIngest, AiTransformVoice |
0.1.0 |
VL-1 |
18 |
18 |
0 |
2026-05-02 |
| VB-nodes-ui |
DashboardNode, NavMenuNode, ListViewNode |
0.1.0 |
VL-2 |
45 |
45 |
0 |
2026-05-02 |
2.4 Runtime Bundles
| Bundle ID |
Component |
Version |
Level |
Tests |
Pass |
Fail |
Date |
| VB-runtime-local-0.2.0 |
Local Runtime |
0.2.0 |
VL-4 |
412 |
412 |
0 |
2026-05-02 |
| VB-runtime-gpu-0.1.0 |
deltaPrism GPU |
0.1.0 |
VL-2 |
34 |
34 |
0 |
2026-05-02 |
| VB-runtime-databricks-0.0.1 |
Databricks Runtime |
0.0.1 |
VL-1 |
0 |
0 |
0 |
-- |
| VB-runtime-snowflake-0.0.1 |
Snowflake Runtime |
0.0.1 |
VL-1 |
0 |
0 |
0 |
-- |
2.5 API and Admin Bundles
| Bundle ID |
Component |
Version |
Level |
Tests |
Pass |
Fail |
Date |
| VB-api-0.2.0 |
FastAPI App |
0.2.0 |
VL-4 |
156 |
156 |
0 |
2026-05-02 |
| VB-admin-0.2.0 |
Admin Web App |
0.2.0 |
VL-3 |
67 |
67 |
0 |
2026-05-02 |
| VB-auth-0.2.0 |
Auth + API Keys |
0.2.0 |
VL-3 |
43 |
43 |
0 |
2026-05-02 |
3. Conformance Test Categories (CT-)
Referenced from Deployment Matrix Section 5. Each target stack must pass applicable CT categories.
| Category |
Code |
Tests |
Applicable To |
Status |
| Rule Determinism |
CT-RULE |
100 seeds x 874 rules |
Controller |
Passing (python-fastapi, docker-onprem) |
| Formula Coverage |
CT-FORM |
11 functions x 5 cases |
Controller |
Passing (python-fastapi, docker-onprem) |
| BDT Validation |
CT-BDT |
21 types x constraints |
Controller |
Passing (python-fastapi, docker-onprem) |
| Lifecycle Enforcement |
CT-LIFE |
5 classes x operations |
Model |
Passing (delta-prism-python, postgresql) |
| RBAC Enforcement |
CT-RBAC |
Role matrix x fields |
Model + View |
Passing (delta-prism-python) |
| Audit Trail |
CT-AUDIT |
Mutation coverage |
Model |
Passing (delta-prism-python) |
| Flow Execution |
CT-FLOW |
DAG patterns x 10 |
Orchestration |
Passing (nexus-runtime-python) |
| Style Tokens |
CT-STYLE |
Theme variables |
View |
Passing (chameleonv2-react) |
| Error Handling |
CT-ERR |
Invalid input suite |
All layers |
Passing (all Python targets) |
| Performance |
CT-PERF |
Batch SLA targets |
Controller |
Passing (1K records < 2s) |
4. Validation Process
4.1 New Component Validation Workflow
- Implement -- Write code, register in Component Status as "Implemented"
- Write tests -- Create test file in
tests/, target VL-2 minimum
- Run suite -- Execute
pytest with coverage measurement
- Record bundle -- Add entry to Section 2 of this registry
- Conformance -- If applicable, run CT-* tests for the target contract
- Certify -- Update Component Status to "Production" if VL-3+ achieved
4.2 Validation on Change
When a component is modified:
- Run existing validation bundle
- If any test fails: component reverts to "Implemented" status
- Fix and re-validate before re-certifying as "Production"
- Update
validated_date in this registry
4.3 Cooperator Validation Requirements
Each work package delivered by a cooperator must include:
| Deliverable |
Required |
| Implementation code |
Yes |
| Unit tests (VL-1) |
Yes |
| Integration tests (VL-2) |
Yes |
| Conformance tests (VL-3) |
If contract-implementing |
| Test evidence (output log) |
Yes |
| Coverage report |
Yes (minimum 85%) |
| Bundle entry update |
Yes (this document) |
5. Aggregate Metrics
| Metric |
Value |
Date |
| Total test count |
2,325 |
2026-05-02 |
| Total validation bundles |
18 |
2026-05-02 |
| Bundles at VL-3+ |
7 |
2026-05-02 |
| Bundles at VL-4+ |
4 |
2026-05-02 |
| Components certified Production |
14 |
2026-05-02 |
| Components awaiting validation |
10 scaffolds |
2026-05-02 |
| Overall pass rate |
100% (on passing bundles) |
2026-05-02 |
6. Version History
| Version |
Date |
Changes |
| VR-1.0 |
2026-05-02 |
Initial registry: 18 validation bundles, 10 CT categories, cooperator requirements defined. |
Referencing This Registry
The NSD deployment section references validation registry version via: "validation_registry_version": "VR-1.0". Component Status Matrix references bundle IDs for certification evidence.