Nexus evolves from a rule-validation engine into a full orchestration platform: absorbing Nexus as its canvas visualisation layer, gaining multi-tenant organisation/team/user management, and a visual node-based pipeline editor capable of modelling the Nesto worker onboarding compliance workflow and any future regulated-process deployment. Comparable in scope to N8N or Make.com, but domain-native to compliance and data governance.
Nexus's responsibility ends precisely here:
Pipeline definition (nodes + edges + configuration). Orchestration execution (trigger, route, condition, parallel). Multi-tenant org/team/user auth. Rule engine (existing + extended). Canvas visualisation (absorbs Nexus). Pipeline state SSE streaming. Compliance checklist engine. Node registry and type system.
deltaPrism: Encrypted document storage, immutable audit trail, tamper-evidence, government integration adapter, AES-256 at rest.
Chameleon: Portal rendering, multi-lingual UI, worker-facing self-service views, employer dashboard skin.
Atlas: OCR extraction, cross-document consistency, SuccessFactors gate, CV screening API.
Payroll processing. Immigration legal advice. Document translation services. Physical courier. End-of-service gratuity calculation. SMS/email delivery (Nexus triggers; Nesto's gateway delivers). Infrastructure provisioning (Nesto owns UAE-region cloud).
| Layer | Component | Responsibility | Status |
|---|---|---|---|
| L1 — Tenancy | _tbl_orch_organisations_tbl_orch_teams_tbl_orch_accounts_tbl_orch_users |
Multi-tenant hierarchy. API key scoped to org. RBAC per user role within team. Organisation owns pipelines. User belongs to team within org. | M-ORCH-1 Build |
| L2 — Pipeline | _tbl_orch_pipelines_tbl_orch_node_types_tbl_orch_pipeline_runs |
Pipeline graph stored as JSON (nodes + edges). Node type registry. Run history with per-node timing telemetry. Pipeline versioning. | M-ORCH-2 Build |
| L3 — Canvas | WORMWOOD_APP.htmlSSE /run/telemetry/streamNexus SSE contract |
Visual graph editor connected to Nexus API. Load/save pipeline definitions. Live node status streaming. Absorbs Nexus pipeline-status-stream contract. | M-ORCH-3 Build |
-- Organisation hierarchy Organisation -- top-level tenant (e.g. "Nesto Group") id, slug, name, plan_tier, api_key_hash, created_at Team -- subdivision within an org (e.g. "Nesto HR / Nesto IT") id, org_id, slug, name, created_at Account -- service or admin account (holds API key material) id, team_id, name, type (admin|api|service), created_at User -- human user scoped to team id, account_id, email, display_name, role (owner|admin|editor|viewer), api_key_hash, created_at -- Pipeline objects Pipeline -- named pipeline definition owned by an org id, org_id, slug, name, version, graph_json, domain, status (draft|active|archived), created_at, updated_at NodeType -- registered node type in the global registry id, category (source|transform|rule|condition|output|trigger|wait|call), name, description, schema_json (input/output pin definitions), created_at PipelineRun -- execution record for a pipeline run id, pipeline_id, user_id, status (running|complete|failed), started_at, finished_at, run_json (full RunTracer output), entity_count, duration_ms
| Level | Category | Node Types | Nesto Example |
|---|---|---|---|
| L0 — Data Flow | SOURCE TRANSFORM OUTPUT | trigger.webhook trigger.schedule trigger.eventrouter.condition router.switchparallel.fan_out parallel.fan_incall.http call.db_query call.file_readwait.timer wait.approval wait.external_event |
trigger.event(new_worker_record) → router(source_country) → parallel.fan_out(UAE_track, source_country_track) |
| L1 — Classes | CLASS | entity_class transform.map transform.filtertransform.enrich validate.schema aggregate.group |
entity_class(WorkerProfile) → validate.schema(permit_type_mandatory_fields) → transform.enrich(source_country_rules) |
| L2 — Properties | PROPERTY | property_def lookup.table lookup.referencecompute.formula compute.expression compute.deadline |
compute.deadline(entry_permit_expiry = arrival_date + 60d) → alert.threshold(days_remaining < 7) |
| L3 — Rules | RULE | rule.compliance_check rule.validationrule.scoring rule.alert rule.deadline |
rule.compliance_check(MoHRE_work_permit_valid) → rule.deadline(residence_visa_within_60d) → alert.regulatory(violation) |
Three core pipelines required for the Nesto PoC. These become Nexus pipeline definitions stored in
_tbl_orch_pipelines under the nesto-group organisation.
trigger.event(new_worker_record) → validate.schema(WorkerProfile: source_country, permit_type, passport_type) → router.switch(source_country: India | Philippines | Nepal | Bangladesh | Pakistan | Sri Lanka | Egypt) → [India] call.http(eMigrate_ECR_check) → rule.compliance_check(GAMCA_medical_required) → [Philippines] call.http(DMW_POLO_accreditation) → rule.compliance_check(OWWA_membership) → [Nepal] call.http(FEIMS_permit) → rule.compliance_check(welfare_fund_paid) → parallel.fan_out → rule.compliance_check(UAE_work_bundle: work_permit, entry_permit, medical, emirates_id, visa) → rule.compliance_check(WPS_enrolled, ILOE_subscribed, Tawjeeh_completed, health_insurance_active) → parallel.fan_in → compute.deadline(all_deadlines_calendar) → rule.alert(any_deadline_within_7d) → output(employer_dashboard_update)
trigger.event(document_uploaded) → validate.schema(doc_type, worker_id, expiry_date) → router.switch(doc_type) → rule.validation(mandatory_doc_present_for_worker_profile) → compute.deadline(expiry_date − today) → rule.alert(expiry_within_30d → notify_hr) (expiry_within_7d → notify_ops_lead)
trigger.event(offer_requested) → validate.schema(worker_nationality, permit_type, role, salary) → lookup.table(MoHRE_approved_worker_language[nationality]) → call.http(Atlas.generate_offer: Arabic + English + worker_language) → validate.schema(UAE_Labour_Law_clauses_present) → call.http(deltaPrism.store_offer_with_audit_trail) → output(offer_download_url)
The user referenced "Endo and Nesto projects" as use case sources for the orchestration platform design.
No repository named "Endo" exists in repos/UV/ or nearby.
Nesto is fully understood (UAE worker onboarding, this document). Endo is unknown.
Decision: Copy the Nexus canvas visualisation JS patterns into Nexus's existing
WORMWOOD_APP.html and connect it to the Nexus API via the existing SSE
/run/telemetry/stream endpoint plus new /pipelines CRUD. Nexus continues as a separate
service for FreelancePipeline; Nexus does not depend on it.
Rationale: WORMWOOD_APP.html already implements the 4-level blueprint graph editor. The missing piece is
API persistence (save/load from _tbl_orch_pipelines) and live telemetry binding. Forking Nexus
would duplicate a 7,500-line server unnecessarily.
Decision: Nexus defines pipelines and orchestrates execution by making HTTP calls to external
services (deltaPrism, Atlas, Chameleon, government APIs). Nexus does NOT host execution runtime for
arbitrary user code. Each node in the registry declares an execution handler:
built_in (rule engine, router, condition) or http_call (POST to configured endpoint).
This keeps Nexus stateless and deployment-simple. Complex compute stays in deltaPrism/Atlas.
Decision: Add M-ORCH tables to the existing SQLite (admin.db) schema first via SQLAlchemy models
and migration script, with PostgreSQL DDL in admin/sql/migration_m_orch.sql for production parity.
This follows the existing pattern: all admin tables live in admin.db for dev, schema_postgres.sql for prod.
Decision: The Nexus demo will use a standalone quick-and-dirty forms solution instead of ChameleonV2 integration. ChameleonV2 has been separated into its own project. The demo forms layer will be implemented as minimal HTML forms embedded directly in the Nexus demo UI, with simple schema-to-form rendering without the full ChameleonV2 feature set.
Rationale: The demo needs to be self-contained and quick to iterate. ChameleonV2 integration adds external dependency complexity that slows demo development velocity. A lightweight forms solution can validate the orchestration flow end-to-end without waiting for full ChameleonV2 integration.
Impact on milestones:
Note: This decision applies ONLY to the demo implementation. Production Nexus pipelines (Nesto, ENDO, ARC) will still integrate with ChameleonV2 as the production forms layer per original architecture.
| Milestone | Name | Deliverables | Status | Depends On |
|---|---|---|---|---|
| M-ORCH-1 | Multi-Tenant Foundation | Organisation / Team / Account / User tables + SQLAlchemy models + CRUD endpoints + tests | Complete | None |
| M-ORCH-2 | Pipeline Definition Engine | Pipeline + NodeType + PipelineRun tables + graph JSON API + pipeline versioning + tests | Complete | M-ORCH-1 |
| M-ORCH-3 | Canvas API Integration | WORMWOOD_APP.html connected to GET/POST /pipelines. Save/load graph. Live SSE telemetry overlaid. Node status dots live. |
Queued | M-ORCH-2 |
| M-ORCH-4 | Node Registry & Built-in Types | All 16 built-in node types registered. trigger, router, parallel, call.http, rule.*, compute.*, alert.* nodes seeded. Tests validate schema. | Queued | M-ORCH-2 |
| M-ORCH-5 | Pipeline Execution Engine | Pipeline runner: walks graph, executes built-in nodes, makes HTTP calls for external nodes, emits SSE events per node, persists PipelineRun. | Queued | M-ORCH-4 |
| M-ORCH-6 | Nesto Pipeline Definitions | 3 Nesto pipelines seeded (Worker Journey, Document Lifecycle, Offer Generation). 9 compliance NodeTypes registered (EventTrigger, RouterSwitch, ComplianceCheck, ParallelFanOut/In, DeadlineCompute, AlertRule, LookupTable, OutputGateway). 43 tests in test_nesto_pipelines.py. | Complete | M-ORCH-5, D-01 resolved |
| M-ORCH-7 | Endo Pipeline Definitions | 3 Endo pipelines seeded (Marcin Lead Qualification, Irena Document Processing, Przemek Voice Brief). Slugs: endo-marcin-lead-qualify-full, endo-irena-doc-full, endo-przemek-voice-brief. 62 tests in tests/test_endo_pipelines.py. |
Complete | M-ORCH-5, D-01 resolved |
| Task | Deliverable | Status |
|---|---|---|
| T-01 | SQLAlchemy models: Organisation, OrgTeam, OrgUser, Pipeline, NodeType, PipelineRun in admin/models.py | Done |
| T-02 | Migration script: admin/migrate_m_orch.py (creates tables + seeds 11 built-in NodeTypes) | Done |
| T-03 | PostgreSQL DDL: admin/sql/migration_m_orch.sql | Done |
| T-04 | FastAPI router: nexus/api/routes_orgs.py — CRUD for orgs/teams/users (API key returned once on create) | Done |
| T-05 | FastAPI router: nexus/api/routes_pipelines.py — CRUD pipelines + node types + run history | Done |
| T-06 | Tests: tests/test_orchestration_tenancy.py — 34 tests, 34 passing | Done |
| Task | Deliverable | Status |
|---|---|---|
| T-06 | SQLAlchemy models: Pipeline, NodeType, PipelineRun | Queued |
| T-07 | FastAPI router: nexus/api/routes_pipelines.py — GET/POST/PUT/DELETE /pipelines | Queued |
| T-08 | Node type seed: 16 built-in types seeded in DB via admin/seed_node_types.py | Queued |
| T-09 | Tests: tests/test_orchestration_pipelines.py | Queued |
| Task | Deliverable | Status |
|---|---|---|
| T-10 | WORMWOOD_APP.html: add Save/Load buttons wired to /pipelines API. Graph data round-trips to DB. | Queued |
| T-11 | WORMWOOD_APP.html: SSE binding to /run/telemetry/stream — node status dots go live on run. | Queued |
| T-12 | Pipeline runner: nexus/core/pipeline_runner.py — graph walk, built-in node execution, HTTP call nodes, SSE emit, PipelineRun record. | Queued |
| T-13 | POST /pipelines/{id}/run endpoint triggering pipeline runner. | Queued |
| T-14 | Tests: runner unit tests with mock HTTP nodes. | Queued |
All three Nesto pipeline definitions seeded into Nexus pipeline store (nesto-worker-journey, nesto-doc-lifecycle, nesto-offer-multilingual).
9 compliance-domain NodeTypes registered: EventTrigger, RouterSwitch, ComplianceCheck, ParallelFanOut, ParallelFanIn, DeadlineCompute, AlertRule, LookupTable, OutputGateway.
Worker Journey graph: EventTrigger → ValidateNode → RouterSwitch (7 source countries) → country compliance checks → ParallelFanOut/FanIn → DeadlineCompute → AlertRule(7d).
Document Lifecycle: event-triggered, RouterSwitch by doc_type, DeadlineCompute(expiry), dual AlertRules (30d HR, 7d ops).
Multilingual Offer: LookupTable(MoHRE nationality→language) → Atlas generate_offer(ar+en+worker_lang) → UAE Labour Law validation → deltaPrism audit trail.
43 tests in tests/test_nesto_pipelines.py — all passing.
| ID | As a… | I want to… | So that… | Acceptance Criteria |
|---|---|---|---|---|
| US-01 | Platform admin | Create an Organisation with a unique slug and API key | A new client (e.g. Nesto) can be onboarded as an isolated tenant | POST /organisations returns 201 with org_id + api_key. Subsequent requests with api_key are scoped to this org only. |
| US-02 | Org admin | Create Teams within my Organisation | HR, IT, and Compliance teams can have separate pipeline access | POST /organisations/{id}/teams returns 201. Team is scoped to that org. Teams from other orgs are not visible. |
| US-03 | Team admin | Invite Users with role (admin|editor|viewer) | HR Manager gets editor access; worker gets viewer access | POST /teams/{id}/users returns 201. User.role enforced: viewer cannot write pipelines. |
| US-04 | Editor | Create a Pipeline definition with a graph (nodes + edges) | I can model the Dubai worker onboarding compliance workflow as a visual pipeline | POST /pipelines accepts graph_json. GET /pipelines/{id} returns same graph. Canvas loads it in WORMWOOD_APP. |
| US-05 | Editor | Add a rule.compliance_check node referencing an existing Nexus rule profile |
Compliance rules I have already modelled in the rule engine are reusable as pipeline nodes | Node of type rule.compliance_check with profile_id set executes the EDR/PFP rule profile inline during pipeline run. |
| US-06 | Admin | Run a pipeline against an input payload | I can trigger the worker journey pipeline for a specific worker record | POST /pipelines/{id}/run with input JSON. Run completes, PipelineRun record shows status=complete, all node timings recorded. |
| US-07 | Ops Lead | Watch the pipeline canvas live during a run | I can see each node light up as it processes, in real time | WORMWOOD_APP.html connects to SSE stream. Node status dots update to running/complete/failed as pipeline executes. |
| US-08 | Compliance Officer | See which compliance rules fired for a given worker onboarding run | I have an immutable audit trail of which checks ran and what they found | GET /pipelines/{id}/runs/{run_id} returns full run JSON with per-node results + rule violations if any. Passed to deltaPrism for formal audit trail. |
| Item | Description | Priority | Blocked By |
|---|---|---|---|
| BL-01 | Webhook trigger: incoming HTTP POST starts a pipeline run | P1 | M-ORCH-5 |
| BL-02 | Schedule trigger: cron-based pipeline runs (deadline sweep daily) | P1 | M-ORCH-5 |
| BL-03 | Pipeline versioning: promote draft to active, archive old versions | P1 | M-ORCH-2 |
| BL-04 | Endo pipeline definitions (pending D-01) | P2 | D-01 |
| BL-05 | Government API integration nodes: MoHRE Work Bundle, eMigrate, FEIMS, DMW/POLO | P2 | M-ORCH-5, external API access |
| BL-06 | Atlas integration node: call.atlas(ocr_extract | consistency_check | cv_screen) | P2 | M-ORCH-5, Atlas M-ATL-1 |
| BL-07 | Org-level audit log: every pipeline run, user action, config change immutably logged | P1 | M-ORCH-2 |
| BL-08 | Pipeline import/export as JSON (portability between Nexus instances) | P2 | M-ORCH-2 |
| BL-09 | Native mobile app consideration for Nesto worker portal (per Commercial Proposal §05.7) | P3 Deferred | Phase 1 stable, month 4+ |
Endo = endo/ — ENDO AI by Pryzmat Media sp. z o.o.
Polish AI automation SaaS for SMBs. Three AI employees: Marcin (Sales & SMS lead qualification, calendar booking), Irena (OCR document processing, email categorisation), Przemek (CEO assistant, voice interface, analytics). Live at endo.pryzmat.media. M-ORCH-7 scoped as three pipeline definitions corresponding to each AI employee workflow.