Use Cases / CreativeAnswer

Use Case: CreativeAnswer Knowledge Base Graph Builder

Automated pipeline that reads CA's historical project archive (Z1.4 Azure SQL), classifies and extracts structured entities via AI, builds a versioned knowledge graph in deltaPrism (CA tenant), and enables Relica to serve pre-processed context for offer writing. Runs in full-rebuild and incremental modes.

Version UC4-CA-v2.0 Date 2026-05-03 Owner Uued Viljapuuaiad Status Specified -- Implementation Pending Authority Tier 4

Section 01 -- Overview [M]

FieldValue
Use Case IDUC4
Client Reference [M]CreativeAnswer (CA) -- internal UV software agency knowledge base
System Name [M]CA Knowledge Base Graph Builder
Domain [M]Knowledge Management / AI-Driven Entity Extraction / Graph Database Population
One-Line Description [M]AI-powered pipeline that transforms unstructured email attachment text into a typed, encrypted, versioned knowledge graph in deltaPrism for Relica consumption
Status [M]Specified -- implementation pending
Target Delivery [M]Q3 2026
ResolvesRAID DP-007 (pipeline specification), RAID DP-008 (provider abstraction)

Summary [M]

The Source: CA's historical project knowledge -- contracts, client briefs, deliverables, project scopes, pricing structures -- is locked inside an Azure SQL email attachment archive (dbo.AttachmentIndex, "Z1.4"). Each row contains extracted plain text from an email attachment. Unstructured: no schema beyond extracted text and a status flag.

The Target: A typed, structured knowledge graph in the deltaPrism CA tenant containing five object classes (Contract, ClientBrief, Deliverable, ProjectScope, Client) with six relation types, PebbledGlass bins for encrypted access, and versioning. This is the pre-processed, persistent knowledge base that the offer-writing-webdev Relica profile traverses to generate context for offer writing.

The Gap: ca_graph_ingest.py is a prototype script that demonstrates the data flow. It is not a Nexus pipeline node, has no pipeline YAML, no caching contract, and no incremental update logic. This use case specifies the GraphBuildExecutor (new Nexus node type) and two pipeline definitions that replace it entirely.


Section 02 -- Actors [M]

Role [M] Description [M] Access Level [D] Frequency [D]
System (Pipeline Nodes) Automated pipeline execution: trigger, ingest, classify, extract, graph build, cache refresh, egress. No human interaction during runs. Pipeline-internal only Scheduled (nightly incremental, weekly full rebuild)
Admin (Pipeline Operator) Triggers manual full rebuild runs. Monitors pipeline run logs. Configures batch size, provider, thresholds. Pipeline trigger, run log access, config edit Weekly / on-demand
Relica (Consumer) Read-only consumer of the deltaPrism CA tenant graph. Never writes. Never calls Azure SQL directly. Read-only graph traversal (offer-writing-webdev profile) Per offer generation request (real-time)

Section 03 -- User Stories [M/D]

Actor: System (Pipeline)

Actor: Admin

Actor: Relica (Consumer)


Section 04 -- System Flows [M/D]

Flow A: UC4-1a -- Full Rebuild Pipeline (7 Steps) [M]

Trigger [M]: Manual trigger or scheduled (weekly Sunday 02:00). Passes run context: run_id, run_mode=full, started_at.

# Step [M] Actor [M] Input [M] Output [D] Decision [M]
1 Scheduler Trigger System / Admin Manual button or cron schedule run_id, run_mode=full, started_at --
2 SourceIngest -- Azure SQL (Full Scan) System CA_SQL_CONN_STR, query: WHERE TextExtractionStatus='done' AND LEN(ExtractedText) > 100 Batch of {source_id, raw_text} events (batch size: CA_BATCH_SIZE, default 20) --
3 AiTransform -- Classify System First 3000 chars of raw_text {object_class, confidence}. Classes: Contract/ClientBrief/Deliverable/ProjectScope/Client/unknown unknown: skip to Egress (ingestion_status=skipped_unknown)
4 AiTransform -- Extract (V2 Dynamic Schema) System First 6000 chars of raw_text + object_class. Class schema injected via prompt_template_fields. {properties: {...}} JSON per class schema Empty JSON: write with gl_status=error, log error
5 GraphBuild -- Encrypt + Upsert + Relate System Extracted properties + source_id + content_hash object_id, properties_written, relations_written, skipped (bool) Existing + unchanged hash: skipped=true. Changed hash: overwrite + increment gl_version.
6 CacheRefresh -- deltaPrism Graph Stats System GET /stats on CA tenant Object counts by class. Run summary: total, inserted, updated, skipped, errored. --
7 Egress -- Mark Source Row System source_id + result status Updates dbo.AttachmentIndex: ingestion_status, ingested_at, graph_object_id --

Result [M]: All qualifying attachment rows classified, extracted, encrypted, and written to deltaPrism CA tenant graph with relations. Source rows watermarked. Relica can read the graph.

Flow B: UC4-1b -- Incremental Update Pipeline [M]

Trigger [M]: Scheduled nightly (or hourly). Identical 7-step node sequence to UC4-1a.

Difference: SourceIngest query predicate reads only:

Zero-row exit: If no matching rows, pipeline exits with status success and total_processed=0. CacheRefresh still runs to confirm graph health. No error raised.

GraphBuildExecutor Idempotency Contract [D]

Every object written carries mandatory system properties:


Section 05 -- Data Model [M/D]

Object Classes (5) [M]

Class [M] Description [M] Key Properties [M] PebbledGlass Bins [D]
Contract Legal/commercial agreement between CA and a client client_name, contract_value_pln, contract_type, start_date, end_date, scope_summary gl_doc_type=contract, gl_client_name, gl_status
ClientBrief Client-authored requirements or project brief project_name, client_name, key_requirements, budget_range_pln, technology_preferences gl_doc_type=brief, gl_client_name, gl_status
Deliverable Specific named work output with acceptance criteria name, description, acceptance_criteria, delivery_date, parent_contract_ref gl_doc_type=deliverable, gl_client_name, gl_status
ProjectScope Vendor-authored technical/functional specification project_name, scope_description, technology_stack, exclusions, assumptions gl_doc_type=scope, gl_client_name, gl_status
Client Legal entity. Upserted from every extracted client_name. company_name, registry_code, vat_number, country, contact_email, industry gl_doc_type=client, gl_client_name, gl_status

Relations (6) [M]

Relation [M]FromToSemantic [D]
ContractPartyContractClientThis contract names this client as a party
ScopeAttachmentContractProjectScopeThis contract covers this scope of work
BriefSourceClientBriefClientThis brief was submitted by this client
BriefOutcomeClientBriefContractThis brief resulted in this contract
DeliverableScopeDeliverableProjectScopeThis deliverable is part of this scope
DeliverableContractDeliverableContractThis deliverable is specified in this contract

System Properties (per object) [D]

PropertyTypePurpose
source_idString (from AttachmentID PK)Idempotency key for upserts
content_hashCHAR(64) -- SHA-256Change detection between runs
gl_versionInteger (starts 1)Incremented on content change. Plaintext bin.
gl_ingested_atISO 8601 DateTimeLast successful write timestamp. Plaintext bin.
gl_statusEnum: ingested/updated/skipped/errorProcessing result. Plaintext bin.
gl_doc_typeStringObject class as bin. Plaintext.
gl_client_nameStringClient name bin for traversal. Plaintext.

Section 06 -- Screens [D]

Screen [D] Actor [M] Purpose [D] Key Elements [D]
Pipeline Canvas Admin Monitor UC4-1a and UC4-1b pipeline execution 7-node graph with green/amber/red status per node. Run summary panel: total, inserted, updated, skipped, errored. Last run timestamp.
Run Log View Admin Historical pipeline run records Table: run_id, run_mode (full/incremental), started_at, completed_at, total_processed, last_successful_run_at. Filter by date.
Graph Stats Dashboard Admin / Relica deltaPrism CA tenant health check Object counts by class (5). Relation counts by type (6). Last rebuild/incremental timestamps. Staleness warnings.

Note: This is a backend data pipeline with no end-user forms. All screens are operational monitoring views within the Nexus admin interface.


Section 07 -- Integrations [M]

System [M] Direction [M] Data [M] Format [D] Frequency [M] Fallback [D]
Azure SQL (Z1.4 Archive) In (read) + Out (watermark write-back) Read: AttachmentID, ExtractedText. Write: ingestion_status, ingested_at, graph_object_id. pyodbc / ODBC Driver 18 (CA_SQL_CONN_STR) Per pipeline run (batch of 20 rows) Connection failure: pipeline fails with explicit error. No cached fallback.
AI Provider (Classification + Extraction) Out (API call) raw_text excerpt (3000-6000 chars) + class-specific prompt REST API. Providers: anthropic (default), openai, azure_openai, ollama, http (sovereign) Per source row (2 calls: classify + extract) API failure: row marked gl_status=error. Pipeline continues with next row.
deltaPrism (CA Tenant) Out (write objects + relations) + In (stats query) Object upserts (5 classes), relation creation (6 types), GET /stats for cache validation deltaPrism BL15 transaction API Per classified row + CacheRefresh per run Write failure: pipeline halts for that row. No silent data loss.
Relica (offer-writing-webdev profile) Downstream consumer (read-only) Reads graph for offer context generation. Never writes. deltaPrism graph traversal API Per offer generation request Empty graph: raises GraphCacheEmpty. Stale graph: logs WARN, still reads.

Section 08 -- Infrastructure [D]

Layer Choice [D] Reason [D]
Pipeline Engine Nexus Core (Python rule engine + pipeline orchestrator) Scheduler, batching, node composition, run records, observable execution
New Executor GraphBuildExecutor (wormwood/core/node_executors/graph_build_node.py) Combines encrypt + upsert + relate in one atomic graph write per row. Registered as "GraphBuild" node type.
AI Transform AiTransformExecutor with prompt_template_fields + provider config V2 dynamic schema injection. Provider-agnostic (DP-008 compliance).
Entity Store deltaPrism (CA tenant, graph database) Typed objects, relations, PebbledGlass bins, versioning, encrypted properties
Source DB Azure SQL (CA existing infrastructure) No migration needed beyond column additions (Section 06 original doc)
Encryption AES-256-GCM per field, key: CA_RELICA_KEY Sensitive fields encrypted at rest. PebbledGlass gl_ bins always plaintext for traversal.

Required Source Schema Changes [D]

ColumnTypePurposeRequired For
ingestion_statusNVARCHAR(30) NULLProcessing result: ingested/updated/skipped_unknown/error. NULL = not processed.UC4-1a + UC4-1b
ingested_atDATETIME2 NULLTimestamp of last successful Egress writeUC4-1a + UC4-1b
graph_object_idNVARCHAR(64) NULLdeltaPrism object ID for direct graph lookup from sourceUC4-1a + UC4-1b
updated_atDATETIME2 NULLRow modification timestamp (for incremental watermark)UC4-1b only
content_hashCHAR(64) NULLOptional: SHA-256 for fast change detectionOptimisation

Section 09 -- Critical Path [D]

# Item [D] Ref Depends On Status [D]
1 Add ingestion columns to dbo.AttachmentIndex Schema Changes CA Azure SQL access Pending
2 Implement GraphBuildExecutor class REQ-GB-01 deltaPrism API available Pending
3 Extend AiTransformExecutor with prompt_template_fields REQ-GB-02 -- Pending
4 Extend AiTransformExecutor with provider config key REQ-GB-03 / DP-008 -- Active RAID
5 Add run watermark persistence to pipeline run records REQ-GB-04 #2 Pending
6 Write UC4-1a pipeline YAML definition Flow A #2, #3, #4 Pending
7 Write UC4-1b pipeline YAML definition Flow B #5, #6 Pending
8 Run UC4-1a full rebuild against CA deltaPrism dev tenant DP-007 #1, #6 Pending
9 Verify Relica offer-writing-webdev reads populated graph Relica M-06 #8 Pending
10 Deprecate ca_graph_ingest.py Artefacts #9 Pending

Section 10 -- What Is Needed [M]

# Item [M] From [M] Blocking [M] Status [M]
1 deltaPrism CA tenant operational with BL15 transaction support Nexus Platform team Yes -- blocks all graph writes In Development
2 AiTransformExecutor extended (prompt_template_fields + provider) Wormwood engine team Yes -- blocks AI classification and extraction Pending (REQ-GB-02, REQ-GB-03)
3 Azure SQL schema migration (4 columns added to dbo.AttachmentIndex) CA DevOps / DB admin Yes for UC4-1b. UC4-1a can run without (Egress no-op mode). Pending
4 CA_RELICA_KEY provisioned for AES-256-GCM encryption UV Security Yes -- blocks encrypted writes Pending
5 AI provider API key (Anthropic or configured alternative) UV -- already provisioned for ENDO pipeline No -- key exists (shared with Przemek/ARC) Available

Section 11 -- Acceptance Criteria [M/D]

Definition of Done [D]

Criterion [D]Verified By [D]
UC4-1a full rebuild processes all qualifying rows and exits with status successRun against dev CA tenant; run log shows total > 0, errored = 0
deltaPrism CA tenant contains objects in all 5 classes with correct PebbledGlass binsGET /stats returns non-zero counts for Contract, ClientBrief, Deliverable, ProjectScope, Client
Relations created between objects per schema (6 types)Query deltaPrism for relation counts; verify ContractParty, ScopeAttachment, etc. exist
Idempotency: re-running UC4-1a produces skipped=true for unchanged rows (no duplicates)Run twice; second run shows total_processed = N, skipped = N, inserted = 0
Incremental: UC4-1b processes only new/changed rowsAdd new row to source; run UC4-1b; verify only 1 row processed
Encryption: sensitive fields encrypted with CA_RELICA_KEY; gl_ bins remain plaintextDirect deltaPrism query: gl_doc_type readable, contract_value_pln encrypted
Relica offer-writing-webdev profile reads graph and generates contextIntegration test: Relica returns non-empty context from CA tenant graph
Zero-row exit: incremental run with nothing new exits success with total_processed=0Run UC4-1b when no new rows; verify clean exit and CacheRefresh still executes

Caching Contract [M]

ConditionRelica Behavior
Cache Valid: At least one UC4-1a completed with exit 0, inserted > 0, GET /stats returns objects in >= 3 classesRelica reads normally
Cache Stale: Last incremental > 48h old OR last full rebuild > 30d oldRelica reads + logs WARN: graph_cache_stale with timestamp
Cache Empty: Zero objects in CA tenant graphRelica raises GraphCacheEmpty -- hard error, no fallback data

Section 12 -- Risks [D]

# Risk [D] Likelihood [D] Impact [D] Mitigation [D]
R1 AI classification accuracy: some documents may be misclassified Medium Medium -- wrong class properties extracted Confidence threshold; low-confidence rows logged for manual review. Versioning allows correction on re-run.
R2 deltaPrism CA tenant not ready (BL15 transaction support) Medium High -- blocks all graph writes GraphBuildExecutor can be unit-tested against mock deltaPrism interface
R3 Azure SQL schema migration blocked by CA production freeze Low Medium -- blocks incremental mode (UC4-1b) UC4-1a can run without new columns (Egress no-op with EGRESS_WRITE_BACK=0)
R4 AI provider cost escalation on large archive (full rebuild) Low Low -- one-time cost Batch size limits (CA_BATCH_SIZE). Haiku model for classification (cheap). Skip unchanged rows via idempotency.

Open Questions [D]

IDQuestionImpact
OQ-01Pricing structures and offer templates: separate source? New object classes (OfferTemplate, PricingStructure) needed?May require second SourceIngest variant + 2 additional classes in CA tenant schema
OQ-02Issues tracking: project issues from Jira/Linear or from email threads in Z1.4?Determines whether new object class + source integration needed
OQ-03Z1.4 updated_at column: does it exist and is it populated on row changes?Required for UC4-1b incremental watermarking. May need CA app code change.
OQ-04GraphBuildExecutor boundary: combined classify+extract+write vs split with AiTransform standalone nodesArchitecture decision. Current spec: AiTransform classify + AiTransform extract are separate nodes; GraphBuild handles encrypt+upsert+relate only.

Section 13 -- Cooperator Brief [D]

FieldValue
Scope [D] Two pipeline variants (full rebuild UC4-1a + incremental UC4-1b) with 7 nodes each: Trigger, SourceIngest (Azure SQL), AiTransform (Classify), AiTransform (Extract), GraphBuild (Encrypt+Upsert+Relate), CacheRefresh (Stats), Egress (Watermark). New GraphBuildExecutor node type implementation required.
Stack [D] Nexus pipeline engine (Python), GraphBuildExecutor (new), AiTransformExecutor (extended), deltaPrism CA tenant (graph DB), Azure SQL (source via pyodbc), AES-256-GCM encryption, AI provider (Anthropic/configurable).
Timeline [D] Specification: Complete. Implementation: ~3 weeks after deltaPrism CA tenant + AiTransform extensions are ready.
Dependencies [D] deltaPrism BL15 transaction support (blocker). AiTransformExecutor prompt_template_fields + provider (blocker). Azure SQL schema migration (blocker for UC4-1b). CA_RELICA_KEY (blocker for encryption).
Interfaces [D] Reads: Azure SQL dbo.AttachmentIndex (text). Writes: deltaPrism CA tenant (5 object classes, 6 relation types). Consumed by: Relica offer-writing-webdev profile (read-only).
Quality Bar [D] Idempotent (re-run safe). Versioned (gl_version tracks changes). Encrypted (AES-256-GCM). Zero-fallback (errors are explicit). Incremental (only new/changed rows processed). Cache contract enforced.
Handover [D] Pipeline YAML definitions (2) + GraphBuildExecutor implementation + AiTransformExecutor extensions + CA tenant schema doc + encryption key provisioning doc + Azure SQL migration script + integration test suite.

Implementation Requirements (Executor Spec)

IDRequirementStatus
REQ-GB-01GraphBuildExecutor(BaseNodeExecutor) in graph_build_node.py, registered as "GraphBuild"Pending
REQ-GB-02AiTransformExecutor: prompt_template_fields config key (list of input field names for str.format_map() injection into system_prompt)Pending
REQ-GB-03AiTransformExecutor: provider config key (anthropic/openai/azure_openai/ollama/http). Resolves DP-008.Active RAID
REQ-GB-04Run record persists last_successful_run_at + last_run_stats (JSON). SourceIngest reads watermark from most recent UC4-1a or UC4-1b run.Pending
REQ-GB-05Zero-row exit: pipeline exits success with total_processed=0 when no matching rows. No error raised.Pending
REQ-GB-06No fallback data: unknown classification routes to Egress (skipped_unknown). Empty extraction writes system props only (gl_status=error). No invented values.Pending
REQ-GB-07Relation creation via deltaPrism POST /relations after primary object write. Unresolvable relations = warning (not error). No rollback.Pending
REQ-GB-08Client upsert: SHA-256 of normalised company name as key. Merge properties (not overwrite). Increment gl_version only if changed.Pending

Related Artefacts [D]

ArtefactLocationRelationship
CA Tenant SchemaRelica/docs/RELICA_CA_TENANT_SCHEMACanonical object class and property schema. AI prompt content source.
RAID DP-007Relica/docs/RAID_LOGResolved by this use case spec. Status: ACTIVE -> RESOLVED when UC4-1a completes.
RAID DP-008Relica/docs/RAID_LOGResolved by REQ-GB-03. Status: ACTIVE -> RESOLVED when provider config implemented.
Relica HLDRelica/docs/RELICA_HLDRelica reads from deltaPrism CA tenant (this pipeline's output).
Prototype scriptWormwood/scripts/ca_graph_ingest.pyReference implementation. Deprecated when UC4-1a pipeline succeeds.