Locus Technical Documentation

This book is the technical reference for teams evaluating, integrating, and operating Locus in real systems.

Who This Is For

• Application developers building products on top of Locus
• Platform engineers integrating memory infrastructure into existing stacks
• Technical teams preparing architecture proposals and implementation plans

What You Can Decide With This Book

• Whether Locus fits your reliability and integration requirements
• Which runtime shape fits your environment (SDK, gateway, MCP)
• How to implement safely without rewriting contracts later

What You Get Here

• Architecture and protocol model details
• Storage and schema specifications
• SDK design and composition patterns
• Mini orchestration cookbooks with prompts and composition flows
• Agent blueprint patterns by domain and task model
• Deployment and operations guidance
• Troubleshooting and versioning policy

API Documentation

Generated Rust API docs are published alongside this book so implementation teams can move from architecture to code-level integration without switching references.

• Local build output: docs/technical/api/index.html
• Typical crate roots:
  • docs/technical/api/locus_core_rs/index.html
  • docs/technical/api/locus_sdk/index.html
  • docs/technical/api/locus_mcp/index.html
  • docs/technical/api/locus_gateway/index.html
  • docs/technical/api/locus_cli/index.html

Build

From repository root:

./docs/build-technical-docs.sh

Then open:

• docs/technical/index.html

Environment Setup

Audience and Use

This page is for developers and platform teams preparing a reliable local or shared environment for Locus evaluation and integration.

Use it to:

1. Confirm supported tooling and runtime combinations.
2. Validate environment readiness before implementation work starts.
3. Standardize local, CI, and staging setup patterns.

Support Matrix

Tooling Prerequisites

Install and verify:

rustc --version
cargo --version
docker --version
mdbook --version
mdbook-mermaid --version

If docs tools are missing:

cargo install mdbook
cargo install mdbook-mermaid

Recommended Environment Profiles

Local Development Profile

Use when iterating on features and integration code.

1. Storage: in-memory or local SurrealDB.
2. Host runtime: single process.
3. Objective: speed and repeatability.

CI Validation Profile

Use for pull requests and release gating.

1. Storage: deterministic in-memory fixtures.
2. Host runtime: test-only jobs.
3. Objective: compatibility and regression detection.

Staging Profile

Use for pre-production verification.

1. Storage: isolated remote SurrealDB.
2. Host runtime: gateway and MCP parity checks.
3. Objective: rollout rehearsal and migration validation.

Baseline Verification

From repository root:

cargo check --workspace --examples
cargo test --workspace
cargo check --examples -p locus-sdk
./docs/build-technical-docs.sh

A setup is considered ready when all commands complete successfully.

Runtime Smoke Start

Image-based MCP:

docker run --rm -i -v "$PWD/locus-data:/data" ghcr.io/entasislabs/locus-mcp:0.1.0

Image-based gateway:

docker run --rm -p 8080:8080 -p 8081:8081 -v "$PWD/locus-data:/data" ghcr.io/entasislabs/locus-gateway:2.0.0

Environment Hygiene Controls

1. Keep secrets outside repository files.
2. Keep development and production data paths separate.
3. Pin model and provider settings in shared environments.
4. Use explicit tenant and session scope in integration tests.

Exit Criteria

Environment setup is complete when:

1. Workspace build and tests pass.
2. At least one host runtime starts successfully.
3. Technical docs site builds successfully.
4. Team members can reproduce setup using this page only.

Architecture

Audience and Use

This page is for engineers deciding how to adopt Locus in production systems.

Use it to:

1. Choose where Locus should sit in your stack.
2. Understand component responsibilities before integration starts.
3. Avoid coupling host transports to internal implementation details.

Purpose

Locus is the Rust implementation layer for STTP memory systems. STTP remains the protocol layer. Locus provides reusable components for parsing, validation, storage, retrieval, and composition with stable contract boundaries.

Layering

1. Protocol layer: STTP grammar and four-layer node contract.
2. Implementation core: locus-core-rs.
3. SDK/application layer: locus-sdk.
4. Host transport layer: MCP, HTTP, gRPC hosts that consume Locus crates.

Crate Responsibilities

locus-core-rs

1. Domain models and contracts.
2. STTP parsing and validation.
3. Storage adapters (in-memory and SurrealDB).
4. Core services for calibration, store/query, rollups, and sync-ready primitives.

locus-sdk

1. Primitive memory workflows: find, recall, aggregate, transform, explain, schema.
2. Composition workflows for multi-step operations.
3. Deterministic manual compression and recursive content construction.
4. Transport DTOs and typed request/response boundaries.

Design Principles

1. Deterministic-first behavior where possible.
2. Explicit policy controls over hidden defaults.
3. Transport-neutral workflows.
4. Additive evolution with compatibility preservation.
5. Observable retrieval paths with explainable outcomes.

Data Contract Boundaries

1. STTP node text is validated against the protocol structure.
2. Core models carry parsed and computed fields.
3. SDK contracts define application-level behavior and policy.
4. Transport DTOs convert cleanly into SDK contracts.

Dependency Direction

1. locus-sdk depends on locus-core-rs.
2. locus-core-rs has no dependency on locus-sdk.
3. Hosts should depend on locus-sdk for workflow orchestration and on locus-core-rs only when low-level access is required.

This dependency model keeps host contracts stable while internal implementations evolve.

Locus Core Database Schema and Data Governance

Document Control

• Document ID: LOCUS-CORE-DATA-ARCH-001
• Status: Current
• Intended Audience: Platform Engineers, Data Engineers, Reliability Engineers, Security Reviewers
• Source of Truth: locus-core-rs implementation (domain/models.rs, storage/surrealdb/*, domain/contracts.rs)
• Last Updated: 2026-05-04

Audience and Use

This document is for teams evaluating data durability, query behavior, and operational safety before adopting Locus in production systems.

Use it to:

1. Validate persistence and indexing behavior against your workload.
2. Understand multi-tenant and compatibility rules.
3. Review storage-level guarantees before integration and rollout.

1. Purpose and Scope

This document specifies the persistence model used by locus-core-rs for STTP memory nodes, calibration records, and synchronization checkpoints.

It covers:

• Physical schema implemented in SurrealDB.
• Logical entity relationships.
• Idempotency and uniqueness rules.
• Multi-tenant and legacy-compatibility behavior.
• Query, indexing, and lifecycle constraints.

It does not cover:

• HTTP/gRPC transport contracts (documented in gateway docs).
• UI-facing DTO contracts (documented in SDK interface docs).

2. Architectural Context

locus-core-rs defines a storage abstraction (NodeStore) with two concrete implementations:

• InMemoryNodeStore for tests and ephemeral execution.
• SurrealDbNodeStore for persistent runtime.

All persisted shape definitions and indexes in this document correspond to SurrealDbNodeStore schema initialization.

3. Canonical Logical Model

erDiagram
    TEMPORAL_NODE {
        string id PK
        string tenant_id
        string session_id
        string tier
        datetime timestamp
        string sync_key
        float psi
        float rho
        float kappa
    }

    CALIBRATION {
        string id PK
        string tenant_id
        string session_id
        float stability
        float friction
        float logic
        float autonomy
        float psi
        datetime created_at
    }

    SYNC_CHECKPOINT {
        string id PK
        string tenant_id
        string session_id
        string connector_id
        datetime cursor_updated_at
        string cursor_sync_key
        datetime updated_at
    }

      TEMPORAL_NODE }o--|| CALIBRATION : "session_scope"
      TEMPORAL_NODE }o--|| SYNC_CHECKPOINT : "sync_scope"

4. Domain Model Alignment (UML)

classDiagram
    class AvecState {
      +float stability
      +float friction
      +float logic
      +float autonomy
      +float psi()
      +float drift_from(previous)
      +DriftClassification classify_drift(previous)
    }

    class SttpNode {
      +string raw
      +string session_id
      +string tier
      +datetime timestamp
      +int compression_depth
      +string parent_node_id
      +string sync_key
      +datetime updated_at
      +object source_metadata
      +string context_summary
      +float[] embedding
      +string embedding_model
      +int embedding_dimensions
      +datetime embedded_at
      +AvecState user_avec
      +AvecState model_avec
      +AvecState compression_avec
      +float rho
      +float kappa
      +float psi
      +string canonical_sync_key()
    }

    class SyncCursor {
      +datetime updated_at
      +string sync_key
    }

    class SyncCheckpoint {
      +string session_id
      +string connector_id
      +SyncCursor cursor
      +datetime updated_at
      +object metadata
    }

    class NodeStore {
      <<interface>>
      +query_nodes_async(query)
      +upsert_node_async(node)
      +get_by_resonance_async(...)
      +get_by_hybrid_async(...)
      +query_changes_since_async(...)
      +get_checkpoint_async(...)
      +put_checkpoint_async(...)
      +batch_rekey_scopes_async(...)
    }

    SttpNode --> AvecState
    SyncCheckpoint --> SyncCursor
    NodeStore ..> SttpNode
    NodeStore ..> SyncCheckpoint

5. Physical Schema (SurrealDB)

5.1 Table: temporal_node

Storage purpose:

• Authoritative persisted STTP node record.
• Supports retrieval, ranking, sync, and transform operations.

Key characteristics:

• SCHEMAFULL table.
• Unique identity by (tenant_id, session_id, sync_key).
• Temporal ordering by timestamp and updated_at.

Required fields:

• Identity and scope: tenant_id, session_id, sync_key.
• Protocol payload: raw, tier, timestamp, compression_depth.
• Metrics: psi, rho, kappa.
• AVEC projections: user_*, model_*, comp_*.

Optional fields:

• Hierarchy: parent_node_id.
• Source lineage: source_metadata.
• Retrieval acceleration: context_summary, embedding*, embedded_at.

5.2 Table: calibration

Storage purpose:

• Session-level AVEC measurement history.

Key characteristics:

• SCHEMAFULL table.
• Ordered by created_at for latest state and trigger timeline.

5.3 Table: sync_checkpoint

Storage purpose:

• Connector/session cursor materialization for incremental pull.

Key characteristics:

• SCHEMAFULL table.
• Unique by (tenant_id, session_id, connector_id).

6. Index and Access Pattern Mapping

• idx_node_sync_identity (tenant_id, session_id, sync_key UNIQUE)
  • Supports idempotent upsert semantics.
• idx_node_change_cursor (tenant_id, session_id, updated_at, sync_key)
  • Supports stable incremental reads (updated_at, tie-break by sync_key).
• idx_node_tenant_session (tenant_id, session_id) and idx_node_session (session_id)
  • Supports scoped listing and filtering.
• idx_node_tier (tier), idx_node_timestamp (timestamp)
  • Supports filtering across tier/time windows.
• idx_cal_tenant_session, idx_cal_session
  • Supports latest calibration and trigger history.
• idx_checkpoint_scope (tenant_id, session_id, connector_id UNIQUE)
  • Supports deterministic checkpoint upsert.

7. Write Path Rules

7.1 Idempotent Upsert

upsert_node_async behavior:

1. Normalize/derive sync_key if missing (canonical_sync_key).
2. Determine tenant from session_id scope (tenant:<id>::session:<id>), else fallback default.
3. Probe existing exact scope and fallback any-tenant records by session_id + sync_key.
4. If fully equivalent payload/metadata exists -> Duplicate.
5. If record exists with differences -> Updated.
6. Otherwise create new -> Created.

7.2 Conflict Handling

On unique index conflict:

• Attempt to resolve conflict record id and update in place.
• Re-read existing record if needed and classify as Updated or Duplicate.

7.3 Compression AVEC Rule

If candidate compression_avec is absent or zeroed, model AVEC is used as persistence fallback for comp_* fields.

8. Read Path Rules

8.1 Query (query_nodes_async)

Supports:

• Optional session scope.
• Optional UTC lower/upper bounds.
• Optional tier set (case-insensitive).
• Descending timestamp order.

8.2 Resonance Retrieval

Distance formula (per record): ResonanceDelta = (|model_stability-s| + |model_friction-f| + |model_logic-l| + |model_autonomy-a|) / 4

Ordering:

• Ascending ResonanceDelta.

8.3 Hybrid Retrieval

NodeStore defines hybrid retrieval APIs and graceful fallback behavior to resonance-only in implementations that cannot apply semantic ranking.

9. Sync and Change Data Capture

flowchart TD
  A[Node upsert] --> B[Set updated_at and sync_key]
  B --> C[Index supports cursor ordering]
  C --> D[Query changes since cursor]
  D --> E{More rows}
  E -- yes --> F[Return page ordered by updated_at then sync_key]
  E -- no --> G[has_more false]
    F --> H[Consumer applies rows]
  H --> I[Store checkpoint with next cursor]

Determinism contract:

• Cursor is composite: (updated_at, sync_key).
• Ordering is total for equal timestamps due to sync_key tie-break.

10. Multi-Tenant and Legacy Compatibility

Tenant rules:

• Derived tenant for scoped sessions: tenant:<tenant_id>::session:<session_id>.
• Default tenant fallback: default.

Legacy compatibility behavior:

• Reads include records where tenant_id is NONE or empty for default tenant scope.
• Initializer backfills missing tenant fields and legacy sync fields.

11. Scope Rekey Transaction Model

Batch rekey performs scope migration over full session scope, not anchor-only rows:

• Update temporal_node.tenant_id, temporal_node.session_id.
• Update calibration.tenant_id, calibration.session_id.
• Execute within a transaction block.

Safety semantics:

• Dry-run supported at service level.
• Merge conflict detection for target scope supported.

12. Validation and Integrity Constraints

Protocol-level validation (TreeSitterValidator) enforces:

• Required STTP layers present.
• Layer order correctness.
• Tier in allowed set: raw|daily|weekly|monthly|quarterly|yearly.
• Content nesting depth <= 5.
• Optional PSI coherence verification against compression AVEC.

These validations occur before persistence in StoreContextService workflows.

13. Operational Non-Functional Requirements

• Durability: SurrealDB persistence for production profile.
• Idempotency: guaranteed by sync identity index and upsert semantics.
• Replay safety: cursor/checkpoint design supports restartable sync pulls.
• Backward compatibility: additive schema evolution expected; legacy tenant buckets supported.

14. Schema Evolution Guidelines

Permitted changes:

• Additive nullable fields.
• Additive indexes for new access paths.
• New retrieval predicates that preserve existing defaults.

Controlled changes (require migration and compatibility review):

• Sync identity key shape.
• Cursor ordering columns.
• Tier vocabulary changes.
• AVEC projection field semantics.

15. Traceability to Implementation

Primary implementation locations:

• locus-core-rs/src/storage/surrealdb/raw_queries.rs
• locus-core-rs/src/storage/surrealdb/node_store.rs
• locus-core-rs/src/domain/models.rs
• locus-core-rs/src/domain/contracts.rs
• locus-core-rs/src/application/validation/tree_sitter_validator.rs

Locus SDK Architecture Specification

Document Control

• Document ID: LOCUS-SDK-ARCH-001
• Status: Current
• Intended Audience: Application Developers, Platform Engineers, Solution Architects
• Source of Truth: locus-sdk implementation (application/*, domain/*, infrastructure/*, interface/*)
• Last Updated: 2026-05-04

1. Purpose and Scope

This document defines the architecture of locus-sdk, including module boundaries, service orchestration, extension points, and operational behavior.

Use this document to evaluate integration fit, implementation boundaries, and operational implications before committing to host-level adoption.

Objectives:

• Make component responsibilities explicit.
• Reduce integration risk for host applications.
• Preserve backward-compatible architecture evolution.

2. Layered Architecture Overview

flowchart TD
    A[Host Application] --> B[SDK API Surface - prelude]
    B --> C[Application Services]
    C --> D[Domain Contracts and Models]
    C --> E[Infrastructure Adapters]
    C --> F[Interface DTO Mappers]
    E --> G[locus-core-rs NodeStore]
    E --> H[AI Provider Runtime]

    subgraph SDK
      B
      C
      D
      E
      F
    end

Layer intent:

• prelude: stable ergonomic import surface for consumers.
• application: orchestration services and workflow composition.
• domain: neutral request/result types and provider contracts.
• infrastructure: concrete adapters and registries.
• interface: DTO translation boundary for transport adapters.

3. Module Decomposition

3.1 Domain Layer

• domain/memory.rs: canonical memory query/recall/aggregate/transform contracts.
• domain/ai.rs: provider capability model and routing contracts.
• domain/compression.rs: manual compression request/result schema.

3.2 Application Layer

Primary services:

• MemoryFindService
• MemoryRecallService
• MemoryExplainService
• MemoryAggregateService
• MemoryTransformService
• MemoryCompositionService
• MemorySchemaService
• ManualCompressionService
• AI routing helpers (ai_router.rs, routing_config.rs)

3.3 Infrastructure Layer

• InMemoryAiProviderRegistry
• GenaiProviderAdapter (feature-gated)
• SttpEmbeddingProviderAdapter
• OllamaEmbeddingProvider
• LocalEmbeddingProvider (feature local-embedding)

3.4 Interface Layer

• interface/dto.rs: bidirectional mapping between SDK-native types and DTO structs.

4. Core Service Topology (UML)

flowchart TD
    MCS[MemoryCompositionService]
    MFS[MemoryFindService]
    MRS[MemoryRecallService]
    MES[MemoryExplainService]
    MAS[MemoryAggregateService]
    MTS[MemoryTransformService]
    NS[NodeStore interface]
    APR[AiProviderRegistry interface]
    AP[AiProvider interface]

    MCS --> MRS
    MCS --> MES
    MCS --> MAS
    MFS --> NS
    MRS --> NS
    MES --> NS
    MAS --> NS
    MTS --> NS
    MTS --> APR
    APR --> AP

4.1 Query and Retrieval Services

classDiagram
    class MemoryFindService {
      +execute(request) MemoryFindResult
    }
    class MemoryRecallService {
      +execute(request) MemoryRecallResult
    }
    class MemoryExplainService {
      +execute(request) MemoryExplainResult
    }
    class MemoryAggregateService {
      +execute(request) MemoryAggregateResult
    }
    class NodeStore {
      <<interface>>
      +query_nodes_async(query)
      +get_by_resonance_async(...)
      +get_by_hybrid_async(...)
    }

    MemoryFindService ..> NodeStore
    MemoryRecallService ..> NodeStore
    MemoryExplainService ..> NodeStore
    MemoryAggregateService ..> NodeStore

4.2 Transform and Provider Resolution

classDiagram
    class MemoryTransformService {
      +execute(request) MemoryTransformResult
    }
    class AiProviderRegistry {
      <<interface>>
      +resolve(task, provider_id, policy)
      +list_capabilities()
    }
    class AiProvider {
      <<interface>>
      +embed_semantic(request)
      +embed_avec(request)
      +score_avec(request)
    }
    class NodeStore {
      <<interface>>
      +upsert_node_async(node)
    }

    MemoryTransformService ..> NodeStore
    MemoryTransformService ..> AiProviderRegistry
    AiProviderRegistry ..> AiProvider

4.3 Composition Workflow Wiring

flowchart LR
    MCS[MemoryCompositionService]
    MRS[MemoryRecallService]
    MES[MemoryExplainService]
    MAS[MemoryAggregateService]
    MTS[MemoryTransformService]

    MCS -->|recall_with_explain| MRS
    MCS -->|recall_with_explain| MES
    MCS -->|daily_rollup| MAS
    MCS -->|transform_then_recall_verify| MTS
    MCS -->|transform_then_recall_verify| MRS

5. Request Lifecycle Patterns

5.1 Find Path

flowchart TD
    A[MemoryFindRequest] --> B[Clamp limit]
    B --> C[NodeStore.query_nodes_async]
    C --> D[Apply common filters]
    D --> E[Sort by selected field]
    E --> F[Compute has_more and next_cursor]
    F --> G[MemoryFindResult]

5.2 Recall Path with Fallback

flowchart TD
    A[MemoryRecallRequest] --> B{query_embedding present?}
    B -- yes --> C[Hybrid retrieval]
    B -- no --> D[Resonance retrieval]
    C --> E[Apply common filters]
    D --> E
    E --> F{Fallback policy allows lexical fallback?}
    F -- no --> J[Return recall result]
    F -- yes --> G[Expanded retrieval]
    G --> H[Lexical scoring over summary/session/raw]
    H --> I[Merge or replace depending on policy]
    I --> J[Return recall result with retrieval_path]

5.3 Transform Path

flowchart TD
    A[MemoryTransformRequest] --> B[Query and filter candidate nodes]
    B --> C{operation == embed_backfill}
    C -- yes --> D[Keep nodes missing embeddings]
    C -- no --> E[Use selected set]
    D --> E
    E --> F{dry_run?}
    F -- yes --> G[Return projected counts]
    F -- no --> H[Batch through selected nodes]
    H --> I[Route semantic embedding via provider registry]
    I --> J[Apply embedding fields to node]
    J --> K[NodeStore.upsert_node_async]
    K --> L[Accumulate updated/skipped/failed/duplicate]
    L --> M[Return MemoryTransformResult]

6. AI Routing and Provider Resolution

Routing semantics:

• route_embedding dispatches by AiTask (SemanticEmbedding or AvecEmbedding).
• route_avec_score dispatches scoring requests.

Provider selection (AiProviderRegistry.resolve):

• Explicit provider_id must exist and support requested task.
• ProviderPolicy::Required requires explicit provider id.
• Auto and Preferred select first provider supporting the task.

AiRoutingConfig responsibilities:

• Default provider fallback.
• Task-specific model selection per provider.
• Request mutation helper methods (apply_to_embed_request, apply_to_score_request).

7. Data Contracts and Guardrails

Memory contract characteristics:

• Scope controls: tenant/session/tier/time.
• Filter controls: embedding presence/model, metric ranges, lexical contains.
• Scoring controls: alpha/beta and fallback policy.

Clamp guardrails:

• clamp_limit: 1..200.
• clamp_groups: 1..5000.
• clamp_nodes: 1..50000.
• clamp_batch_size: 1..500.

These guardrails are enforced before expensive operations.

8. Composition Workflows

MemoryCompositionService encapsulates higher-level workflows:

• recall_with_explain: single request emits both retrieval and explain trace.
• daily_rollup: aggregate by day over filtered scope.
• transform_then_recall_verify: execute transform, then immediate recall verification.
• build_content_from_text: deterministic structured content generation using manual compression and role-based AVEC resolution.

Composite content building constraints:

• Maximum recursion depth clamped to 5.
• Can require explicit AVEC overrides unless LLM fallback is allowed.
• Emits confidence-annotated structured JSON fields.

9. Infrastructure Adapters and Feature Gating

• GenaiProviderAdapter behind genai-provider feature.
• LocalEmbeddingProvider behind local-embedding feature.
• OllamaEmbeddingProvider always available.
• SttpEmbeddingProviderAdapter wraps locus-core-rs EmbeddingProvider into SDK AiProvider shape.

Operational note:

• Local embedding uses CPU-bound worker and blocking-task isolation for runtime safety.

10. DTO Boundary and Integration Strategy

interface/dto.rs provides:

• Inbound request conversion DTO -> domain model.
• Outbound response conversion domain model -> DTO.
• Composite workflow DTOs for composed service responses.

Design intent:

• Keep transport concerns out of domain/application layers.
• Preserve ability to evolve transport fields with additive mapping.

11. Reliability and Change Management Rules

Compatibility rules:

• Additive enums/fields are preferred evolution path.
• Existing defaults remain stable unless a versioned schema contract is introduced.
• New provider adapters must satisfy AiProvider capabilities explicitly.

Failure management principles:

• Partial batch failures are surfaced in MemoryTransformResult.failures.
• Fallback behavior is explicit and observable in explain outputs.
• Provider misconfiguration fails fast with structured errors.

12. Security and Operational Considerations

• Provider API keys and endpoints are externalized in host configuration.
• Text sent to embedding/scoring providers should be minimized to needed content.
• MemoryTransformService dry-run mode should be default for first execution in new environments.

13. Traceability to Implementation

Primary implementation locations:

• locus-sdk/src/domain/memory.rs
• locus-sdk/src/domain/ai.rs
• locus-sdk/src/application/memory_find.rs
• locus-sdk/src/application/memory_recall.rs
• locus-sdk/src/application/memory_explain.rs
• locus-sdk/src/application/memory_aggregate.rs
• locus-sdk/src/application/memory_transform.rs
• locus-sdk/src/application/memory_composition.rs
• locus-sdk/src/application/routing_config.rs
• locus-sdk/src/infrastructure/registry.rs
• locus-sdk/src/infrastructure/embeddings.rs
• locus-sdk/src/interface/dto.rs

STTP SDK Recursive Node-From-Text Composite Guide

Date: 2026-05-03 Status: Implemented (initial slice)

1. Purpose

This guide documents the recursive deterministic composition workflow in locus-sdk.

The workflow lets callers build a spec-safe STTP content-layer payload from plain text entries without requiring model summarization for the core compression path.

2. What It Solves

1. Build content payloads from ordered role-tagged text input.
2. Support recursive context trees while staying validator-safe.
3. Resolve AVEC using explicit deterministic policy order.
4. Keep an optional LLM path only for unresolved AVEC, not core compression.

3. Core Types

Defined in locus-sdk/src/application/memory_composition.rs:

1. CompositeRole
2. CompositeInputItem
3. CompositeRoleAvecOverrides
4. CompositeNodeFromTextOptions
5. CompositeNodeFromTextRequest
6. CompositeNodeFromTextResult

DTO equivalents are defined in locus-sdk/src/interface/dto.rs:

1. CompositeRoleDto
2. CompositeInputItemDto
3. CompositeRoleAvecOverridesDto
4. CompositeNodeFromTextOptionsDto
5. CompositeNodeFromTextRequestDto
6. CompositeNodeFromTextResponseDto

4. Deterministic AVEC Resolution

For each input item, AVEC resolution order is:

1. item-level override (CompositeInputItem.avec_override)
2. role-level override (CompositeNodeFromTextOptions.role_avec)
3. global override (CompositeNodeFromTextOptions.global_avec)
4. unresolved

If unresolved item count is greater than zero:

1. if allow_llm_avec_fallback is false, the workflow fails with a typed error,
2. if allow_llm_avec_fallback is true, requires_llm_avec is true in the result.

5. Recursion and Validation Constraints

1. Requested depth is clamped to [1, 5].
2. Build fails if item context exceeds max_recursion_depth.
3. The content object uses confidence-scored keys to remain strict-parser compatible.
4. This aligns with validator nesting limits in core validator logic.

Reference: locus-core-rs/src/application/validation/tree_sitter_validator.rs

6. Compression Strategy

Each input/context item is compressed with ManualCompressionService to derive:

1. anchor_topic
2. key_points

This keeps the composite deterministic and model-optional for compression.

Reference: locus-sdk/src/application/manual_compression.rs

7. Quick Example (Domain Models)

use std::sync::Arc;

use anyhow::Result;
use locus_core_rs::{InMemoryNodeStore, NodeStore};
use locus_core_rs::domain::models::AvecState;
use locus_sdk::prelude::{
    CompositeInputItem, CompositeNodeFromTextOptions, CompositeNodeFromTextRequest,
    CompositeRole, CompositeRoleAvecOverrides, MemoryCompositionService,
};

fn main() -> Result<()> {
    let store: Arc<dyn NodeStore> = Arc::new(InMemoryNodeStore::new());
    let composition = MemoryCompositionService::new(store);

    let req = CompositeNodeFromTextRequest {
        items: vec![CompositeInputItem {
            role: CompositeRole::Conversation,
            text: "user asks about fallback policy and model explains ranked retrieval".to_string(),
            avec_override: None,
            context: vec![CompositeInputItem {
                role: CompositeRole::Document,
                text: "design notes mention deterministic lexical fallback".to_string(),
                avec_override: None,
                context: Vec::new(),
            }],
        }],
        options: CompositeNodeFromTextOptions {
            role_avec: CompositeRoleAvecOverrides {
                conversation: Some(AvecState {
                    stability: 0.82,
                    friction: 0.20,
                    logic: 0.86,
                    autonomy: 0.74,
                }),
                ..Default::default()
            },
            global_avec: Some(AvecState {
                stability: 0.75,
                friction: 0.25,
                logic: 0.80,
                autonomy: 0.70,
            }),
            allow_llm_avec_fallback: false,
            max_recursion_depth: 5,
        },
    };

    let result = composition.build_content_from_text(&req)?;

    println!("resolved_avec_count={}", result.resolved_avec_count);
    println!("requires_llm_avec={}", result.requires_llm_avec);
    println!("content={}", result.content);

    Ok(())
}

8. Quick Example (DTO Boundary)

#![allow(unused)]
fn main() {
use locus_sdk::prelude::{
    CompositeNodeFromTextRequestDto, CompositeNodeFromTextResponseDto,
};

fn convert_boundary(req_dto: CompositeNodeFromTextRequestDto) -> CompositeNodeFromTextResponseDto {
    let domain_req = req_dto.into();
    let domain_result = service.build_content_from_text(&domain_req).unwrap();
    domain_result.into()
}
}

9. Test Coverage

Current coverage includes:

1. role and global AVEC resolution chain
2. unresolved AVEC failure behavior
3. recursion depth enforcement
4. strict parser and validator conformance for generated content payload

Reference tests: locus-sdk/src/application/memory_composition.rs

10. Integration Notes

1. This workflow currently returns content-layer payload plus AVEC resolution stats.
2. It does not replace typed IR grammar; it composes spec-safe content for insertion into full STTP node workflows.
3. Recommended next step is a full store-ready recipe that assembles the complete four-layer node from this content payload.

Deployment

Audience and Use

This page is for teams deploying Locus in development, staging, and production environments.

Use it to:

1. Select an operating model by environment.
2. Apply release checks before rollout.
3. Keep runtime behavior consistent across deployment targets.

For prerequisite tooling and baseline readiness checks, see Environment Setup.

Supported Runtime Modes

1. Embedded/local development mode.
2. Remote SurrealDB mode.
3. Hybrid host mode where gateway and MCP share the same underlying Locus behavior.

Environment Matrix

Build Prerequisites

1. Rust toolchain (stable).
2. cargo available in PATH.
3. Optional: SurrealDB endpoint for integration environments.

Platform and Runtime Support

Build Commands

From repository root:

cargo check --workspace --examples
cargo test -p locus-core-rs --lib
cargo test -p locus-sdk

Packaging

Core package helper:

./locus-core-rs/build-package.sh

Publish preflight helper:

./locus-core-rs/publish-crates.sh

Configuration Guidance

1. Keep environment-specific endpoint credentials out of repository files.
2. Set explicit session/tenant scoping in host integrations.
3. Prefer pinned model/provider settings for repeatable production behavior.
4. Separate development and production provider endpoints.
5. Use explicit tenant/session scoping in host runtime configs.

These controls reduce configuration drift and support predictable operations during upgrades.

Host Deployment Profiles

1. MCP host profile: MCP prioritizes tool contract stability and strict parser and validator checks in smoke tests.

2. Gateway host profile: Gateway prioritizes stable HTTP and gRPC mapping to SDK primitives with health checks and retrieval policy observability.

Release Readiness Checklist

1. Workspace compile and tests pass.
2. Example binaries compile and execute in CI smoke path.
3. Integration hosts confirm SDK contract compatibility.
4. Changelog and migration notes updated.
5. Versioning and compatibility policy reviewed for this release.
6. Security checklist validated for secrets and logging hygiene.

Releases that do not meet this checklist should not be promoted to production.

Release Gates by Environment

Local Development Gate

1. Workspace compile succeeds.
2. Targeted crate tests pass for changed modules.
3. At least one example path executes without contract errors.

CI Validation Gate

1. Workspace test matrix passes.
2. Lint and docs build complete in CI.
3. No parser or validator regressions in changed paths.

Staging Gate

1. Gateway and MCP parity checks pass against the same dataset.
2. Retrieval path distribution stays within expected baseline variance.
3. Rollback procedure is rehearsed and time-bounded.

Production Promotion Gate

1. Staging sign-off is recorded.
2. Observability alerts and dashboards are active.
3. Release note includes compatibility and rollback instructions.

Operations

Observability Priorities

1. Retrieval path visibility for recall workflows.
2. Fallback policy trigger visibility.
3. Transform execution counters and failure reasons.
4. Parser and validator error surface for malformed nodes.

Golden Signals

1. Latency: recall and transform response times by percentile.
2. Errors: parser, validator, retrieval, and transform failure rates.
3. Throughput: query volume and transform batch throughput.
4. Saturation: host and storage pressure indicators under load.

Runtime Health Signals

1. Parse success ratio.
2. Validation failure ratio.
3. Recall latency percentiles.
4. Transform batch success and failure counts.
5. Embedding backfill throughput and error rates.

Alert Triggers

1. Sudden rise in validation failures.
2. Recall latency p95 breach sustained over multiple intervals.
3. Unexpected shift in retrieval_path distribution.
4. Transform failure ratio above configured threshold.

Incident Severity Model

1. SEV-1: broad service impact or data integrity risk.
2. SEV-2: degraded behavior with viable workaround.
3. SEV-3: localized or non-critical workflow degradation.

Severity guides response urgency, escalation path, and communication cadence.

Suggested Starting Thresholds

These baseline thresholds are starting points and should be tuned per environment.

1. Recall p95 latency: page at sustained breach over 15 minutes.
2. Validation failure ratio: page when sustained above 2 percent.
3. Transform failure ratio: page when sustained above 1 percent.
4. Retrieval path drift: investigate when lexical fallback share doubles from baseline.

Severity-to-Response Mapping

1. SEV-1: immediate incident bridge, continuous updates, executive and stakeholder notification.
2. SEV-2: active triage with scheduled updates and rollback decision checkpoint.
3. SEV-3: normal triage queue with documented mitigation and scheduled follow-up.

Logging Recommendations

1. Log explicit policy settings for recall and transform requests.
2. Log retrieval path and fallback reason for each recall request.
3. Log session and tenant scope used for each operation.
4. Keep request payload snapshots in redacted form for incident replay.

Incident Response Playbook

1. Determine whether failure is parser, policy, or storage related.
2. Reproduce with the same request payload and explicit limits.
3. Use explain workflows to isolate ranking or fallback regressions.
4. Run dry-run transforms before any broad mutation in recovery paths.

Runbooks

Runbook A: Parser/Validation Spike

1. Capture representative failing payloads.
2. Validate four-layer ordering and content confidence key format.
3. Compare with last known-good release behavior.
4. Roll back host version if failures are release-correlated.

Runbook B: Retrieval Regression

1. Capture request payload, scoring settings, and retrieval_path.
2. Execute explain workflow with same request.
3. Compare channel scores and fallback behavior against baseline.
4. Patch policy defaults only with regression test coverage.

Runbook C: Transform Job Instability

1. Switch affected operations to dry-run mode.
2. Validate selected node set and provider capabilities.
3. Reduce batch_size and re-run with checkpoint controls.
4. Resume full run only after zero-failure dry-run parity.

SLO Suggestions

1. Recall p95 latency target per environment.
2. Parse/validate success target for ingested nodes.
3. Transform job success target and bounded failure budget.

Teams should define environment-specific numeric targets and review them at each release cycle.

Maintenance

1. Keep crate versions aligned within workspace releases.
2. Re-run examples as part of regression verification.
3. Audit docs links and command validity each release cycle.
4. Rehearse incident runbooks quarterly.

Integration

Audience and Use

This page is for developers integrating Locus into applications, services, and agent tooling.

Use it to:

1. Define a stable integration boundary.
2. Migrate existing retrieval logic without contract breakage.
3. Validate behavior parity before cutover.

Goal

Integrate host transports with Locus crates while keeping STTP protocol semantics stable.

Recommended Integration Boundary

1. Parse and validate at ingestion boundary.
2. Convert transport payloads into SDK DTOs.
3. Execute SDK workflows in application services.
4. Map results back to transport contracts.

Host Migration Pattern

1. Replace transport-owned retrieval logic with locus-sdk primitives.
2. Keep endpoint names stable during migration.
3. Add compatibility tests for behavior parity.
4. Remove deprecated transport-local logic after one release cycle.

Compatibility Policy

1. Keep external MCP tool and gateway endpoint contracts stable during internal rewires.
2. Prefer additive fields and optional settings over contract replacement.
3. Any unavoidable breaking change requires migration notes and version bump alignment.
4. Retrieval policy defaults must remain explicit and test-covered.

This policy enables staged migrations while preserving downstream client compatibility.

MCP Integration Notes

1. Prefer SDK primitives for list and context tools.
2. Keep tool contracts stable; change only internals first.
3. Add tests for fallback policy behavior and explain parity.

Gateway Integration Notes

1. Route HTTP and gRPC handlers into SDK service layer.
2. Share one request mapping strategy across transports.
3. Keep response payload fields stable until formal API revision.

Safety Checks During Migration

1. Compare retrieval_path results before and after migration.
2. Compare transform counters and side effects in dry-run first.
3. Validate strict parser compatibility for generated nodes.

Migration Acceptance Criteria

1. Contract parity tests pass for migrated host routes/tools.
2. Explain outputs are consistent with recall behavior before and after migration.
3. Transform counters and dry-run selection remain behaviorally equivalent.
4. Compile and runtime smoke checks pass in workspace and host deployment jobs.

Migration is complete only when all acceptance criteria are met.

Integration Acceptance Templates

Contract Parity Template

1. Endpoint or tool name unchanged.
2. Request field semantics unchanged or additive.
3. Response field semantics unchanged or additive.
4. Error code and message class behavior unchanged.

Retrieval Behavior Template

1. retrieval_path distribution aligned with baseline behavior.
2. Explain output aligns with returned recall results.
3. Fallback activation remains policy-consistent.
4. Latency profile remains within agreed operational bounds.

Rollout Gate Template

1. Local and CI validation complete.
2. Staging parity checks complete.
3. Rollback path validated.
4. Monitoring and alert hooks confirmed in target environment.

Cutover Checklist Template

Pre-Cutover Checks

1. Freeze incompatible contract changes during the cutover window.
2. Confirm request and response snapshots for baseline comparison.
3. Confirm on-call ownership and escalation path.

Parity Checks During Cutover

1. Compare retrieval_path and explain behavior between old and new integration paths.
2. Compare latency and error rate against baseline windows.
3. Validate parser and validator success ratios remain stable.

Rollback Criteria

1. Error-rate increase exceeds agreed threshold.
2. Contract mismatch is observed in client-visible payloads.
3. Retrieval behavior diverges from baseline without approved policy change.

Post-Cutover Verification

1. Re-run acceptance templates in production mode.
2. Confirm no delayed transform or ingestion side effects.
3. Publish cutover summary with observed deltas and follow-up actions.

Examples

Audience and Use

This page is for developers using Locus patterns in their own systems.

Use examples to:

1. Start from known-good integration patterns.
2. Validate behavior quickly in local environments.
3. Translate reference workflows into production service code.

Available Example Binaries

1. ../locus-sdk/examples/provider_registry_setup.rs
2. ../locus-sdk/examples/memory_composition.rs
3. ../locus-sdk/examples/recursive_composite_pipeline.rs
4. ../locus-sdk/examples/generate_faker_fixture.rs

Run Examples

From repository root:

cargo run -p locus-sdk --example provider_registry_setup
cargo run -p locus-sdk --example memory_composition
cargo run -p locus-sdk --example recursive_composite_pipeline
cargo run -p locus-sdk --example generate_faker_fixture -- --help

Example Coverage

1. Provider capability registry wiring.
2. Composition workflows and multi-step orchestration.
3. Recursive deterministic content construction plus strict parser and validator checks.
4. Deterministic fixture generation for load and quality testing.

For domain-specific multi-agent setup patterns, see Agent Blueprints by Domain.

For practical GenAI plus STTP memory orchestration recipes, see Mini Orchestration Cookbooks.

If you want a fast path chooser by outcome and build style, start with Cookbook Overview.

Example Decision Guide

provider_registry_setup

Use when:

1. You are wiring provider capability routing for the first time.
2. You need to validate task-to-provider matching rules.

Expected outcome:

1. Registry contains expected providers and capability visibility.
2. Unsupported provider and task combinations fail with explicit errors.

Common failure checks:

1. Provider IDs differ between config and runtime registration.
2. Required capabilities are missing from selected providers.

Next integration step:

1. Wire registry initialization into your service startup path and fail fast on missing capabilities.

memory_composition

Use when:

1. You want a reference for multi-step memory workflows.
2. You need recall and explain behavior in one composed flow.

Expected outcome:

1. Composed workflow returns stable result shapes.
2. Explain path aligns with recall outcome.

Common failure checks:

1. Request policy settings are implicit instead of explicit.
2. Scope filters are omitted, causing noisy result sets.

Next integration step:

1. Promote composed request defaults into a shared app-level policy module.

recursive_composite_pipeline

Use when:

1. You are building structured content from recursive composition.
2. You need deterministic recursion-depth behavior.

Expected outcome:

1. Recursive output respects depth and schema constraints.
2. Content remains parser and validator compatible.

Common failure checks:

1. Recursion depth exceeds configured limits.
2. Output shape drifts from expected typed content structure.

Next integration step:

1. Add schema-level response assertions in CI to prevent output drift before deployment.

generate_faker_fixture

Use when:

1. You need reproducible fixture data for tests and benchmarking.
2. You want fast local workload simulation.

Expected outcome:

1. Fixture generation is deterministic for the same parameters.
2. Generated payloads are valid for ingest and retrieval tests.

Common failure checks:

1. Parameter ranges produce unrealistic payload distributions.
2. Fixture output is not validated before being used in regression tests.

Next integration step:

1. Add fixture generation to CI pre-test setup with deterministic seed controls.

Production Usage Guidance

1. Treat examples as reference patterns, not deployment templates.
2. Externalize credentials and environment-specific endpoints.
3. Keep explicit limits and fallback policies in runtime requests.
4. Add environment-specific observability and failure handling before production rollout.

Promotion Checklist

Before promoting an example pattern into production code:

1. Add scope controls and policy defaults explicitly.
2. Add structured error handling and retry boundaries.
3. Add telemetry for latency, fallback behavior, and failures.
4. Add regression tests for request and response shape stability.

Cookbook Overview

Start Here

This is the fastest way to choose the right Locus cookbook for your use case.

Pick by outcome, not by internal architecture.

Choose by Outcome

I need better answers in customer-facing workflows

Start with:

1. Mini Orchestration Cookbooks
2. Agent Blueprints by Domain

I need safer incident or operations decisions

Start with:

1. Mini Orchestration Cookbooks
2. Troubleshooting
3. Operations

I need rollout-safe automation with dry-run controls

Start with:

1. Mini Orchestration Cookbooks
2. Integration
3. Deployment

I need research and synthesis across sessions

Start with:

1. Mini Orchestration Cookbooks
2. Agent Blueprints by Domain

Choose by Build Style

I want code-first SDK examples

Start with Examples.

I want orchestration recipes plus prompt templates

Start with Mini Orchestration Cookbooks.

I want domain presets and starter config values

Start with Agent Blueprints by Domain.

Quick Path Matrix

Implementation Flow We Recommend

1. Start with one cookbook and keep scope narrow.
2. Run recall plus explain before writing any policy automation.
3. Add transform dry-run as a safety default.
4. Promote to production only after integration acceptance checks.

Competitive Positioning

Locus is optimized for teams that need:

1. memory behavior that is observable and explainable,
2. deterministic workflow composition options,
3. transport-neutral integration paths across SDK, MCP, and gateway.

Use this hub as your entrypoint, then deepen into the linked pages.

Mini Orchestration Cookbooks

Why This Exists

This chapter is for builders who want working patterns, not theory.

Use these cookbooks when you want:

1. GenAI to handle model-facing intelligence.
2. Locus to handle memory, recall policy, explainability, and safe transforms.
3. A repeatable orchestration shape you can adapt by domain.

For a fast picker by outcome, go to Cookbook Overview.

Core Pattern

Every cookbook in this chapter follows the same loop:

1. Ingest events into STTP memory.
2. Recall context with explicit scoring and fallback policy.
3. Explain the retrieval path before high-impact actions.
4. Optionally run transform in dry-run before mutation.
5. Pass grounded context into a prompt template for model response.

GenAI Hookup: Provider and Routing

Use GenAI as the model interface and keep memory orchestration in Locus.

#![allow(unused)]
fn main() {
use std::collections::HashMap;
use std::sync::Arc;

use anyhow::Result;
use locus_sdk::prelude::{
    AiRoutingConfig, InMemoryAiProviderRegistry, ProviderModelProfile,
};
#[cfg(feature = "genai-provider")]
use locus_sdk::prelude::GenaiProviderAdapter;

fn build_registry_and_routing() -> Result<(Arc<InMemoryAiProviderRegistry>, AiRoutingConfig)> {
    let mut registry = InMemoryAiProviderRegistry::new();

    #[cfg(feature = "genai-provider")]
    registry.register(GenaiProviderAdapter::new(
        "genai",
        Some("text-embedding-3-small".to_string()),
    ));

    let mut providers = HashMap::new();
    providers.insert(
        "genai".to_string(),
        ProviderModelProfile {
            semantic_model: Some("text-embedding-3-small".to_string()),
            avec_embedding_model: Some("text-embedding-3-small".to_string()),
            avec_scoring_model: Some("gpt-4o-mini".to_string()),
        },
    );

    let routing = AiRoutingConfig {
        default_provider_id: Some("genai".to_string()),
        providers,
    };

    Ok((Arc::new(registry), routing))
}
}

Integration note:

1. Configure credentials for your selected GenAI provider in environment variables.
2. Keep provider IDs stable across environments for predictable routing behavior.

Composition Primer

The MemoryCompositionService gives you high-level orchestration:

1. recall_with_explain for grounded context with retrieval diagnostics.
2. daily_rollup for memory compression over time windows.
3. transform_then_recall_verify for safe write-preview and immediate validation.
4. build_content_from_text for deterministic content construction.

Cookbook 1: Support Copilot (Fast Resolution)

Intent:

1. Pull the smallest high-confidence context window.
2. Generate a customer-safe next action grounded in prior resolutions.

use anyhow::Result;
use std::sync::Arc;
use locus_core_rs::{InMemoryNodeStore, NodeStore};
use locus_sdk::prelude::{
    FallbackPolicy, MemoryCompositionService, MemoryRecallRequest, MemoryScoring,
};

#[tokio::main]
async fn main() -> Result<()> {
    let store: Arc<dyn NodeStore> = Arc::new(InMemoryNodeStore::new());
    let composition = MemoryCompositionService::new(store);

    let result = composition
        .recall_with_explain(&MemoryRecallRequest {
            query_text: Some("billing mismatch after plan upgrade".to_string()),
            scoring: MemoryScoring {
                alpha: 0.70,
                beta: 0.30,
                fallback_policy: FallbackPolicy::OnEmpty,
                ..Default::default()
            },
            ..Default::default()
        })
        .await?;

    println!(
        "retrieved={}, path={:?}, stages={}",
        result.recall.retrieved,
        result.explain.retrieval_path,
        result.explain.stages.len()
    );

    Ok(())
}

Prompt template:

system:
You are a support resolution assistant.
Use only provided context. If evidence is weak, say so explicitly.

user:
Issue: {{live_issue_summary}}
Top memory context:
{{ranked_context_snippets}}

Return:
1) likely root cause
2) next best action
3) confidence (0-1)
4) what evidence is still missing

Cookbook 2: Incident Triage Captain (Explain Before Escalate)

Intent:

1. Keep response speed high.
2. Require explain trace before severity escalation.

use anyhow::Result;
use std::sync::Arc;
use locus_core_rs::{InMemoryNodeStore, NodeStore};
use locus_sdk::prelude::{
    FallbackPolicy, MemoryCompositionService, MemoryRecallRequest, MemoryScoring,
    StrictnessMode,
};

#[tokio::main]
async fn main() -> Result<()> {
    let store: Arc<dyn NodeStore> = Arc::new(InMemoryNodeStore::new());
    let composition = MemoryCompositionService::new(store);

    let triage = composition
        .recall_with_explain(&MemoryRecallRequest {
            query_text: Some("unusual outbound auth failures".to_string()),
            scoring: MemoryScoring {
                alpha: 0.75,
                beta: 0.25,
                fallback_policy: FallbackPolicy::Always,
                strictness: StrictnessMode::Precision,
                ..Default::default()
            },
            ..Default::default()
        })
        .await?;

    println!("path={:?}", triage.explain.retrieval_path);
    println!("fallback_triggered={}", triage.explain.fallback_triggered);
    Ok(())
}

Prompt template:

system:
You are an incident triage assistant. Never recommend irreversible action without evidence.

user:
Signal summary: {{signal_summary}}
Explain trace:
{{explain_stages}}
Top evidence:
{{context_snippets}}

Return:
1) severity recommendation
2) containment candidates
3) risks of false positive
4) evidence citations

Cookbook 3: Release Reliability Coach (Dry-Run First)

Intent:

1. Preview transform impact before writes.
2. Validate retrieval quality immediately after transform simulation.

use anyhow::Result;
use std::sync::Arc;
use locus_core_rs::{InMemoryNodeStore, NodeStore};
use locus_sdk::prelude::{
    InMemoryAiProviderRegistry, MemoryCompositionService, MemoryRecallRequest,
    MemoryTransformOperation, MemoryTransformRequest, MemoryTransformThenRecallRequest,
};

#[tokio::main]
async fn main() -> Result<()> {
    let store: Arc<dyn NodeStore> = Arc::new(InMemoryNodeStore::new());
    let composition = MemoryCompositionService::new(store);
    let providers = Arc::new(InMemoryAiProviderRegistry::new());

    let result = composition
        .transform_then_recall_verify(
            providers,
            &MemoryTransformThenRecallRequest {
                transform: MemoryTransformRequest {
                    operation: MemoryTransformOperation::EmbedBackfill,
                    dry_run: true,
                    batch_size: 25,
                    max_nodes: 500,
                    ..Default::default()
                },
                recall: MemoryRecallRequest {
                    query_text: Some("release rollback patterns".to_string()),
                    ..Default::default()
                },
            },
        )
        .await?;

    println!("selected={}, updated={}", result.transform.selected, result.transform.updated);
    println!("retrieved={}", result.recall.retrieved);
    Ok(())
}

Prompt template:

system:
You are a release reliability assistant.
Prefer conservative recommendations when regression evidence is incomplete.

user:
Release context: {{release_context}}
Transform preview: {{transform_summary}}
Recall context: {{recall_summary}}

Return:
1) promote / hold / rollback recommendation
2) top risk factors
3) confidence
4) mandatory verification checks

Cookbook 4: Research Synthesizer (Deterministic Memory Build)

Intent:

1. Structure raw research notes into deterministic memory content.
2. Ask the model to synthesize only after context is structured and scoped.

use anyhow::Result;
use std::sync::Arc;
use locus_core_rs::{InMemoryNodeStore, NodeStore};
use locus_sdk::prelude::{
    CompositeInputItem, CompositeNodeFromTextOptions, CompositeNodeFromTextRequest,
    CompositeRole, MemoryCompositionService,
};

fn main() -> Result<()> {
    let store: Arc<dyn NodeStore> = Arc::new(InMemoryNodeStore::new());
    let composition = MemoryCompositionService::new(store);

    let structured = composition.build_content_from_text(&CompositeNodeFromTextRequest {
        items: vec![CompositeInputItem {
            role: CompositeRole::Document,
            text: "Policy memo highlights housing supply bottlenecks".to_string(),
            avec_override: None,
            context: vec![],
        }],
        options: CompositeNodeFromTextOptions {
            allow_llm_avec_fallback: false,
            max_recursion_depth: 3,
            ..Default::default()
        },
    })?;

    println!("resolved_avec_count={}", structured.resolved_avec_count);
    println!("requires_llm_avec={}", structured.requires_llm_avec);
    Ok(())
}

Prompt template:

system:
You are a research synthesis assistant.
Preserve uncertainty and disagreement; do not collapse conflicting evidence.

user:
Question: {{research_question}}
Structured context payload:
{{structured_content}}

Return:
1) synthesis
2) competing interpretations
3) confidence per claim
4) additional evidence needed

Prompting Rules That Work Well With STTP

1. Require evidence-cited outputs for high-impact decisions.
2. Ask for explicit confidence fields rather than vague certainty language.
3. Pass retrieval metadata to the prompt when using explain-driven guardrails.
4. Preserve contradictions in output instead of forcing a single narrative.

Anti-Patterns

1. Letting model responses bypass retrieval and explain checks.
2. Running transforms without dry-run in fresh environments.
3. Using broad session scopes that mix unrelated workflows.
4. Tuning alpha and beta without checking retrieval_path drift.

Where To Go Next

1. For domain presets, continue with Agent Blueprints by Domain.
2. For API-level examples, use Examples.
3. For rollout safeguards, use Integration and Operations.

Agent Blueprints by Domain

Audience and Use

This page is for teams designing task-specific agents that share one STTP memory backbone.

Use it to:

1. Map business tasks to concrete agent memory behavior.
2. Reuse reliable STTP patterns across multiple domains.
3. Start with practical defaults before deep customization.

Blueprint Format

Each blueprint defines:

1. Primary task and success criteria.
2. Memory ingestion pattern across raw, daily, and weekly tiers.
3. Retrieval policy defaults.
4. Operational guardrails and handoff points.

Shared Foundation for All Blueprints

Start each agent with these defaults:

1. Parse and validate all stored nodes before persistence.
2. Require explicit tenant and session scope on write and recall.
3. Store key runtime policy values alongside each operation.
4. Keep explain-capable recall enabled for incident diagnostics.

Blueprint 1: Customer Support Resolution Agent

Primary task:

1. Resolve customer issues faster while preserving account context.

Memory pattern:

1. raw: ticket transcript fragments, action decisions, workaround attempts.
2. daily: condensed issue timeline and resolution outcomes.
3. weekly: recurring issue clusters and escalation themes.

Retrieval defaults:

1. Start with hybrid retrieval when embeddings exist.
2. Enable lexical fallback for sparse or new accounts.
3. Limit recall result set to a small operator-reviewable window.

Guardrails:

1. Escalate when confidence on root-cause fields falls below policy threshold.
2. Require human approval before customer-visible policy exceptions.

Blueprint 2: Revenue and Account Strategy Agent

Primary task:

1. Prepare account intelligence summaries for sales and success teams.

Memory pattern:

1. raw: call notes, objections, product-fit signals, next-step commitments.
2. daily: account momentum summary with confidence annotations.
3. weekly: pipeline risk and expansion opportunity themes.

Retrieval defaults:

1. Prioritize recent daily and weekly tiers for executive summaries.
2. Pull raw evidence snippets for every high-impact recommendation.

Guardrails:

1. Block outbound recommendations lacking supporting evidence links.
2. Flag contradictory account signals for manual review.

Blueprint 3: Security Incident Triage Agent

Primary task:

1. Speed investigation and containment decisions during active incidents.

Memory pattern:

1. raw: alerts, timeline events, containment steps, IOC references.
2. daily: incident narrative with mitigation state.
3. weekly: recurring attack pattern and control-gap rollups.

Retrieval defaults:

1. Bias toward strict time-window filtering during active incidents.
2. Use explain flows on every severity upgrade recommendation.

Guardrails:

1. Require dual confirmation before irreversible containment actions.
2. Freeze autonomous remediation if retrieval path deviates from baseline.

Blueprint 4: Clinical Operations Coordination Agent

Primary task:

1. Improve handoff quality across clinical operations teams.

Memory pattern:

1. raw: triage notes, operational handoff details, care coordination tasks.
2. daily: shift-level summary and unresolved coordination items.
3. weekly: recurring workflow bottlenecks and staffing friction themes.

Retrieval defaults:

1. Prefer deterministic recall windows scoped to active handoff queues.
2. Keep retrieval limits tight for auditability and review speed.

Guardrails:

1. Restrict memory access by role and operational scope.
2. Require explicit redaction policy for sensitive fields before storage.

Blueprint 5: Engineering Release Reliability Agent

Primary task:

1. Reduce regressions by preserving release and rollback context.

Memory pattern:

1. raw: deploy events, test failures, rollback attempts, mitigation notes.
2. daily: release status summary and unresolved risks.
3. weekly: stability trend and recurring failure-class rollup.

Retrieval defaults:

1. Compare current release signals against recent weekly baselines.
2. Use explain traces to justify promote, hold, or rollback decisions.

Guardrails:

1. Require rollback recommendation when failure classes repeat beyond threshold.
2. Block promotion if validation success metrics regress from baseline.

Blueprint 6: Research and Policy Analysis Agent

Primary task:

1. Build traceable recommendations from large document sets.

Memory pattern:

1. raw: source excerpts with provenance and extraction rationale.
2. daily: topic summaries with confidence-scored claims.
3. weekly: cross-source synthesis with contradiction tracking.

Retrieval defaults:

1. Retrieve source-backed evidence first, synthesized claims second.
2. Use lexical fallback for low-embedding or highly technical terminology.

Guardrails:

1. Do not emit conclusions without source-linked evidence fields.
2. Route high-impact policy conclusions through human review.

Implementation Template

Use this checklist when instantiating any blueprint:

1. Define task-specific session naming and tenant strategy.
2. Define required content fields and confidence expectations.
3. Define fallback policy and acceptable retrieval_path behavior.
4. Define transform workflows and dry-run policy.
5. Define escalation, rollback, and audit logging requirements.

Starter Request Shape

Use a request profile like this as a baseline for domain agents:

session scope: <tenant>::<agent-role>::<workstream>
tiers: ["raw", "daily", "weekly"]
retrieval policy:
  mode: hybrid-preferred
  lexical_fallback: enabled
  limit: 5-20 depending on operator review mode
explain: enabled for all high-impact actions
transform mode: dry-run first in new environments

Drop-In Starter Configs

Use these as first-pass defaults. Tune only after collecting baseline metrics.

Config A: Customer Support Resolution

session pattern: tenant::<account-id>::agent::support-resolution::<ticket-id>
tiers: ["raw", "daily", "weekly"]
limit: 8
alpha: 0.70
beta: 0.30
retrieval mode: hybrid-preferred
lexical fallback: enabled
time window: last 30 days by default
explain: required for escalation and exception decisions
transform mode: dry-run for new policy revisions

Config B: Revenue and Account Strategy

session pattern: tenant::<account-id>::agent::revenue-strategy::<quarter>
tiers: ["daily", "weekly", "raw"]
limit: 12
alpha: 0.60
beta: 0.40
retrieval mode: hybrid-preferred
lexical fallback: enabled
time window: last 90 days by default
explain: required for outbound recommendation generation
transform mode: dry-run for scoring-model changes

Config C: Security Incident Triage

session pattern: tenant::<org-id>::agent::security-triage::<incident-id>
tiers: ["raw", "daily"]
limit: 6
alpha: 0.75
beta: 0.25
retrieval mode: hybrid-preferred with strict time filtering
lexical fallback: enabled
time window: incident start to now
explain: required for severity upgrade and containment recommendations
transform mode: dry-run first, then controlled batch execution

Config D: Clinical Operations Coordination

session pattern: tenant::<facility-id>::agent::clinical-ops::<shift-id>
tiers: ["raw", "daily"]
limit: 7
alpha: 0.65
beta: 0.35
retrieval mode: deterministic-first with hybrid fallback
lexical fallback: enabled
time window: active shift plus previous shift handoff window
explain: required for unresolved handoff item recommendations
transform mode: dry-run until handoff schema is stable

Config E: Engineering Release Reliability

session pattern: tenant::<org-id>::agent::release-reliability::<release-id>
tiers: ["raw", "daily", "weekly"]
limit: 10
alpha: 0.68
beta: 0.32
retrieval mode: hybrid-preferred
lexical fallback: enabled
time window: release window plus previous two release cycles
explain: required for promote, hold, and rollback recommendations
transform mode: dry-run for migration and backfill operations

Config F: Research and Policy Analysis

session pattern: tenant::<program-id>::agent::policy-analysis::<topic-id>
tiers: ["raw", "daily", "weekly"]
limit: 14
alpha: 0.55
beta: 0.45
retrieval mode: hybrid-preferred with source-first ordering
lexical fallback: enabled
time window: topic-defined research horizon
explain: required for all high-impact conclusions
transform mode: dry-run for synthesis template updates

Tuning Sequence

When adjusting any config:

1. Tune limit first to match operator review capacity.
2. Tune alpha and beta second to control resonance versus semantic emphasis.
3. Tune time window third to reduce noise without losing critical context.
4. Keep explain enabled while tuning to verify retrieval_path behavior.

End-to-End Sample Flows

Each flow uses the same four-step runtime pattern:

1. Ingest: store raw node events.
2. Recall: retrieve relevant context using scoped policy.
3. Explain: inspect retrieval path and score behavior.
4. Transform dry-run: preview rollups or embedding updates before mutation.

Flow A: Customer Support Resolution

step 1 ingest
  session_id: tenant::acme::agent::support-resolution::ticket-1042
  tier: raw
  content focus: issue summary, attempted fix, customer impact

step 2 recall
  tiers: ["raw", "daily", "weekly"]
  limit: 8
  alpha: 0.70
  beta: 0.30
  lexical fallback: enabled

step 3 explain
  verify retrieval_path is hybrid or lexical_fallback as expected
  confirm top results include prior resolutions for same issue family

step 4 transform dry-run
  operation: monthly_rollup
  dry_run: true
  expected: projected counts and no persisted mutations

Flow B: Revenue and Account Strategy

step 1 ingest
  session_id: tenant::acme::agent::revenue-strategy::2026-q2
  tier: raw
  content focus: objection notes, expansion signals, decision stakeholders

step 2 recall
  tiers: ["daily", "weekly", "raw"]
  limit: 12
  alpha: 0.60
  beta: 0.40
  lexical fallback: enabled

step 3 explain
  verify top recommendations are linked to supporting evidence
  verify contradictory signals are surfaced in results

step 4 transform dry-run
  operation: weekly_rollup
  dry_run: true
  expected: account momentum summary preview without writes

Flow C: Security Incident Triage

step 1 ingest
  session_id: tenant::acme-sec::agent::security-triage::inc-8891
  tier: raw
  content focus: alert details, IOC artifacts, containment actions

step 2 recall
  tiers: ["raw", "daily"]
  limit: 6
  alpha: 0.75
  beta: 0.25
  lexical fallback: enabled
  time window: incident_start..now

step 3 explain
  verify severity recommendation is traceable to matching indicators
  verify retrieval_path remains stable across repeated calls

step 4 transform dry-run
  operation: embed_backfill
  dry_run: true
  expected: selected node count for embedding without updates

Flow D: Clinical Operations Coordination

step 1 ingest
  session_id: tenant::north-hospital::agent::clinical-ops::shift-2026-05-05-n1
  tier: raw
  content focus: handoff notes, unresolved tasks, time-critical blockers

step 2 recall
  tiers: ["raw", "daily"]
  limit: 7
  alpha: 0.65
  beta: 0.35
  lexical fallback: enabled
  time window: active_shift_plus_prior_handoff

step 3 explain
  verify unresolved handoff items rank ahead of historical summaries
  verify retrieval_path reflects deterministic-first policy intent

step 4 transform dry-run
  operation: daily_rollup
  dry_run: true
  expected: shift summary preview for coordinator review

Flow E: Engineering Release Reliability

step 1 ingest
  session_id: tenant::acme-eng::agent::release-reliability::release-2.7.0
  tier: raw
  content focus: test failures, deploy checkpoints, rollback notes

step 2 recall
  tiers: ["raw", "daily", "weekly"]
  limit: 10
  alpha: 0.68
  beta: 0.32
  lexical fallback: enabled
  time window: current_release_plus_two_prior

step 3 explain
  verify recurring failure classes are present in top-ranked context
  verify promote or rollback suggestion is evidence-backed

step 4 transform dry-run
  operation: monthly_rollup
  dry_run: true
  expected: release stability trend preview with no writes

Flow F: Research and Policy Analysis

step 1 ingest
  session_id: tenant::policy-lab::agent::policy-analysis::housing-supply
  tier: raw
  content focus: source excerpts, claims, contradiction annotations

step 2 recall
  tiers: ["raw", "daily", "weekly"]
  limit: 14
  alpha: 0.55
  beta: 0.45
  lexical fallback: enabled
  ordering intent: source-first then synthesis

step 3 explain
  verify every high-impact conclusion maps to source-backed entries
  verify contradiction nodes are retained in top context window

step 4 transform dry-run
  operation: weekly_rollup
  dry_run: true
  expected: synthesis preview with confidence-bearing claim structure

Fast Adaptation Notes

1. Keep the four-step flow constant across domains.
2. Change session naming and time windows first for domain fit.
3. Change alpha and beta only after baseline explain traces are stable.
4. Keep transform in dry-run mode until rollback criteria are documented.

Choosing Your First Blueprint

1. High-volume interaction workflows: start with Customer Support Resolution.
2. Time-critical response workflows: start with Security Incident Triage.
3. Cross-team handoff workflows: start with Clinical Operations Coordination.
4. Decision-justification workflows: start with Research and Policy Analysis.

Then adapt the same STTP patterns to your domain-specific contracts and controls.

Troubleshooting

Triage Workflow

Use this sequence to reduce time-to-diagnosis:

1. Reproduce with the smallest failing input.
2. Classify failure domain: build/test, parser/validator, retrieval, or transform.
3. Run the command pack for that domain.
4. Capture escalation bundle before changing runtime defaults.

Command Packs

Build and Test Command Pack

Run from repository root:

cargo check --workspace --examples
cargo test --workspace
cargo test -p locus-core-rs --lib
cargo test -p locus-sdk

Use when:

1. CI fails but local state is unclear.
2. A release candidate needs a fast baseline confidence pass.

Parser and Validator Command Pack

Run from repository root:

cargo test -p locus-core-rs parser_tests
cargo test -p locus-core-rs validator_tests
cargo test -p locus-core-rs end_to_end_parsing_tests

Use when:

1. Ingestion rejects node payloads.
2. Strict profile compatibility appears unstable.

Retrieval Command Pack

Run from repository root:

cargo test -p locus-core-rs context_query_service_tests
cargo test -p locus-sdk --example memory_composition
cargo test -p locus-sdk --example provider_registry_setup

Use when:

1. retrieval_path behavior diverges from expected baseline.
2. recall and explain outputs are not aligned.

Transform and Sync Command Pack

Run from repository root:

cargo test -p locus-core-rs monthly_rollup_service_tests
cargo test -p locus-core-rs sync_coordinator_service_tests
cargo test -p locus-core-rs surrealdb_node_store_tests
cargo test -p locus-core-rs surrealdb_runtime_tests

Use when:

1. Backfill, rollup, or sync progression stalls.
2. selected/updated/skipped counts are inconsistent.

Build and Test Failures

Build/Test Symptom

cargo test fails in integration targets while unit tests pass.

Build/Test Actions

1. Run crate-scoped tests first to isolate failure domain.
2. Run failing target with single thread when test state leakage is suspected.
3. Compare behavior against known baseline in source repository if migrating.
4. Capture failing command output and the exact crate version set.

Build/Test Common Signatures

1. Integration tests fail while crate lib tests pass: likely environment or fixture state isolation issue.
2. Workspace build passes but host runtime fails: likely endpoint/config mismatch.

Parser and Validation Errors

Parser/Validation Symptom

Stored node rejected or strict parser fails.

Parser/Validation Actions

1. Confirm four-layer ordering is preserved.
2. Confirm content keys use field_name(.confidence) format.
3. Confirm content nesting depth does not exceed five.
4. Re-run parser and validator command pack and compare strict versus tolerant outcomes.

Parser/Validation Common Signatures

1. Missing required layer errors indicate malformed STTP spine.
2. Invalid key format errors indicate content keys not using field_name(.confidence).
3. Nesting depth errors indicate recursive payload exceeded protocol constraints.

Retrieval Behavior Mismatch

Retrieval Symptom

Unexpected recall ranking or fallback path.

Retrieval Actions

1. Log full scoring policy and scope.
2. Verify alpha and beta values are explicitly set.
3. Use explain workflow to inspect fallback triggers and stage counts.
4. Compare retrieval_path distribution to recent baseline window before changing policy defaults.

Retrieval Common Signatures

1. Unexpected lexical_fallback path often indicates sparse embeddings or strict filter scope.
2. Score drift with unchanged input usually indicates policy defaults changed between versions.

Transform/Migration Issues

Transform/Migration Symptom

Embedding backfill updates fewer nodes than expected.

Transform/Migration Actions

1. Run dry-run first and inspect selected counts.
2. Validate filter scope and has_embedding settings.
3. Validate provider capability routing and model configuration.
4. Verify checkpoint progression and sync cursor ordering when running incremental jobs.

Transform/Migration Common Signatures

1. Selected count near zero in dry-run usually indicates filter constraints are too narrow.
2. Update counts diverge from selected counts when provider or storage writes are failing.

Escalation Data To Capture

1. Exact request payload.
2. Retrieval path and fallback reason.
3. Validation and parser error text.
4. Relevant crate versions and commit sha.
5. Environment profile and runtime mode used during failure.

Rollback Triggers

Initiate rollback when any of the following occur:

1. Client-visible contract mismatch appears in migrated host routes.
2. Error rate or latency breaches remain sustained after initial mitigation window.
3. Parser or validator strict failures correlate with the latest release.

Escalation Bundle Template

Capture the following in one incident record:

1. Failing command and timestamp.
2. Environment profile: local, CI, staging, or production.
3. Request payload and redacted response sample.
4. retrieval_path or transform result counters.
5. Recent change references and suspected blast radius.

This bundle enables rapid handoff across development, platform, and on-call teams.

Versioning and Compatibility

Scope

This policy defines versioning expectations for Locus crates and host integrations.

It also defines dependency governance requirements used to keep releases stable over time.

Package Versioning

1. locus-core-rs and locus-sdk use SemVer.
2. Pre-1.0 versions may evolve rapidly but should still document behavior changes clearly.
3. Breaking changes require explicit migration notes in release documentation.

Compatibility Rules

1. SDK should remain additive where possible.
2. Core model changes should preserve parse/load compatibility for legacy node rows.
3. Host integrations should keep external endpoint/tool contracts stable during internal rewires.
4. Deprecated contract paths should remain available for at least one release cycle unless security or correctness requires immediate removal.

Release Channels

1. Patch: bug fixes, non-breaking behavior corrections, docs-only changes.
2. Minor: additive APIs, new optional workflows, new integration hooks.
3. Major: intentional API/contract breaks with migration guide.

Migration Guarantees

1. Every breaking change requires a migration section with before/after payload examples.
2. Behavior changes in retrieval or fallback policy must include validation evidence and parity notes.
3. Parser and validator contract shifts must include strict-profile regression tests.

Version Alignment

1. Workspace releases should keep crate versions intentionally aligned when shipped together.
2. If independent crate version bumps are required, release notes must explain compatibility implications.

Dependency Governance

Dependency Update Cadence

1. Security updates: prioritized and released on an expedited path.
2. Patch and minor dependency updates: batched on regular release cadence.
3. Major dependency updates: planned with explicit compatibility validation and rollback criteria.

Dependency Change Controls

1. Dependency changes must include rationale and impact scope in release notes.
2. Runtime-facing dependencies require integration smoke checks before release.
3. Changes affecting transport, storage, or crypto behavior require targeted regression coverage.

Security Patch Handling

1. Vulnerabilities with active exposure are patched on the nearest release path.
2. Mitigations and operational impact are documented with the release.
3. Consumers receive upgrade guidance when patch behavior changes are observable.

Compatibility Validation for Dependency Changes

1. Workspace compile and tests must pass.
2. Host runtime smoke checks must pass for MCP and gateway.
3. SDK example workflows must pass for reference integration patterns.
4. Generated technical docs must build without warnings requiring manual interpretation.

SemVer Range Management Policy

1. Internal workspace crate dependencies should use intentionally aligned versions at release time.
2. External dependencies should avoid overly broad ranges that mask breaking behavior.
3. Major version upgrades require explicit compatibility notes and rollout strategy.
4. Lockfile and resolved-version deltas should be reviewed with dependency change rationale.

Security and Data Handling

Security Posture

Locus is a memory implementation layer and may process sensitive conversational state. Treat all stored node content as potentially sensitive.

Data Classification Guidance

1. Treat raw node text and content layer fields as confidential by default.
2. Treat session identifiers and tenant identifiers as sensitive metadata.
3. Treat provider credentials and endpoint secrets as secrets at all times.

Secret Management

1. Never commit credentials, tokens, or endpoint secrets to source control.
2. Use environment variables or secret managers for runtime configuration.
3. Rotate credentials on incident response or role changes.

PII and Sensitive Content

1. Avoid storing direct PII when not required for product behavior.
2. Apply redaction/minimization before persistence where possible.
3. Ensure logs do not include raw secrets or unredacted sensitive payloads.

Transport and Storage Controls

1. Use TLS-enabled transports for remote database and provider calls.
2. Restrict database access using least privilege credentials.
3. Scope operations by tenant/session to avoid cross-tenant data leakage.

Operational Security Checklist

1. Validate dependency updates and lockfile changes during review.
2. Run parser/validator tests to avoid unsafe ingestion regressions.
3. Audit error paths for accidental raw payload leakage.
4. Keep incident response runbooks updated with containment steps.

Incident Handling Baseline

1. Contain by rotating keys and restricting affected endpoints.
2. Identify scope using session/tenant trace data.
3. Patch and verify with deterministic replay payloads.
4. Document remediation and prevention actions in release notes.

STTP Typed IR Language Specification

Formal Contract for Cognitive State Transfer

Status: Current Version: 1.1.0 Last Updated: 2026-04-25

1. Purpose and Position

STTP defines a typed intermediate representation for cognition state transfer between agents.

This specification defines:

1. the language surface,
2. the structural and semantic contract,
3. conformance profiles,
4. security and verification obligations.

This specification does not define:

1. storage backend implementation details,
2. sync authority policy,
3. retrieval ranking policy.

2. Normative Keywords

Keywords have normative meaning:

1. MUST and MUST NOT indicate mandatory requirements.
2. SHOULD and SHOULD NOT indicate strong recommendations.
3. MAY indicates optional behavior.

3. Conceptual Model

An STTP node is a four-layer typed artifact with ordered semantics.

Required semantic order:

1. Provenance: orientation and transmission contract.
2. Envelope: temporal identity and dual actor state.
3. Content: compressed meaning with confidence typing.
4. Metrics: quality and compression state verification.

Layer order is part of meaning. Reordering alters interpretation and invalidates strict conformance.

4. Structural Symbols

Reserved protocol symbols:

1. Node marker: ⏣
2. Provenance marker: ⊕⟨
3. Envelope marker: ⦿⟨
4. Content marker: ◈⟨
5. Metrics marker: ⍉⟨
6. Layer stop marker: ⟩

Implementations MUST treat these tokens as structural control symbols when encountered in structural positions.

5. Lexical Domain

5.1 Required Token Classes

A conforming lexer MUST recognize:

1. protocol markers,
2. braces, brackets, and parentheses,
3. delimiters,
4. identifiers,
5. numeric literals,
6. string literals,
7. boolean and null literals.

5.2 Identifiers

Canonical identifier regex:

• [A-Za-z_][A-Za-z0-9_]*

5.3 Scalars and Containers

Scalar values:

1. String
2. Number
3. Boolean
4. Null

Container values:

1. Object
2. Array

6. Grammar Skeleton

The following grammar is normative at the layer level.

Node             := ProvenanceLayer EnvelopeLayer ContentLayer MetricsLayer

ProvenanceLayer  := PROVENANCE_START NodeObject LAYER_END
EnvelopeLayer    := ENVELOPE_START   NodeObject LAYER_END
ContentLayer     := CONTENT_START    NodeObject LAYER_END
MetricsLayer     := METRICS_START    NodeObject LAYER_END

NodeObject       := OptionalNodeMarker Object
OptionalNodeMarker := NODE_MARKER [NodeOrdinal]
NodeOrdinal      := Integer

Object           := '{' Members? '}'
Members          := Pair (',' Pair)*
Pair             := Key ':' Value
Key              := Identifier | String
Value            := Object | Array | String | Number | Boolean | Null
Array            := '[' (Value (',' Value)*)? ']'

Note:

1. legacy compact separators are non-canonical and profile-scoped to tolerant mode.

7. Layer Schema Contract

7.1 Provenance Layer

Required keys:

1. trigger
2. response_format
3. origin_session
4. compression_depth
5. parent_node
6. prime

trigger enum:

1. scheduled
2. threshold
3. resonance
4. seed
5. manual

response_format enum:

1. temporal_node
2. natural_language
3. hybrid

prime required keys:

1. attractor_config
2. context_summary
3. relevant_tier
4. retrieval_budget

attractor_config required keys:

1. stability
2. friction
3. logic
4. autonomy

7.2 Envelope Layer

Required keys:

1. timestamp
2. tier
3. session_id
4. user_avec
5. model_avec

Optional keys:

1. schema_version

tier enum:

1. raw
2. daily
3. weekly
4. monthly
5. quarterly
6. yearly

AVEC required keys:

1. stability
2. friction
3. logic
4. autonomy
5. psi

7.3 Content Layer

Required shape:

1. typed object containing one or more semantic fields.

Canonical field signature:

1. field_name(.confidence): value

confidence rules:

1. confidence MUST parse as float.
2. strict range is [0.0, 1.0].

depth rule:

1. strict maximum nesting depth is 5.

7.4 Metrics Layer

Required keys:

1. rho
2. kappa
3. psi
4. compression_avec

compression_avec required keys:

1. stability
2. friction
3. logic
4. autonomy
5. psi

8. Semantic Invariants

Strict conformance requires all invariants below.

S1 Layer order invariant:

1. provenance precedes envelope.
2. envelope precedes content.
3. content precedes metrics.

S2 Completeness invariant:

1. all required keys exist in all required layers.

S3 Type invariant:

1. enum values are members of defined enum sets.
2. required numeric fields parse as numeric values.

S4 Confidence invariant:

1. every canonical content field includes confidence annotation.
2. every confidence annotation is range-valid in strict mode.

S5 Structural depth invariant:

1. content object nesting depth is less than or equal to 5.

A node that violates any invariant MUST be marked strict-invalid.

9. Compliance Profiles

9.1 Strict Profile

Strict profile MUST:

1. enforce all syntax and invariants,
2. fail closed on unresolved structure,
3. reject non-canonical legacy forms.

9.2 Tolerant Profile

Tolerant profile MAY accept legacy forms including:

1. compact pipe-delimited key-value payloads,
2. omitted internal node wrappers,
3. unquoted enum-like values where unambiguous,
4. mixed formatting and spacing variants.

Tolerant profile MUST:

1. preserve raw source,
2. emit diagnostics for every deviation,
3. produce canonical AST when recoverable,
4. never claim strict-valid unless strict invariants pass.

10. Diagnostics Specification

Recommended severities:

1. Fatal
2. Error
3. Warning
4. Info

Diagnostic payload SHOULD include:

1. code,
2. message,
3. source span,
4. recovery action,
5. strict impact.

11. Canonicalization Rules

When tolerant recovery is used, implementation SHOULD canonicalize:

1. layer ordering in AST output,
2. known aliases into canonical keys,
3. parent_node into normalized representation,
4. unambiguous numeric strings into numeric values.

Canonicalization MUST NOT:

1. modify original raw text,
2. alter identity-sensitive semantics,
3. fabricate missing semantic values.

12. Adversarial and Security Considerations

Threat classes:

1. marker spoofing and layer confusion,
2. malformed confidence signatures,
3. delimiter smuggling in compact forms,
4. layer bleed-through via unbalanced containers.

Required response:

1. strict profile fails closed,
2. tolerant profile fails informative with bounded recovery,
3. diagnostics remain deterministic for equal input.

13. Backward Compatibility Contract

Language evolution policy is additive.

Rules:

1. optional fields MAY be introduced without invalidating old nodes.
2. old nodes MAY be ingested via tolerant profile.
3. strict profile remains the canonical generation target.

14. Conformance Testing Requirements

A conformance suite SHOULD include:

1. strict canonical fixtures,
2. tolerant legacy fixtures,
3. negative fixtures by invariant class,
4. adversarial fixtures by threat class,
5. canonical AST snapshot expectations.

15. Canonical Example

⊕⟨ ⏣0{ trigger: manual, response_format: temporal_node, origin_session: "session-abc", compression_depth: 1, parent_node: null, prime: { attractor_config: { stability: 0.90, friction: 0.20, logic: 0.98, autonomy: 0.85 }, context_summary: "parser hardening session", relevant_tier: raw, retrieval_budget: 8 } } ⟩
⦿⟨ ⏣0{ timestamp: "2026-04-25T00:00:00Z", tier: raw, session_id: "session-abc", schema_version: "sttp-1.0", user_avec: { stability: 0.90, friction: 0.20, logic: 0.98, autonomy: 0.85, psi: 2.93 }, model_avec: { stability: 0.90, friction: 0.20, logic: 0.98, autonomy: 0.85, psi: 2.93 } } ⟩
◈⟨ ⏣0{ focus(.99): "grammar update", decision(.96): { parser_mode(.95): "strict_and_tolerant" } } ⟩
⍉⟨ ⏣0{ rho: 0.95, kappa: 0.94, psi: 2.93, compression_avec: { stability: 0.90, friction: 0.20, logic: 0.98, autonomy: 0.85, psi: 2.93 } } ⟩

16. Change Control

When schema or grammar rules change, maintainers SHOULD update:

1. this language specification,
2. sttp_grammar_blueprint.md,
3. parser conformance fixtures,
4. parser implementation notes.

17. Language Promise

STTP as typed IR is complete only when it can do all three simultaneously:

1. represent cognition state faithfully,
2. validate structure deterministically,
3. survive historical reality without semantic drift.