MnemoLayer

Sub-300ms recall p95

Zero data loss (3 regions)

79 tests / 17 properties

7 serverless endpoints

Overview

Drop-in persistent memory layer for AI agents, built on CockroachDB and deployed serverless on AWS. Agents get four human-like memory tiers (working, episodic, semantic, procedural) that survive region failure with zero data loss, queryable by any MCP-compatible client. The demo vehicle is an SRE on-call agent that recalls a past incident to resolve a new one in real time. When Bedrock embedding times out, recall degrades gracefully to keyword search instead of failing. The entire system runs serverless: Lambda behind API Gateway with Terraform IaC, CockroachDB Cloud across us-east-1, us-west-2, and eu-west-1, and a Next.js frontend on Vercel.

Architecture Diagram

System Architecture

Memory Tiers

Data Flow

AWS Deployment

Design Decisions

• →CockroachDB as the sole system of record for all memory tiers. Not DynamoDB, not Redis, not a separate vector database. CockroachDB's multi-region Raft consensus gives automatic failover with zero data loss, and its distributed vector index (pgvector-compatible) handles semantic similarity search without another service.
• →Four distinct memory tiers modeled on human cognition: working (session scratchpad with TTL eviction), episodic (time-ordered event log), semantic (meaning-indexed long-term knowledge via 1024-dim Titan Embed v2 vectors), and procedural (learned skills, upserted by skill name). Each tier maps to a dedicated table with appropriate indexes.
• →Serverless Memory API on Lambda behind API Gateway instead of embedding memory logic inside the agent. This makes memory framework-agnostic: any HTTP or MCP client can use it. The agent writes via HTTP, reads via the CockroachDB MCP Server (read-only, audited).
• →Bedrock Converse API for all LLM calls, invoke_model for embeddings (Titan Embed doesn't support Converse). Guardrails with a numbered version filter prompt injection and system-prompt extraction. The agent has no destructive tools: it can recall, write memory, and consolidate. Nothing else.
• →Property-based testing (fast-check + hypothesis) for 17 correctness properties including: the API key never reaches the browser, request_id is forwarded byte-for-byte, mock mode issues zero network calls, recall input is validated and clamped, and keyword-fallback maps to degraded state (not error).
• →Server-side proxy pattern: the frontend calls same-origin Next.js Route Handlers, which attach the API key and forward to API Gateway. The browser never sees the key. CORS, auth, timeouts, and error mapping all handled at the proxy layer.

Deployment

Infrastructure defined in Terraform (lambda.tf, api_gateway.tf, iam.tf, secrets.tf, s3.tf). A single terraform apply deploys: Lambda (Python 3.12, 60s timeout, 512MB), API Gateway (REST, API key auth, rate limiting at 50 req/s burst 100), IAM (least-privilege scoped to exact Bedrock model ARNs with confused-deputy protection), Secrets Manager (CockroachDB DSN), and S3 (encrypted artifact storage). CockroachDB Cloud is provisioned separately via ccloud CLI across us-east-1, us-west-2, eu-west-1. The frontend deploys to Vercel with zero config. Lambda dependencies (psycopg2, pydantic) are pip-installed into the zip at build time via a Terraform null_resource provisioner.

Lessons Learned

The biggest decision was making CockroachDB the sole memory store instead of splitting across DynamoDB and a vector database. CockroachDB's vector index handles similarity search, its multi-region replication handles resilience, and parameterized SQL keeps things simple. One database, one source of truth. The keyword-fallback pattern for recall proved essential during development: Bedrock embedding calls occasionally take 5+ seconds under load, and a hard failure there would kill the demo. Instead, the handler falls back to substring search over episodic memory and returns mode: keyword-fallback so the UI shows a degraded notice instead of an error. The server-side proxy pattern solved a real security problem: the .env.example originally had the API key in a NEXT_PUBLIC_ variable, which would have inlined it into the browser bundle for every visitor to extract. Moving to Route Handlers with a non-public env var fixed it without changing the frontend's calling pattern.