Skip to main content

Architecture

Technical overview of how Align works under the hood.

System Overview

┌─────────────────────────────────────────────────────────────────────┐
│ Your Tools │
│ Slack · Teams · Jira · GitHub · Linear │
└──────────────────────────────┬──────────────────────────────────────┘
│ webhooks / APIs

┌─────────────────────────────────────────────────────────────────────┐
│ Connectors (MCP) │
│ Each connector speaks one platform's API and normalizes data │
└──────────────────────────────┬──────────────────────────────────────┘


┌─────────────────────────────────────────────────────────────────────┐
│ Gateway (Fastify) │
│ API server · Job orchestration · SSE streaming · Business logic │
│ │
│ ┌─────────────┐ ┌──────────────┐ ┌───────────────────────────┐ │
│ │ Import │ │ Bulk Approval│ │ Relationship Candidate │ │
│ │ Worker │ │ Worker │ │ Detector │ │
│ └─────────────┘ └──────────────┘ └───────────────────────────┘ │
└──────────────────────────────┬──────────────────────────────────────┘

┌──────────┴──────────┐
▼ ▼
┌──────────────────────────┐ ┌──────────────────────────┐
│ PostgreSQL (RLS) │ │ Brain (FastAPI) │
│ Decisions · Snapshots │ │ LLM analysis │
│ Relationships · Jobs │ │ Embeddings │
│ Candidates · Telemetry │ │ Relationship detection │
└──────────────────────────┘ └──────────────────────────┘

Services

ServiceStackPurpose
UISvelteKit 5Frontend application
GatewayTypeScript / FastifyAPI server, job orchestration, business logic
BrainPython / FastAPILLM analysis, embeddings, AI-powered relationship detection
ConnectorsTypeScript / MCPPlatform-specific integrations (Slack, Teams, Jira, GitHub, Linear)
PostgreSQL15+Primary data store with Row-Level Security (RLS)

Data Flow

Real-time Capture

When someone mentions @align in Slack or Teams:

User mentions @align → Connector receives webhook → Gateway creates Decision Snapshot
→ Brain analyzes for relationships
→ Links to related decisions created

Discover (Historical Scan)

Discover is the pipeline for importing historical decisions from your existing tools. It runs in three stages:

Stage 1: Scan

UI starts scan → Gateway creates import job
→ Connector fetches items from platform API (paginated)
→ Brain analyzes each batch for decision-like content
→ Gateway stores suggestions in import_suggestions table
→ SSE streams progress to UI in real-time

The scan pipeline processes items in batches with adaptive chunking that limits both item count (30 per chunk) and payload size (500KB per chunk) to prevent timeouts.

Stage 2: Cross-Reference Detection

During the scan, the gateway automatically detects cross-platform references between suggestions:

┌──────────────────────────────────────────────────────────────┐
│ Suggestion A (Jira) Suggestion B (Slack) │
│ source_url: .../PROJ-123 text: "...see PROJ-123..." │
│ metadata: {issue_key: metadata: {issue_key: │
│ "PROJ-123"} "PROJ-123"} │
│ │
│ ──── deterministic match ──── │
│ Stored as relationship_candidate │
│ confidence: 1.0 │
└──────────────────────────────────────────────────────────────┘

Deterministic matchers (confidence = 1.0):

  • Jira keys - PROJ-123 found in both a Jira ticket and a Slack message
  • GitHub PR/Issue references - org/repo#42 or full GitHub URLs
  • Slack thread timestamps - Same thread referenced from multiple sources

These precomputed candidates are stored in the relationship_candidates table, ready for instant promotion when the user approves.

Stage 3: Review & Approve

The user reviews AI-suggested decisions and approves them. Approval uses a two-phase flow for maximum speed:

User clicks "Approve All"


┌─── Phase 1: Instant (< 1 second) ────────────────────────────┐
│ 1. Create all Decision Snapshots from approved suggestions │
│ 2. Look up precomputed relationship_candidates │
│ 3. Resolve suggestion IDs → decision IDs │
│ 4. Create decision_links for all precomputed matches │
│ 5. Stream "Linked: relates (N instant connections)" via SSE │
└──────────────────────────────────────────────────────────────┘


┌─── Phase 2: Background (seconds to minutes) ─────────────────┐
│ 1. Filter out suggestions that already have precomputed links│
│ 2. Send ONLY unlinked decisions to Brain for LLM analysis │
│ 3. Brain discovers semantic/contextual relationships │
│ 4. Stream "Discovered: relates (N connections found)" via SSE│
└──────────────────────────────────────────────────────────────┘

Why two phases? Deterministic cross-references (same Jira key in two items) are facts, not opinions - they don't need LLM analysis. By promoting them instantly, the user sees connections appear within milliseconds of clicking approve. The LLM only runs for items where no obvious link exists, keeping bulk approval fast even for hundreds of decisions.

Database Schema (Key Tables)

TablePurpose
decisionsCore decision snapshots
decision_linksRelationships between decisions (relates, supersedes, conflicts, duplicates)
import_jobsDiscover scan job state and progress
import_suggestionsAI-suggested decisions pending review
relationship_candidatesPrecomputed cross-platform links (pending promotion at approval time)
connectorsConfigured integrations
tenantsOrganizations / workspaces

Row-Level Security (RLS)

All tenant data is isolated using PostgreSQL RLS. Every query runs within a withTenant() wrapper that sets app.current_tenant for the session:

-- Automatically applied to all queries
CREATE POLICY tenant_isolation ON decisions
USING (tenant_id = current_setting('app.current_tenant')::uuid);

Job Processing

Architecture Options

DeploymentQueueWorkerUse Case
Single-podIn-memoryEmbedded in gatewayDevelopment, small teams
Multi-podAWS SQSEmbedded in gatewayProduction, scaling

The gateway includes an embedded worker that processes jobs directly when no external queue is configured. This means self-hosted single-pod deployments work out of the box with zero additional infrastructure.

Job Types

JobTriggerWhat It Does
Import JobUser starts Discover scanFetches items from connector, sends to Brain for analysis, stores suggestions
Bulk ApprovalUser approves suggestionsTwo-phase: instant link promotion + background LLM analysis

SSE (Server-Sent Events)

Both scan and approval progress are streamed to the UI via SSE:

Gateway ──SSE──▶ UI

├─ scan_progress: items scanned, suggestions found
├─ scan_complete: scan finished
├─ approval_started: bulk approval begun
├─ relationship_found: links discovered (Phase 1 or Phase 2)
├─ analyzing: LLM analysis progress
└─ approval_complete: all done

For multi-pod deployments, Redis pub/sub ensures SSE events reach the correct pod regardless of which pod processes the job.

Self-Hosting Considerations

What Works Out of the Box

Self-hosted deployments get the full Discover and bulk approval pipeline with zero additional configuration:

  • Embedded worker - No separate worker process needed
  • In-memory job queue - No SQS/Redis required for single-pod
  • Idempotent migrations - All migrations use IF NOT EXISTS / CREATE OR REPLACE for safe re-runs
  • Automatic fallbacks - No Redis? In-memory pub/sub. No SQS? Direct processing.

When to Add Infrastructure

ScenarioAddWhy
Multiple gateway podsRedisSSE pub/sub across pods, shared job state
High-volume (1000s decisions/day)SQSDurable job queue, retry handling
Large scans (10k+ items)PgBouncerConnection pooling for parallel processing

Performance Expectations

OperationSingle-PodMulti-Pod
Scan (100 items)~30 seconds~30 seconds
Scan (1000 items)~3 minutes~3 minutes
Bulk approve (50 items, with precomputed links)< 2 seconds (Phase 1)< 2 seconds (Phase 1)
Bulk approve (50 items, LLM analysis)~30 seconds (Phase 2)~30 seconds (Phase 2)

LLM Configuration

The Brain service requires an LLM for:

  • Scan analysis - Identifying decisions in source content
  • Relationship detection - Finding semantic connections (Phase 2 only)
  • Summarization - Generating decision titles and descriptions

See LLM Setup for configuring OpenAI, Anthropic, or self-hosted models (Ollama, vLLM).

Cost optimization: Precomputed deterministic links (Phase 1) skip LLM entirely, reducing API costs. For teams with strong cross-referencing habits (linking Jira tickets in Slack, referencing PRs in issues), the majority of relationships are detected deterministically.

Relationship Detection

Align detects relationships between decisions using multiple strategies:

Deterministic (Phase 1 - Instant)

Cross-platform reference matching with 100% confidence:

PatternExampleDetected From
Jira keyPROJ-123URLs, text content, metadata
GitHub PRorg/repo#42 or full URLURLs, text content
GitHub Issueorg/repo#10 or full URLURLs, text content
Slack threadThread timestampURLs

LLM-Powered (Phase 2 - Background)

For decisions without deterministic links, the Brain service uses LLM analysis to find:

  • Semantic similarity - Decisions about the same topic
  • Supersession - Newer decisions replacing older ones
  • Conflicts - Contradictory decisions
  • Duplicates - Same decision captured from different sources

Relationship Types

TypeMeaning
relatesGeneral relationship (most common)
supersedesThis decision replaces an older one
conflicts_withThese decisions contradict each other
duplicatesSame decision captured from different sources
refinesAdds detail or narrows scope of another decision
clarifiesResolves ambiguity in another decision
questionsRaises concerns or open questions about a decision