Current operating model
Refresh scheduling is request-driven. The inventory API materializes a workspace, and candidate generation has bounded pair examination but is not an incremental background index. PostgreSQL stores ordinary audit records without an external immutable archive. There is no measured production-scale claim in these docs.
Target architecture — planned
Bounded scheduler + durable collection jobs↓ scoped credentials + leases + retry budgetValidated source snapshot / monotonic generation↓ atomic observations + transactional outboxTenant-scoped, rule-versioned candidate index↓ paginated review + explicit decision commandBinding + decision + audit commit
These are design goals, not components implied by the current deployment. Keep collection and candidate generation separate from the decision authority. A failed worker must never invent ownership or automatically resolve a conflict.
Durable job requirements
- Partition work by tenant, connection and source generation.
- Use leases, bounded retries, deadlines, rate limits and operator-visible dead letters.
- Define idempotency by completed snapshot generation, not by mutable display name.
- Publish invalidations only after commit through an outbox; tolerate duplicate delivery.
- Keep credentials outside job payloads and matching indexes.
Before claiming scale
- Load-test realistic tenant sizes, duplicate-name groups and saved decision histories.
- Measure queue completeness/limit notices, memory, lock wait, refresh latency and retry pressure.
- Add paginated evidence/history and stable cursor semantics.
- Test crash recovery between collection, validation, commit and event publication.
- Version rule/index migrations and prove that stale confirmations fail safely.
- Set source-specific freshness and completeness contracts before introducing age-based link policies.