--- url: /events/appending.md --- # Appending Events Polecat provides several ways to append events to streams. ## Starting a New Stream Create a new event stream with initial events: ```cs // With explicit ID var streamId = Guid.NewGuid(); session.Events.StartStream(streamId, new QuestStarted("Destroy the Ring"), new MembersJoined("Rivendell", ["Frodo", "Sam"]) ); // With auto-generated ID var streamId = session.Events.StartStream( new QuestStarted("Destroy the Ring") ); // String stream IDs (when StreamIdentity = AsString) session.Events.StartStream("quest-123", new QuestStarted("Destroy the Ring") ); ``` `StartStream` will throw if a stream with the same ID already exists. ## Appending to an Existing Stream ```cs session.Events.Append(streamId, new MembersJoined("Moria", ["Gimli", "Legolas"]), new MembersDeparted("Moria", ["Gandalf"]) ); await session.SaveChangesAsync(); ``` ## Optimistic Concurrency Append with an expected version to detect concurrent modifications: ```cs session.Events.Append(streamId, expectedVersion: 4, new MembersDeparted("Amon Hen", ["Boromir"]) ); // Throws EventStreamUnexpectedMaxEventIdException // if current stream version != 4 await session.SaveChangesAsync(); ``` ## FetchForWriting Load an aggregate and append events with built-in version checking: ```cs var stream = await session.Events.FetchForWriting(streamId); // stream.Aggregate is the current state // stream.CurrentVersion is the current version stream.AppendOne(new MembersDeparted("Amon Hen", ["Boromir"])); await session.SaveChangesAsync(); ``` ## FetchForExclusiveWriting Pessimistic locking with `UPDLOCK HOLDLOCK` for exclusive access: ```cs var stream = await session.Events.FetchForExclusiveWriting(streamId); // The stream row is locked until the transaction completes stream.AppendOne(new MembersJoined("Gondor", ["Faramir"])); await session.SaveChangesAsync(); ``` ## Enforcing Consistency Without Appending Events In some command handling scenarios, your business logic may evaluate the current aggregate state and decide that no new events need to be emitted. By default, if no events are appended to the stream returned by `FetchForWriting()`, Polecat will not perform any concurrency check when `SaveChangesAsync()` is called. This means that if another process has modified the stream between your fetch and save, you won't know about it. If you need to guarantee that the stream has not been modified even when your handler doesn't emit events, you can set `AlwaysEnforceConsistency = true` on the stream: ```cs var stream = await session.Events.FetchForWriting(command.OrderId); // Tell Polecat to enforce the optimistic concurrency check // even if we don't append any events stream.AlwaysEnforceConsistency = true; var order = stream.Aggregate; // Business logic that may or may not produce events if (order.NeedsUpdate(command)) { stream.AppendOne(new OrderUpdated(command.Data)); } // If no events were appended, Polecat will still verify that the // stream version hasn't changed since FetchForWriting() was called. // Throws EventStreamUnexpectedMaxEventIdException if another process modified the stream. await session.SaveChangesAsync(); ``` When `AlwaysEnforceConsistency` is `true`: * **If events are appended**, Polecat behaves exactly as before -- the normal optimistic concurrency check is applied. * **If no events are appended**, Polecat issues an `AssertStreamVersion` check that reads the current stream version from the database and throws an `EventStreamUnexpectedMaxEventIdException` if it doesn't match the version that was fetched. This is useful in workflows where: * A command handler conditionally emits events and you need to know if another process raced ahead * You want to implement "read-then-validate" patterns where consistency of the read matters even without writes * You're building saga or process manager patterns where skipping an event is a valid but concurrency-sensitive outcome ## WriteToAggregate Fetch, apply, and save in a single call: ```cs await session.Events.WriteToAggregate(streamId, stream => { stream.AppendOne(new MembersDeparted("Mordor", ["Frodo", "Sam"])); }); await session.SaveChangesAsync(); ``` ## QuickAppend Polecat uses **QuickAppend** exclusively -- all event appending is done via direct SQL `INSERT` statements with an `UPDATE...OUTPUT` pattern for atomic version management. No stored procedures are involved. The flow: 1. `INSERT` new events into `pc_events` 2. `UPDATE pc_streams SET version = version + @count OUTPUT INSERTED.version` for version management 3. Both operations run in the same transaction as document operations via `SaveChangesAsync()` ## Event Metadata Set correlation and causation IDs via session options: ```cs await using var session = store.LightweightSession(new SessionOptions { CorrelationId = "request-123", CausationId = "command-456" }); session.Events.Append(streamId, new QuestEnded("Destroy the Ring")); await session.SaveChangesAsync(); ``` Custom headers can be added to individual events via the `Headers` property on `StreamAction`. --- --- url: /events/archiving.md --- # Archiving Streams Polecat supports archiving event streams to logically remove them from active queries without permanently deleting the data. ## Archiving a Stream ```cs session.Events.ArchiveStream(streamId); await session.SaveChangesAsync(); ``` This sets `is_archived = 1` on both the `pc_streams` and `pc_events` tables for the stream. ## Effects of Archiving When a stream is archived: * `FetchStreamAsync` excludes the archived stream * The async daemon's event loader skips archived events * Attempting to append to an archived stream throws `InvalidStreamException` ## Unarchiving a Stream Restore an archived stream: ```cs session.Events.UnArchiveStream(streamId); await session.SaveChangesAsync(); ``` This sets `is_archived = 0` on both tables, making the stream active again. ## Tombstoning (Hard Delete) For permanent removal of a stream and all its events: ```cs session.Events.TombstoneStream(streamId); await session.SaveChangesAsync(); ``` ::: warning Tombstoning permanently `DELETE`s the stream record and all associated events from the database. This cannot be undone. ::: Tombstoning works with both Guid and string stream IDs. ## Archiving vs Tombstoning | Operation | Reversible | Data Preserved | Use Case | | :--- | :--- | :--- | :--- | | Archive | Yes | Yes | Soft removal, compliance holds | | Tombstone | No | No | GDPR right to erasure, cleanup | --- --- url: /events/projections/async-daemon.md --- # Asynchronous Projections The async daemon is a background service that processes events and applies projections asynchronously, providing eventually consistent read models. ## How It Works 1. The **High Water Mark Detector** monitors `pc_events` for new events using SQL `LEAD()` window functions to detect gaps 2. The **Event Loader** fetches batches of events for processing 3. Each projection processes its batch and updates its read model 4. Progress is tracked in `pc_event_progression` via atomic `MERGE` statements ## Enabling the Async Daemon Register projections with async lifecycle: ```cs var store = DocumentStore.For(opts => { opts.Connection("..."); opts.Projections.Snapshot(SnapshotLifecycle.Async); opts.Projections.Add(ProjectionLifecycle.Async); }); ``` When wired up through `AddPolecat()`, opt the daemon into the host's lifetime explicitly with `AddAsyncDaemon(DaemonMode)`: ```cs builder.Services.AddPolecat(opts => { opts.Connection("..."); opts.Projections.Snapshot(SnapshotLifecycle.Async); opts.Projections.Add(ProjectionLifecycle.Async); }) .AddAsyncDaemon(DaemonMode.Solo) // start the daemon as IHostedService .ApplyAllDatabaseChangesOnStartup(); // run schema migration at boot ``` ::: tip Async projections do **not** run unless you call `AddAsyncDaemon(...)`. Use `DaemonMode.Solo` for single-node deployments and `DaemonMode.HotCold` for multi-node deployments where only one host should own each projection shard. ::: ## Daemon Settings Configure daemon behavior: ```cs opts.DaemonSettings.StaleSequenceThreshold = 1000; ``` ### Polling Unlike Marten's PostgreSQL `LISTEN/NOTIFY`, Polecat uses **polling** to detect new events: ```cs // The daemon polls for new events at a configurable interval // Default: 500ms ``` ## Graceful Shutdown and the Drain Timeout When a projection or subscription shard is stopped, the daemon does not simply cancel it. It first tries to *drain* the agent: let the in-flight page of events finish being applied, then flush the shard's progression row so the next start picks up exactly where this one left off. `StopAndDrainTimeout` bounds how long the daemon waits for that drain on **a single** shard: ```cs // The default is 5 seconds opts.Projections.StopAndDrainTimeout = TimeSpan.FromSeconds(30); ``` The bound applies to every stop path: stopping one agent, stopping all agents (the `SIGTERM`/host shutdown path), and the internal stop-if-already-running replacement that happens when an agent is reassigned. **Why you would raise it.** If the drain is cut off before the progression flush lands, the shard restarts against a stale progression row and throws `ProgressionProgressOutOfOrderException` on its next start. Raise the timeout when in-flight batches legitimately take longer than five seconds — a large `BatchSize`, expensive projection code, heavy rebuild load, or a slow or contended SQL Server. This is most visible shutting down a host with a large agent universe: a [database-per-tenant](/documents/multi-tenancy) deployment with thousands of (projection × tenant) shards all draining inside a Kubernetes termination grace window. ::: tip A per-shard bound is only useful if the process lives long enough to spend it. Match a raised `StopAndDrainTimeout` with the host's own `HostOptions.ShutdownTimeout` and, on Kubernetes, the pod's `terminationGracePeriodSeconds`. ::: **Why you would lower it.** A deployment that would rather cut a wedged shard loose quickly and take the progression replay hit — to keep node failover and reassignment latency low, for instance — can set it below the default. **Opting out.** `Timeout.InfiniteTimeSpan`, or any non-positive value, removes the separate bound so the drain is limited only by the daemon's own cancellation. Be aware that this means a genuinely wedged shard can hold up shutdown indefinitely. ## Waiting for Non-Stale Data ### CatchUpAsync Wait for all projections to catch up to the current high water mark: ```cs await store.WaitForNonStaleProjectionDataAsync(TimeSpan.FromSeconds(30)); ``` ### Per-Query Wait for projections before a specific query: ```cs var orders = await session.Query() .QueryForNonStaleData() .Where(x => x.Status == "Active") .ToListAsync(); ``` ## Event Progression Track daemon progress: ```cs // The pc_event_progression table stores: // - name: Projection/subscription name // - last_seq_id: Last processed sequence ID // - last_updated: When last updated ``` ## High Water Mark Detection The high water mark detector uses SQL Server's `LEAD()` window function to detect sequence gaps in the event log. This prevents the daemon from processing events out of order when concurrent writers create gaps. ## Listening for Daemon Commits Sometimes you need to run a side effect **after** an aggregate has been durably updated by the async daemon — the canonical case is invalidating a cache key. Doing this before the commit would open a window where a concurrent read could repopulate the cache with stale state. Subscriptions, composite projections, and projection side effects all run *before* the batch is committed, so they can't close that window. Register an `IChangeListener` on `Projections.AsyncListeners` to hook the commit boundary of each daemon projection batch: ```cs public class CacheFlushingListener : IChangeListener { // Runs AFTER the batch is committed → "at most once". Ideal for cache invalidation. public async Task AfterCommitAsync(IDocumentSession session, IChangeSet commit, CancellationToken token) { foreach (var party in commit.Updated.OfType()) { await _cache.RemoveAsync($"quest-party:{party.Id}", token); } } // Runs BEFORE the batch is committed → "at least once". public Task BeforeCommitAsync(IDocumentSession session, IChangeSet commit, CancellationToken token) => Task.CompletedTask; } ``` ```cs var store = DocumentStore.For(opts => { opts.ConnectionString = connectionString; opts.Projections.Snapshot(SnapshotLifecycle.Async); // Fires only within the async daemon, once per committed projection batch opts.Projections.AsyncListeners.Add(new CacheFlushingListener()); }); ``` The `commit` parameter is an [`IChangeSet`](/documents/sessions#the-ichangeset) describing the projected documents written in that batch (`Inserted` / `Updated` / `Deleted`). Delivery semantics mirror Marten: * **`AfterCommitAsync`** runs once, *after* the transaction commits — **at most once**. A faulting after-commit listener is swallowed so the batch is not reprocessed (which would re-fire the side effect); the data is already durable. * **`BeforeCommitAsync`** runs *before* the commit — **at least once**. A throw here aborts the batch before anything is committed. ::: tip Async listeners are **suppressed during projection rebuilds**. A full replay re-applies every event, so firing post-commit side effects for each historical batch is almost never what you want. ::: ## Blue/Green Deployments with Projection Versioning Every projection carries a `Version` (default `1`). The version is baked into the shard's progression identity, so different versions of the same projection track their progress independently: ```cs public class TripProjection : SingleStreamProjection { public TripProjection() { // Bump the version when you change the projection's logic or shape Version = 3; } } ``` | Projection | `Version` | Progression identity (`pc_event_progression.name`) | |------------|-----------|-----------------------------------------------------| | `Trip` | `1` | `Trips:All` | | `Trip` | `2` | `Trips:V2:All` | | `Trip` | `3` | `Trips:V3:All` | Because each version has its own progression row, you can stand up a new version alongside the old one and let it rebuild from zero while the old version keeps serving reads — a blue/green deployment. ### Gating Side Effects Behind the Prior Version The catch with a blue/green rebuild is [side effects](side-effects.md). If your projection publishes messages or emits other side effects from `RaiseSideEffects()`, a new version replaying the full event history from zero would **re-fire every side effect** for events the previous version already processed — duplicate emails, duplicate messages, and so on. Opt into the **side-effect gate** to prevent that: ```cs public class TripProjection : SingleStreamProjection { public TripProjection() { Version = 3; // Suppress side effects for events the prior version already processed Options.GateSideEffectsBehindPriorVersion = true; } } ``` When the daemon starts a gated shard whose own progression is **behind** the highest prior version's persisted mark `N`, it: 1. Replays the new version from its current position up to `N` in **Rebuild mode**, with side effects **suppressed** (the projected documents are still built — only the side effects are gated). 2. Hands off to **Continuous** execution from `N`, where side effects fire normally. The net effect: side effects fire exactly once, only for events past `N` — the events the previous version never saw. ### Semantics and Edge Cases * **Trigger is "own progress `< prior mark`".** The gate keys off the new version's own persisted progression, so an interrupted warm-up **resumes suppressed** rather than re-emitting. Restarting the daemon after a crash mid-warm-up picks up from the recorded position with side effects still gated. * **Failed warm-up pauses the shard.** If the suppressed replay throws (e.g. a poison event under a pausing error policy), the shard is left **paused** with the exception attached, no continuous execution starts, and no side effects fire. Restarting the daemon resumes the warm-up from its persisted progress. * **`Version == 1` and the flag-off case are inert.** The gate is skipped entirely unless `Version > 1` **and** `GateSideEffectsBehindPriorVersion` is set — a v1 projection behaves exactly as it always has. * **`SubscribeFromPresent` is incompatible.** A shard that subscribes from "present" ignores persisted progression, so the gate cannot reason about a prior mark. The gate is skipped and a warning is logged; use versioning-with-gate *or* subscribe-from-present, not both. * **Overlap window.** The prior mark `N` is snapshotted when the new version starts. If the **old** version is still running and advances past `N` while the new version warms up, the events in `(N, old_final]` can be processed by both versions — an accepted duplicate window. Stop the old version (or accept the small overlap) when you need exactly-once across the cutover. ::: tip The gate suppresses **side effects**, not the projection write itself — the new version's documents are fully rebuilt over the entire history. Only `RaiseSideEffects()` output (published messages, emitted events, etc.) is held back during the warm-up. ::: ## Error Handling The daemon uses Polly resilience pipelines for error handling. See [Resiliency Policies](/configuration/retries) for configuration. ## Architecture ```text pc_events │ (Polling) ▼ High Water Mark Detector │ (Sequence Range) ▼ Event Loader ├──► Projection A ──► pc_doc_summary ├──► Projection B ──► pc_doc_dashboard └──► Subscription C ──► External System │ ▼ pc_event_progression (tracks progress for all) ``` --- --- url: /documents/querying/batched-queries.md --- # Batched Queries Polecat supports batching multiple queries into a single database round-trip using `IBatchedQuery`. ## Creating a Batch ```cs var batch = session.CreateBatchQuery(); ``` ## Batch Operations ### Load by ID ```cs var userTask = batch.Load(userId); ``` ### Load Many ```cs var usersTask = batch.LoadMany(userId1, userId2, userId3); ``` ### LINQ Query ```cs var ordersTask = batch.Query() .Where(x => x.Status == "Active") .ToList(); ``` ## Executing the Batch ```cs await batch.Execute(); // Now resolve the results var user = await userTask; var users = await usersTask; var orders = await ordersTask; ``` All queries in the batch execute in a single database call, significantly reducing latency when you need to load multiple independent pieces of data. ## Query Plans For reusable query specifications, implement `IBatchQueryPlan`: ```cs public class ActiveOrdersPlan : QueryListPlan { protected override IQueryable Query(IQuerySession session) { return session.Query().Where(x => x.Status == "Active"); } } // Use in a batch var ordersTask = batch.QueryByPlan(new ActiveOrdersPlan()); await batch.Execute(); var orders = await ordersTask; ``` Query plans can also be used independently: ```cs var orders = await session.QueryByPlanAsync(new ActiveOrdersPlan()); ``` ## Batched Event Store Fetches `batch.Events` exposes the batched counterparts of `FetchStreamStateAsync` and `FetchStreamAsync`, so a raw event-stream read can share the batch's single round trip with document loads and LINQ queries: ```cs var batch = session.CreateBatchQuery(); var stateTask = batch.Events.FetchStreamState(streamId); var eventsTask = batch.Events.FetchStream(streamId); var orderTask = batch.Load(orderId); await batch.Execute(); var state = await stateTask; // StreamState?, null when the stream does not exist var events = await eventsTask; // IReadOnlyList, empty when the stream does not exist var order = await orderTask; ``` Both come in `Guid` and `string` overloads for the two stream identity modes, and `FetchStream` carries the same optional `version` / `timestamp` / `fromVersion` filters as `FetchStreamAsync`: ```cs // Events up to and including version 5 var capped = batch.Events.FetchStream(streamId, version: 5); // Everything appended from version 10 onward var tail = batch.Events.FetchStream(streamId, fromVersion: 10); ``` ## Event Stream Query Plans `FetchStreamStatePlan` and `FetchStreamPlan` wrap those fetches as query plans. Both implement **both** `IQueryPlan` and `IBatchQueryPlan`, so the same plan instance works standalone or in a batch: ```cs // Standalone var state = await session.QueryByPlanAsync(new FetchStreamStatePlan(streamId)); var events = await session.QueryByPlanAsync(new FetchStreamPlan(streamId, version: 5)); // Batched — one round trip var batch = session.CreateBatchQuery(); var stateFetcher = batch.QueryByPlan(new FetchStreamStatePlan(streamId)); var eventsFetcher = batch.QueryByPlan(new FetchStreamPlan(streamId)); await batch.Execute(); ``` ::: tip Implementing both interfaces matters beyond convenience. Through Wolverine's fetch-specification feature, a plan that implements only `IBatchQueryPlan` produces uncompilable generated code — so a custom plan you intend to route through a handler's `Load` should implement the pair as well. ::: --- --- url: /events/binary-serialization.md --- # Binary Event Serialization Polecat can store the body of an individual event type in a compact binary format — [MessagePack](https://msgpack.org/), [MemoryPack](https://github.com/Cysharp/MemoryPack), compressed JSON, anything you can express as `byte[]` — instead of the default JSON. The opt-in is **per event type**. Binary-serialized and JSON-serialized events coexist in the same `pc_events` table, so switching one hot event type to binary is an in-place change with **no migration of existing event data** — and switching it back off is equally safe. ::: tip Why bother This is a pure storage and IO win with no application-visible behaviour change. The motivating measurement, from CritterWatch's telemetry ingest, was a representative fleet going from **904 KB/min to 19 KB/min** of event storage by moving a single high-volume event type off JSON. That is the difference between retaining a month and retaining a day at the same disk budget. ::: ## The contract The seam is [`JasperFx.Events.IEventBinarySerializer`](https://github.com/JasperFx/jasperfx), shared by every Critter Stack store: ```cs public interface IEventBinarySerializer { byte[] Serialize(Type type, object data); object Deserialize(Type type, byte[] data); } ``` Two methods, deliberately. Everything else about the feature — which column carries the bytes, how a row records that it is binary, how the serializer for a type is resolved — is a storage concern that differs per store and stays there. ::: warning It is shared on purpose The interface and the `[BinaryEvent]` attribute both live in `JasperFx.Events`, not in Polecat. An application that compiles one body of source against Marten *and* Polecat *and* Fisher writes **one** serializer and registers it three times, rather than carrying three identical two-method classes that differ only in which namespace the interface came from. See [polecat#475](https://github.com/JasperFx/polecat/issues/475). Polecat 5.17 **removed** the earlier `Polecat.Events.IEventBinarySerializer` and `Polecat.Events.BinaryEventAttribute` (added in 5.12 by [#388](https://github.com/JasperFx/polecat/issues/388)) rather than keeping them as aliases: any file with both `JasperFx.Events` and `Polecat.Events` in scope — which is exactly the code this feature exists for — got `CS0104` on the bare name. Upgrading means re-pointing the `using` at `JasperFx.Events`. It is a compile error, never a silent behaviour change, and no stored data is affected. ::: ## How it works on SQL Server `pc_events` carries a nullable binary column alongside the JSON one: | Column | Type | | | :--- | :--- | :--- | | `data` | `json` (SQL Server 2025) / `nvarchar(max)` | the JSON payload, `NOT NULL` | | `bdata` | `varbinary(max)` | the binary payload, `NULL` | The discriminator is **per row**: `bdata IS NULL`. | When | `data` | `bdata` | | :--- | :--- | :--- | | The event's type stays on the JSON path | the full JSON payload | `NULL` | | The event's type has a binary serializer | the placeholder `{}` | the serialized bytes | `data` holds `{}` rather than `NULL` for a binary event because the column is `NOT NULL` and, on SQL Server 2025, typed `json` — the row still has to hold something the engine will parse. No schema relaxation was needed. On read, Polecat inspects `bdata`: * `NULL` → the ordinary JSON deserialization path. Rows written before the feature existed keep working, forever, with no conversion. * non-null → `IEventBinarySerializer.Deserialize(eventType, bytes)`. Because the discriminator is on the row and the serializer is resolved per event type, one stream can carry rows of either format with no special handling at the call site. This holds across every read path: `FetchStreamAsync`, live aggregation, inline projections, the async daemon, the event LINQ provider, and event data masking. ## Opting an event type in Two equivalent routes. **By attribute**, resolved against the store-wide default serializer: ```cs using JasperFx.Events; [BinaryEvent] public record TelemetrySampled(string Channel, int[] Samples); ``` ```cs var store = DocumentStore.For(opts => { opts.Connection(connectionString); // Every [BinaryEvent]-marked type with no explicit registration uses this. opts.Events.DefaultBinarySerializer = new MessagePackEventSerializer(); }); ``` **By explicit registration**, which wins over the attribute and needs no attribute at all — useful when the event type lives in an assembly you do not own: ```cs opts.Events.UseBinarySerializer(new MessagePackEventSerializer()); ``` Resolution order for an event type: 1. An explicit `opts.Events.UseBinarySerializer(...)` registration. 2. `[BinaryEvent]` on the type, plus `opts.Events.DefaultBinarySerializer`. 3. Otherwise plain JSON — the existing path, and the default for everything. A type marked `[BinaryEvent]` with **neither** configured is a configuration error: Polecat throws when it first resolves that event type, naming both registration entry points. It deliberately does not fall back to JSON — a silent fallback would give you a store whose write amplification quietly does not match the configuration you believe you are running. ## Bring your own serializer The interface is small enough to implement directly against any binary format. The serializer is a singleton — one instance serves every session in the store — so **it must be thread-safe**: ```cs using JasperFx.Events; public sealed class MessagePackEventSerializer : IEventBinarySerializer { private static readonly MessagePackSerializerOptions _options = MessagePackSerializerOptions.Standard .WithCompression(MessagePackCompression.Lz4BlockArray) .WithResolver(ContractlessStandardResolver.Instance); public byte[] Serialize(Type type, object data) => MessagePackSerializer.Serialize(type, data, _options); public object Deserialize(Type type, byte[] data) => MessagePackSerializer.Deserialize(type, data, _options); } ``` ::: danger A serializer is part of your read path forever A serializer stays load-bearing for as long as any row it wrote still exists. Removing a registration does not restore those rows to the JSON path — it makes them **unreadable**, and the read fails with an exception naming the type and both registration entry points. Keep the registration after the event type stops being written as binary; retire it only once the rows are archived or compacted away. ::: ## Schema migration Purely additive. The only schema change is the nullable `bdata varbinary(max)` column on `pc_events`, and Polecat's ordinary schema migration adds it on the next apply — Weasel produces an additive delta, so `ApplyAllConfiguredChangesToDatabaseAsync()` or the `polecat` CLI's `db-apply` handles an existing store with no downtime and no data conversion. The column is created **unconditionally**, whether or not a binary serializer is configured. That keeps the read projection one fixed shape rather than two, and it means turning the feature on later is a code change only — the column is already there. Existing rows have `bdata = NULL` and keep reading through the JSON path. There is nothing to backfill. ## On-disk shape ```sql -- a binary-serialized event select [type], cast([data] as nvarchar(max)) as data, iif([bdata] is null, 1, 0) as is_json from [polecat].[pc_events] where seq_id = 42; -- type | data | is_json -- ---------------------|------|-------- -- telemetry_sampled | {} | 0 -- a JSON-serialized event on the same stream select [type], cast([data] as nvarchar(max)) as data, iif([bdata] is null, 1, 0) as is_json from [polecat].[pc_events] where seq_id = 43; -- type | data | is_json -- ---------------------|-------------------------------|-------- -- trip_comment_added | {"comment":"looking good",…} | 1 ``` ## Schema evolution — use versioned event types Polecat's event upcasters operate on the JSON wire form and do not generalize to a `byte[]` payload, so they do not apply to binary events. The recommended pattern for evolving a binary event's shape is to **introduce a new event type per version** rather than transforming in place: ```cs // Original [BinaryEvent] public record TripStarted(Guid TripId, string DriverName); // Schema change. Don't edit TripStarted — add a new type. [BinaryEvent] public record TripStartedV2(Guid TripId, string DriverName, DateTimeOffset StartedAt); ``` Have the aggregate handle both versions, and the coexistence design carries old rows and new rows on the same stream with no migration: ```cs public class Trip { public Guid Id { get; set; } public string DriverName { get; set; } = ""; public DateTimeOffset? StartedAt { get; set; } public void Apply(TripStarted e) { Id = e.TripId; DriverName = e.DriverName; } public void Apply(TripStartedV2 e) { Id = e.TripId; DriverName = e.DriverName; StartedAt = e.StartedAt; } } ``` You *can* instead lean on your serializer's own version tolerance — MessagePack and MemoryPack both have backward-compatible field evolution for additive-only changes. That works right up to the edge of the serializer's tolerance rules; renames, type changes and field splits have no JSON-style upcaster to fall back on. Versioned event types work for every shape of change and stay explicit about which version each row was written with. The same pattern covers going binary for an event type that is already JSON: define a new `[BinaryEvent]`-marked type for the new version, leave the old type and its upcasters alone, and let the per-row dispatch cope with both formats on one stream. ## What this is not * **Not** a store-wide or per-stream setting. There is no flag on the store, the stream or the table — only per-event-type registration and the per-row `bdata` discriminator. That is deliberate; it is what makes adoption and rollback both free. * **Not** a change to querying event *metadata*. Sequence, version, stream, type name, timestamp, correlation, causation and headers are all still ordinary columns and still queryable. * ⚠️ **Not** queryable *content*. Anything that reaches into the event body through LINQ resolves against the `data` column, which holds `{}` for a binary event — so a `Where` on a body member never matches a binary row, and `QueryRawEventDataOnly()` (which deserializes straight from `data`) hands back an empty instance. Both fail *silently*: no exception, just no results. Do not make an event type binary if you query its body. `QueryAllRawEvents()` is the exception and is safe — it materializes `IEvent` wrappers through the same per-row dispatch as `FetchStreamAsync`, so binary payloads hydrate correctly. Only the *filtering* is blind to them. ## Cross-store parity The behaviour on this page is held to a shared definition by `BinaryEventSerializationCompliance` in `JasperFx.Events.ComplianceTests`, which Marten and Polecat both run against their own storage. The fact that pins the headline promise is `json_and_binary_events_coexist_in_one_stream` — a store that got everything else right but kept a per-store or per-stream format flag fails exactly there. ## See also * [Event Storage](/events/storage) — the `pc_events` table in full * [Appending Events](/events/appending) --- --- url: /configuration/hostbuilder.md --- # Bootstrapping Polecat Polecat provides `AddPolecat()` extension methods on `IServiceCollection` for easy integration with .NET's dependency injection. ## Basic Registration The simplest way to register Polecat: ```cs builder.Services.AddPolecat(options => { options.Connection("Server=localhost;Database=myapp;User Id=sa;Password=YourStrong!Password;TrustServerCertificate=True"); }); ``` ## Registration Overloads Polecat offers several `AddPolecat()` overloads: ```cs // Connection string only builder.Services.AddPolecat("Server=localhost;Database=myapp;..."); // Action-based configuration builder.Services.AddPolecat(options => { options.Connection("..."); options.DatabaseSchemaName = "myschema"; }); // Pre-built StoreOptions var storeOptions = new StoreOptions(); storeOptions.Connection("..."); builder.Services.AddPolecat(storeOptions); // Factory-based (access IServiceProvider) builder.Services.AddPolecat(sp => { var config = sp.GetRequiredService(); var opts = new StoreOptions(); opts.Connection(config.GetConnectionString("SqlServer")!); return opts; }); ``` ## Registered Services `AddPolecat()` registers the following services: | Service | Lifetime | Description | | :--- | :--- | :--- | | `IDocumentStore` | Singleton | Main entry point, creates sessions | | `ISessionFactory` | Singleton | Factory for creating sessions (default: lightweight) | | `IDocumentSession` | Scoped | Read/write session with unit of work | | `IQuerySession` | Scoped | Read-only session for queries | ## ConfigurePolecat `ConfigurePolecat()` registers a post-configuration action that runs against `StoreOptions` after `AddPolecat()` has built them. Use it when a module other than the one that called `AddPolecat()` owns part of the store configuration: ```cs builder.Services.ConfigurePolecat(options => { options.CommandTimeout = 120; }); ``` There is a second overload that also hands you the built `IServiceProvider`, for configuration that depends on other registered services: ```cs builder.Services.ConfigurePolecat((services, options) => { var settings = services.GetRequiredService>().Value; options.Schema.For().PartitionOn(x => x.Timestamp) .ByRollingRange(RollingPeriod.Day, ahead: 2, behind: settings.DaysRetained); }); ``` Both overloads have a `ConfigurePolecat()` counterpart that targets an ancillary store registered with `AddPolecatStore()`. ## IConfigurePolecat You can implement `IConfigurePolecat` to modularize your configuration: ```cs public class MyPolecatConfig : IConfigurePolecat { public void Configure(IServiceProvider services, StoreOptions options) { // Apply configuration here } } ``` Register it before `AddPolecat()`: ```cs builder.Services.AddSingleton(); builder.Services.AddPolecat(options => { options.Connection("..."); }); ``` ## Session Factory By default, Polecat creates lightweight sessions (no identity tracking). You can change this by providing a custom `ISessionFactory`: ```cs builder.Services.AddPolecat(options => { options.Connection("..."); }); ``` ::: tip Lightweight sessions are recommended for most use cases. Only use `IdentityMap` sessions when you need to ensure the same document instance is returned for repeated loads within a session. ::: --- --- url: /documents/bulk-insert.md --- # Bulk Insert Polecat provides a high-performance bulk insert API for inserting large numbers of documents efficiently. ## Basic Usage ```cs var users = Enumerable.Range(0, 1000) .Select(i => new User { FirstName = $"User{i}", LastName = "Bulk" }) .ToList(); await store.Advanced.BulkInsertAsync(users); ``` ## Bulk Insert Modes ### InsertsOnly (Default) Inserts all documents. Throws on duplicate IDs: ```cs await store.Advanced.BulkInsertAsync(users, BulkInsertMode.InsertsOnly); ``` ### IgnoreDuplicates Inserts new documents and silently skips duplicates: ```cs await store.Advanced.BulkInsertAsync(users, BulkInsertMode.IgnoreDuplicates); ``` ### OverwriteExisting Inserts new documents and updates existing ones (upsert): ```cs await store.Advanced.BulkInsertAsync(users, BulkInsertMode.OverwriteExisting); ``` ## Batch Size Control the number of documents per batch (default: 200): ```cs await store.Advanced.BulkInsertAsync(users, BulkInsertMode.InsertsOnly, batchSize: 500); ``` ## Multi-Tenant Bulk Insert Specify a tenant ID for conjoined tenancy: ```cs await store.Advanced.BulkInsertAsync(users, BulkInsertMode.InsertsOnly, 200, tenantId: "tenant-a"); ``` ## Automatic ID Assignment Bulk insert automatically handles ID assignment: * **Guid IDs**: Auto-generated for empty Guids * **Numeric IDs**: Assigned via HiLo sequences * **Strongly Typed IDs**: Inner values are auto-assigned ## Metadata Sync During bulk insert, Polecat automatically syncs metadata interfaces: * `ITenanted.TenantId` is set from the tenant parameter * `ISoftDeleted` properties are initialized * `ITracked` properties are synced if set on the session --- --- url: /events/projections/composite.md --- # Composite Projections Composite projections orchestrate multiple projection stages that must run in a specific order. Within each stage, projections can run in parallel. ## How It Works A composite projection defines stages: 1. **Stage 1**: Projections A and B run in parallel 2. **Stage 2**: Projection C runs after Stage 1 completes (it depends on A and B) 3. **Stage 3**: Projections D and E run in parallel after Stage 2 All stages share a single `IProjectionBatch` that is flushed to SQL Server **once**, after every stage has run. This is what makes a composite update atomic. ## Defining a Composite Projection ```cs opts.Projections.CompositeProjectionFor("OrderComposite", composite => { composite.Add(); // stage 1 composite.Add(); // stage 1 (parallel) composite.Add(stageNumber: 2); // stage 2 (depends on stage 1) }); ``` You can also register self-aggregating snapshot projections directly: ```cs opts.Projections.CompositeProjectionFor("OrderSnapshots", composite => { composite.Snapshot(); // stage 1 composite.Snapshot(stageNumber: 2); }); ``` ## Lifecycle Composite projections always run asynchronously. They flow through the standard async daemon, so you must enable it on the host: ```cs builder.Services.AddPolecat(opts => { opts.Connection("..."); opts.Projections.CompositeProjectionFor("OrderComposite", composite => { composite.Add(); composite.Add(2); }); }) .AddAsyncDaemon(DaemonMode.Solo) .ApplyAllDatabaseChangesOnStartup(); ``` ## Thread Safety The `PolecatProjectionBatch` used by composite projections uses `ConcurrentBag` and `ConcurrentQueue` internally to safely handle parallel projections within a stage. ## Cross-stage document visibility ::: warning A downstream stage **cannot** see the document writes of an upstream stage by issuing a SQL query against `IQuerySession` — those writes are still queued on the shared in-memory projection batch and have not been committed yet. The query goes to SQL Server, which has not received them. ::: All stages of a composite share one `IProjectionBatch` that flushes once, after every stage has run. This is what makes a composite atomic, but it also means that during a later stage's `EnrichEventsAsync`, the document writes produced by earlier stages are still queued in memory. A query like: ```cs // Inside a stage-2 projection's EnrichEventsAsync — DOES NOT see Order // rows written by an upstream stage-1 projection in this same batch var orders = await querySession.Query().ToListAsync(); ``` will return only what was committed by **previous** batches. During a projection rebuild, where every event is replayed from scratch, neither the upstream nor the downstream documents have been committed yet, so the query returns an empty result. Polecat (via JasperFx.Events 1.35.0+) supports four ways for a downstream stage to consume upstream stage output: * **`Updated` and `ProjectionDeleted` synthetic events.** When an upstream `SingleStreamProjection` or `MultiStreamProjection` updates or deletes a document, JasperFx injects a synthetic event into the downstream stage's event stream. The current snapshot of `T` is carried directly on the event payload, so no database lookup is needed. * **`EnrichWith().ForEvent().ForEntityId(...).AddReferences()`** (and the related `EnrichAsync` overloads). These walk the upstream's in-memory aggregate cache for `T` rather than the database, so they observe in-flight writes from earlier stages in the same batch. * **`group.TryFindUpstreamCache(out var cache)`** for custom enrichment callbacks (notably inside `EnrichUsingEntityQuery`) that need to look up an in-flight upstream entity by id when it isn't the type of the enclosing `EnrichWith`. Returns `false` when no upstream stage of this composite produces entities of that type — see [the example below](#looking-up-arbitrary-upstream-entities). * **`group.ReferencePeerView()`** for a parallel projected view that shares the same identity as the projection being built. Direct use of `querySession.Query()` from inside `EnrichEventsAsync` is appropriate for **static reference data committed in earlier batches** and not for documents produced by upstream stages of the *current* batch. ### Looking up arbitrary upstream entities `EnrichUsingEntityQuery`'s callback receives a cache parameter typed for the enclosing `EnrichWith`. When the callback also needs to read an in-flight upstream entity of a *different* type — for example an `OrderShippingNotification` enrichment that needs to consult the upstream `Order` that is being projected in the same batch — call `group.TryFindUpstreamCache` against the captured `SliceGroup` to reach into the upstream stage's in-memory aggregate cache. ```cs public partial class OrderShippingNotificationProjection : MultiStreamProjection { public OrderShippingNotificationProjection() { Identity>(e => e.StreamId); } public override Task EnrichEventsAsync(SliceGroup group, IQuerySession querySession, CancellationToken cancellation) { // Ask the upstream CompositeOrderProjection (running earlier in the same composite stage) // for its in-memory aggregate cache. A SQL query for CompositeOrder in this same batch // would return nothing — those writes are still queued on the shared IProjectionBatch // and have not been committed to SQL Server yet. if (!group.TryFindUpstreamCache(out var upstreamOrders)) { return Task.CompletedTask; } foreach (var slice in group.Slices) { if (upstreamOrders.TryFind(slice.Id, out var order)) { // Stamp a synthetic References event onto the slice so // the Evolve method can read the upstream entity's data. slice.Reference(order); } } return Task.CompletedTask; } public override OrderShippingNotification? Evolve(OrderShippingNotification? snapshot, Guid id, IEvent e) { switch (e.Data) { case CompositeOrderShipped shipped: snapshot ??= new OrderShippingNotification { Id = id }; snapshot.Carrier = shipped.Carrier; break; case References orderRef: snapshot ??= new OrderShippingNotification { Id = id }; snapshot.CustomerId = orderRef.Entity.CustomerId; snapshot.OrderTotal = orderRef.Entity.Total; break; } return snapshot; } } ``` snippet source | anchor `TryFindUpstreamCache` returns `false` when no upstream stage of this composite is registered as producing entities of that type, and the cache it returns is a hint — `IAggregateCache.TryFind` may still miss for entities outside the cache window (`Options.CacheLimitPerTenant`), in which case the caller should fall back to whatever is appropriate for that data. ::: tip `CacheLimitPerTenant` is a *memory* tunable, not a correctness knob. As of JasperFx.Events 1.35.0 the upstream cache is held until every composite stage has completed, so a tiny `CacheLimitPerTenant` no longer starves a downstream stage. See `composite_try_find_upstream_cache_tests.tiny_upstream_cache_limit_does_not_starve_downstream_stage` for the regression coverage. ::: ### Fan-out enrichment with `ForEntityIds` When a single event references **multiple** entities of the same type — for example a `BatchTransfer` event carrying a list of account ids — use the `ForEntityIds` (or `ForEntityIdsFromEvent`) variant of `EnrichWith` to fan out the lookup: ```cs public class TransferEnrichingProjection : MultiStreamProjection { public TransferEnrichingProjection() { EnrichWith() .ForEvent() .ForEntityIds(e => e.AccountIds) .AddReferences(); } } ``` This produces one `References` event per id per slice, so the projection's `Apply` / `Evolve` method can read each referenced entity directly. Duplicates within a single event are passed through to the application callback as-is; ids are de-duplicated only when fetching from storage to avoid redundant loads. ## Use Cases * **Dependent read models** — Dashboard that depends on individual aggregates * **Multi-stage processing** — Transform data through a pipeline * **Performance optimization** — Parallelize independent projections within a stage ## Things to Know * Composite projections can include any kind of projection (single-stream, multi-stream, event projections, flat-table projections). * Composite projections can only run asynchronously. * In `pc_event_progression`, you will see rows for both the parent composite and every constituent projection — they should never disagree. * You can use as many stages as you wish, but two or three is usually enough. * If you rebuild a composite projection, you have to rebuild every constituent projection. --- --- url: /documents/indexing/computed-indexes.md --- # Computed Indexes Polecat supports computed indexes on document properties. These indexes use SQL Server's **persisted computed columns** backed by `JSON_VALUE` expressions, giving you the performance of a traditional column index without duplicating data outside of the JSON document. ## How It Works When you define a computed index, Polecat: 1. Adds a **persisted computed column** to the document table using `JSON_VALUE(data, '$.path')` 2. Creates a standard **nonclustered index** on that computed column For example, indexing `UserName` on a `User` document produces: ```sql ALTER TABLE [myschema].[pc_doc_user] ADD [cc_username] AS CAST(JSON_VALUE(data, '$.userName') AS varchar(250)) PERSISTED; CREATE NONCLUSTERED INDEX [ix_pc_doc_user_username] ON [myschema].[pc_doc_user] ([cc_username]); ``` ## Simple Indexes Use the fluent API via `StoreOptions.Schema.For().Index()`: ```cs var store = DocumentStore.For(opts => { opts.ConnectionString = "..."; opts.Schema.For().Index(x => x.UserName); }); ``` SQL Server will use this index when querying: ```cs var user = await session.Query() .FirstOrDefaultAsync(x => x.UserName == "somebody"); ``` ## Composite (Multi-Column) Indexes Create a single index across multiple properties using an anonymous type: ```cs opts.Schema.For().Index(x => new { x.FirstName, x.LastName }); ``` This produces one nonclustered index with both columns: ```sql CREATE NONCLUSTERED INDEX [ix_pc_doc_user_firstname_lastname] ON [myschema].[pc_doc_user] ([cc_firstname], [cc_lastname]); ``` ## Unique Indexes ```cs opts.Schema.For().UniqueIndex(x => x.Email); ``` Attempting to store two documents with the same email will throw a SQL Server unique constraint violation. ## Covering Indexes (INCLUDE Columns) A query that filters on one property but also *selects* others can still pay for a lookup back into the table to fetch the extra columns. You can avoid that by carrying those extra members in the index as non-key **`INCLUDE`** columns, so SQL Server satisfies the whole query from the index alone. Pass the `include:` argument — a single member or an anonymous type, just like the key expression: ```cs opts.Schema.For().Index(x => x.UserName, include: x => new { x.FirstName, x.LastName }); ``` Each include member gets its own persisted computed column, and the index gains an `INCLUDE` clause: ```sql ALTER TABLE [myschema].[pc_doc_user] ADD [cc_firstname] AS CAST(JSON_VALUE(data, '$.firstName') AS varchar(250)) PERSISTED; ALTER TABLE [myschema].[pc_doc_user] ADD [cc_lastname] AS CAST(JSON_VALUE(data, '$.lastName') AS varchar(250)) PERSISTED; CREATE NONCLUSTERED INDEX [ix_pc_doc_user_username] ON [myschema].[pc_doc_user] ([cc_username]) INCLUDE ([cc_firstname], [cc_lastname]); ``` This maps to the `IncludeColumns` property on `DocumentIndex` (named after Marten/Weasel's `IndexDefinition.IncludeColumns`), which you can also set directly in the `configure` action if you prefer to work in raw JSON paths: ```cs opts.Schema.For().Index(x => x.UserName, idx => idx.IncludeColumns = ["$.firstName", "$.lastName"]); ``` `UniqueIndex(...)` accepts the same `include:` argument. Include columns are always stored with default casing (they are payload, not keys), and covering indexes work on both native `json` and `nvarchar(max)` storage. ## Customizing an Index The `Index()` and `UniqueIndex()` methods accept an optional `Action` to customize the index: ```cs opts.Schema.For().Index(x => x.UserName, idx => { // Force the indexed value to lowercase for case-insensitive lookups idx.Casing = IndexCasing.Lower; // Override the index name idx.IndexName = "ix_user_name_ci"; // Use a different SQL type (default: varchar(250)) idx.SqlType = "varchar(500)"; // Change sort order (default: Ascending) idx.SortOrder = SortOrder.Descending; // Scope uniqueness per tenant (for conjoined tenancy) idx.TenancyScope = TenancyScope.PerTenant; // Add a WHERE clause for a filtered (partial) index idx.Predicate = "tenant_id <> 'EXCLUDED'"; }); ``` ## Case Transformations For case-insensitive lookups, you can apply `UPPER()` or `LOWER()` transformations to string-typed index columns. This wraps the `JSON_VALUE` expression so the persisted computed column stores the normalized value: ```cs // Lowercase index — stores "john.doe@example.com" even if original is "John.Doe@Example.COM" opts.Schema.For().Index(x => x.Email, idx => { idx.Casing = IndexCasing.Lower; }); // Uppercase index opts.Schema.For().Index(x => x.UserName, idx => { idx.Casing = IndexCasing.Upper; }); ``` The generated SQL for a lowercase index: ```sql ALTER TABLE [myschema].[pc_doc_user] ADD [cc_email_lower] AS LOWER(CAST(JSON_VALUE(data, '$.email') AS varchar(250))) PERSISTED; CREATE NONCLUSTERED INDEX [ix_pc_doc_user_email_lower] ON [myschema].[pc_doc_user] ([cc_email_lower]); ``` ::: tip Case transformations only apply to string-typed columns. Non-string columns (int, Guid, etc.) ignore the `Casing` setting. ::: ### Case-Insensitive Unique Indexes Combine casing with unique indexes to enforce uniqueness regardless of case: ```cs opts.Schema.For().UniqueIndex(x => x.Email, idx => { idx.Casing = IndexCasing.Lower; }); ``` This rejects both `"test@example.com"` and `"Test@Example.COM"` as duplicates. ## Attribute-Based Indexes Instead of (or in addition to) the fluent API, you can declare indexes directly on your document properties using attributes. ### \[Index] Attribute Marks a property for a computed index: ```cs using Polecat.Attributes; public class User { public Guid Id { get; set; } [Index] public string UserName { get; set; } = ""; [Index(Casing = IndexCasing.Lower)] public string Email { get; set; } = ""; [Index(SqlType = "int")] public int Age { get; set; } } ``` ### \[UniqueIndex] Attribute Marks a property for a unique computed index: ```cs public class User { public Guid Id { get; set; } [UniqueIndex] public string Email { get; set; } = ""; public string Name { get; set; } = ""; } ``` ### Composite Unique Indexes with Attributes Use the `IndexName` property to group multiple properties into a single composite unique index: ```cs public class User { public Guid Id { get; set; } [UniqueIndex(IndexName = "ux_fullname")] public string FirstName { get; set; } = ""; [UniqueIndex(IndexName = "ux_fullname")] public string LastName { get; set; } = ""; } ``` This creates one unique index across both `FirstName` and `LastName`. ### Attribute Options Both `[Index]` and `[UniqueIndex]` support these options: | Option | Type | Default | Description | | :--- | :--- | :--- | :--- | | `IndexName` | `string?` | Auto-generated | Explicit index name | | `Casing` | `IndexCasing` | `Default` | Case transformation (`Upper`, `Lower`, `Default`) | | `SqlType` | `string?` | `varchar(250)` | SQL type for the computed column | | `SortOrder` | `SortOrder` | `Ascending` | Sort order (Index only) | | `TenancyScope` | `TenancyScope` | `Global` | Per-tenant scoping (UniqueIndex only) | ::: tip Attribute-based indexes are discovered automatically when the document type is first used. They can be combined with fluent API indexes on the same document type. ::: ## Tenancy-Scoped Indexes For multi-tenant applications using conjoined tenancy, you can scope unique indexes per tenant: ```cs opts.Schema.For().UniqueIndex(x => x.Email, idx => { idx.TenancyScope = TenancyScope.PerTenant; }); ``` This includes `tenant_id` in the index columns, allowing the same email across different tenants while enforcing uniqueness within each tenant. ## Serialized Names and `[JsonPropertyName]` The computed column reads the JSON path the **serializer** writes, not the CLR property name. A member carrying `[JsonPropertyName]` is indexed under its alias: ```cs public class Metric { public Guid Id { get; set; } [JsonPropertyName("bucket_label")] public string Label { get; set; } = string.Empty; } opts.Schema.For().Index(x => x.Label); ``` produces a column over `$.bucket_label`, matching what queries on `x.Label` translate to, so the index is usable. ::: warning Upgrading from before 5.20 Prior to 5.20 the index path was built from the CLR member name, so an aliased member got a column over a path the serializer never writes — permanently `NULL`, and an index that could never match. Queries kept returning correct results by scanning the `data` column, so there was no error to notice. On upgrade the correct column and index are created additively; the stale ones are left in place, since Polecat's migrations never drop. For a member previously indexed under an alias you will find an orphaned `cc_` column and its `ix__` index, both safe to drop by hand once you no longer need to roll back. ::: ## Naming Policies The index path follows the store's configured naming policy, so a non-default casing indexes the path the serializer actually writes: ```cs opts.ConfigureSerialization(casing: Casing.SnakeCase); opts.Schema.For().Index(x => x.ServiceName); ``` produces a `cc_service_name` column over `$.service_name`, matching both the document and the queries translated against it. Every segment of a nested path is converted, and native `JsonIndex(...)` paths follow the same rule. An explicit `[JsonPropertyName]` still wins verbatim and is **not** put through the policy on top, matching how `System.Text.Json` itself behaves. ::: warning Upgrading from before 5.20 Prior to 5.20 index paths were always camelCased regardless of the configured policy, so on a `Casing.SnakeCase` store every computed column read a path the document did not contain — `NULL` for every row, and an index that could never match. As with the alias case above, queries returned correct results by scanning `data`, so there was nothing to notice. On upgrade the correctly-named column and index are created additively and the stale ones are left in place. Expect an orphaned `cc_` column and its index per affected member, both safe to drop by hand once you no longer need to roll back. ::: --- --- url: /configuration/storeoptions.md --- # Configuring Document Storage The `StoreOptions` class is the central configuration object for Polecat. It controls connection settings, schema management, serialization, and more. ## Connection String ```cs var store = DocumentStore.For(opts => { opts.Connection("Server=localhost;Database=myapp;User Id=sa;Password=YourStrong!Password;TrustServerCertificate=True"); }); ``` ## Schema Name By default, Polecat uses the `dbo` schema. You can change this: ```cs opts.DatabaseSchemaName = "myschema"; ``` ## Auto-Create Schema Objects Control how Polecat manages database schema: ```cs // Default: CreateOrUpdate - auto-creates and updates tables opts.AutoCreateSchemaObjects = AutoCreate.CreateOrUpdate; // CreateOnly - only creates new tables, never modifies existing ones opts.AutoCreateSchemaObjects = AutoCreate.CreateOnly; // None - never creates or modifies schema objects opts.AutoCreateSchemaObjects = AutoCreate.None; ``` ::: warning In production environments, consider setting `AutoCreate.None` and managing schema migrations separately. ::: ## Table Prefix All Polecat tables use the `pc_` prefix: * `pc_events` -- Event log * `pc_streams` -- Stream metadata * `pc_event_progression` -- Async daemon progression * `pc_hilo` -- HiLo sequence storage * `pc_doc_{typename}` -- Document tables ## Native JSON Column Type By default, Polecat uses SQL Server 2025's native `json` data type for document bodies, event data, and headers. To fall back to `nvarchar(max)` for pre-2025 SQL Server instances: ```cs opts.UseNativeJsonType = false; ``` See [JSON Serialization](/configuration/json#falling-back-to-nvarcharmax) for more details. ## Store Policies Apply policies across all document types: ```cs opts.Policies.AllDocumentsSoftDeleted(); opts.Policies.ForDocument(mapping => { mapping.DeleteStyle = DeleteStyle.SoftDelete; }); ``` ## Listeners Register global session listeners: ```cs opts.Listeners.Add(new MySessionListener()); ``` See [Session Listeners](/documents/sessions#session-listeners) for more details. ## Logging Configure store-level logging: ```cs opts.Logger(new MyPolecatLogger()); ``` ## HiLo Sequence Defaults Configure default HiLo sequence settings for numeric identity generation: ```cs opts.HiloSequenceDefaults.MaxLo = 500; // default is 1000 ``` ## Initial Data Seeding Register data to be seeded on application startup: ```cs opts.InitialData.Add(async (store, ct) => { await using var session = store.LightweightSession(); session.Store(new User { FirstName = "Admin", LastName = "User" }); await session.SaveChangesAsync(ct); }); ``` See [Initial Baseline Data](/documents/initial-data) for more details. --- --- url: /schema.md --- # Database Management Polecat uses [Weasel.SqlServer](https://github.com/JasperFx/weasel) for all database schema management. Tables are automatically created and updated as needed. ## Auto-Create Modes Control schema management behavior via `StoreOptions.AutoCreateSchemaObjects`: | Mode | Behavior | | :--- | :--- | | `CreateOrUpdate` (default) | Creates new tables, adds new columns to existing tables | | `CreateOnly` | Only creates new tables, never modifies existing ones | | `None` | No automatic schema management | ```cs opts.AutoCreateSchemaObjects = AutoCreate.CreateOrUpdate; ``` ## Schema Objects Polecat manages these SQL Server objects: ### Document Tables * `pc_doc_{typename}` -- One table per document type * Created on first use (first `Store()`, `Query()`, or `LoadAsync()`) ### Event Store Tables * `pc_events` -- Global event log * `pc_streams` -- Stream metadata * `pc_event_progression` -- Async daemon progress ### Support Tables * `pc_hilo` -- HiLo sequence values for numeric IDs ## Default Schema All tables are created in the `dbo` schema by default. Change this with: ```cs opts.DatabaseSchemaName = "myschema"; ``` ## Weasel Integration Polecat delegates all DDL generation and execution to Weasel.SqlServer. This provides: * Diff-based migrations (compares desired vs actual schema) * Safe column additions (never drops columns) * Index management * Foreign key management ## PolecatActivator On application startup, Polecat runs the `PolecatActivator` which: 1. Ensures all configured schema objects exist 2. Runs schema migrations if `AutoCreate` is enabled 3. Executes [initial data seeding](/documents/initial-data) if configured 4. Starts the async daemon if async projections are registered --- --- url: /documents/storage.md --- # Database Storage Polecat stores each document type in its own dedicated SQL Server table. ## Table Naming Document tables follow the pattern `pc_doc_{typename}` where `{typename}` is the lowercase, simple name of the .NET type. For example: * `User` → `pc_doc_user` * `Order` → `pc_doc_order` * `InvoiceLineItem` → `pc_doc_invoicelineitem` ## Table Structure A typical document table includes: ```sql CREATE TABLE dbo.pc_doc_user ( id uniqueidentifier NOT NULL PRIMARY KEY, data json NOT NULL, type nvarchar(250) NULL, last_modified datetimeoffset NOT NULL DEFAULT SYSDATETIMEOFFSET(), created datetimeoffset NOT NULL DEFAULT SYSDATETIMEOFFSET(), dotnet_type nvarchar(500) NULL ); ``` ### Additional Columns Depending on configuration, additional columns may be present: | Column | Type | When Added | | :--- | :--- | :--- | | `tenant_id` | `nvarchar(250)` | Conjoined tenancy | | `is_deleted` | `bit` | Soft deletes enabled | | `deleted_at` | `datetimeoffset` | Soft deletes enabled | | `guid_version` | `uniqueidentifier` | `IVersioned` interface | | `version` | `bigint` | Revision counter (`IRevisioned` int / `ILongVersioned` long) | | `correlation_id` | `nvarchar(250)` | Metadata tracking | | `causation_id` | `nvarchar(250)` | Metadata tracking | ## JSON Storage Document bodies are stored using SQL Server 2025's native `json` data type. This provides: * Server-side JSON validation * Efficient `JSON_VALUE()` extraction for WHERE clauses * `JSON_MODIFY()` for [partial updates](/documents/partial-updates-patching) * Compact storage format ## Auto-Create Behavior By default (`AutoCreate.CreateOrUpdate`), Polecat will: 1. Create the table if it doesn't exist on first use 2. Add new columns if the configuration changes (e.g., enabling soft deletes) 3. Never drop existing columns See [Schema Migrations](/schema/migrations) for more details. --- --- url: /documents/deletes.md --- # Deleting Documents Polecat supports both hard deletes (permanent removal) and soft deletes (logical deletion). ## Hard Deletes By default, `Delete()` performs a permanent deletion: ```cs // Delete by ID session.Delete(userId); // Delete by document instance session.Delete(user); // Delete by predicate session.DeleteWhere(x => x.Internal == true); await session.SaveChangesAsync(); ``` ## Soft Deletes Soft deletes mark documents as deleted without removing them from the database. Enable soft deletes in one of three ways: ### Via Attribute ```cs [SoftDeleted] public class Order { public Guid Id { get; set; } public string Description { get; set; } = ""; } ``` ### Via Interface ```cs public class Order : ISoftDeleted { public Guid Id { get; set; } public bool Deleted { get; set; } public DateTimeOffset? DeletedAt { get; set; } } ``` ### Via Policy ```cs // For a specific type opts.Policies.ForDocument(mapping => { mapping.DeleteStyle = DeleteStyle.SoftDelete; }); // For all document types opts.Policies.AllDocumentsSoftDeleted(); ``` ### How Soft Deletes Work When soft deletes are enabled: * `Delete()` sets `is_deleted = 1` and `deleted_at = SYSDATETIMEOFFSET()` in the database * If the document implements `ISoftDeleted`, the in-memory properties are also updated * All queries automatically filter out soft-deleted documents * `HardDelete()` still performs a permanent removal ### Querying Soft-Deleted Documents LINQ extensions allow querying deleted documents: ```cs // Include deleted documents in results var all = await session.Query() .Where(x => x.MaybeDeleted()) .ToListAsync(); // Only return deleted documents var deleted = await session.Query() .Where(x => x.IsDeleted()) .ToListAsync(); // Deleted since a specific time var recentlyDeleted = await session.Query() .Where(x => x.DeletedSince(cutoff)) .ToListAsync(); // Deleted before a specific time var oldDeleted = await session.Query() .Where(x => x.DeletedBefore(cutoff)) .ToListAsync(); ``` ### Undoing Soft Deletes Restore soft-deleted documents: ```cs session.UndoDeleteWhere(x => x.Description == "Restore me"); await session.SaveChangesAsync(); ``` ## Hard Delete (Force) Even when soft deletes are enabled, you can force a permanent deletion: ```cs session.HardDelete(orderId); await session.SaveChangesAsync(); ``` --- --- url: /diagnostics.md --- # Diagnostics and Instrumentation Polecat provides several tools for monitoring, debugging, and understanding what's happening in your application. ## Session Logging ### IPolecatLogger Implement `IPolecatLogger` at the store level to create per-session loggers: ```cs public class ConsolePolecatLogger : IPolecatLogger { public IPolecatSessionLogger StartSession(IQuerySession session) { return new ConsoleSessionLogger(); } } public class ConsoleSessionLogger : IPolecatSessionLogger { public void OnBeforeExecute(string sql) { Console.WriteLine($"Executing: {sql}"); } public void LogSuccess(string sql) { Console.WriteLine($"Success: {sql}"); } public void LogFailure(string sql, Exception ex) { Console.WriteLine($"Failed: {sql} - {ex.Message}"); } public void RecordSavedChanges(IDocumentSession session) { Console.WriteLine($"Saved changes ({session.RequestCount} requests)"); } } ``` Register the logger: ```cs opts.Logger(new ConsolePolecatLogger()); ``` ### Request Counting Track database requests per session: ```cs await using var session = store.LightweightSession(); await session.LoadAsync(id); Console.WriteLine(session.RequestCount); // 1 ``` ## SQL Preview ### ToSql Preview the SQL generated by a LINQ query: ```cs var sql = session.Query() .Where(x => x.LastName == "Smith") .ToSql(); Console.WriteLine(sql); // SELECT data FROM pc_doc_user WHERE JSON_VALUE(data, '$.lastName') = @p0 ``` ## Schema Diagnostics ### ToDatabaseScript Generate the complete DDL script for all Polecat tables: ```cs var script = await store.Advanced.ToDatabaseScript(); Console.WriteLine(script); ``` ### WriteCreationScriptToFileAsync Save the schema script to a file: ```cs await store.Advanced.WriteCreationScriptToFileAsync("/path/to/schema.sql"); ``` ## Data Cleanup ### CleanAllDocumentsAsync Delete all data from all document tables: ```cs await store.Advanced.CleanAllDocumentsAsync(); ``` ### CleanAsync Delete all data from a specific document table: ```cs await store.Advanced.CleanAsync(); ``` ### CleanAllEventDataAsync Delete all event data (events, streams, progressions): ```cs await store.Advanced.CleanAllEventDataAsync(); ``` ::: warning These cleanup methods permanently delete data. They are intended for testing and development, not production use. ::: ## Projection Daemon Monitoring ### Extended Progression Tracking Polecat can extend its event-progression table (`pc_event_progression`) with additional columns that the asynchronous projection daemon uses to report per-shard health — the same surface monitoring tools such as **CritterWatch** read to show heartbeat liveness, agent status, pause reasons, and per-shard alert thresholds. Opt in via the event store options: ```cs var store = DocumentStore.For(opts => { opts.Connection("..."); opts.Events.EnableExtendedProgressionTracking = true; }); ``` When enabled, the next schema apply adds six nullable columns to `pc_event_progression` (`heartbeat`, `agent_status`, `pause_reason`, `running_on_node`, `warning_behind_threshold`, `critical_behind_threshold`); the daemon writes its runtime agent state to them, and they are read back into `JasperFx.Events.Projections.ShardState`. The default is `false`, in which case no extra columns are created. `StoreOptions.Events` also implements the storage-agnostic `JasperFx.Events.IEventStoreInstrumentation` interface, whose `ExtendedProgressionEnabled` property is an alias for `EnableExtendedProgressionTracking`. This lets store-agnostic tooling toggle the same monitoring fidelity across Marten and Polecat without referencing store-specific types: ```cs IEventStoreInstrumentation instrumentation = store.Options.Events; instrumentation.ExtendedProgressionEnabled = true; ``` --- --- url: /documents/identity.md --- # Document Identity Every document in Polecat must have a unique identity. Polecat supports several identity strategies. ## Supported ID Types ### Guid (Default) ```cs public class User { public Guid Id { get; set; } public string Name { get; set; } = ""; } ``` When `Id` is `Guid.Empty`, Polecat will automatically assign a new `Guid` on `Store()`. ### String ```cs public class UserByEmail { public string Id { get; set; } = ""; public string Name { get; set; } = ""; } ``` String IDs must be assigned by the application before storing. ### Int / Long with HiLo ```cs public class Invoice { public int Id { get; set; } public decimal Amount { get; set; } } ``` Numeric IDs are automatically assigned using the [HiLo algorithm](#hilo-sequences). ## Strongly Typed IDs Polecat supports [strong typed identifiers](https://en.wikipedia.org/wiki/Strongly_typed_identifier) — types that wrap one of the supported primitive ID types (`Guid`, `string`, `int`, or `long`) so that an `OrderId` can never be passed where a `CustomerId` was meant. ### Supported Patterns Polecat detects wrapper types through JasperFx's `ValueTypeInfo`. A type qualifies when it exposes exactly one public, readable property wrapping a supported inner type, plus a way to build it from that value — either a matching constructor or a public static factory method. Two shapes follow: **1. Record struct with constructor:** ```cs public record struct OrderId(Guid Value); public class Order { public OrderId Id { get; set; } public string Name { get; set; } = ""; } ``` **2. Struct with a static builder method:** ```cs public readonly struct TaskId { private TaskId(Guid value) => Value = value; public Guid Value { get; } public static TaskId From(Guid value) => new TaskId(value); } public class TaskDoc { public TaskId Id { get; set; } public string Title { get; set; } = ""; } ``` There is **no naming requirement** — neither the wrapper type nor its inner property has to be called anything in particular. `Value` and `From` are simply the conventional names, and the ones both source generators below emit. ### Using Vogen or StronglyTypedId Hand-rolled wrappers work, but a generator gives you equality, comparison, `ToString()`, and — most importantly — a `System.Text.Json` converter that writes the *inner* value. That last part is what lets Polecat treat the member as a scalar in SQL. Polecat's test suite exercises both libraries directly. [Vogen](https://github.com/SteveDunn/Vogen) emits a private constructor plus a static `From` factory: ```cs [ValueObject] public readonly partial struct InvoiceId; public class Invoice { // Polecat will use this as the Invoice's identity public InvoiceId? Id { get; set; } public string Name { get; set; } = ""; } ``` [StronglyTypedId](https://github.com/andrewlock/StronglyTypedId) emits a public constructor: ```cs [StronglyTypedId(Template.Int)] public readonly partial struct OrderId; public class Order { public OrderId Id { get; set; } public string Name { get; set; } = ""; } ``` ::: warning **Vogen identities must be declared `Nullable`** — `InvoiceId? Id`, as above. Vogen forbids an uninitialized value object, so reading `.Value` off a `default` instance throws. Polecat inspects the identity to decide whether one needs assigning, and on a non-nullable Vogen id that inspection throws before Polecat can see that the id was unset. StronglyTypedId permits `default` and so can be declared either way. ::: ### Identity Assignment | Inner type | ID generation | | --- | --- | | `Guid` | Auto-assigned sequential `Guid` | | `int` | [HiLo sequence](#hilo-sequences) | | `long` | [HiLo sequence](#hilo-sequences) | | `string` | You assign it — Polecat never generates string identities | The identity column is created with the **inner** type (`uniqueidentifier`, `int`, `bigint`, `varchar`), not a serialized form of the wrapper. ### Usage ```cs // Store with an auto-assigned Guid wrapper var order = new Order { Name = "Widget" }; session.Store(order); await session.SaveChangesAsync(); // order.Id is now assigned // Load by the wrapper itself var loaded = await query.LoadAsync(order.Id); // ...or by the inner value; both resolve the same row var alsoLoaded = await query.LoadAsync(order.Id.Value); // LINQ queries take the wrapper type directly var result = await query.Query() .Where(x => x.Id == order.Id) .FirstOrDefaultAsync(); // IsOneOf for multiple ids var results = await query.Query() .Where(x => x.Id.IsOneOf(id1, id2, id3)) .ToListAsync(); // Select projects the wrapper back out var ids = await query.Query() .Select(x => x.Id) .ToListAsync(); // Delete by inner value session.Delete(order.Id.Value); // Check existence var exists = await query.CheckExistsAsync(order.Id.Value); ``` Supported across: * `Store()` / `Insert()` / `Update()`, with automatic identity assignment * `LoadAsync()` by the wrapper **or** by the inner value; `LoadManyAsync()` by inner value * `Delete()` by inner value or by document * `CheckExistsAsync()` by inner value * LINQ `Where`, `OrderBy` / `OrderByDescending`, `IsOneOf`, `Select`, `Count` * Paging via `ToPagedListAsync()` * Identity map sessions * Bulk insert via `BulkInsertAsync()` * Batched queries — `Load`, `LoadMany`, `CheckExists`, and `Query()` * Aggregate identities in event projections ::: warning `Delete()` and `CheckExistsAsync()` take the **inner** value (`order.Id.Value`), not the wrapper. Only `LoadAsync()` accepts either — see below. ::: #### Loading by the wrapper `LoadAsync(object id)` is the overload a strong typed identifier binds to, and it accepts either spelling: the wrapper (`order.Id`) or the inner value (`order.Id.Value`) resolve the same row, identity map included. The `Guid` / `string` / `int` / `long` overloads stay preferred by overload resolution, so adding it moved no existing call site — only an argument that fits none of them lands here. ```cs var byWrapper = await query.LoadAsync(order.Id); var byInner = await query.LoadAsync(order.Id.Value); ``` Passing an identity of any other type throws `ArgumentException` naming the document type and both id types, rather than failing later as an id-type mismatch. This is also the store-agnostic spelling: it satisfies `JasperFx.Events.Documents.IDocumentReadOperations.LoadAsync(object, CancellationToken)`, so source shared with Marten can do a by-id load of a strong-typed-id document against either store ([#472](https://github.com/JasperFx/polecat/issues/472), [jasperfx#665](https://github.com/JasperFx/jasperfx/issues/665)). ### Value Types on Other Members A wrapper is not limited to the identity. Any document member can be one, and it stays queryable: ```cs [ValueObject] public readonly partial struct TeacherId; public class ClassRoom { public Guid Id { get; set; } public TeacherId Teacher { get; set; } public string Subject { get; set; } = ""; } // Filter, order and project by the wrapper var rooms = await query.Query() .Where(x => x.Teacher == teacherId) .OrderBy(x => x.Teacher) .ToListAsync(); var teacherIds = await query.Query() .Select(x => x.Teacher) .ToListAsync(); ``` This works because the generator's JSON converter writes the inner value, so the member lands in the document JSON as a bare scalar. Polecat types the SQL from the inner type and unwraps the parameter for you. The same unwrapping applies to: * **Computed indexes** — `Schema.For().Index(x => x.Teacher)` creates a `uniqueidentifier` computed column, and the LINQ predicate matches it so the index stays seekable * **Flat table projections** — `map.Map(x => x.Account)` creates a column of the inner type ::: tip A wrapper written by hand, with no JSON converter, serializes as a nested object (`{"value": ...}`) and will **not** match a scalar predicate. Use Vogen or StronglyTypedId for any value type you intend to query on a non-identity member. ::: A type with more than one public property — `record struct Money(decimal Amount, Guid CurrencyId)` — is not a strong typed identifier and is left alone as a nested JSON object, so `x.Amount.CurrencyId` keeps resolving as a nested path. ### Registering a Value Type Polecat discovers value types on its own, so **you never need to register one**. The API exists purely so that a single store-configuration file can compile against both Marten and Polecat: ```cs // Compiles and means the same thing under either store opts.RegisterValueType(); opts.RegisterValueType(typeof(AlertId)); ``` Both overloads return JasperFx's `ValueTypeInfo`, matching Marten's signature. Registration resolves the type eagerly and validates it, so passing something that cannot be a value wrapper throws `InvalidValueTypeException` at configuration time rather than failing at the first query. ::: tip Marten's docs describe a timing issue where value types used in LINQ have to be registered before the first expression is evaluated. Polecat has no such issue — it resolves on demand and caches — so registration here is genuinely optional. ::: ### Not Currently Supported * No `LoadManyAsync()`, `Delete()` or `CheckExistsAsync()` overload taking the wrapper itself (`LoadAsync()` does — see [Loading by the wrapper](#loading-by-the-wrapper)) * No `Include()` LINQ operator * No compiled queries * F# single-case discriminated unions as identities ## HiLo Sequences For `int` and `long` ID types, Polecat uses the HiLo algorithm to generate unique IDs efficiently without round-tripping to the database for every insert. ### How It Works 1. The application reserves a block of IDs (the "Hi" value) from the `pc_hilo` table 2. IDs within the block are assigned sequentially in memory (the "Lo" values) 3. When the block is exhausted, a new "Hi" value is reserved ### Configuration ```cs // Global defaults opts.HiloSequenceDefaults.MaxLo = 500; // default is 1000 // Per-document type via attribute [HiloSequence(MaxLo = 100)] public class Invoice { public int Id { get; set; } public decimal Amount { get; set; } } ``` ### Resetting the Sequence Floor ```cs await store.Advanced.ResetHiloSequenceFloor(); ``` This scans existing documents and resets the HiLo sequence to start above the highest existing ID. ## Natural Keys with the \[Identity] Attribute If your document type uses a property name other than `Id` for its identity, you can use the `[Identity]` attribute to designate the identity property. This is common when migrating from other databases or when the natural key of the document has a more descriptive name. ```cs using Polecat.Attributes; public class Customer { [Identity] public string CustomerCode { get; set; } = ""; public string Name { get; set; } = ""; } ``` With the `[Identity]` attribute, Polecat will use `CustomerCode` as the document identity instead of looking for a conventional `Id` property. All standard operations work the same way: ```cs // Store var customer = new Customer { CustomerCode = "CUST-001", Name = "Acme" }; session.Store(customer); await session.SaveChangesAsync(); // Load by the identity value var loaded = await query.LoadAsync("CUST-001"); // Delete session.Delete("CUST-001"); ``` ### Priority When both an `[Identity]` attribute and a conventional `Id` property exist on the same document type, the `[Identity]` attribute takes priority: ```cs public class LegacyDoc { public Guid Id { get; set; } // Ignored by Polecat [Identity] public string DocumentId { get; set; } = ""; // Used as the identity } ``` ### Supported Types The `[Identity]` attribute works with all the same ID types as the conventional `Id` property: `Guid`, `string`, `int`, `long`, and [strongly typed ID wrappers](#strongly-typed-ids). ## ID Member Resolution Polecat resolves the identity property in the following priority order: 1. A property marked with `[Identity]` attribute 2. A public property named `Id` The property must be public with both a getter and setter. --- --- url: /events/dcb.md --- # Dynamic Consistency Boundary (DCB) The Dynamic Consistency Boundary (DCB) pattern allows you to query and enforce consistency across events from multiple streams using **tags** -- strong-typed identifiers attached to events at append time. This is useful when your consistency boundary doesn't align with a single event stream. ## Concept In traditional event sourcing, consistency is enforced per-stream using optimistic concurrency on the stream version. DCB extends this by letting you: 1. **Tag** events with one or more strong-typed identifiers 2. **Query** events across streams by those tags 3. **Aggregate** tagged events into a view (like a live aggregation, but cross-stream) 4. **Enforce consistency** at save time -- detecting if new matching events were appended since you last read Polecat uses a single append strategy (direct `INSERT` with `OUTPUT inserted.seq_id`), so DCB tags are always persisted immediately after each event insert. There are no separate append modes to configure -- tags just work. ## Registering Tag Types Tag types are strong-typed identifiers (typically `record` types wrapping a primitive). Register them during store configuration: ```cs public override async ValueTask InitializeAsync() { await StoreOptions(opts => { // Register tag types -- each gets its own table (pc_event_tag_student, pc_event_tag_course) opts.Events.RegisterTagType("student") .ForAggregate(); opts.Events.RegisterTagType("course") .ForAggregate(); }); } ``` snippet source | anchor Each tag type gets its own table (`pc_event_tag_student`, `pc_event_tag_course`, etc.) with a composite primary key of `(value, seq_id)`. ### Tag Type Requirements Tag types should be simple wrapper records around a primitive value: ```cs // Strong-typed tag identifiers public record StudentId(Guid Value); public record CourseId(Guid Value); ``` snippet source | anchor Supported inner value types: `Guid`, `string`, `int`, `long`, `short`. ## Tagging Events Use `BuildEvent` and `WithTag` to attach tags before appending: ```cs var enrolled = theSession.Events.BuildEvent(new StudentEnrolled("Alice", "Math")); enrolled.WithTag(studentId, courseId); theSession.Events.Append(streamId, enrolled); await theSession.SaveChangesAsync(TestContext.Current.CancellationToken); ``` snippet source | anchor Events can have multiple tags of different types. Tags are persisted to their respective tag tables in the same transaction as the event. ## Querying Events by Tags Use `EventTagQuery` to build a query, then execute it with `QueryByTagsAsync`: ```cs var query = new EventTagQuery().Or(studentId); var events = await session2.Events.QueryByTagsAsync(query, TestContext.Current.CancellationToken); ``` snippet source | anchor ### Multiple Tags (OR) ```cs var query = new EventTagQuery() .Or(student1) .Or(student2); var events = await session2.Events.QueryByTagsAsync(query, TestContext.Current.CancellationToken); ``` snippet source | anchor ### Filtering by Event Type ```cs var query = new EventTagQuery() .Or(studentId); var events = await session2.Events.QueryByTagsAsync(query, TestContext.Current.CancellationToken); ``` snippet source | anchor Events are always returned ordered by sequence number (global append order). ### Tag Predicates in LINQ (`HasTag`) Tag predicates can also compose directly into a LINQ `Where()` over an event query, alongside ordinary event predicates (timestamp, event type, stream). Use the `IEvent.HasTag(value)` marker method: ```cs var events = await session.Events.QueryAllRawEvents() .Where(e => e.HasTag(alice)) .ToListAsync(TestContext.Current.CancellationToken); ``` snippet source | anchor `HasTag` compiles to the same tag-table subquery that `QueryByTagsAsync` emits (a correlated `seq_id IN (SELECT seq_id FROM pc_event_tag_{suffix} WHERE value = @p)` predicate), and under conjoined tenancy it is automatically tenant-scoped. AND-ing several `HasTag` calls with normal predicates is supported: ```cs var events = await session.Events.QueryAllRawEvents() .Where(e => e.HasTag(alice) && e.HasTag(math) && e.Timestamp > cutoff) .ToListAsync(); ``` For OR-across-tags or the richer tag/event-type interplay, keep using the `EventTagQuery` builder with `QueryByTagsAsync` — that remains the rich escape hatch. `HasTag` is a marker method recognized by the LINQ provider; calling it outside a Polecat event query throws `NotSupportedException`, and using an unregistered tag type throws `InvalidOperationException`. ## Aggregating by Tags Build an aggregate from tagged events, similar to `AggregateStreamAsync` but across streams. First define an aggregate that applies the tagged events: ```cs // Aggregate for DCB public partial class StudentCourseEnrollment { public Guid Id { get; set; } public string StudentName { get; set; } = ""; public string CourseName { get; set; } = ""; public List Assignments { get; set; } = new(); public bool IsDropped { get; set; } public void Apply(StudentEnrolled e) { StudentName = e.StudentName; CourseName = e.CourseName; } public void Apply(AssignmentSubmitted e) { Assignments.Add(e.AssignmentName); } public void Apply(StudentDropped e) { IsDropped = true; } } ``` snippet source | anchor Then aggregate across streams by tag query: ```cs var query = new EventTagQuery() .Or(studentId) .Or(courseId); var aggregate = await session2.Events.AggregateByTagsAsync(query, TestContext.Current.CancellationToken); ``` snippet source | anchor Returns `null` if no matching events are found. ### Identity-less Boundary Aggregates The `StudentCourseEnrollment` aggregate above carries an `Id` property, so Polecat's source generator emits its evolver automatically and no extra annotation is needed. Some DCB aggregates, though, are *pure boundary aggregates*: they span streams only by tag and have **no single-stream identity** -- no `Id` property and no `[AggregateIdentity]`. For these, mark the aggregate type with `[BoundaryAggregate]` (from `JasperFx.Events.Aggregation`) so the source generator emits an evolver for it: ```csharp using JasperFx.Events.Aggregation; [BoundaryAggregate] public partial class CourseEnrollmentSummary { public int EnrolledCount { get; private set; } public List Students { get; } = new(); public void Apply(StudentEnrolled e) { Students.Add(e.StudentName); EnrolledCount++; } public void Apply(StudentDropped e) { EnrolledCount--; } } ``` Register it exactly like any other DCB aggregate -- the registration API is unchanged: ```csharp opts.Events.RegisterTagType("course") .ForAggregate(); ``` **Why the marker is required.** Without a single-stream identity the source generator can't infer a `TId`, so it *intentionally* emits nothing. A bare no-`Id` aggregate is far more often a forgotten `Id` property than a deliberate boundary aggregate, and silently generating an evolver would mask that mistake -- so `[BoundaryAggregate]` is the explicit opt-in. Without it, the DCB fetch/aggregate path fails fast at runtime: ``` InvalidProjectionException: No source-generated dispatcher found for SingleStreamProjection ``` (The `string` type argument is vestigial -- it matches the `SingleStreamProjection` the DCB aggregator builds and is never used by boundary-aggregate dispatch.) **Placement.** Put `[BoundaryAggregate]` on the aggregate type itself, and keep the type `partial` so the generator can attach the emitted dispatcher. The attribute must sit on the type **in its own defining assembly** -- that is the compilation the generator emits the `[assembly: GeneratedEvolver]` into, and it is the assembly the runtime scans (`typeof(T).Assembly`) when resolving the evolver. **Aggregates with an `Id` need no marker** and keep working unchanged -- `[BoundaryAggregate]` is only for the identity-less case. ::: tip `[BoundaryAggregate]` requires JasperFx.Events 2.0.0-alpha.21 / JasperFx.Events.SourceGenerator 2.0.0-alpha.13 or later. The marker is a JasperFx.Events source-generator concern, so its behavior is backend-agnostic -- it works identically across the Critter Stack event stores; only the surrounding `RegisterTagType<...>().ForAggregate()` registration is Polecat's SQL Server-backed API. ::: ## Fetch for Writing (Consistency Boundary) `FetchForWritingByTags` loads the aggregate and establishes a consistency boundary. At `SaveChangesAsync` time, Polecat checks whether any new events matching the query have been appended since the read, throwing `DcbConcurrencyException` if so: ```cs await using var session2 = theStore.LightweightSession(); var query = new EventTagQuery().Or(studentId); var boundary = await session2.Events.FetchForWritingByTags(query, TestContext.Current.CancellationToken); // Read current state var aggregate = boundary.Aggregate; // may be null if no events yet var lastSequence = boundary.LastSeenSequence; // Append via boundary var assignment = session2.Events.BuildEvent(new AssignmentSubmitted("HW1", 95)); assignment.WithTag(studentId, courseId); boundary.AppendOne(assignment); // Save -- will throw DcbConcurrencyException if another session // appended matching events after our read await session2.SaveChangesAsync(TestContext.Current.CancellationToken); ``` snippet source | anchor ### Handling Concurrency Violations ```cs public static async Task handling_a_concurrency_violation(IDocumentSession session) { try { await session.SaveChangesAsync(); } catch (DcbConcurrencyException violation) { // Reload and retry -- the boundary's tag query had new matching events since the read. // violation.Query is the original tag query, violation.LastSeenSequence the sequence // the boundary was read at. Console.WriteLine($"DCB violation on {violation.Query} at {violation.LastSeenSequence}"); } } ``` snippet source | anchor ::: tip The consistency check only detects events that match the **same tag query**. Events appended to unrelated tags or streams will not cause a violation. ::: ::: warning A session can establish more than one consistency boundary. If **several** of them are violated in the same `SaveChangesAsync()`, Polecat throws an `AggregateException` carrying one `DcbConcurrencyException` per violated boundary rather than a single exception. A session with one boundary -- the common case, and what the sample above shows -- always throws the `DcbConcurrencyException` directly. ::: ## Checking Event Existence If you only need to know whether any events matching a tag query exist -- without loading or deserializing them -- use `EventsExistAsync`. This is a lightweight existence check that avoids the overhead of fetching and materializing event data: ```cs [Fact] public async Task events_exist_returns_true_when_matching_events_found() { var studentId = new StudentId(Guid.NewGuid()); var courseId = new CourseId(Guid.NewGuid()); var streamId = Guid.NewGuid(); var enrolled = theSession.Events.BuildEvent(new StudentEnrolled("Alice", "Math")); enrolled.WithTag(studentId, courseId); theSession.Events.Append(streamId, enrolled); await theSession.SaveChangesAsync(TestContext.Current.CancellationToken); // Check existence -- lightweight, no event loading await using var session2 = theStore.LightweightSession(); var query = new EventTagQuery().Or(studentId); var exists = await session2.Events.EventsExistAsync(query, TestContext.Current.CancellationToken); exists.ShouldBeTrue(); } ``` snippet source | anchor This is useful for guard clauses and validation logic in DCB workflows where you need to check preconditions before appending new events. `EventsExistAsync` is also available in batch queries via `batch.EventsExist(query)`. ## How It Works ### Storage Each registered tag type creates a table: ```sql CREATE TABLE [dbo].[pc_event_tag_student] ( value uniqueidentifier NOT NULL, seq_id bigint NOT NULL, CONSTRAINT pk_pc_event_tag_student PRIMARY KEY (value, seq_id), CONSTRAINT fk_pc_event_tag_student_events FOREIGN KEY (seq_id) REFERENCES [dbo].[pc_events](seq_id) ON DELETE CASCADE ); ``` ### Consistency Check Polecat keeps a side table, `pc_dcb_tag_version`, holding one row per tag boundary with a counter of the commits that have touched it. `FetchForWritingByTags` reads the counter for every tag the query names; at `SaveChangesAsync` time, before any events are inserted, Polecat asserts the counter has not moved and bumps it with a single `UPDATE ... WHERE version = @captured`. Concurrent savers queue on that row's exclusive lock, and the loser gets a `DcbConcurrencyException`. Every save that appends a tagged event bumps the counter, boundary or not — so an ordinary `session.Events.Append(streamId, taggedEvent)` invalidates an in-flight boundary just as another boundary save would. A save that appends nothing does **not** assert its boundaries. A boundary is a condition on an append, so a session that read one and then wrote nothing has nothing for it to guard, and bumping the counter would invalidate other sessions over a save that changed nothing. ::: warning Through 5.20.0 this check was an `EXISTS` query over the tag tables. That is a non-locking predicate read: concurrent savers each ran it before any of them had committed, so none could see the others, and their event inserts did not collide either because a boundary routinely spans distinct streams. Every racer committed. If you rely on DCB boundaries under concurrent load, upgrade — and note that the new table is created by the ordinary schema migration, with no data migration needed. ::: ### Tag Routing Events appended via `IEventBoundary.AppendOne()` are automatically routed to streams based on their tags. Each tag value becomes the stream identity, so events with the same tag value end up in the same stream. --- --- url: /events/projections/efcore.md --- # EF Core Projections Polecat integrates with Entity Framework Core, allowing you to use DbContext within your event projections. This is provided by the `Polecat.EntityFrameworkCore` package. ## Installation ```shell dotnet add package Polecat.EntityFrameworkCore ``` ## Single Stream Projection with EF Core ```cs public class OrderDbContext : DbContext { public DbSet Orders { get; set; } = null!; public OrderDbContext(DbContextOptions options) : base(options) { } } public class OrderEfProjection : EfCoreSingleStreamProjection { public OrderReadModel Create(OrderCreated e, OrderDbContext db) { var model = new OrderReadModel { Status = "Created", Amount = e.Amount }; db.Orders.Add(model); return model; } public void Apply(OrderShipped e, OrderReadModel current, OrderDbContext db) { current.Status = "Shipped"; current.ShippedDate = e.ShippedAt; } } ``` ## Multi Stream Projection with EF Core ```cs public class CustomerEfProjection : EfCoreMultiStreamProjection { public CustomerEfProjection() { Identity(e => e.CustomerId); } public CustomerDashboard Create(OrderCreated e, AppDbContext db) { return new CustomerDashboard { TotalOrders = 1, TotalSpent = e.Amount }; } public void Apply(OrderCreated e, CustomerDashboard current, AppDbContext db) { current.TotalOrders++; current.TotalSpent += e.Amount; } } ``` ## Event Projection with EF Core ```cs public class AuditEfProjection : EfCoreEventProjection { public void Project(IEvent @event, AppDbContext db) { db.AuditEntries.Add(new AuditEntry { Action = "OrderCreated", Timestamp = @event.Timestamp }); } } ``` ## Schema Migration with AddEntityTablesFromDbContext By default, EF Core entity tables must be created separately (e.g., via `dotnet ef database update` or `Database.GenerateCreateScript()`). If you want Polecat's Weasel migration pipeline to manage your EF Core entity tables alongside Polecat's own schema objects, use `AddEntityTablesFromDbContext`: ```cs var store = DocumentStore.For(opts => { opts.ConnectionString = connectionString; opts.AddEntityTablesFromDbContext(); }); ``` This reads the entity model from your `DbContext` and registers each table with Polecat's Weasel migration system. When `ApplyAllConfiguredChangesToDatabaseAsync()` runs, EF Core entity tables will be created and migrated automatically. ### Schema Handling `AddEntityTablesFromDbContext` respects explicit schema configuration in your EF Core model: * **Entities with an explicit schema** (via `HasDefaultSchema()` or `ToTable("name", "schema")`) will retain their configured schema * **Entities without an explicit schema** will use the SQL Server default schema (`dbo`) ```cs // This DbContext places entities in a custom schema public class MyDbContext : DbContext { protected override void OnModelCreating(ModelBuilder modelBuilder) { modelBuilder.HasDefaultSchema("my_schema"); modelBuilder.Entity().ToTable("orders"); } } var store = DocumentStore.For(opts => { opts.ConnectionString = connectionString; opts.DatabaseSchemaName = "polecat_schema"; // The "orders" table will be created in "my_schema", NOT "polecat_schema" opts.AddEntityTablesFromDbContext(); }); ``` ### Combining with Projections You can use `AddEntityTablesFromDbContext` alongside EF Core projections for a fully integrated setup: ```cs var store = DocumentStore.For(opts => { opts.ConnectionString = connectionString; opts.Projections.Add( opts, new OrderEfProjection(), ProjectionLifecycle.Inline); // Also register the entity tables for Weasel migration opts.AddEntityTablesFromDbContext(); }); // This single call now creates both Polecat event tables and EF Core entity tables await store.Database.ApplyAllConfiguredChangesToDatabaseAsync(); ``` ## How It Works 1. Polecat creates a placeholder `SqlConnection` and `DbContext` for each projection batch 2. During `SaveChangesAsync`, the placeholder connection is swapped for the real connection and transaction 3. `DbContext.SaveChangesAsync()` is called as a `ITransactionParticipant` within Polecat's transaction 4. Both Polecat document operations and EF Core changes commit atomically ## Registration ```cs opts.Projections.Add(ProjectionLifecycle.Inline); // or opts.Projections.Add(ProjectionLifecycle.Async); ``` ## Lifecycle Support EF Core projections support all lifecycle modes: | Mode | Description | | :--- | :--- | | Inline | DbContext changes commit in same transaction as events | | Async | Daemon processes events and applies DbContext changes | | Live | Aggregate built on demand with DbContext | ## Tenanted Projections EF Core projections work with multi-tenancy -- the DbContext receives the correct connection for the tenant's database. ::: tip The `ITransactionParticipant` pattern ensures that EF Core's `SaveChanges` runs within Polecat's transaction boundary, providing atomic consistency between event-sourced projections and EF Core-managed tables. ::: --- --- url: /events/metadata.md --- # Event Metadata Polecat stores rich metadata alongside each event. ## Built-in Metadata Every event automatically includes: | Field | Description | | :--- | :--- | | `Id` | Unique event identifier (Guid) | | `Sequence` | Global sequence number (auto-incremented) | | `Version` | Position within the stream | | `Timestamp` | When the event was recorded | | `EventTypeName` | Snake\_case type name | | `DotNetType` | Full .NET type name | ## Correlation and Causation Track request flow and event chains: ```cs await using var session = store.LightweightSession(new SessionOptions { CorrelationId = "http-request-abc", CausationId = "command-create-order" }); session.Events.Append(streamId, new OrderCreated(...)); await session.SaveChangesAsync(); ``` These values are stored in the `correlation_id` and `causation_id` columns on `pc_events`. ## Custom Headers Attach arbitrary key-value metadata to events: ```cs var action = session.Events.Append(streamId, new OrderCreated(...)); action.Headers = new Dictionary { ["user_agent"] = "MyApp/1.0", ["ip_address"] = "192.168.1.1" }; await session.SaveChangesAsync(); ``` Headers are stored as JSON in the `headers` column. ## Tenant ID In multi-tenant configurations, the tenant ID is automatically recorded: ```cs await using var session = store.LightweightSession(new SessionOptions { TenantId = "tenant-abc" }); session.Events.Append(streamId, new OrderCreated(...)); // Event is stored with tenant_id = "tenant-abc" ``` ## Accessing Metadata When loading events, all metadata is available on the `IEvent` wrapper: ```cs var events = await session.Events.FetchStreamAsync(streamId); foreach (var @event in events) { Console.WriteLine($"Seq: {@event.Sequence}"); Console.WriteLine($"Version: {@event.Version}"); Console.WriteLine($"Time: {@event.Timestamp}"); Console.WriteLine($"Correlation: {@event.CorrelationId}"); Console.WriteLine($"Causation: {@event.CausationId}"); Console.WriteLine($"Tenant: {@event.TenantId}"); Console.WriteLine($"Headers: {JsonSerializer.Serialize(@event.Headers)}"); } ``` ## Using Metadata in Projections Projections can access event metadata through the `IEvent` wrapper: ```cs public class OrderSummary { public Guid Id { get; set; } public DateTimeOffset CreatedAt { get; set; } public string? CreatedBy { get; set; } public void Apply(IEvent @event) { CreatedAt = @event.Timestamp; CreatedBy = @event.Headers?["user"]?.ToString(); } } ``` --- --- url: /events/multitenancy.md --- # Event Multi-Tenancy Polecat supports multi-tenancy in the event store through conjoined tenancy (shared tables) or separate database tenancy. ## Conjoined Tenancy Enable tenant isolation within shared event tables: ```cs var store = DocumentStore.For(opts => { opts.Connection("..."); opts.Events.TenancyStyle = TenancyStyle.Conjoined; }); ``` With conjoined tenancy: * All events include a `tenant_id` column * Stream queries automatically filter by tenant * Event appending records the session's tenant ID * The async daemon processes events per-tenant ### Using Tenanted Events ```cs await using var session = store.LightweightSession(new SessionOptions { TenantId = "tenant-abc" }); // Events are stored with tenant_id = "tenant-abc" session.Events.StartStream( new OrderCreated(100m, "Widget") ); await session.SaveChangesAsync(); // Only loads events for "tenant-abc" var order = await session.Events.AggregateStreamAsync(streamId); ``` ## Separate Database Tenancy Each tenant gets its own database with independent event stores: ```cs var store = DocumentStore.For(opts => { opts.MultiTenantedDatabases(databases => { databases.AddTenant("tenant-a", "Server=localhost;Database=events_tenant_a;..."); databases.AddTenant("tenant-b", "Server=localhost;Database=events_tenant_b;..."); }); }); ``` With separate database tenancy: * Each tenant has completely isolated event data * The async daemon runs one daemon per tenant database, each with its own high-water mark and projection progress * Schema management is independent per database; `ApplyAllDatabaseChangesOnStartup()` migrates every tenant database ::: warning No default tenant — and no top level connection string `MultiTenantedDatabases()` (and `MultiTenantedMasterTable()`) sets `StoreOptions.DefaultTenantUsageEnabled` to `false`. Do **not** register a placeholder `*DEFAULT*` tenant, and you do **not** need to set `StoreOptions.ConnectionString` — the tenancy supplies one. Opening a session or building a daemon without a tenant throws `DefaultTenantUsageDisabledException` rather than quietly using whichever database happened to be first. The async daemon starts one daemon per tenant database with the default tenant disabled, and `ApplyAllDatabaseChangesOnStartup()` migrates every tenant database. See [No default tenant is required](/configuration/multitenancy#no-default-tenant-is-required). ::: ## Default Tenant When no tenant ID is specified, events are stored with `tenant_id = 'DEFAULT'`. In single-tenant mode, all queries use this default value. Under **separate database** tenancy the default tenant is disabled, so there is no `'DEFAULT'` fallback — every session must name its tenant. See the warning above. ## Per-Tenant Event Partitioning ::: tip This is an advanced, opt-in option for large multi-tenanted event stores where a single shared event sequence becomes a scalability bottleneck. It builds on conjoined event tenancy by giving each tenant its own event numbering, so a tenant-scoped projection rebuild doesn't have to pay for a database-wide event-sequence scan. Tracked in [polecat#163](https://github.com/JasperFx/polecat/issues/163) / [CritterStack #209](https://github.com/JasperFx/CritterWatch/issues/209). ::: By default every tenant shares the global `seq_id BIGINT IDENTITY` on `pc_events`. With **per-tenant event partitioning**, each tenant instead gets its own event sequence: ```cs var store = DocumentStore.For(opts => { opts.Connection("..."); // Per-tenant partitioning builds on conjoined event tenancy opts.Events.TenancyStyle = TenancyStyle.Conjoined; // Opt into per-tenant event numbering opts.EventGraph.UseTenantPartitionedEvents = true; }); ``` When enabled, Polecat: * **Maintains a tenant registry** — `pc_tenant_partitions` maps each `tenant_id` to a compact integer `ordinal`, populated the first time a tenant appends events (via Weasel.SqlServer's managed tenant partitioning). * **Gives each tenant its own sequence** — `seq_id` is drawn from a per-tenant `pc_events_sequence_{ordinal}` object (created on demand) via `NEXT VALUE FOR`, rather than a single global `IDENTITY`. `seq_id` is therefore unique only *within* a tenant, so the `pc_events` primary key becomes composite `(tenant_ordinal, seq_id)`. * **Physically partitions `pc_events` and `pc_streams` by tenant** — both tables are `RANGE RIGHT` partitioned on the tenant `ordinal`, and a new partition is split in as each tenant registers. A tenant's events and stream rows live in their own physical partitions, so per-tenant scans and rebuilds touch only those partitions (Marten parity: `mt_streams` rides `mt_events`' tenant partitioning). ```cs // Each tenant's seq_id starts at 1 and advances independently await using var red = store.LightweightSession(new SessionOptions { TenantId = "Red" }); red.Events.StartStream(redStream, new QuestStarted("Red")); // Red seq_id 1 await red.SaveChangesAsync(); await using var blue = store.LightweightSession(new SessionOptions { TenantId = "Blue" }); blue.Events.StartStream(blueStream, new QuestStarted("Blue")); // Blue seq_id 1 — independent await blue.SaveChangesAsync(); ``` ### Tenant-aware async daemon The asynchronous projection daemon is per-tenant aware under this flag. Rather than one database-wide high-water scan, it polls a **per-tenant high-water vector** in a single round-trip (joining `pc_tenant_partitions` → each tenant's `pc_events_sequence` value → `pc_event_progression`), and tracks each tenant's projection progress independently. The headline benefit is **bounded, isolated per-tenant rebuilds**: ```cs using var daemon = (IProjectionDaemon)await store.BuildProjectionDaemonAsync(); // Rebuild a projection for ONE tenant — replays only that tenant's bounded sequence range and // resets only that tenant's (projection, tenant) progression. Other tenants keep running untouched. await daemon.RebuildProjectionAsync("QuestParty", "Red", CancellationToken.None); // Or fan out an isolated rebuild across every registered tenant: await CrossTenantRebuild.RebuildEverywhereAsync( daemon, "QuestParty", timeout: 1.Minutes(), CancellationToken.None); ``` A tenant-scoped rebuild never pays for a database-wide event scan and never disturbs other tenants — exactly the [#209](https://github.com/JasperFx/CritterWatch/issues/209) win for very large multi-tenanted stores. ::: warning `UseTenantPartitionedEvents` defaults to `false`; existing stores keep the global `IDENTITY` append path byte-for-byte. The flag **requires** `TenancyStyle.Conjoined` (there is nothing to partition by otherwise) and is currently incompatible with `UseArchivedStreamPartitioning` — a SQL Server table supports only one partition scheme; both raise an error at store construction. Physical partitioning applies to `pc_events` and (since #335) `pc_streams`. The partition function/scheme are database-global objects, so a single database should host one tenant-partitioned event store. Document tables can join the same managed per-tenant partitioning — including runtime tenant onboarding/removal via `store.Advanced.AddPolecatManagedTenantsAsync` / `RemovePolecatManagedTenantsAsync` — see [Document multi-tenancy partitioning](/documents/partitioning#managed-per-tenant-partitioning-335). ::: --- --- url: /events/projections/event-projections.md --- # Event Projections Event projections process individual events to create, modify, or delete documents. Unlike aggregate projections, they don't maintain per-stream state. ## Defining an Event Projection ```cs public class AuditLogProjection : EventProjection { public void Project(IEvent @event, IDocumentOperations ops) { ops.Store(new AuditEntry { Id = Guid.NewGuid(), Action = "OrderCreated", StreamId = @event.StreamId, Timestamp = @event.Timestamp, Details = $"Order created for {@@event.Data.Amount:C}" }); } public void Project(IEvent @event, IDocumentOperations ops) { ops.Store(new AuditEntry { Id = Guid.NewGuid(), Action = "OrderCancelled", StreamId = @event.StreamId, Timestamp = @event.Timestamp, Details = "Order was cancelled" }); } } ``` The examples name the event parameter `@event`, but that is not required: Polecat identifies the event argument **by type**. In `Project(IEvent @event, IDocumentOperations ops)`, the `IEvent` parameter is the event regardless of its name (`IDocumentOperations` is an interface and is never treated as the event). You only need a conventional event parameter **name** — `@event`, `event`, `e`, or `ev` — to disambiguate a signature in which more than one parameter could be the event. This is the same rule used by aggregate projections; see [Identifying the Event Parameter](/events/projections/single-stream-projections#identifying-the-event-parameter). ## Registration ```cs opts.Projections.Add(ProjectionLifecycle.Inline); // or opts.Projections.Add(ProjectionLifecycle.Async); ``` ## Use Cases Event projections are ideal for: * **Audit logs** -- Create a record for each significant event * **Search indexes** -- Maintain denormalized documents for search * **Notifications** -- Create notification records per event * **Cross-cutting concerns** -- Track metrics, analytics, or compliance data ## Accessing Session Operations The `IDocumentOperations` parameter gives you full access to document operations: ```cs public void Project(IEvent @event, IDocumentOperations ops) { // Store new documents ops.Store(new DeactivationRecord { ... }); // Delete documents ops.Delete(@event.Data.UserId); // Patch existing documents ops.Patch(@event.Data.UserId) .Set(x => x.IsActive, false); } ``` ## Event Enrichment `EventProjection` supports an `EnrichEventsAsync` hook that runs **before** individual events are processed. This allows you to batch-load reference data from the database and enrich events with it, avoiding N+1 query problems. ::: warning Event enrichment is designed for **read model / query model** projections processed by the async daemon or inline during `SaveChangesAsync`. It is **not** called during `FetchForWriting()` or `FetchLatest()`. Avoid depending on enriched data in write model aggregates used with those APIs. ::: Override `EnrichEventsAsync` in your `EventProjection` subclass: ```cs public class TaskSummaryProjection : EventProjection { public TaskSummaryProjection() { Project((e, ops) => { ops.Store(new TaskSummary { Id = e.TaskId, AssignedUserName = e.UserName // Set by enrichment }); }); } public override async Task EnrichEventsAsync( IQuerySession querySession, IReadOnlyList events, CancellationToken cancellation) { var assigned = events.OfType>().ToArray(); if (assigned.Length == 0) return; var userIds = assigned.Select(e => e.Data.UserId).Distinct().ToArray(); foreach (var userId in userIds) { var user = await querySession.LoadAsync(userId, cancellation); if (user != null) { foreach (var e in assigned.Where(a => a.Data.UserId == userId)) { e.Data.UserName = user.Name; } } } } } ``` The method is called once per tenant batch before any `Project` handlers run. Modifications to event data properties are visible to all subsequent handlers in the batch. --- --- url: /events/storage.md --- # Event Storage Polecat stores events in SQL Server 2025 using three core tables. ## pc\_events The main event log table: | Column | Type | Description | | :--- | :--- | :--- | | `seq_id` | `bigint IDENTITY(1,1)` | Global sequence number (PK) | | `id` | `uniqueidentifier` | Unique event ID | | `stream_id` | `uniqueidentifier` or `nvarchar(250)` | Stream identifier | | `version` | `int` | Position within the stream | | `data` | `json` | Serialized event body | | `bdata` | `varbinary(max)` | Binary event body, `NULL` unless the event type opted in — see [Binary Event Serialization](/events/binary-serialization) | | `type` | `nvarchar(250)` | Event type name (snake\_case) | | `timestamp` | `datetimeoffset` | When the event was recorded | | `tenant_id` | `nvarchar(250)` | Tenant identifier | | `dotnet_type` | `nvarchar(500)` | Full .NET type name | | `correlation_id` | `nvarchar(250)` | Request correlation | | `causation_id` | `nvarchar(250)` | Event causation chain | | `headers` | `json` | Custom metadata headers | | `is_archived` | `bit` | Archive flag | ## pc\_streams Stream metadata: | Column | Type | Description | | :--- | :--- | :--- | | `id` | `uniqueidentifier` or `nvarchar(250)` | Stream identifier (PK) | | `type` | `nvarchar(250)` | Aggregate type name | | `version` | `int` | Current stream version | | `timestamp` | `datetimeoffset` | Last event timestamp | | `created` | `datetimeoffset` | Stream creation time | | `tenant_id` | `nvarchar(250)` | Tenant identifier | | `is_archived` | `bit` | Archive flag | ## pc\_event\_progression Async daemon progress tracking: | Column | Type | Description | | :--- | :--- | :--- | | `name` | `nvarchar(250)` | Projection/subscription name (PK) | | `last_seq_id` | `bigint` | Last processed sequence ID | | `last_updated` | `datetimeoffset` | Last update timestamp | ## Event Type Naming Polecat converts .NET event type names to snake\_case for storage: * `QuestStarted` → `quest_started` * `MembersJoined` → `members_joined` * `InvoiceLineItemAdded` → `invoice_line_item_added` ## Pre-registering Event Types Event types are registered with the store on the fly the first time they are appended, so registering them up front is optional. It is worth doing when the *reading* side may meet an event type before the writing side does -- most commonly the async daemon in a separate process, which has to map an `event_type_name` back to a .NET type it has not yet seen appended: ```cs var store = DocumentStore.For(opts => { opts.Connection(connectionString); opts.Events.AddEventType(); opts.Events.AddEventType(typeof(MembersJoined)); opts.Events.AddEventTypes([typeof(MembersDeparted), typeof(QuestEnded)]); }); ``` This mirrors Marten's `StoreOptions.Events.AddEventType()` / `AddEventTypes(...)`. ## Sequence IDs Events are assigned a global, monotonically increasing sequence ID via SQL Server's `IDENTITY(1,1)`. This provides a total ordering of all events across all streams, which is critical for the async daemon's event processing. ## JSON Storage Event data and headers are stored using SQL Server 2025's native `json` type, serialized with System.Text.Json. --- --- url: /events/quickstart.md --- # Event Store Quick Start This guide walks you through setting up event sourcing with Polecat. ## Define Your Events Events are simple .NET classes or records that describe something that happened: ```cs public record QuestStarted(string Name); public record MembersJoined(string Location, string[] Members); public record MembersDeparted(string Location, string[] Members); public record QuestEnded(string Name); ``` ## Define Your Aggregate An aggregate is a projection of events into a domain object: ```cs public class QuestParty { public Guid Id { get; set; } public string Name { get; set; } = ""; public List Members { get; set; } = new(); public void Apply(QuestStarted started) { Name = started.Name; } public void Apply(MembersJoined joined) { Members.AddRange(joined.Members); } public void Apply(MembersDeparted departed) { foreach (var member in departed.Members) Members.Remove(member); } } ``` ## Start a Stream and Append Events ```cs var store = DocumentStore.For(opts => { opts.Connection("Server=localhost,1433;Database=myapp;User Id=sa;Password=YourStrong!Password;TrustServerCertificate=True"); }); await using var session = store.LightweightSession(); // Start a new stream with initial events var questId = session.Events.StartStream( new QuestStarted("Destroy the Ring"), new MembersJoined("Rivendell", ["Frodo", "Sam", "Aragorn", "Gandalf"]) ); await session.SaveChangesAsync(); ``` ## Append More Events ```cs await using var session = store.LightweightSession(); session.Events.Append(questId, new MembersJoined("Moria", ["Gimli", "Legolas"]), new MembersDeparted("Moria", ["Gandalf"]) ); await session.SaveChangesAsync(); ``` ## Live Aggregation Replay all events to build the current state: ```cs var party = await session.Events.AggregateStreamAsync(questId); // party.Name == "Destroy the Ring" // party.Members == ["Frodo", "Sam", "Aragorn", "Gimli", "Legolas"] ``` ## Inline Projections For strong consistency, register an inline projection that updates automatically: ```cs var store = DocumentStore.For(opts => { opts.Connection("..."); opts.Projections.Add>(ProjectionLifecycle.Inline); }); // Now QuestParty is automatically updated in the same transaction await using var session = store.LightweightSession(); session.Events.StartStream( new QuestStarted("Destroy the Ring") ); await session.SaveChangesAsync(); // Load the projection directly as a document var party = await session.LoadAsync(questId); ``` ## Optimistic Concurrency Append with an expected version to prevent lost updates: ```cs session.Events.Append(questId, expectedVersion: 3, new MembersDeparted("Moria", ["Gandalf"]) ); // Throws EventStreamUnexpectedMaxEventIdException if version doesn't match await session.SaveChangesAsync(); ``` ## Next Steps * [Appending Events](/events/appending) -- Detailed append options * [Querying Events](/events/querying) -- Loading streams and fetching events * [Projections](/events/projections/) -- Building read models from events * [Async Daemon](/events/projections/async-daemon) -- Background projection processing --- --- url: /events/subscriptions.md --- # Event Subscriptions Polecat supports event subscriptions for push-based processing of events as they are appended. ## ISubscription Interface Implement `ISubscription` to process events: ```cs public class OrderNotificationSubscription : SubscriptionBase { public override async Task ProcessEventsAsync( EventRange page, ISubscriptionController controller, IDocumentOperations operations, CancellationToken ct) { foreach (var @event in page.Events) { if (@event.Data is OrderCreated created) { // Send notification, update external system, etc. await SendNotification(created); } } } } ``` ## Registering Subscriptions ```cs var store = DocumentStore.For(opts => { opts.Connection("..."); opts.Projections.Subscribe(new OrderNotificationSubscription()); }); ``` ## How Subscriptions Work Subscriptions are processed by the async daemon alongside projections: 1. The daemon tracks progression via `pc_event_progression` 2. Events are loaded in batches 3. Your subscription's `ProcessEventsAsync` is called for each batch 4. Progression is updated after successful processing ## Subscriptions vs Projections | Feature | Subscription | Projection | | :--- | :--- | :--- | | Purpose | Side effects (notifications, external systems) | Read model construction | | Output | Arbitrary | Documents or flat tables | | Replay | May not be idempotent | Should be idempotent | | Processing | Sequential batches | Sequential batches | ## SubscriptionBase The `SubscriptionBase` class provides a convenient base with default implementations. Override `ProcessEventsAsync` to handle events. ::: tip Unlike projections, subscriptions are intended for side effects like sending emails, updating external systems, or triggering workflows. They are not expected to be idempotent or replayable. ::: --- --- url: /schema/exporting.md --- # Exporting Schema Definition Polecat can export the complete database schema as SQL scripts for review, version control, or manual deployment. ## ToDatabaseScript Generate the complete DDL as a string: ```cs var script = await store.Advanced.ToDatabaseScript(); Console.WriteLine(script); ``` This returns `CREATE TABLE` statements for all configured document tables and event store tables. ## WriteCreationScriptToFileAsync Save the schema script to a file: ```cs await store.Advanced.WriteCreationScriptToFileAsync("schema.sql"); ``` ## Use Cases * **Code review** -- Include schema scripts in pull requests * **Version control** -- Track schema changes over time * **Manual deployment** -- Apply scripts to production databases manually * **Documentation** -- Understand the database structure * **DBA review** -- Allow database administrators to review before deployment ## Example Output ```sql -- Polecat Schema Script CREATE TABLE dbo.pc_streams ( id uniqueidentifier NOT NULL PRIMARY KEY, type nvarchar(250) NULL, version int NOT NULL DEFAULT 0, ... ); CREATE TABLE dbo.pc_events ( seq_id bigint IDENTITY(1,1) NOT NULL PRIMARY KEY, id uniqueidentifier NOT NULL, stream_id uniqueidentifier NOT NULL, ... ); CREATE TABLE dbo.pc_event_progression ( name nvarchar(250) NOT NULL PRIMARY KEY, last_seq_id bigint NOT NULL DEFAULT 0, ... ); CREATE TABLE dbo.pc_doc_user ( id uniqueidentifier NOT NULL PRIMARY KEY, data json NOT NULL, ... ); ``` --- --- url: /events/projections/flat.md --- # Flat Table Projections Flat table projections write events directly to SQL Server tables with defined columns, rather than as JSON documents. This is ideal for reporting and analytics scenarios. ## Defining a Flat Table Projection ```cs public class OrderFlatTableProjection : FlatTableProjection { public OrderFlatTableProjection() : base("order_history") { // Define table columns Table.AddColumn("id").AsPrimaryKey(); Table.AddColumn("stream_id"); Table.AddColumn("status"); Table.AddColumn("amount"); Table.AddColumn("event_time"); // Map events to columns Project(map => { map.Map(e => e.Amount, "amount"); map.SetValue("status", "Created"); }); Project(map => { map.SetValue("status", "Shipped"); }); // Delete rows on certain events Delete(); } } ``` ## Registration ```cs opts.Projections.Add(ProjectionLifecycle.Async); ``` ## Column Mapping ### Map Event Properties ```cs map.Map(e => e.Amount, "amount"); map.Map(e => e.CustomerName, "customer_name"); ``` ### Set Static Values ```cs map.SetValue("status", "Shipped"); map.SetValue("processed", true); ``` ## SQL Generation Flat table projections use SQL Server `MERGE` statements for upsert behavior, ensuring idempotent processing. ## Use Cases * **Reporting tables** -- Denormalized data for BI tools * **Analytics** -- Pre-computed metrics per event * **Audit trails** -- Structured event history in relational format * **Integration** -- Data accessible by non-.NET systems via standard SQL ::: tip Flat table projections bypass the document storage layer entirely, writing directly to user-defined SQL tables. This makes the data accessible to any SQL tool without understanding JSON document structure. ::: ## Using EventProjection for Flat Tables ::: tip The `EventProjection` approach shown below is more explicit code than `FlatTableProjection`, but it is also more flexible. Use `EventProjection` when you need full control over the SQL being generated, need to access event metadata through the `IEvent` envelope, or when the declarative `FlatTableProjection` API does not support your use case. The tradeoff is that you are writing raw SQL yourself, so you are responsible for getting the SQL correct and handling upsert logic on your own. ::: As an alternative to the more rigid `FlatTableProjection` approach, you can use Polecat's `EventProjection` as a base class and write explicit SQL to project events into a flat table. This gives you complete control over the SQL operations and full access to event metadata: ```cs public partial class ImportSqlProjection: EventProjection { // Use the IEvent envelope to access event metadata // like stream identity and timestamps public void Project(IEvent e, IDocumentSession ops) { ops.QueueSqlCommand( "insert into import_history (id, activity_type, customer_id, started) values (?, ?, ?, ?)", e.StreamId, e.Data.ActivityType, e.Data.CustomerId, e.Data.Started ); } public void Project(IEvent e, IDocumentSession ops) { ops.QueueSqlCommand( "update import_history set finished = ? where id = ?", e.Data.Finished, e.StreamId ); } // You can use any SQL operation, including deletes public void Project(IEvent e, IDocumentSession ops) { ops.QueueSqlCommand( "delete from import_history where id = ?", e.StreamId ); } } ``` snippet source | anchor A couple notes about the `EventProjection` approach: * **Batched execution** -- The `QueueSqlCommand()` method doesn't execute inline. Instead, it adds the SQL to be executed in a batch when you call `IDocumentSession.SaveChangesAsync()`. This batching reduces network round trips to the database and is a consistent performance win. * **Event metadata access** -- The `Project()` methods use `IEvent` envelope types, giving you access to event metadata like timestamps, version information, and stream identity. This is something the declarative `FlatTableProjection` cannot currently provide. * **Full SQL control** -- You can write any SQL you need: inserts, updates, deletes, or even complex statements with subqueries. This is useful when your projection logic doesn't fit the `Map`/`Increment`/`SetValue` patterns of `FlatTableProjection`. --- --- url: /getting-started.md --- # Getting Started Polecat integrates with the standard .NET `IServiceCollection` abstractions for IoC registration. Most features work without IoC, but the async daemon and schema management leverage the `IHost` model. ## Installation ::: code-group ```shell [.NET CLI] dotnet add package Polecat ``` ```powershell [Powershell] PM> Install-Package Polecat ``` ::: ## SQL Server 2025 Polecat requires **SQL Server 2025** (v17) or later for its native JSON type support. For local development, the easiest approach is Docker: ```yaml # docker-compose.yml services: sqlserver: image: mcr.microsoft.com/mssql/server:2025-CU1-ubuntu-24.04 environment: ACCEPT_EULA: "Y" MSSQL_SA_PASSWORD: "YourStrong!Password" ports: - "1433:1433" ``` ## Registering Polecat In your application startup, call `AddPolecat()` to register all services: ```cs builder.Services.AddPolecat(options => { // Connection string to your SQL Server 2025 database options.Connection("Server=localhost;Database=myapp;User Id=sa;Password=YourStrong!Password;TrustServerCertificate=True"); // Optionally change the default schema (default is "dbo") options.DatabaseSchemaName = "myschema"; }) // Run Weasel schema migration on startup so the pc_* tables exist // before the first session is opened. Recommended in development; // in production you'll typically generate scripts ahead of time instead. .ApplyAllDatabaseChangesOnStartup(); ``` If you have async projections registered, also opt the daemon into the host: ```cs builder.Services.AddPolecat(options => { options.Connection("..."); options.Projections.Add(ProjectionLifecycle.Async); }) .ApplyAllDatabaseChangesOnStartup() .AddAsyncDaemon(DaemonMode.Solo); ``` See [Bootstrapping Polecat](/configuration/hostbuilder) for more options. ::: tip `AddPolecat()` registers `IDocumentStore` as a singleton, and `IDocumentSession` / `IQuerySession` as scoped services. In most cases you should inject a session directly. ::: ::: warning `AddPolecat()` only registers services. The pc\_\* event store and document tables are not created until either `ApplyAllDatabaseChangesOnStartup()` runs (recommended) or the first session triggers `AutoCreateSchemaObjects`. Calling `store.Advanced.CleanAllEventDataAsync()` before either has happened on a brand-new database is a no-op rather than an error. ::: ## Working with Documents Define a simple document type: ```cs public class User { public Guid Id { get; set; } public required string FirstName { get; set; } public required string LastName { get; set; } public bool Internal { get; set; } } ``` *For more information on document identity, see [identity](/documents/identity).* Use `IDocumentSession` to store and query documents: ```cs // Store a document app.MapPost("/user", async (CreateUserRequest create, IDocumentSession session) => { var user = new User { FirstName = create.FirstName, LastName = create.LastName, Internal = create.Internal }; session.Store(user); await session.SaveChangesAsync(); }); // Query with LINQ app.MapGet("/users", async (bool internalOnly, IDocumentSession session, CancellationToken ct) => { return await session.Query() .Where(x => x.Internal == internalOnly) .ToListAsync(ct); }); // Load by ID app.MapGet("/user/{id:guid}", async (Guid id, IQuerySession session, CancellationToken ct) => { return await session.LoadAsync(id, ct); }); ``` For more information on querying, check [document querying](/documents/querying/). ## Working with Events Please check the [Event Store quick start](/events/quickstart). ## Creating a Standalone Store You can create a document store outside of the generic host infrastructure using `DocumentStore.For`: ```cs var store = DocumentStore.For("Server=localhost;Database=myapp;User Id=sa;Password=YourStrong!Password;TrustServerCertificate=True"); ``` Or with full configuration: ```cs var store = DocumentStore.For(opts => { opts.Connection("Server=localhost;Database=myapp;User Id=sa;Password=YourStrong!Password;TrustServerCertificate=True"); // Configure additional options... }); ``` --- --- url: /schema/storage.md --- # How Documents are Stored Polecat stores documents as JSON in SQL Server 2025 using the native `json` data type. ## Document Table Structure Each document type gets its own table with the prefix `pc_doc_`. A plain, single-tenant document type produces exactly these columns — no `tenant_id` (that is conjoined-tenancy only, see below): ```sql CREATE TABLE dbo.pc_doc_user ( id uniqueidentifier NOT NULL, data json NOT NULL, version bigint NOT NULL, last_modified datetimeoffset NOT NULL DEFAULT SYSDATETIMEOFFSET(), created_at datetimeoffset NOT NULL DEFAULT SYSDATETIMEOFFSET(), dotnet_type varchar(500) NULL, CONSTRAINT pkey_pc_doc_user_id PRIMARY KEY (id) ); ``` The `version` column is always present (a `bigint` that carries optimistic/numeric concurrency revisions when configured); `last_modified`, `created_at`, and `dotnet_type` are the standard metadata columns. The `tenant_id` column is **only** created for conjoined multi-tenancy — a single-tenant store keeps every document under the default tenant implicitly, with no `tenant_id` column and no per-query tenant filter. ## ID Column Types The `id` column type varies based on the document's ID property: | .NET Type | SQL Server Type | | :--- | :--- | | `Guid` | `uniqueidentifier` | | `string` | `nvarchar(250)` | | `int` | `int` | | `long` | `bigint` | ## Optional Columns Additional columns are added based on document configuration: ### Soft Deletes ```sql is_deleted bit NOT NULL DEFAULT 0, deleted_at datetimeoffset NULL ``` ### Guid Versioning (IVersioned) ```sql guid_version uniqueidentifier NULL ``` ### Numeric Revisions (IRevisioned / ILongVersioned) The `version` column is always `bigint` (Decision D2), carrying both `IRevisioned` (int, downcast on read) and `ILongVersioned` (long) revisions. Every write sets it explicitly, so it has no default. ```sql version bigint NOT NULL ``` ### Conjoined Tenancy ```sql tenant_id nvarchar(250) NOT NULL DEFAULT 'DEFAULT' ``` The primary key becomes composite: `PRIMARY KEY (tenant_id, id)`. ### Metadata Tracking ```sql correlation_id nvarchar(250) NULL, causation_id nvarchar(250) NULL ``` ## JSON Storage The `data` column uses SQL Server 2025's native `json` type. This provides: * **Server-side validation** -- Invalid JSON is rejected at the database level * **JSON\_VALUE()** -- Extract scalar values for WHERE clauses * **JSON\_QUERY()** -- Extract objects and arrays * **JSON\_MODIFY()** -- Partial updates without full document rewrite * **Efficient storage** -- Compact representation compared to `nvarchar(max)` ## Upsert Strategy Polecat uses SQL Server's `MERGE` statement for upsert operations: ```sql MERGE pc_doc_user AS target USING (VALUES (@id, @data, @type, ...)) AS source (id, data, type, ...) ON target.id = source.id WHEN MATCHED THEN UPDATE SET data = source.data, ... WHEN NOT MATCHED THEN INSERT (id, data, type, ...) VALUES (...); ``` --- --- url: /documents/indexing.md --- # Indexing Documents ::: warning Polecat owns the indexes on its own tables, so any custom index needs to be declared through Polecat itself. An index created out of band may be dropped the next time Polecat reconciles the schema. ::: Like Marten, Polecat gives you several ways to speed up document queries — all of which trade slightly slower inserts for faster reads. Polecat supports: * [Computed indexes](/documents/indexing/computed-indexes) — persisted computed columns backed by `JSON_VALUE`, with standard nonclustered indexes. Supports composite keys, uniqueness, case transformations, filtered (partial) indexes, per-tenant scoping, and covering `INCLUDE` columns. * [JSON indexes](/documents/indexing/json-indexes) — native SQL Server 2025 `CREATE JSON INDEX` over the `data` column, covering many JSON paths with one index (the SQL Server counterpart to Marten's `GinIndexJsonData`). Indexes can be declared with the fluent `Schema.For()` API or, for computed indexes, with `[Index]` / `[UniqueIndex]` attributes on the document type itself. As in Marten, the choice between keeping persistence concerns off your document types (fluent API) and colocating them for traceability (attributes) is yours — both are supported and can be combined on the same type. --- --- url: /documents/initial-data.md --- # Initial Baseline Data Polecat can automatically seed data on application startup using the `IInitialData` interface. ## IInitialData Interface ```cs public class SeedUsers : IInitialData { public async Task Populate(IDocumentStore store, CancellationToken ct) { await using var session = store.LightweightSession(); session.Store(new User { FirstName = "Admin", LastName = "User" }); session.Store(new User { FirstName = "Test", LastName = "User" }); await session.SaveChangesAsync(ct); } } ``` Register in configuration: ```cs opts.InitialData.Add(new SeedUsers()); ``` ## Lambda-Based Seeding For simple cases, use a lambda: ```cs opts.InitialData.Add(async (store, ct) => { await using var session = store.LightweightSession(); session.Store(new DefaultSettings { Id = "default", Theme = "dark" }); await session.SaveChangesAsync(ct); }); ``` ## Execution Order Initial data runs after schema migration on application startup, before the application starts accepting requests. All registered `IInitialData` implementations are executed in the order they were registered. ::: tip Initial data seeding runs via the `PolecatActivator` which is triggered by the .NET host's startup pipeline. ::: --- --- url: /events/projections/inline.md --- # Inline Projections Inline projections run in the **same transaction** as the event append, providing strong consistency between events and their read models. ## How It Works When `SaveChangesAsync()` is called: 1. Events are inserted into `pc_events` 2. Stream versions are updated in `pc_streams` 3. Inline projections process the new events 4. Projection documents are upserted into their tables 5. All operations commit in a single transaction If any step fails, the entire transaction rolls back -- events and projections are always consistent. ## Registration Register a projection for inline processing: ```cs // Single stream projection opts.Projections.Add>(ProjectionLifecycle.Inline); // Event projection opts.Projections.Add(ProjectionLifecycle.Inline); // Multi stream projection opts.Projections.Add(ProjectionLifecycle.Inline); ``` ## When to Use Inline Projections Inline projections are ideal when: * **Read models must always be consistent** with events * **Queries immediately follow writes** in the same request * **The projection is simple** and fast (doesn't slow down writes) ## Trade-offs | Advantage | Disadvantage | | :--- | :--- | | Strong consistency | Adds latency to writes | | No staleness window | Transaction grows larger | | Simpler architecture | No async daemon needed, but more write contention | ## Inline vs Async If your projections are complex or slow, consider [async projections](/events/projections/async-daemon) instead. The async daemon processes events in the background, decoupling write performance from projection processing. ::: tip Start with inline projections for simplicity. Move to async if write performance becomes a concern. ::: --- --- url: /testing/integration.md --- # Integration Testing Polecat provides patterns for writing integration tests against a real SQL Server database. ## Test Infrastructure ### Docker Compose Use Docker Compose to run SQL Server 2025 for testing: ```yaml services: sqlserver: image: mcr.microsoft.com/mssql/server:2025-CU1-ubuntu-24.04 environment: ACCEPT_EULA: "Y" MSSQL_SA_PASSWORD: "Polecat#Dev2025" ports: - "11433:1433" ``` ### Connection String ```cs private const string ConnectionString = "Server=localhost,11433;User Id=sa;Password=Polecat#Dev2025;Database=polecat_testing;TrustServerCertificate=True"; ``` ## IntegrationContext Base Class Create a base class for your integration tests: ```cs public abstract class IntegrationContext : IAsyncLifetime { protected IDocumentStore Store { get; private set; } = null!; protected virtual void ConfigureStore(StoreOptions opts) { // Override to customize store configuration } public async Task InitializeAsync() { Store = DocumentStore.For(opts => { opts.Connection(ConnectionString); ConfigureStore(opts); }); // Clean slate for each test await Store.Advanced.CleanAllDocumentsAsync(); await Store.Advanced.CleanAllEventDataAsync(); } public async Task DisposeAsync() { await Store.DisposeAsync(); } protected IDocumentSession OpenSession() => Store.LightweightSession(); protected IQuerySession QuerySession() => Store.QuerySession(); } ``` ## Example Test ```cs public class UserStorageTests : IntegrationContext { [Fact] public async Task can_store_and_load_document() { var user = new User { FirstName = "Alice", LastName = "Smith" }; await using (var session = OpenSession()) { session.Store(user); await session.SaveChangesAsync(); } await using (var session = QuerySession()) { var loaded = await session.LoadAsync(user.Id); loaded.ShouldNotBeNull(); loaded.FirstName.ShouldBe("Alice"); } } } ``` ## Event Sourcing Tests ```cs public class EventStoreTests : IntegrationContext { [Fact] public async Task can_append_and_aggregate() { Guid streamId; await using (var session = OpenSession()) { streamId = session.Events.StartStream( new QuestStarted("Test Quest"), new MembersJoined("Start", ["Alice", "Bob"]) ); await session.SaveChangesAsync(); } await using (var session = QuerySession()) { var party = await session.Events.AggregateStreamAsync(streamId); party.ShouldNotBeNull(); party.Members.Count.ShouldBe(2); } } } ``` ## Cleaning Data Between Tests ```cs // Clean all documents await store.Advanced.CleanAllDocumentsAsync(); // Clean specific document type await store.Advanced.CleanAsync(); // Clean all event data await store.Advanced.CleanAllEventDataAsync(); ``` ## Testing Async Projections For async daemon tests, use `CatchUpAsync` to wait for projections: ```cs [Fact] public async Task async_projection_catches_up() { await using var session = OpenSession(); session.Events.StartStream(new OrderCreated(100m, "Acme")); await session.SaveChangesAsync(); // Wait for the daemon to process await Store.WaitForNonStaleProjectionDataAsync(TimeSpan.FromSeconds(10)); await using var query = QuerySession(); var summary = await query.LoadAsync(streamId); summary.ShouldNotBeNull(); } ``` --- --- url: /introduction.md --- # Introduction Welcome to the Polecat documentation! ## What is Polecat? **Polecat is a .NET library for building applications using a [document-oriented database approach](https://en.wikipedia.org/wiki/Document-oriented_database) and [Event Sourcing](https://martinfowler.com/eaaDev/EventSourcing.html), backed by SQL Server 2025.** Polecat is part of the [Critter Stack](https://jasperfx.net) ecosystem and mirrors the API patterns of [Marten](https://martendb.io) (the PostgreSQL equivalent), making it easy for teams already familiar with Marten to adopt SQL Server as their backing store. ::: tip If you've used Marten before, you'll feel right at home with Polecat. The API surface is intentionally similar -- same interface names, same session patterns, same projection model. ::: Under the hood, Polecat is built on top of [SQL Server 2025](https://www.microsoft.com/en-us/sql-server/sql-server-2025), leveraging its native JSON type to provide: * a [document database](/documents/), * an [event store](/events/). Polecat uses SQL Server 2025's native `JSON` data type for storing document bodies, event data, and headers. Combined with modern T-SQL features, this provides strong data consistency for both document storage and event sourcing approaches. ## Main Features | Feature | Description | | :---: | :---: | | [Document Storage](/documents/) | Store your entities as JSON documents in SQL Server with full LINQ querying support. | | [Event Store](/events/) | Full-fledged event store for Event Sourcing with stream management, projections, and subscriptions. | | [Strong Consistency](/documents/sessions#unit-of-work) | Uses SQL Server transactions for ACID compliance across both document and event operations. | | [LINQ Querying](/documents/querying/) | Filter documents using LINQ queries with string searching, child collection queries, paging, and more. | | [Event Projections](/events/projections/) | Build read models from events using inline, async, or live projection strategies. | | [Automatic Schema Management](/schema/migrations) | Polecat manages SQL Server table creation and migrations automatically via Weasel.SqlServer. | | [Optimistic Concurrency](/documents/concurrency) | Built-in support for both Guid-based and numeric revision concurrency control. | | [Multi-Tenancy](/configuration/multitenancy) | Multiple tenancy strategies: conjoined (same tables), separate databases, or single tenant. | | [Async Daemon](/events/projections/async-daemon) | Background projection processing for eventually consistent read models. | | [EF Core Integration](/events/projections/efcore) | Use Entity Framework Core DbContext within your event projections. | ## Critter Stack Ecosystem Polecat is designed to work alongside other Critter Stack libraries: | Library | Purpose | | :---: | :---: | | [Marten](https://martendb.io) | PostgreSQL document database and event store | | [Wolverine](https://wolverinefx.net) | Messaging and command processing framework | | [JasperFx](https://jasperfx.net) | Core framework and event sourcing abstractions | | [Weasel](https://github.com/JasperFx/weasel) | Database schema management | ## Polecat vs Marten Polecat mirrors Marten's API but targets SQL Server 2025 instead of PostgreSQL: | Feature | Marten (PostgreSQL) | Polecat (SQL Server) | | :---: | :---: | :---: | | JSON storage | `jsonb` type | `json` type (SQL Server 2025) | | Sequences | `bigserial` / sequences | `bigint IDENTITY(1,1)` | | Upsert | `INSERT ... ON CONFLICT` | `MERGE` statement | | Change notification | `LISTEN/NOTIFY` | Polling (configurable interval) | | Advisory locks | `pg_advisory_lock` | `sp_getapplock` / `sp_releaseapplock` | | Timestamps | `timestamptz` | `datetimeoffset` | | Serialization | STJ or Newtonsoft | System.Text.Json only | ## History and Origins The project name *Polecat* follows the Critter Stack tradition of naming projects after animals. A [polecat](https://en.wikipedia.org/wiki/Polecat) is a member of the mustelid family -- close relatives of the marten, making it a fitting name for Marten's SQL Server cousin. --- --- url: /configuration/aspire-commands.md --- # JasperFx Commands in the Aspire Dashboard The optional **`JasperFx.Aspire`** package surfaces a Polecat application's command-line verbs as clickable **custom commands** on each resource tile in the [.NET Aspire dashboard](https://learn.microsoft.com/en-us/dotnet/aspire/fundamentals/dashboard). With one extension call in your AppHost project, an operator running the Aspire dashboard against a local or staging environment can run **`check-env`**, **`describe`**, **`resources`**, or **`projections`** against a live Polecat service without dropping to a terminal — output streams back into the dashboard's resource console. Polecat apps inherit this for free because they build on the same JasperFx command layer that Marten and Wolverine use. Because the projection commands are defined on the shared `JasperFx.Events` abstractions, the `projections` and `resources` buttons behave identically whether your event store is backed by SQL Server (Polecat) or PostgreSQL (Marten). See the [JasperFx.Aspire package README](https://github.com/JasperFx/jasperfx/tree/master/src/JasperFx.Aspire) for the full, store-agnostic reference. ## Quick start Add the package to your **Aspire AppHost** project (not the Polecat service project itself): ```shell dotnet add package JasperFx.Aspire ``` Then opt in on the Polecat service resource: ```csharp using JasperFx.Aspire; var builder = DistributedApplication.CreateBuilder(args); builder.AddProject("api") .WithJasperFxCommands(); builder.Build().Run(); ``` That adds the **safe-by-default** command buttons — `check-env`, `describe`, and `codegen` (preview only) — to the `api` resource tile. Click any of them in the dashboard, the verb runs against the live service with the same environment Aspire injects, and the output streams into the resource's console log view. This works because the target Polecat service already ends its `Program.cs` with `RunJasperFxCommands(args)` (the standard JasperFx bootstrap) — the integration reuses that exact CLI path, so the service itself needs no changes. ## The verbs that matter for Polecat users * **`check-env`** *(read-only)* — runs every registered environment check. Confirms the Polecat service can reach its SQL Server database, that all projection dependencies are wired, that the required schemas exist, etc. * **`describe`** *(read-only)* — dumps the resolved Polecat `StoreOptions` (document mappings, event store config, projections, retry policies, tenancy strategy, …). Useful for verifying composite configuration at a glance. * **`resources`** *(mutating)* — applies / patches Polecat's schema objects (`pc_events`, `pc_streams`, `pc_event_progression`, document tables, indexes, projection tables). Equivalent to applying all configured changes to the database. * **`projections`** *(mutating)* — runs or **rebuilds** asynchronous projections. The rebuild path reprocesses the event store — long-running and disruptive on a populated store, which is why it prompts for confirmation. ## Opting in to mutating verbs Mutating verbs are off by default. Adding them is a one-liner: ```csharp builder.AddProject("api") .WithJasperFxCommands(opts => { // Adds resources + projections + codegen-write buttons. opts.IncludeMutatingCommands = true; }); ``` When `IncludeMutatingCommands = true`, every mutating verb requires an explicit **confirmation dialog** in the Aspire dashboard before it runs. The default confirmation copy is generic ("Run `projections` on `api`?"); customize per-verb when the impact is non-obvious: ```csharp builder.AddProject("api") .WithJasperFxCommands(opts => { opts.IncludeMutatingCommands = true; opts.For("projections").ConfirmationMessage = "Rebuild ALL projections for 'api'? This reprocesses the entire event store."; opts.For("resources").ConfirmationMessage = "Apply pending schema changes to the 'api' SQL Server database?"; }); ``` ## Per-verb tweaks `opts.For("verb")` returns a registration object that lets you override the dashboard presentation per verb: | Property | Use | | --------------------- | ----------------------------------------------------------------------------------------- | | `DisplayName` | Button label (defaults to a humanized verb name). | | `DisplayDescription` | Tooltip / extended description. | | `IconName` | Fluent UI icon name; sensible defaults per verb. | | `ConfirmationMessage` | Required for mutating verbs; setting this opts a non-mutating verb into confirmation too. | | `IsHighlighted` | Pins the button to the front of the strip. | | `IsEnabled` | Predicate over the resource's current state — useful for gating verbs to `Running`. | ## Adding a single verb When the curated default set isn't quite what you want, register verbs one-at-a-time with `WithJasperFxCommand` instead of the batch helper: ```csharp builder.AddProject("api") .WithJasperFxCommand("projections", "rebuild MyProjection", registration => { registration.DisplayName = "Rebuild MyProjection"; registration.ConfirmationMessage = "Rebuild MyProjection for 'api'? This reprocesses the event store."; registration.IsHighlighted = true; }); ``` The second argument is the verb's fixed argument string — handy for locking a button down to one specific projection rather than exposing the full `projections` surface. ## Constraints * **`JasperFx.Aspire` runs at the AppHost project layer**, not inside the Polecat service itself. Adding it as a `` of the service project is a no-op. * The verbs run in a **child process** of the Polecat service, with the same environment Aspire injects into the resource. If `check-env`, `resources`, or `projections` fail to reach Aspire-managed dependencies, verify the dashboard shows the resource as `Running` first. * Buttons require `RunJasperFxCommands(args)` to already be wired in the Polecat service's `Program.cs`. Without that wiring the verb spawn succeeds but the child process won't recognize the verb. --- --- url: /documents/indexing/json-indexes.md --- # JSON Indexes Polecat can create a native SQL Server 2025 **`CREATE JSON INDEX`** over a document's `data` column — the SQL Server counterpart to Marten's `GinIndexJsonData()` GIN index on the JSONB column. Unlike a [computed index](/documents/indexing/computed-indexes), which materializes one persisted computed column per path, a **single** JSON index covers **many** JSON paths at once and accelerates a wide range of ad-hoc predicates against the document body — `JSON_VALUE` equality (including the `JSON_VALUE(data, '$.x' RETURNING type)` form used on native `json` storage), `JSON_PATH_EXISTS`, and `JSON_CONTAINS` — with no per-path computed columns. ::: warning Native JSON indexes require the native `json` column type, so they are only available when `UseNativeJsonType = true` (the default) against **SQL Server 2025**. `CREATE JSON INDEX` is invalid against `nvarchar(max)` storage, so Polecat throws a clear `InvalidOperationException` if you configure one on a store that isn't using the native `json` type — use a [computed index](/documents/indexing/computed-indexes) there instead. JSON indexes are a preview, on-prem SQL Server 2025 feature. ::: ## Indexing Specific Paths Use `JsonIndex(...)` in `Schema.For()`, passing a single member or an anonymous type for the paths you want covered: ```cs var store = DocumentStore.For(opts => { opts.ConnectionString = "..."; // Index two paths with one JSON index opts.Schema.For().JsonIndex(x => new { x.UserName, x.Department }); }); ``` This produces one JSON index over both paths: ```sql SET QUOTED_IDENTIFIER ON; CREATE JSON INDEX [jidx_pc_doc_user] ON [myschema].[pc_doc_user] (data) FOR ('$.userName', '$.department'); ``` SQL Server can then use the index for queries against either path: ```cs var users = await session.Query() .Where(x => x.Department == "Engineering") .ToListAsync(); ``` ## Indexing the Whole Document Omit the expression to index the entire JSON body (no `FOR` clause) — the direct analog of Marten's `GinIndexJsonData()`: ```cs opts.Schema.For().JsonIndex(); ``` ```sql SET QUOTED_IDENTIFIER ON; CREATE JSON INDEX [jidx_pc_doc_user] ON [myschema].[pc_doc_user] (data); ``` ## Optimizing for Array Search When you query *inside* a JSON array property (for example `JSON_CONTAINS` over a list), set `OptimizeForArraySearch` to emit `WITH (OPTIMIZE_FOR_ARRAY_SEARCH = ON)`: ```cs opts.Schema.For().JsonIndex(x => x.Roles, idx => idx.OptimizeForArraySearch = true); ``` ```sql SET QUOTED_IDENTIFIER ON; CREATE JSON INDEX [jidx_pc_doc_user] ON [myschema].[pc_doc_user] (data) FOR ('$.roles') WITH (OPTIMIZE_FOR_ARRAY_SEARCH = ON); ``` ## Customizing a JSON Index The `JsonIndex()` methods accept an optional `Action` to customize the index: ```cs opts.Schema.For().JsonIndex(x => x.UserName, idx => { // Override the auto-derived index name (default: jidx_
) idx.IndexName = "jidx_user_custom"; // Tune for array searches idx.OptimizeForArraySearch = true; // Set a FILLFACTOR (1–100) idx.FillFactor = 80; }); ``` ## Constraints Native JSON indexes carry a number of SQL Server restrictions, all documented on the `JsonIndex` type: * Native `json` storage only (`UseNativeJsonType = true`). * The table's clustered primary key must be ≤ 128 bytes. Polecat's single-tenant `id` primary key satisfies this, but **per-tenant tables whose primary key prepends a `varchar` `tenant_id` can exceed the limit**, in which case SQL Server rejects the index. * Only **one** JSON index is allowed per `json` column, so a table has at most one JSON index. Polecat keys idempotency on the table, so re-applying the schema never creates a duplicate. * Indexed paths may not overlap (e.g. `$.a` and `$.a.b`). * No `UNIQUE`, filtered (`WHERE`), `INCLUDE`, range/`ORDER BY`, or `LIKE` / `IS NULL` support — use a [computed index](/documents/indexing/computed-indexes) for any of those. JSON indexes are therefore **additive** to computed-column indexes, not a replacement: reach for a JSON index to accelerate many ad-hoc paths at once, and a computed index when you need uniqueness, filtering, covering columns, ordering, or portable `nvarchar(max)` storage. --- --- url: /configuration/json.md --- # JSON Serialization Polecat uses **System.Text.Json** exclusively for all JSON serialization. Newtonsoft.Json is not supported. ## Default Configuration Out of the box, Polecat uses camelCase property naming: ```cs var store = DocumentStore.For(opts => { opts.Connection("..."); // These are the defaults: opts.Serializer(new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }); }); ``` ## Serializer Options Configure serialization behavior through `StoreOptions`: ### Enum Storage ```cs // Store enums as integers (default) opts.UseSystemTextJsonSerializerOptions(o => { o.Converters.Add(new JsonStringEnumConverter()); // Store as strings instead }); ``` ### Casing ```cs // CamelCase (default) - "firstName" opts.Serializer(new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }); // Keep original casing - "FirstName" opts.Serializer(new JsonSerializerOptions { PropertyNamingPolicy = null }); ``` ### Non-Public Members ```cs // Include non-public properties in serialization opts.UseSystemTextJsonSerializerOptions(o => { o.IncludeFields = true; }); ``` ## SQL Server 2025 JSON Type Polecat stores all document bodies and event data using SQL Server 2025's native `json` data type by default. This provides: * Native JSON validation at the database level * Efficient JSON path queries via `JSON_VALUE()` and `JSON_QUERY()` * `JSON_MODIFY()` for partial updates (used by the [patching API](/documents/partial-updates-patching)) * Smaller storage footprint compared to `nvarchar(max)` ::: tip The `json` type in SQL Server 2025 is analogous to PostgreSQL's `jsonb` type used by Marten, but without the binary storage optimization. ::: ### Falling Back to nvarchar(max) If you are running against a pre-2025 SQL Server instance that does not support the native `json` data type, disable it with: ```cs opts.UseNativeJsonType = false; ``` When set to `false`, Polecat uses `nvarchar(max)` for all JSON columns instead. All JSON querying, patching, and projection features continue to work identically with either column type. --- --- url: /events/projections/live-aggregates.md --- # Live Aggregations Live aggregation replays events on demand to build the current aggregate state without persisting a read model. ## Basic Usage ```cs var order = await session.Events.AggregateStreamAsync(streamId); ``` This loads all events for the stream and applies them through the aggregate's `Create` and `Apply` methods. ## When to Use Live Aggregation Live aggregation is best for: * **Infrequently accessed aggregates** -- No need to maintain a persistent read model * **Always-current state** -- No staleness, always reflects the latest events * **Streams with few events** -- Low replay cost * **Testing and debugging** -- Verify aggregate behavior ## Version Cap Replay only up to a specific version: ```cs var orderAtV5 = await session.Events.AggregateStreamAsync(streamId, version: 5); ``` ## Timestamp Cap Replay only events before a specific time: ```cs var orderAtTime = await session.Events.AggregateStreamAsync( streamId, timestamp: DateTimeOffset.Parse("2024-06-15") ); ``` ## UseIdentityMapForAggregates Polecat offers a performance optimization that caches aggregates in a session-level identity map when using `FetchForWriting()`. When enabled, subsequent calls to `FetchLatest()` within the same session will return the cached instance instead of re-querying the database. ```cs opts.Projections.UseIdentityMapForAggregates = true; ``` This is particularly valuable in CQRS command handlers that fetch an aggregate for writing, append events, and then need to return the updated aggregate state — avoiding a redundant database round trip. ::: warning Only use this optimization if you are NOT mutating the aggregate outside of Polecat internals. This is safe with immutable event application patterns (Apply methods that set properties from event data). ::: ::: tip Unlike Marten, Polecat defaults to lightweight sessions (no identity map tracking). This optimization adds aggregate-specific caching on top of lightweight sessions without switching to full identity map sessions. ::: ## Live vs Inline vs Async | Strategy | Consistency | Storage | Performance | | :--- | :--- | :--- | :--- | | Live | Always current | None | Replay cost per read | | Inline | Always current | Document table | Write overhead | | Async | Eventually consistent | Document table | Background processing | --- --- url: /documents/querying/byid.md --- # Loading Documents by Id The most direct way to load documents is by their ID. ## LoadAsync Load a single document by its ID: ```cs var user = await session.LoadAsync(userId); ``` Returns `null` if the document doesn't exist. ## LoadManyAsync Load multiple documents by their IDs in a single query: ```cs var users = await session.LoadManyAsync(userId1, userId2, userId3); ``` Returns a list containing only the documents that were found. Missing documents are silently omitted. ## Identity Map Behavior With `DocumentTracking.IdentityMap`, repeated loads of the same document return the same instance: ```cs await using var session = store.OpenSession(DocumentTracking.IdentityMap); var user1 = await session.LoadAsync(userId); var user2 = await session.LoadAsync(userId); // user1 and user2 are the same instance Assert.Same(user1, user2); ``` With lightweight sessions, each load returns a new instance. ## Strongly Typed IDs Loading with strongly typed IDs works the same way: ```cs public record struct UserId(Guid Value); var user = await session.LoadAsync(new UserId(guid)); ``` ## Soft Delete Filtering When soft deletes are enabled, `LoadAsync` automatically excludes soft-deleted documents: ```cs // Returns null if the document was soft-deleted var order = await session.LoadAsync(orderId); ``` ## JSON Loading Load a document as a raw JSON string: ```cs string? json = await session.LoadJsonAsync(userId); ``` --- --- url: /configuration/mcp.md --- # MCP Server Polecat ships with a built-in [Model Context Protocol](https://modelcontextprotocol.io/) (MCP) server that exposes your event store configuration as read-only tools. This lets AI assistants and agents introspect your Polecat setup for diagnostics, code generation guidance, and operational visibility. The MCP server uses the **Streamable HTTP** transport (stateless POST-based JSON-RPC) and is implemented as ASP.NET Core Minimal API endpoints in the `Polecat.AspNetCore` package. ## Installation Install the `Polecat.AspNetCore` NuGet package: ```bash dotnet add package Polecat.AspNetCore ``` ## Setup Register the MCP endpoints in your ASP.NET Core application: ```csharp using Polecat.AspNetCore; var app = builder.Build(); app.MapPolecatMcp(); ``` This registers a single POST endpoint at `/polecat/mcp/` that handles all MCP JSON-RPC requests. ### Custom Route Prefix You can change the default route prefix: ```csharp app.MapPolecatMcp("/api/polecat-mcp"); ``` ## Authorization `MapPolecatMcp()` returns a `RouteGroupBuilder`, so you can chain standard ASP.NET Core endpoint configuration including authorization policies: ```csharp app.MapPolecatMcp() .RequireAuthorization("AdminPolicy"); ``` Or with a specific authorization policy: ```csharp app.MapPolecatMcp() .RequireAuthorization(policy => { policy.RequireRole("admin"); }); ``` ::: warning The MCP endpoints expose internal configuration details about your event store schema, projections, and event types. You **should** apply authorization to these endpoints in production environments. ::: ## Available Tools The MCP server exposes three read-only tools: ### get\_event\_store\_configuration Returns the full event store options snapshot including: | Property | Description | | :--- | :--- | | `streamIdentity` | `AsGuid` or `AsString` | | `tenancyStyle` | `Single` or `Conjoined` | | `databaseSchemaName` | Schema for event tables | | `enableCorrelationId` | Correlation ID tracking enabled | | `enableCausationId` | Causation ID tracking enabled | | `enableHeaders` | Custom event headers enabled | ### list\_known\_event\_types Lists all event types registered with the event store. Each entry includes: | Property | Description | | :--- | :--- | | `eventTypeName` | The alias stored in the database `type` column (e.g. `members_joined`) | | `dotNetTypeName` | The full .NET type name | ### list\_projections Lists all projections and subscriptions. Each entry includes: | Property | Description | | :--- | :--- | | `name` | Projection class name | | `implementationType` | Full .NET type name | | `type` | Subscription type (e.g. `Snapshot`, `MultiStream`, `FlatTableProjection`) | | `shards` | Array of shard identifiers | ## MCP Protocol Details The endpoint implements the MCP [Streamable HTTP transport](https://modelcontextprotocol.io/specification/2025-03-26/basic/transports#streamable-http). All requests are JSON-RPC 2.0 POST requests to the single endpoint. ### Supported Methods | Method | Description | | :--- | :--- | | `initialize` | Handshake — returns server capabilities | | `tools/list` | Lists available tools with descriptions and input schemas | | `tools/call` | Executes a tool by name and returns results | ### Example Request ```bash curl -X POST http://localhost:5000/polecat/mcp/ \ -H "Content-Type: application/json" \ -d '{ "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": "get_event_store_configuration" } }' ``` ### Example Response ```json { "jsonrpc": "2.0", "id": 1, "result": { "content": [ { "type": "text", "text": "{\"streamIdentity\":\"AsGuid\",\"tenancyStyle\":\"Single\",\"databaseSchemaName\":\"dbo\",...}" } ] } } ``` --- --- url: /migration-guide.md --- # Migration Guide ## Key Changes in 4.0.0 Polecat 4.0 ships in lockstep with [Marten 9.0](https://martendb.io) and [JasperFx 2.0](https://github.com/JasperFx/jasperfx/issues/215) as part of the [Critter Stack 2026](https://github.com/JasperFx/jasperfx/issues/217) release wave. The 4.0 line builds on the same JasperFx 2.0 / JasperFx.Events 2.0 / Weasel 9.0 release-candidate matrix that Marten 9 consumes — this is the consolidation cycle, not a rewrite. Most upgrades are pin bumps; the breaking surface is small and localized, and the bulk of the rc-line dedup work is source-compatible for typical applications (see [rc dedup-wave relocations](#rc-dedup-wave-relocations)). ### Foundation pin bumps Polecat 4 consumes the 2026-wave release-candidate line for the shared substrate. Update your `Directory.Packages.props` (or the equivalent in your csproj): | Package | 3.x | 4.0 (current rc) | |---|---|---| | `JasperFx` | `1.31.0` | `2.0.0-rc.3` | | `JasperFx.Events` | `1.36.0` | `2.0.0-rc.3` | | `JasperFx.Events.SourceGenerator` | `1.4.0` | `2.0.0-rc.3` | | `JasperFx.RuntimeCompiler` *(transitive, but pin centrally for matrix coherence)* | *(no equivalent)* | `5.0.0-rc.3` | | `JasperFx.SourceGeneration` *(transitive, but pin centrally)* | *(no equivalent)* | `2.0.0-rc.3` | | `Weasel.SqlServer` | `8.15.2` | `9.0.0-alpha.8` | | `Weasel.EntityFrameworkCore` | `8.15.2` | `9.0.0-alpha.8` | Polecat 4 itself is at `4.0.0-rc.2` (the consolidation wave converged through an alpha series into the rc line). The five `JasperFx.*` packages move as a matrix and must be pinned to the **same** rc together; the two `Weasel.*` packages share their own version and currently trail one wave behind on `alpha.8` (the rc.3 line resolves cleanly against them). Keep all seven in one renovate/dependabot group. Expect a final tick to the matched `9.0`/`2.0` GA set as the 2026 wave closes out. ::: warning Pin the **`5.x` line** of `JasperFx.RuntimeCompiler` — that's the active continuation of the 4.x lineage. A parallel `2.0.x` series exists on NuGet but is **stale**; don't pin against it. ::: Target frameworks (`net9.0;net10.0`) are unchanged from late Polecat 3.x; .NET 8 was already dropped before Polecat 3.2. ### Dedup audit relocations The Marten ↔ Polecat dedup audit ([JasperFx/jasperfx#218](https://github.com/JasperFx/jasperfx/issues/218)) consolidated several enums that had parallel definitions in Polecat and Marten into the canonical Weasel / JasperFx homes. The relocations are mechanical — same values, different namespace. #### `Polecat.BulkInsertMode` → `Weasel.Core.BulkInsertMode` [#50](https://github.com/JasperFx/polecat/pull/50), audit row [weasel#264](https://github.com/JasperFx/weasel/issues/264). Third-party consumers that referenced `Polecat.BulkInsertMode` by full name need to update their `using` directive: ```csharp // before (Polecat 3.x) using Polecat; await store.Advanced.BulkInsertAsync(docs, BulkInsertMode.OverwriteExisting); // after (Polecat 4.0) using Weasel.Core; await store.Advanced.BulkInsertAsync(docs, BulkInsertMode.OverwriteExisting); ``` The enum gained a fourth value — `OverwriteIfVersionMatches` — that did not exist on the Polecat 3.x enum. See the new-behavior section below. #### `Polecat.Storage.CascadeAction` → `Weasel.Core.CascadeAction` [#47](https://github.com/JasperFx/polecat/issues/47) / [#61](https://github.com/JasperFx/polecat/pull/61), audit row 2 in [JasperFx/jasperfx#218](https://github.com/JasperFx/jasperfx/issues/218). `DocumentForeignKey.OnDelete` is now typed as `Weasel.Core.CascadeAction`. The same four values (`NoAction`, `Cascade`, `SetNull`, `SetDefault`) remain available; Weasel.Core's version additionally exposes `Restrict`. Existing call sites only need a `using Weasel.Core;` addition: ```csharp // before (Polecat 3.x) mapping.ForeignKey(x => x.AssigneeId, fk => fk.OnDelete = CascadeAction.Cascade); // ^ resolved Polecat.Storage.CascadeAction // after (Polecat 4.0) — add `using Weasel.Core;` to the file mapping.ForeignKey(x => x.AssigneeId, fk => fk.OnDelete = CascadeAction.Cascade); // ^ resolves Weasel.Core.CascadeAction ``` #### `Polecat.Metadata.ITenanted` now extends `JasperFx.MultiTenancy.IHasTenantId` Multi-tenancy dedup audit slice ([jasperfx#224](https://github.com/JasperFx/jasperfx/issues/224), row 1). `Polecat.Metadata.ITenanted` no longer declares its own `TenantId` property — it inherits it from `IHasTenantId`. Polecat-side framework code that accepts `IHasTenantId` now accepts any `ITenanted` document type. Source-compatible for the typical case (your document class implementing `ITenanted` keeps a single `string TenantId { get; set; }` property and compiles unchanged). Only affects code that referenced the explicit `ITenanted.TenantId` property symbol via reflection or interface forwarding. #### `Polecat.Exceptions.UnknownTenantException` → `JasperFx.MultiTenancy.UnknownTenantIdException` Multi-tenancy dedup audit slice ([jasperfx#224](https://github.com/JasperFx/jasperfx/issues/224), row 2). The local `Polecat.Exceptions.UnknownTenantException` was removed; throw sites and consumers use the canonical `JasperFx.MultiTenancy.UnknownTenantIdException`. In Polecat 2.0.0-alpha.7 the JasperFx exception gained a `TenantId` property so consumers can `catch (UnknownTenantIdException ex) { ex.TenantId }` without parsing the message string. ```csharp // before (Polecat 3.x) catch (Polecat.Exceptions.UnknownTenantException ex) { var id = ex.TenantId; } // after (Polecat 4.0) catch (JasperFx.MultiTenancy.UnknownTenantIdException ex) { var id = ex.TenantId; } ``` ### rc dedup-wave relocations The rc.1 → rc.3 cycle finished the [Critter Stack 2026 dedup pillar](https://github.com/JasperFx/jasperfx/issues/214): a batch of types that Polecat and Marten each declared in parallel were consolidated into the canonical JasperFx / Weasel homes ([Polecat #125–#141](https://github.com/JasperFx/polecat/issues/46)). The values, members, and ordinals are unchanged — only the namespace moved. Inside Polecat the old unqualified names keep resolving (the assembly carries `global using` aliases / empty inheriting interfaces), so the framework's own API surface is unaffected. **The break, if any, is in your code**: if you referenced one of these types by its old `Polecat.*` name — most often by implementing a marker interface on a document class, or by a fully-qualified `catch` / config reference — update the `using` directive (or the fully-qualified name) to the new home. | Old name (Polecat 3.x / early 4.0) | New canonical home | PR | |---|---|---| | `TenancyStyle` *(inline in `StoreOptions`)* | `JasperFx.MultiTenancy.TenancyStyle` | [#127](https://github.com/JasperFx/polecat/issues/127) | | `Polecat.Metadata.DeleteStyle` | `JasperFx.DeleteStyle` | [#127](https://github.com/JasperFx/polecat/issues/127) | | `Polecat.Events.Dcb.DcbConcurrencyException` | `JasperFx.Events.DcbConcurrencyException` | [#128](https://github.com/JasperFx/polecat/issues/128) | | `Polecat.Exceptions.ProgressionProgressOutOfOrderException` | `JasperFx.Events.Daemon.ProgressionProgressOutOfOrderException` | [#128](https://github.com/JasperFx/polecat/issues/128) | | `Polecat.Metadata.ISoftDeleted` | `JasperFx.Metadata.ISoftDeleted` | [#130](https://github.com/JasperFx/polecat/issues/130) | | `Polecat.Metadata.IVersioned` | `JasperFx.Metadata.IVersioned` | [#130](https://github.com/JasperFx/polecat/issues/130) | | `Polecat.Metadata.ITracked` ⚠️ | `JasperFx.Metadata.ITracked` *(non-nullable members — see below)* | [#130](https://github.com/JasperFx/polecat/issues/130) | | `Polecat.Patching.RemoveAction` | `JasperFx.Events.RemoveAction` | [#131](https://github.com/JasperFx/polecat/issues/131) | | `Polecat.Patching.IPatchExpression` ⚠️ | `JasperFx.Events.IPatchExpression` *(predicate overloads — see below)* | [#131](https://github.com/JasperFx/polecat/issues/131) | | `Polecat.Internal.OpenTelemetry.TrackLevel` | `JasperFx.OpenTelemetry.TrackLevel` | [#132](https://github.com/JasperFx/polecat/issues/132) | | `Polecat.Attributes.IdentityAttribute` *(document `[Identity]` — marks a non-`Id` identity member on a plain document)* | `JasperFx.IdentityAttribute` | [#135](https://github.com/JasperFx/polecat/issues/135) | | `Polecat.Schema.Identity.Sequences.{ISequence, HiloSettings, IReadOnlyHiloSettings}` | `Weasel.Core.Sequences.*` | [#137](https://github.com/JasperFx/polecat/issues/137) | | `Polecat.Serialization.{Casing, CollectionStorage, NonPublicMembersStorage, EnumStorage}` | `Weasel.Core.*` | [#137](https://github.com/JasperFx/polecat/issues/137) | | `Polecat.Exceptions.DocumentAlreadyExistsException` ⚠️ | `JasperFx.DocumentAlreadyExistsException` *(message text — see below)* | [#140](https://github.com/JasperFx/polecat/issues/140) | The two you are most likely to touch directly are the **document marker interfaces** (`ISoftDeleted` / `IVersioned` / `ITracked` — you implement these on your own document classes) and the **serialization-config enums** (`Casing` / `CollectionStorage` / `NonPublicMembersStorage` / `EnumStorage`, set via `StoreOptions`). Both just need their `using` swapped: ```csharp // before (Polecat 3.x) using Polecat.Metadata; public class Invoice : ISoftDeleted { /* ... */ } // after (Polecat 4.0) using JasperFx.Metadata; public class Invoice : ISoftDeleted { /* ... */ } ``` Three of the relocations carry a behavior change beyond the namespace move: #### `ITracked` members are now non-nullable [#130](https://github.com/JasperFx/polecat/issues/130). The lifted `JasperFx.Metadata.ITracked` declares `CorrelationId` / `CausationId` / `LastModifiedBy` as non-nullable `string` (Marten's canonical shape); Polecat's old interface declared them `string?`. If your document class declared those members nullable to match the old interface, the compiler now warns (`CS8766`) that the nullability doesn't line up with the interface. Drop the `?` (and add a default) on the implementing class — the session always copies non-null values onto them: ```csharp // before public string? CorrelationId { get; set; } // after public string CorrelationId { get; set; } = string.Empty; ``` #### `IPatchExpression` predicate overloads throw on SQL Server [#131](https://github.com/JasperFx/polecat/issues/131). Polecat adopted Marten's superset `IPatchExpression`, which adds predicate-based overloads — `AppendIfNotExists(expr, element, predicate)`, `InsertIfNotExists(expr, element, predicate, index)`, `Remove(expr, predicate, action)` — and the object-shaped `Rename(string, Expression>)` (replacing Polecat's generic `Rename`). The predicate variants require translating the predicate into a JSON-array search, which Polecat's `JSON_MODIFY` patch translation does not implement yet, so **they throw `NotSupportedException`** with a message pointing at the supported non-predicate overload. The element/key/index overloads you were already using are unchanged; only the new predicate forms are unimplemented. #### `DocumentAlreadyExistsException` message text changed [#140](https://github.com/JasperFx/polecat/issues/140). The lifted `JasperFx.DocumentAlreadyExistsException` keeps Polecat's `DocumentType` (Type) and `Id` properties and the `(Type, object)` ctor, so the duplicate-key `catch` path is a near-no-op rebind. But the thrown **`Message`** now uses Marten's FullName-based format — `"Document already exists {FullName}: {id}"` (was `"A document of type '{Name}' with id '{id}' already exists."`). If you assert on or parse `.Message`, update the expectation; assertions on `.DocumentType` / `.Id` are unaffected. ### Event-sourcing API changes #### Projections and self-aggregating documents must be `partial` [JasperFx/jasperfx#276](https://github.com/JasperFx/jasperfx/issues/276) (FEC elimination, Phase 1) removed the FastExpressionCompiler fallback in `JasperFx.Events`'s projection apply-method dispatch. Source-generated dispatchers emitted by [`JasperFx.Events.SourceGenerator`](https://www.nuget.org/packages/JasperFx.Events.SourceGenerator) are now the only path — runtime registration **fails fast** when no `[GeneratedEvolver]` is found for a projection that uses conventional `Apply` / `Create` / `ShouldDelete` methods: ``` JasperFx.Events.Projections.InvalidProjectionException: No source-generated dispatcher found for MyApp.MyProjection. When using conventional Apply/Create/ShouldDelete methods, the projection class must be declared `partial` in an assembly that references the JasperFx.Events.SourceGenerator analyzer, or alternatively override Evolve / EvolveAsync / DetermineAction / DetermineActionAsync directly. ``` To clear this exception, **every projection class that uses conventional apply-method discovery must be `partial`**, including: * Subclasses of `SingleStreamProjection`, `MultiStreamProjection`, `EventProjection`, and `PolecatCompositeProjection`. * **Self-aggregating document types** registered via `opts.Projections.Snapshot(...)`, queried via `session.Events.AggregateStreamAsync(...)`, or live-projected via `session.Events.FetchLatest(...)` / `FetchForWriting(...)`. The source generator emits a standalone `TEvolver` class for the closed `SingleStreamProjection` runtime instance, and it can only do that when `T` is `partial`. ```csharp // before (Polecat 3.x — works because of FEC fallback) public class QuestParty { public Guid Id { get; set; } public List Members { get; } = []; public static QuestParty Create(QuestStarted e) => new(); public void Apply(MembersJoined e) => Members.AddRange(e.Members); } // after (Polecat 4.0 — `partial` required) public partial class QuestParty { public Guid Id { get; set; } public List Members { get; } = []; public static QuestParty Create(QuestStarted e) => new(); public void Apply(MembersJoined e) => Members.AddRange(e.Members); } ``` Every assembly that defines such a projection or aggregate also needs to reference the `JasperFx.Events.SourceGenerator` NuGet as an **analyzer-only** package — Polecat itself already pulls it in through `JasperFx.Events` for projection types declared in your store-host project, but any sibling assembly that declares its own projection / aggregate types needs its own reference: ```xml ``` If you have a projection that **cannot** be made `partial` (e.g. a sealed class from a third-party library, or one you want to keep as a non-source-generator path), the escape hatch is to override the dispatch methods directly: ```csharp public class MyProjection : SingleStreamProjection { public override ValueTask EvolveAsync( MyAggregate? snapshot, Guid id, IEvent e, CancellationToken ct) { // hand-written dispatch — no SG, no FEC } } ``` This matches Marten 9's adoption story for the same JasperFx.Events Phase-1 change. The shared consumer-side contract is documented at greater length in [Marten's migration guide — "Projection apply dispatch is now source-generator-only"](https://martendb.io/migration-guide#projection-apply-dispatch); the Polecat-side adoption tracking lives in [Polecat#46](https://github.com/JasperFx/polecat/issues/46), with the audit harness from [#110](https://github.com/JasperFx/polecat/pull/110) standing as the pre-release gate against future SG regressions. ##### Convention-method visibility, identity, and required members The source generator inspects only **public** instance / static methods. `Apply` / `Create` / `ShouldDelete` methods that were `private` / `internal` / `protected` under Polecat 3.x (the reflective FEC path picked those up) are silently skipped by the SG and trip the fail-fast above. Make them `public`: ```csharp // before (Polecat 3.x — FEC reflected over non-public members too) public partial class Quest { public Guid Id { get; set; } private void Apply(MembersJoined e) { /* ... */ } // <-- skipped by SG } // after (Polecat 4.0) public partial class Quest { public Guid Id { get; set; } public void Apply(MembersJoined e) { /* ... */ } // <-- discovered } ``` The event argument of those conventional methods is identified **by type, not by parameter name**: a parameter typed `IEvent` is the event, otherwise the single concrete parameter that isn't an interface (`IQuerySession` / `IDocumentOperations`), `IEvent`, `CancellationToken`, or the aggregate type. You do **not** need to name it `@event` — `Apply(MembersJoined e)` and `Apply(MembersJoined joined)` are equivalent. A conventional name (`@event`, `event`, `e`, or `ev`) is only consulted to disambiguate a signature with more than one candidate parameter. See [Identifying the Event Parameter](/events/projections/single-stream-projections#identifying-the-event-parameter). Aggregate identity discovery follows the same convention rule as Marten — by default, the SG looks for a property literally named `Id` (or `Id`). If your aggregate's identity uses a different member name, annotate that member with `[Identity]` from **`JasperFx`** (the same `JasperFx.IdentityAttribute` Polecat uses for document identity — see the [rc dedup-wave relocations](#rc-dedup-wave-relocations) table; it moved out of `Polecat.Attributes` in [#135](https://github.com/JasperFx/polecat/issues/135)): ```csharp using JasperFx; // JasperFx.IdentityAttribute — NOT JasperFx.Events public partial class Quest { [Identity] public Guid QuestKey { get; set; } // <-- explicit identity slot public static Quest Create(QuestStarted e) => new() { QuestKey = e.QuestId }; public void Apply(MembersJoined e) { /* ... */ } } ``` The source generator matches the identity attribute by simple name, so any attribute named `IdentityAttribute` works — but `JasperFx.IdentityAttribute` is the canonical one. As an alternative to a member-level `[Identity]`, you can declare the identity type explicitly at the class level with `[AggregateIdentity(typeof(TId))]` from `JasperFx.Events.Aggregation` — useful when the identity is a strong-typed id or isn't a simple property. For aggregates with `required` members, the SG's null-snapshot "create-from-default + apply" branch can't `new T()` directly — `required` members would leave the compiler complaining. Either provide a static `Create(TEvent e)` factory (preferred — gives full control) or use the `default!` init-pattern Marten's guide documents: ```csharp public partial class Account { public Guid Id { get; set; } public required string Owner { get; init; } // Preferred: explicit factory — SG calls this, no default! required. public static Account Create(AccountOpened e) => new() { Id = e.AccountId, Owner = e.Owner, }; public void Apply(Deposited e) { /* ... */ } } ``` If no `Create` factory is supplied and the SG falls back to the create-from-default path, the emitted code is `var s = new Account { Owner = default! }; Apply(e, s);` — semantically valid, but the `default!` hands the aggregate over to your `Apply` methods in a partially-initialized state. The factory route is cleaner. ##### Surfaces unaffected by the partial requirement * **`FlatTableProjection`** — has its own dictionary-keyed dispatch via `Project(...)` / `Delete(...)`, doesn't go through JasperFx.Events apply-method discovery. No `partial` required. * **`EfCoreSingleStreamProjection` / `EfCoreEventProjection`** (from `Polecat.EntityFrameworkCore`) — override `DetermineActionAsync` directly, also bypass the SG-required path. #### Inline-lambda projection registration APIs removed [JasperFx/jasperfx#276](https://github.com/JasperFx/jasperfx/issues/276) / [#286](https://github.com/JasperFx/jasperfx/issues/286) removed the inline-lambda registration overloads on `EventProjection`, `IAggregationSteps`, and `JasperFxAggregationProjectionBase` — the source generator cannot dispatch a runtime closure, so these were the last thing keeping FEC reachable. Migrate to conventional method declarations on the projection class: ```csharp // before (Polecat 3.x) public class MyEventProjection : EventProjection { public MyEventProjection() { Project((e, ops) => { ops.Store(new OrderSummary { Id = e.OrderId, ... }); }); } } // after (Polecat 4.0) public partial class MyEventProjection : EventProjection { public void Project(OrderPlaced e, IDocumentSession ops) { ops.Store(new OrderSummary { Id = e.OrderId, ... }); } } ``` The same shape — `public void Project(TEvent, IDocumentSession)` for inline mutation, plus the conventional `Create` / `Apply` / `ShouldDelete` methods for aggregations — covers everything the lambda APIs used to do. `DeleteEvent()` (no-args, populates the internal delete-types list) and `TransformsEvent()` remain available; only the lambda-taking overloads were dropped. The migration recipe is identical between Polecat and Marten — see [Marten's migration guide — "Inline-lambda projection registration APIs removed"](https://martendb.io/migration-guide#inline-lambda-removed) for the same content + a longer worked example. #### `IInlineProjection.ApplyAsync` parameter widening JasperFx.Events 2.0 widened `IInlineProjection.ApplyAsync(...)` so the `streams` parameter is `IEnumerable` instead of `IReadOnlyList` (jasperfx-events#4306). Polecat's built-in projection types (`NaturalKeyProjection`, `FlatTableProjection`) were updated to match; third-party `IInlineProjection` implementors must update their parameter type: ```csharp // before public Task ApplyAsync(TOperations operations, IReadOnlyList streams, CancellationToken cancellation) { ... } // after public Task ApplyAsync(TOperations operations, IEnumerable streams, CancellationToken cancellation) { ... } ``` If your implementation relied on `streams.Count` or indexed access, materialize the parameter once with `.ToList()` at the top of the method — the widening is intentional to give callers room to stream without forcing materialization. #### `IJasperFxAggregateGrouper.Group` parameter tightening JasperFx.Events 2.0 promoted the `events` parameter on `IJasperFxAggregateGrouper.Group(...)` from `IEnumerable` to `IReadOnlyList` ([jasperfx#201](https://github.com/JasperFx/jasperfx/issues/201)). Polecat does not ship any `IAggregateGrouper` implementations of its own, so most Polecat users see no impact. If your application has custom groupers, update the parameter type — no logic change is required, and you can drop any defensive `events.ToList()` you'd been doing at the top of `Group`: ```csharp // before public Task Group(IQuerySession session, IEnumerable events, IEventGrouping grouping) { ... } // after public Task Group(IQuerySession session, IReadOnlyList events, IEventGrouping grouping) { ... } ``` ### New behavior worth flagging #### `BulkInsertWithVersionAsync` — optimistic concurrency on bulk inserts [#48](https://github.com/JasperFx/polecat/issues/48) / [#62](https://github.com/JasperFx/polecat/pull/62). The new `BulkInsertMode.OverwriteIfVersionMatches` value (added by the Weasel.Core relocation) is exposed through a dedicated sibling method on `AdvancedOperations` rather than the existing `BulkInsertAsync` surface — the version-check needs per-row expected versions that the versionless overload has no way to thread: ```csharp var batch = new[] { (new User { Id = userId, FirstName = "Updated" }, expectedVersion: 1L), (new User { Id = newUserId, FirstName = "Fresh" }, expectedVersion: 0L), }; await store.Advanced.BulkInsertWithVersionAsync(batch); ``` Semantics: * **Row exists + stored version == expected version** → UPDATE, version bumped to `target.version + 1`. * **Row does not exist** → INSERT at version 1; the expected version is irrelevant on inserts. * **Row exists + stored version != expected version** → MERGE is a no-op; after the batch's reader drains, `JasperFx.ConcurrencyException(typeof(T), id)` is thrown for the first mismatched id. Matches the per-row pattern used by `UpdateOperation` / `UpsertOperation`. Calling the versionless `BulkInsertAsync(docs, BulkInsertMode.OverwriteIfVersionMatches)` overload now throws `InvalidOperationException` pointing callers at `BulkInsertWithVersionAsync` rather than the Polecat-3.x-era `NotSupportedException` stub. ::: warning The concurrency throw is **best-effort, not transactional.** Matched-and-updated rows in the batch commit before the throw fires; the exception is the signal rather than a rollback. If you need transactional semantics, wrap the call in an outer transaction (`TransactionScope` or your own `BEGIN TRAN` / `ROLLBACK` flow). ::: #### `FlatTableProjection` column casing on SQL Server [#49](https://github.com/JasperFx/polecat/pull/49) (Weasel side: `99e40f0 fix(SqlServer): preserve user casing on TableColumn names`, shipped as Weasel 9.0.0-alpha.2). `TableColumn` names declared on a `FlatTableProjection` now preserve the casing you wrote in code; pre-fix the projection was silently lower-casing column names, breaking case-sensitive collations. No user action required — the projection schema is regenerated on next migration. #### `ILongVersioned` — 64-bit document revisions [#136](https://github.com/JasperFx/polecat/issues/136) / [#138](https://github.com/JasperFx/polecat/issues/138) (consumes [jasperfx#348](https://github.com/JasperFx/jasperfx/issues/348)). Polecat already supported `JasperFx.IRevisioned` (an `int Version` optimistic-concurrency counter). 4.0 adds the new `JasperFx.ILongVersioned` (`long Version`) for documents whose version is a global event sequence number that can exceed `Int32.MaxValue` — typically `MultiStreamProjection`-derived views. Implement `ILongVersioned` instead of `IRevisioned` on those document types; the store, concurrency check, and read-back all flow the 64-bit value end to end. `IRevisioned` (int) keeps working unchanged for ordinary per-document revision counters. This required a one-time **schema change** — see below. #### `StoreOptions.SchemaResolver` — table-name diagnostics surface [#133](https://github.com/JasperFx/polecat/issues/133) (implements the lifted `JasperFx.Events.IDocumentSchemaResolver`). A new `StoreOptions.SchemaResolver` resolves the database table backing any document, projection, or event-store table — qualified (`[schema].[table]`) or bare. Useful for schema inspection, diagnostics, and projection-coordinator activity tags: ```csharp var resolver = store.Options.SchemaResolver; resolver.For(); // "[dbo].[pc_doc_customer]" resolver.ForEvents(qualified: false); // "pc_events" ``` (Named `SchemaResolver` rather than `Schema` because `StoreOptions.Schema` is already Polecat's `SchemaConfiguration`.) #### Projection-emitted side-effect messages [#84](https://github.com/JasperFx/polecat/issues/84) / [#98](https://github.com/JasperFx/polecat/pull/98) / [#100](https://github.com/JasperFx/polecat/pull/100). Aggregation projections can now publish side-effect messages through an `IMessageOutbox` / `IMessageBatch`, flushed in the same transaction as the projection's storage writes. This is the foundation for Wolverine integration where a projection raises follow-on messages. Inline projections gained the same side-effect support in [#99](https://github.com/JasperFx/polecat/issues/99) / [#101](https://github.com/JasperFx/polecat/pull/101). #### Hot-cold async-daemon coordination [#96](https://github.com/JasperFx/polecat/pull/96), refined through the rc wave ([#117](https://github.com/JasperFx/polecat/issues/117)–[#120](https://github.com/JasperFx/polecat/pull/120), [#126](https://github.com/JasperFx/polecat/issues/126), [#141](https://github.com/JasperFx/polecat/issues/141)). The async daemon now coordinates across nodes: a `ProjectionCoordinator` (built on the lifted `JasperFx.Events.Daemon.ProjectionCoordinatorBase`) negotiates per-shard / per-database leadership via SQL Server `sp_getapplock` (through `Weasel.SqlServer.AdvisoryLock`), so multiple instances of your app run each projection on exactly one node with automatic failover. No configuration change is required for single-node deployments; multi-node deployments get safe hot-cold handover for free. When the daemon is disabled (`DaemonMode.Disabled`, the default), no coordinator distributor is built at all. ### Schema changes Unlike the 3.x → early-4.0 transition (which touched no table shapes), the `ILongVersioned` work introduced one DDL change: * **Revision column widened `int` → `bigint`.** [#136](https://github.com/JasperFx/polecat/issues/136). The `version` column on document tables (`pc_doc_*`) is now `bigint` to hold 64-bit `ILongVersioned` values. `IRevisioned` (int) reads downcast transparently. The change ships as a **non-destructive widening migration** — existing `int` columns are `ALTER`ed in place (no drop/recreate, no data loss) on the next schema migration. Run your normal `ApplyAllConfiguredChangesToDatabaseAsync()` / migration step after upgrading; no manual SQL is required. ### AOT / codegen posture Polecat 4 inherits the same AOT-friendly posture introduced in JasperFx 2.0 ([jasperfx#213](https://github.com/JasperFx/jasperfx/issues/213) AOT pillar, jasperfx#190 `ITypeLoader` abstraction). Polecat itself has been source-generator-first since 3.x — there is no Roslyn runtime-compile path to disable — so: * `PublishAot=true` is the supported posture for Polecat 4 applications, modulo the usual System.Text.Json `JsonSerializerContext` setup for your document and event types. * `IsAotCompatible=true` is now set on the Polecat assembly (PR [#67](https://github.com/JasperFx/polecat/pull/67)), and the reflective surfaces of Polecat have been progressively annotated for the trimmer / AOT analyzer — Serialization ([#74](https://github.com/JasperFx/polecat/pull/74)), ProjectionReplay ([#75](https://github.com/JasperFx/polecat/pull/75)), LINQ extension/provider surface ([#76](https://github.com/JasperFx/polecat/pull/76)), Storage / Registry / EventStoreExplorer ([#77](https://github.com/JasperFx/polecat/pull/77)), and the class-level → call-site refactor of the remaining reflective surface ([#107](https://github.com/JasperFx/polecat/pull/107)). * **FastExpressionCompiler is no longer pulled in transitively** through `JasperFx.Events` ([jasperfx#276](https://github.com/JasperFx/jasperfx/issues/276)). Projection apply-method dispatch routes exclusively through `[GeneratedEvolver]` outputs from `JasperFx.Events.SourceGenerator` — see the **"Projections and self-aggregating documents must be `partial`"** section above for the consumer-side migration. * An AOT smoke-test consumer ([Polecat.AotSmoke](https://github.com/JasperFx/polecat/tree/main/src/Polecat.AotSmoke), PR [#106](https://github.com/JasperFx/polecat/pull/106)) ships with the repo and runs in CI with `WarningsAsErrors` covering `IL2026 / IL2046 / IL2055 / IL2065 / IL2067 / IL2070 / IL2072 / IL2075 / IL2090 / IL2091 / IL2111 / IL3050 / IL3051`. Regressions in Polecat's AOT-clean surface fail the build. ### Publishing AOT Polecat 4 applications target the same Critter Stack 2026 AOT pre-gen flow Marten 9 uses — Polecat is a thin consumer of `JasperFx.Events`, so the consumer-side recipe (csproj flags, `WarningsAsErrors=IL*` set, source-generator-backed `ISerializer` swap, projection-class partial requirement, smoke-test pattern) lives in JasperFx's docs rather than being forked here: * **[Publishing AOT with JasperFx](https://jasperfx.github.io/codegen/aot)** — the end-to-end Critter Stack guide. Read this first. * **[Marten's AOT publishing walkthrough](https://martendb.io/configuration/aot-publishing)** — longer worked example with full csproj + program snippets. Polecat-applicable apart from the Marten-specific service registrations. Polecat-specific call-outs: * **`Polecat.AotSmoke`** ([source](https://github.com/JasperFx/polecat/tree/main/src/Polecat.AotSmoke), originally [PR #106](https://github.com/JasperFx/polecat/pull/106)) ships in-tree as the canonical AOT consumer example. The csproj sets `IsAotCompatible=true` + `TrimMode=full` and promotes IL2026 / IL2046 / IL2055 / IL2065 / IL2067 / IL2070 / IL2072 / IL2075 / IL2090 / IL2091 / IL2111 / IL3050 / IL3051 to errors — any regression in Polecat's AOT-clean surface fails the build. * **`IsAotCompatible=true`** is set on `Polecat.csproj` itself, plus both shipping extension packages — `Polecat.AspNetCore` and `Polecat.EntityFrameworkCore` (mirrors Marten PR [#4468](https://github.com/JasperFx/marten/pull/4468)'s per-extension audit). Both flagged carry method-level annotations that name the AOT-hostile upstream APIs they wrap: * `Polecat.AspNetCore` — `StreamMany` / `StreamOne` / `StreamAggregate` `ExecuteAsync` overrides carry `[RequiresDynamicCode]` + `[RequiresUnreferencedCode]` for the `JsonSerializer.SerializeToUtf8Bytes` reflective overloads; `McpEndpointExtensions` class-level RUC/RDC for the JSON-RPC payload serialization in `MapPolecatMcp`. AOT consumers building Minimal-API endpoints can return `new StreamMany(query)` from a `[RequiresDynamicCode]`-annotated endpoint, or supply a `JsonSerializerContext` and write a custom `IResult`. * `Polecat.EntityFrameworkCore` — `TDoc` type parameters carry the full EF Core entity DAM set (`PublicConstructors | NonPublicConstructors | PublicFields | NonPublicFields | PublicProperties | NonPublicProperties | Interfaces`) matching `DbContext.Find` requirements; `TDbContext` parameters carry `PublicConstructors` matching `Activator.CreateInstance(typeof(TDbContext), ...)`. Concrete closed generics (`EfCoreSingleStreamProjection`) satisfy these implicitly. * **`Polecat.AotSmoke`** also references both extensions and exercises a constructor / `typeof()` from each, so a future annotation regression in either extension trips the smoke build before downstream consumers see it. * The **post-#276 fail-fast** semantics apply identically to AOT consumers — a projection registered without a `[GeneratedEvolver]` dispatcher throws `InvalidProjectionException` at host build (not at runtime via FEC fallback like Polecat 3.x). The remediation is the same as the non-AOT story — see the [SG-only projection apply dispatch](#projections-and-self-aggregating-documents-must-be-partial) section above. ### Out of scope (no Polecat 4 change) * **Obsolete API removals.** A grep of the 3.x codebase for `[Obsolete]` declarations returned no matches; the 4.0 release does not remove any APIs that were obsoleted in 3.x. * **Schema migrations.** The only `pc_*` table-shape change in the whole 3.x → 4.0 line is the document `version` column widening to `bigint` — covered in [Schema changes](#schema-changes) above, applied non-destructively by the normal migration step. Everything else (event store, streams, tags, hilo, snapshots) is unchanged; a 3.x database upgrades in place. ### Dependency lockstep Polecat 4 ships in lockstep with the rest of the Critter Stack 2026 wave. The supported pairings are: | Polecat | Marten | JasperFx | JasperFx.Events | Weasel | |---|---|---|---|---| | 4.0 | 9.0 | 2.0 | 2.0 | 9.0 | Mixing major versions across products is unsupported in this wave (the dedup work moves abstractions between assemblies and ABI-binds them to specific majors). If you upgrade Polecat to 4.0, plan to upgrade Marten and Weasel-side dependencies to their 9.0 lines at the same time. ## References * [Polecat 4.0 master plan](https://github.com/JasperFx/polecat/issues/46) * [Critter Stack 2026 umbrella](https://github.com/JasperFx/jasperfx/issues/217) * [Marten 8 → 9 migration guide](https://martendb.io/migration-guide) * [JasperFx 2.0 master plan](https://github.com/JasperFx/jasperfx/issues/215) --- --- url: /events/projections/multi-stream-projections.md --- # Multi Stream Projections Multi stream projections aggregate events from multiple streams into a single document. ## Defining a Multi Stream Projection ```cs public class CustomerDashboard { public Guid Id { get; set; } public int TotalOrders { get; set; } public decimal TotalSpent { get; set; } public DateTimeOffset? LastOrderDate { get; set; } } public class CustomerDashboardProjection : MultiStreamProjection { public CustomerDashboardProjection() { // Route events to the correct aggregate by extracting the customer ID Identity(e => e.CustomerId); Identity(e => e.CustomerId); Identity(e => e.CustomerId); } public static CustomerDashboard Create(OrderCreated e) => new() { TotalOrders = 1, TotalSpent = e.Amount, LastOrderDate = DateTimeOffset.UtcNow }; public void Apply(OrderCreated e, CustomerDashboard current) { current.TotalOrders++; current.TotalSpent += e.Amount; current.LastOrderDate = DateTimeOffset.UtcNow; } public void Apply(OrderCancelled e, CustomerDashboard current) { current.TotalOrders--; current.TotalSpent -= e.RefundAmount; } } ``` ## Registration ```cs opts.Projections.Add(ProjectionLifecycle.Async); ``` Multi stream projections can run inline or async, but async is typically recommended since they process events across streams. ## Event Routing The `Identity()` method tells Polecat which aggregate document an event belongs to: ```cs // Route by a property on the event Identity(e => e.CustomerId); // For string IDs Identity(e => e.CustomerKey); ``` ## Time-Based Segmentation Multi-stream projections can segment a single event stream by time period. This is useful for monthly reports, daily summaries, billing periods, or any scenario where you need per-period aggregations of a single stream's events. The key technique is using a composite identity that combines the stream ID with a time bucket (e.g., `"{streamId}:{yyyy-MM}"`), derived from the event's timestamp metadata via `IEvent`. **Events:** ```cs public record AccountOpened(string AccountName); public record AccountDebited(decimal Amount, string Description); public record AccountCredited(decimal Amount, string Description); ``` snippet source | anchor **Read model document:** ```cs public partial class MonthlyAccountActivity { public string Id { get; set; } = ""; public Guid AccountId { get; set; } public string Month { get; set; } = ""; public int TransactionCount { get; set; } public decimal TotalDebits { get; set; } public decimal TotalCredits { get; set; } public decimal NetChange => TotalCredits - TotalDebits; } ``` snippet source | anchor **Projection with time-based routing:** ```cs public partial class MonthlyAccountActivityProjection : MultiStreamProjection { public MonthlyAccountActivityProjection() { // Route each event to a document keyed by "{streamId}:{yyyy-MM}" // This segments a single account stream into monthly summaries Identity>(e => $"{e.StreamId}:{e.Timestamp:yyyy-MM}"); Identity>(e => $"{e.StreamId}:{e.Timestamp:yyyy-MM}"); } public MonthlyAccountActivity Create(IEvent e) => new() { AccountId = e.StreamId, Month = e.Timestamp.ToString("yyyy-MM"), TransactionCount = 1, TotalDebits = e.Data.Amount }; public MonthlyAccountActivity Create(IEvent e) => new() { AccountId = e.StreamId, Month = e.Timestamp.ToString("yyyy-MM"), TransactionCount = 1, TotalCredits = e.Data.Amount }; public void Apply(IEvent e, MonthlyAccountActivity view) { view.TransactionCount++; view.TotalDebits += e.Data.Amount; } public void Apply(IEvent e, MonthlyAccountActivity view) { view.TransactionCount++; view.TotalCredits += e.Data.Amount; } } ``` snippet source | anchor Register the projection as inline (for immediate consistency) or async (for eventual consistency): ```cs // Inline — projected immediately during SaveChangesAsync opts.Projections.Add(ProjectionLifecycle.Inline); // Async — projected by the async daemon in the background opts.Projections.Add(ProjectionLifecycle.Async); ``` Each account stream's events are routed to monthly documents automatically. Querying is straightforward: ```cs // Get all monthly summaries for an account var monthlies = await session.Query() .Where(x => x.AccountId == accountId) .OrderBy(x => x.Month) .ToListAsync(); // Get a specific month var jan = await session.LoadAsync($"{accountId}:2026-01"); ``` ::: tip When using async projections, make sure to start the async daemon — see [Asynchronous Projections](/events/projections/async-daemon) for details. ::: ## ID Types Multi stream projections support any ID type for the aggregate: ```cs // Guid IDs public class MyProjection : MultiStreamProjection { } // String IDs public class MyProjection : MultiStreamProjection { } // Int IDs public class MyProjection : MultiStreamProjection { } ``` --- --- url: /configuration/multitenancy.md --- # Multi-Tenancy Polecat supports several multi-tenancy strategies for isolating data between tenants — from a single shared table to a database per tenant. They differ in **where** a tenant's data lives, **how** the tenant set is managed, and **how** the async projection daemon scales. ## Choosing a Tenancy Strategy | Strategy | Isolation | Tenant set | Best for | | --- | --- | --- | --- | | [Single tenant](#single-tenant-default) | None (shared tables) | n/a | Single-tenant apps | | [Conjoined](#conjoined-tenancy) | `tenant_id` column, shared tables | Open (any id) | Many tenants, shared schema, simplest ops | | [Separate database (static)](#separate-database-tenancy) | One database per tenant | Fixed at startup (`AddTenant`) | Strong isolation, known tenant list | | [Master-table (dynamic)](#dynamic-tenant-management-master-table-tenancy) | One database per tenant | Managed at runtime via a control table | Strong isolation + add/remove/enable/disable tenants without a restart | | [Per-tenant event partitioning](/events/multitenancy#per-tenant-event-partitioning) | Conjoined + per-tenant event sequence | Open (any id) | Very large multi-tenant **event** stores needing bounded, isolated per-tenant projection rebuilds | Conjoined and the two separate-database strategies are mutually exclusive choices for **where data lives**. Per-tenant event partitioning is an **opt-in optimization layered on conjoined event tenancy** — see [Per-Tenant Event Partitioning](/events/multitenancy#per-tenant-event-partitioning). For event-store specifics (tenanted streams, the default tenant), see [Event Multi-Tenancy](/events/multitenancy). ## Tenancy Styles ### Single Tenant (Default) All data lives in one set of tables with no tenant isolation: ```cs var store = DocumentStore.For(opts => { opts.Connection("..."); // This is the default -- no tenant isolation }); ``` ### Conjoined Tenancy All tenants share the same database and tables, but data is isolated by a `tenant_id` column: ```cs var store = DocumentStore.For(opts => { opts.Connection("..."); // Enable conjoined tenancy for events opts.Events.TenancyStyle = TenancyStyle.Conjoined; }); ``` With conjoined tenancy: * All document tables get a `tenant_id` column * Document primary keys become composite: `(tenant_id, id)` * All queries automatically filter by the session's tenant ID * Event streams are isolated per tenant Specify the tenant when creating a session: ```cs await using var session = store.LightweightSession(new SessionOptions { TenantId = "tenant-abc" }); ``` See [Multi-Tenanted Documents](/documents/multi-tenancy) and [Event Multi-Tenancy](/events/multitenancy) for more details. ### Separate Database Tenancy Each tenant gets their own isolated SQL Server database: ```cs var store = DocumentStore.For(opts => { opts.MultiTenantedDatabases(databases => { databases.AddTenant("tenant-a", "Server=localhost;Database=tenant_a;..."); databases.AddTenant("tenant-b", "Server=localhost;Database=tenant_b;..."); }); }); ``` With separate database tenancy: * Each tenant has completely isolated data * Schema management runs independently per database — `ApplyAllDatabaseChangesOnStartup()` migrates every tenant database, not just the one behind `StoreOptions.ConnectionString` * Sessions automatically route to the correct database based on tenant ID * The async daemon runs independently per tenant database: `AddAsyncDaemon(...)` starts one daemon per tenant database, each tracking its own high-water mark and projection progress ### No default tenant is required Calling `MultiTenantedDatabases()` (or `MultiTenantedMasterTable()`) sets `StoreOptions.DefaultTenantUsageEnabled` to `false`. Every tenant has its own database, so there is no database the default tenant could coherently mean — and you should **not** register a placeholder `*DEFAULT*` tenant to satisfy startup. You also do not need to set `StoreOptions.ConnectionString`. A configured tenancy already names every database the store will touch, so it supplies the store's own connection string: the first registered tenant for `MultiTenantedDatabases()`, the control-plane database for `MultiTenantedMasterTable()`. Nominating one of the tenant databases at the top level is not required and makes that tenant's database quietly special. ```cs var store = DocumentStore.For(opts => { // No opts.ConnectionString, and no "*DEFAULT*" tenant. opts.MultiTenantedDatabases(databases => { databases.AddTenant("tenant-a", "Server=localhost;Database=tenant_a;..."); databases.AddTenant("tenant-b", "Server=localhost;Database=tenant_b;..."); }); opts.Projections.Add(ProjectionLifecycle.Async); }); ``` With the default tenant disabled, opening a session or building a projection daemon without a tenant throws `DefaultTenantUsageDisabledException` rather than quietly landing on whichever database happens to back `StoreOptions.ConnectionString`: ```cs // throws DefaultTenantUsageDisabledException await using var session = store.LightweightSession(); // correct — always name the tenant await using var session = store.LightweightSession(new SessionOptions { TenantId = "tenant-a" }); ``` Infrastructure that has already resolved a database — the async daemon, for instance — opts out with `SessionOptions.AllowAnyTenant`, which `SessionOptions.ForDatabase(database)` sets for you. You can also re-enable the default tenant explicitly if your application genuinely wants it: ```cs opts.MultiTenantedDatabases(databases => { /* ... */ }); opts.DefaultTenantUsageEnabled = true; // opt back in, after configuring the tenancy ``` This mirrors Marten's `StoreOptions.Advanced.DefaultTenantUsageEnabled`; Polecat has no `Advanced` sub-object and carries the setting directly on `StoreOptions`. ### Dynamic Tenant Management (Master Table Tenancy) `MultiTenantedDatabases` above is **static** — the full tenant list is fixed when the store is configured. When you need to add, remove, enable, or disable tenants **at runtime** without restarting the service, use the **master table** strategy. A control-plane table (`pc_tenants`) maps each `tenant_id` to its connection string, and Polecat reads from it dynamically: ```cs var store = DocumentStore.For(opts => { // Default/fallback connection opts.Connection("..."); // The control-plane database that holds the pc_tenants registry opts.MultiTenantedMasterTable("Server=localhost;Database=control_plane;..."); }); ``` `MultiTenantedMasterTable` returns a `MasterTableTenancy` you can drive from operational code (for example, a CritterWatch tenant-management handler). The master table is created automatically on first use: ```cs var tenancy = (MasterTableTenancy)store.Options.Tenancy!; // Register a tenant -> connection string mapping at runtime (idempotent upsert) await tenancy.AddDatabaseRecordAsync("tenant-a", "Server=localhost;Database=tenant_a;..."); // Temporarily take a tenant offline without losing its record... await tenancy.DisableTenantAsync("tenant-a"); // ...and bring it back await tenancy.EnableTenantAsync("tenant-a"); // Inspect which tenants are currently disabled IReadOnlyList disabled = await tenancy.AllDisabledAsync(); // Remove a tenant record entirely (the tenant database itself is left untouched) await tenancy.DeleteDatabaseRecordAsync("tenant-a"); // Materialize the full set of currently-enabled tenant databases // (e.g. to apply schema to each) foreach (var db in await tenancy.BuildDatabasesAsync()) { await db.ApplyAllConfiguredChangesToDatabaseAsync(); } ``` Notes: * The master table is `pc_tenants` (`tenant_id`, `connection_string`, `is_disabled`) and lives in the schema you pass to `MultiTenantedMasterTable` (defaults to `StoreOptions.DatabaseSchemaName`). * `AddDatabaseRecordAsync` records the mapping and re-enables a previously-disabled tenant; it does **not** create the tenant database — provision that separately (the connection string must point at an existing database). * Disabled or unknown tenants raise `UnknownTenantIdException` when a session is opened for them, exactly like static separate-database tenancy. * All master-table access flows through `StoreOptions.ResiliencePipeline`. This is the Polecat (SQL Server) equivalent of Marten's `MultiTenantedDatabasesViaMasterTable` / `MasterTableTenancy`. ### Store-Agnostic Runtime Tenant Management The `MasterTableTenancy` methods above are Polecat-specific. `MasterTableTenancy` also implements `JasperFx.MultiTenancy.IDynamicTenantSource` — the same store-agnostic abstraction Marten's `MasterTableTenancy` and `ShardedTenancy` implement — so tooling can drive the runtime tenant lifecycle without taking a dependency on Polecat's concrete tenancy types: ```cs var source = (IDynamicTenantSource)store.Options.Tenancy!; // Always DatabaseCardinality.DynamicMultiple -- the tenant list is read from the // master table and free to change while the store is running var cardinality = source.Cardinality; // Add a tenant with a caller-supplied connection string await source.AddTenantAsync("tenant-a", "Server=localhost;Database=tenant_a;..."); // Resolve a tenant's connection string; throws UnknownTenantIdException for an // unknown *or* disabled tenant var connectionString = await source.FindAsync("tenant-a"); // Soft delete / restore await source.DisableTenantAsync("tenant-a"); IReadOnlyList disabled = await source.AllDisabledAsync(); await source.EnableTenantAsync("tenant-a"); // Re-read the master table, dropping cached entries for tenants that have since // been removed or disabled elsewhere await source.RefreshAsync(); // The currently active tenants. AllActive() returns the tenant *database // identifiers* rather than raw connection strings, so credentials never reach an // admin dashboard IReadOnlyList databases = source.AllActive(); IReadOnlyList> byTenant = source.AllActiveByTenant(); // Remove the registry record entirely. The tenant database itself is untouched await source.RemoveTenantAsync("tenant-a"); ``` When the store is configured with `MultiTenantedMasterTable()`, `AddPolecat()` also registers the tenancy in the container as `IDynamicTenantSource`: ```cs builder.Services.AddPolecat(opts => { opts.Connection(connectionString); opts.MultiTenantedMasterTable(controlPlaneConnectionString); }); // ...elsewhere var source = provider.GetServices>().Single(); ``` This is what makes a Polecat-backed service's **Tenants** tab editable in CritterWatch — add, disable, enable, and remove all round-trip against SQL Server with no CritterWatch release required. Consumers resolve the source with `GetServices>()` and degrade to a read-only tenant list when the collection is empty. ::: tip The registration is deliberately **conditional**: it happens only when the configured tenancy is a dynamic source. Single-database stores and static `MultiTenantedDatabases()` stores leave `GetServices>()` empty, which is the signal consumers use to fall back to a read-only tenant list. ::: Two caveats: * The auto-assign overload, `Task AddTenantAsync(string tenantId, CancellationToken)`, throws `NotSupportedException`. Database-per-tenant has no pool for Polecat to assign from, so the caller must supply a connection string via `AddTenantAsync(tenantId, connectionValue)`. CritterWatch treats an empty connection string as "auto-assign" and skips sources that throw. * The `AddPolecat(Func)` overload cannot inspect the tenancy while the container is still being assembled, so it does not auto-register the source. Configure the store with one of the other `AddPolecat` overloads, or register it yourself: ```cs services.AddSingleton>(sp => (IDynamicTenantSource)((DocumentStore)sp.GetRequiredService()).Options.Tenancy!); ``` Hard-deleting the physical tenant database is out of scope for this abstraction — SQL Server needs `ALTER DATABASE ... SET SINGLE_USER WITH ROLLBACK IMMEDIATE` before `DROP DATABASE`, and that is the consumer's job. Add, disable, enable, and remove all work without it. ## Setting the Tenant ID The tenant ID is set when opening a session: ```cs // Via SessionOptions await using var session = store.LightweightSession(new SessionOptions { TenantId = "my-tenant" }); ``` ::: warning If no tenant ID is specified, Polecat uses `"DEFAULT"` as the tenant ID. In conjoined tenancy mode, this means documents and events will be stored with `tenant_id = 'DEFAULT'`. Under **separate database** tenancy this fallback is switched off — see [No default tenant is required](#no-default-tenant-is-required) — and a tenantless session throws `DefaultTenantUsageDisabledException` instead. ::: ## ITenanted Interface Documents that implement `ITenanted` will have their `TenantId` property automatically synced from the session: ```cs public class Order : ITenanted { public Guid Id { get; set; } public string TenantId { get; set; } = string.Empty; public string Description { get; set; } = string.Empty; } ``` --- --- url: /documents/multi-tenancy.md --- # Multi-Tenanted Documents Polecat supports isolating document data by tenant using conjoined tenancy (shared tables with a `tenant_id` column) or separate database tenancy. ## Conjoined Tenancy When conjoined tenancy is enabled, all document tables include a `tenant_id` column and use a composite primary key of `(tenant_id, id)`: ```cs var store = DocumentStore.For(opts => { opts.Connection("..."); opts.Events.TenancyStyle = TenancyStyle.Conjoined; }); ``` ### Querying All queries automatically filter by the session's tenant ID: ```cs await using var session = store.LightweightSession(new SessionOptions { TenantId = "tenant-a" }); // Only returns documents belonging to "tenant-a" var orders = await session.Query().ToListAsync(); ``` ### Loading by ID ```cs // Only loads if the document belongs to the session's tenant var order = await session.LoadAsync(orderId); ``` ## Separate Database Tenancy Each tenant gets a completely separate SQL Server database: ```cs var store = DocumentStore.For(opts => { opts.MultiTenantedDatabases(databases => { databases.AddTenant("tenant-a", "Server=localhost;Database=tenant_a;..."); databases.AddTenant("tenant-b", "Server=localhost;Database=tenant_b;..."); }); }); ``` Sessions are automatically routed to the correct database. ::: warning No default tenant — and no top level connection string `MultiTenantedDatabases()` (and `MultiTenantedMasterTable()`) sets `StoreOptions.DefaultTenantUsageEnabled` to `false`. Do **not** register a placeholder `*DEFAULT*` tenant, and you do **not** need to set `StoreOptions.ConnectionString` — the tenancy supplies one. Opening a session or building a daemon without a tenant throws `DefaultTenantUsageDisabledException` rather than quietly using whichever database happened to be first. The async daemon starts one daemon per tenant database with the default tenant disabled, and `ApplyAllDatabaseChangesOnStartup()` migrates every tenant database. See [No default tenant is required](/configuration/multitenancy#no-default-tenant-is-required). ::: ## ITenanted Interface Documents implementing `ITenanted` have their `TenantId` property automatically synced: ```cs public class Order : ITenanted { public Guid Id { get; set; } public string TenantId { get; set; } = ""; public string Description { get; set; } = ""; } // When stored, order.TenantId is automatically set to the session's tenant ``` See [Multi-Tenancy Configuration](/configuration/multitenancy) for complete setup details. --- --- url: /events/natural-keys.md --- # Natural Keys Natural keys let you look up an event stream by a domain-meaningful identifier (like an order number or invoice code) instead of by its internal stream id. Polecat maintains a separate lookup table that maps natural key values to stream ids, so you can use `FetchForWriting` and `FetchLatest` with your natural key in a single database round-trip. ## When to Use Natural Keys Use natural keys when: * External systems or users reference aggregates by a business identifier (e.g., `"ORD-12345"`) rather than a `Guid` stream id * You need to look up streams by a human-readable identifier without maintaining your own separate index * Your aggregate has a stable "business key" that may occasionally change (natural keys support mutation) ## Declaring Natural Keys Mark a property on your aggregate with `[NaturalKey]`, and mark the methods that set or change the key value with `[NaturalKeySource]`: ```cs public record OrderNumber(string Value); public partial class OrderAggregate { public Guid Id { get; set; } [NaturalKey] public OrderNumber OrderNum { get; set; } = null!; public decimal TotalAmount { get; set; } public string CustomerName { get; set; } = string.Empty; public bool IsComplete { get; set; } [NaturalKeySource] public void Apply(NkOrderCreated e) { OrderNum = e.OrderNumber; CustomerName = e.CustomerName; } public void Apply(NkOrderItemAdded e) { TotalAmount += e.Price; } [NaturalKeySource] public void Apply(NkOrderNumberChanged e) { OrderNum = e.NewOrderNumber; } public void Apply(NkOrderCompleted e) { IsComplete = true; } } public partial class InvoiceAggregate { public Guid Id { get; set; } [NaturalKey] public string InvoiceCode { get; set; } = string.Empty; public decimal Amount { get; set; } [NaturalKeySource] public void Apply(NkInvoiceCreated e) { InvoiceCode = e.Code; Amount = e.Amount; } } public record NkOrderCreated(OrderNumber OrderNumber, string CustomerName); public record NkOrderItemAdded(string ItemName, decimal Price); public record NkOrderNumberChanged(OrderNumber NewOrderNumber); public record NkOrderCompleted; public record NkInvoiceCreated(string Code, decimal Amount); ``` snippet source | anchor The `[NaturalKeySource]` attribute tells Polecat which `Create` / `Apply` methods produce or change the natural key value. Polecat uses this information to keep the lookup table in sync whenever events are appended. ## Event-to-Key Mappings Every event type that sets or changes the natural key must be declared through the `[NaturalKeySource]` attribute. When Polecat processes events during an append operation, it extracts the key value from these mapped events and writes it to the lookup table. Events that do not affect the natural key (like `NkOrderItemAdded` in the example above) do not need any mapping. A `[NaturalKeySource]` method that cannot be bound to an extraction strategy is now a configuration-time error — `AssembleAndAssertValidity()` throws an `InvalidProjectionException` naming the method and the reason. Previously such a method registered nothing at all, silently, and the lookup table was simply never written for that event type. ## Explicit Mappings with `NaturalKeyFor()` When attribute discovery cannot bind your method — or when you would simply rather be explicit — register the mapping directly from the projection: ```cs public class OrderProjection: SingleStreamProjection { public OrderProjection() { NaturalKeyFor(x => x // From the event body .SetBy(e => new ProductCode(e.Code)) // From the whole IEvent, when the key depends on event metadata .SetByEvent(e => new ProductCode(e.Data.NewCode))); } } ``` An explicit registration replaces any discovered mapping for the same event type, so it always wins over the attribute. ::: warning The natural key has to be a function of **the event alone**. The lookup table is maintained inline at append time, where no prior aggregate exists under an `Async` snapshot lifecycle, so an extraction that depends on the current aggregate state has nothing to read. This is the constraint the attribute path now enforces as well. ::: ## Storage Polecat automatically creates and manages a lookup table (prefixed with `pc_`) for each aggregate type that has a natural key configured. The table maps natural key values to stream ids and is: * Created automatically during schema migrations * Updated transactionally alongside event appends using `MERGE` statements * Archive-aware via an `is_archived` bit column (archived streams are excluded from lookups) You do not need to create or manage this table yourself. ::: info Unlike Marten's PostgreSQL-based implementation which uses table partitioning for multi-tenancy, Polecat's SQL Server implementation uses an `is_archived` bit column and does not use partitioning for the natural key lookup table. ::: ## FetchForWriting by Natural Key The primary use case for natural keys is looking up a stream for writing without knowing its stream id: ```cs // FetchForWriting by the business identifier instead of stream id var stream = await session2.Events.FetchForWriting(orderNumber, TestContext.Current.CancellationToken); stream.Aggregate.ShouldNotBeNull(); stream.Aggregate!.OrderNum.ShouldBe(orderNumber); // Append new events through the stream stream.AppendOne(new NkOrderItemAdded("Gadget", 19.99m)); await session2.SaveChangesAsync(TestContext.Current.CancellationToken); ``` snippet source | anchor This resolves the natural key to a stream id and fetches the aggregate in a single database round-trip. Polecat uses `WITH (UPDLOCK, HOLDLOCK)` hints to ensure safe concurrent access during the lookup. ## FetchLatest by Natural Key For read-only access, you can use `FetchLatest` with a natural key: ```cs // Read-only access by natural key var aggregate = await session2.Events.FetchLatest(orderNumber, TestContext.Current.CancellationToken); ``` snippet source | anchor ## Mutability Natural keys can change over the lifetime of a stream. When an event mapped with `[NaturalKeySource]` is appended, Polecat updates the lookup table with the new value using a `MERGE` statement. The old key value is replaced, so lookups using the previous key will no longer resolve to that stream. ## Null and Default Keys If a mapped event produces a `null` or default key value, Polecat silently skips writing to the lookup table. This means streams where the natural key has not yet been assigned will not appear in natural key lookups, but will still be accessible by stream id. ## Clean and Maintenance Operations The natural key lookup table is maintained automatically as part of normal event appending. If you need to rebuild the lookup table (for example, after a data migration), you can do so through Polecat's schema management tools as part of a projection rebuild. ## Testing Considerations When writing integration tests: * Natural key lookups work against the same session's uncommitted data, so you can append events and look up by natural key within the same unit of work * If you are using `FetchForWriting` with a natural key that does not exist, the behavior is the same as with a stream id that does not exist ## Integration with Wolverine Natural keys integrate with Wolverine's aggregate handler workflow. See the [Wolverine documentation on natural keys with Polecat](https://wolverinefx.io/guide/durability/polecat/event-sourcing#natural-keys) for details on how Wolverine resolves natural keys from command properties. --- --- url: /documents/sessions.md --- # Opening Sessions Polecat uses sessions to manage document operations. Sessions provide a unit of work pattern -- changes are accumulated and flushed to the database in a single transaction when `SaveChangesAsync()` is called. ## Session Types ### Lightweight Session (Default) No identity tracking. Each `LoadAsync()` call returns a new object instance: ```cs await using var session = store.LightweightSession(); ``` ### Identity Map Session Tracks loaded documents by ID. Repeated loads of the same document return the same instance: ```cs await using var session = store.OpenSession(DocumentTracking.IdentityMap); ``` ### Query Session (Read-Only) For read-only operations. Cannot store or delete documents: ```cs await using var session = store.QuerySession(); ``` ## Session Options Configure sessions with `SessionOptions`: ```cs await using var session = store.LightweightSession(new SessionOptions { // Multi-tenancy TenantId = "my-tenant", // Command timeout in seconds Timeout = 30, // Metadata tracking CorrelationId = "request-123", CausationId = "command-456", LastModifiedBy = "user@example.com", // Per-session listeners Listeners = { new MySessionListener() } }); ``` ## OpenSessionAsync For scenarios requiring explicit transaction control, use `OpenSessionAsync` to eagerly open a connection and begin a transaction: ```cs await using var session = await store.OpenSessionAsync(new SessionOptions { IsolationLevel = IsolationLevel.Serializable }); ``` This is particularly useful for: * Exclusive writing patterns (pessimistic locking) * Custom isolation levels * Ensuring a transaction is active before any operations ## Unit of Work Sessions accumulate pending operations: ```cs await using var session = store.LightweightSession(); session.Store(new User { FirstName = "Alice" }); session.Store(new User { FirstName = "Bob" }); session.Delete(orderId); // All operations execute in a single transaction await session.SaveChangesAsync(); ``` ### Inspecting Pending Changes ```cs var pending = session.PendingChanges; ``` ### Ejecting Documents Remove documents from pending changes without affecting the database: ```cs // Eject a specific document session.Eject(document); // Eject all documents of a type session.EjectAllOfType(typeof(User)); // Eject all pending changes session.EjectAllPendingChanges(); ``` ## Session Listeners Implement `IDocumentSessionListener` to hook into session lifecycle events. `AfterCommitAsync` receives an [`IChangeSet`](#the-ichangeset) describing everything that was written in the just-committed unit of work: ```cs public class AuditListener : IDocumentSessionListener { public Task BeforeSaveChangesAsync(IDocumentSession session, CancellationToken ct) { // Called before the transaction begins. Inspect the pending work via session.PendingChanges. return Task.CompletedTask; } public Task AfterCommitAsync(IDocumentSession session, IChangeSet commit, CancellationToken ct) { // Called after the transaction commits. `commit` reports what changed. foreach (var inserted in commit.Inserted) { /* ... */ } foreach (var updated in commit.Updated) { /* ... */ } foreach (var deleted in commit.Deleted) { /* deleted.DocumentType / deleted.Id */ } return Task.CompletedTask; } } ``` Register listeners globally or per-session: ```cs // Global (on all sessions) opts.Listeners.Add(new AuditListener()); // Per-session await using var session = store.LightweightSession(new SessionOptions { Listeners = { new AuditListener() } }); ``` ### The IChangeSet `IChangeSet` is a snapshot of a single committed unit of work, handed to both `IDocumentSessionListener` (session saves) and `IChangeListener` (async daemon batches — see [Listening for Async Daemon Events](/events/projections/async-daemon#listening-for-daemon-commits)): ```cs public interface IChangeSet { IEnumerable Updated { get; } // Update/Upsert operations IEnumerable Inserted { get; } // Insert operations IEnumerable Deleted { get; } // Deletions (DocumentType + Id) IEnumerable GetEvents(); // events appended this unit of work IEnumerable GetStreams(); // stream actions this unit of work IChangeSet Clone(); } ``` ::: warning The live change set is reset immediately after each commit. If you retain it beyond the `AfterCommitAsync` call (e.g. to hand off to background work), call `commit.Clone()` first to take an immutable copy. ::: ## Request Counting Track the number of database requests made by a session: ```cs await using var session = store.LightweightSession(); await session.LoadAsync(id); Console.WriteLine(session.RequestCount); // 1 ``` --- --- url: /documents/concurrency.md --- # Optimistic Concurrency Polecat supports two forms of optimistic concurrency control to prevent lost updates. ## Guid-Based Versioning (IVersioned) Each save generates a new Guid version. Concurrent modifications are detected when the expected version doesn't match: ```cs public class Order : IVersioned { public Guid Id { get; set; } public Guid Version { get; set; } public string Description { get; set; } = ""; } ``` Usage: ```cs // Load and modify var order = await session.LoadAsync(orderId); // order.Version is automatically populated order.Description = "Updated"; session.Store(order); await session.SaveChangesAsync(); // order.Version is now a new Guid // If another session modified the order between load and save, // SaveChangesAsync throws ConcurrencyException ``` ### UpdateExpectedVersion Explicitly set the expected version for concurrency checks: ```cs session.UpdateExpectedVersion(order, expectedGuidVersion); ``` ## Numeric Revisions (IRevisioned) An integer revision counter that increments on each save: ```cs public class Order : IRevisioned { public Guid Id { get; set; } public int Version { get; set; } public string Description { get; set; } = ""; } ``` Usage: ```cs var order = await session.LoadAsync(orderId); // order.Version == 1 (after first save) order.Description = "Updated"; session.Store(order); await session.SaveChangesAsync(); // order.Version == 2 ``` ### UpdateRevision Explicitly set the expected revision: ```cs session.UpdateRevision(order, expectedRevision: 3); ``` ## Long Numeric Revisions (ILongVersioned) `ILongVersioned` is the 64-bit counterpart of `IRevisioned` — identical behavior, but the revision is a `long` instead of an `int`: ```cs public class CustomerOrderHistory : ILongVersioned { public Guid Id { get; set; } public long Version { get; set; } public string Description { get; set; } = ""; } ``` Usage mirrors `IRevisioned`, with a `long` overload of `UpdateRevision` for explicit checks: ```cs var view = await session.LoadAsync(id); view.Description = "Updated"; session.UpdateRevision(view, expectedRevision: 4_000_000_000L); await session.SaveChangesAsync(); ``` ::: tip Prefer `ILongVersioned` over `IRevisioned` for `MultiStreamProjection`-derived views whose `Version` tracks the **global event sequence number**. That sequence is monotonic across every stream the view folds in and can climb past `Int32.MaxValue` (~2.1 billion) on a busy store, where an `int` revision would overflow. A plain single-stream aggregate, whose `Version` is just that stream's event count, is fine with `IRevisioned`. ::: Both interfaces persist into the same `version` column, which is always `bigint` (Decision D2) so the two are storage-compatible: `IRevisioned` values fit and are downcast on read, while `ILongVersioned` carries the full 64-bit value. Existing tables with an `int` version column are widened to `bigint` in place on the next schema migration — a non-destructive `ALTER COLUMN`, never a drop/recreate. ## Configuration ### Auto-Detection Polecat automatically detects concurrency mode from interfaces: * Implements `IVersioned` → Guid-based versioning * Implements `IRevisioned` → Numeric revisions (int) * Implements `ILongVersioned` → Numeric revisions (long) ### Manual Configuration ```cs opts.Policies.ForDocument(mapping => { mapping.UseOptimisticConcurrency = true; // Guid-based // OR mapping.UseNumericRevisions = true; // Integer-based }); ``` ::: warning `UseOptimisticConcurrency` and `UseNumericRevisions` are mutually exclusive. Choose one per document type. ::: ## ConcurrencyException When a concurrent modification is detected, Polecat throws `ConcurrencyException` (from JasperFx): ```cs try { await session.SaveChangesAsync(); } catch (ConcurrencyException ex) { // Handle the conflict -- reload, merge, or notify the user } ``` ## How It Works * **First save** (version is zero/empty): No concurrency check -- the document is inserted * **Subsequent saves**: The MERGE/UPDATE statement includes a version check in its WHERE clause * **Version sync**: After a successful save, the new version is read back from the database via OUTPUT clause and synced to the in-memory document * **LINQ queries**: Version columns are included in SELECT, and version properties are synced on deserialization --- --- url: /documents/querying/linq/paging.md --- # Paging Polecat provides built-in pagination support via the `IPagedList` interface. ## ToPagedListAsync The simplest way to paginate results: ```cs var pagedList = await session.Query() .OrderBy(x => x.LastName) .ToPagedListAsync(pageNumber: 1, pageSize: 20); ``` ::: tip Page numbers are 1-based, not 0-based. ::: ## IPagedList Properties The returned `IPagedList` includes: | Property | Description | | :--- | :--- | | `TotalItemCount` | Total items across all pages | | `PageCount` | Total number of pages | | `PageNumber` | Current page number (1-based) | | `PageSize` | Items per page | | `HasPreviousPage` | Whether a previous page exists | | `HasNextPage` | Whether a next page exists | | `IsFirstPage` | Whether this is the first page | | `IsLastPage` | Whether this is the last page | | `FirstItemOnPage` | 1-based index of first item on this page | | `LastItemOnPage` | 1-based index of last item on this page | ## How It Works `ToPagedListAsync` executes two queries: 1. A `COUNT(*)` query for the total number of matching documents 2. A `SELECT` with `OFFSET/FETCH` for the current page's data Both queries run against the same filter criteria. ## Manual Paging You can also page manually with `Skip` and `Take`: ```cs var page2 = await session.Query() .OrderBy(x => x.LastName) .Skip(20) .Take(10) .ToListAsync(); ``` ::: warning Always use `OrderBy` with paging to ensure consistent results across pages. ::: ## Keyset (Cursor) Pagination Offset paging (`Skip`/`Take`, `ToPagedListAsync`) gets more expensive the deeper you go — the database still scans and discards every skipped row. **Keyset** (a.k.a. seek or cursor) pagination is instead **constant cost at any depth**: each page seeks directly to where the previous one ended. It is ideal for infinite scroll, "load more", and export feeds. `ToJsonPageByCursorAsync` fetches one page as raw JSON plus an opaque continuation cursor: ```cs // First page: cursor = null var page = await session.Query() .OrderBy(x => x.LastName).ThenBy(x => x.Id) .ToJsonPageByCursorAsync(cursor: null, pageSize: 25); // page.ItemsJson -> "[ {...}, {...}, ... ]" (raw persisted document JSON) // page.Count -> number of items on this page // page.NextCursor -> opaque cursor for the next page, or null at the end of the set // Next page: pass the previous page's NextCursor var next = await session.Query() .OrderBy(x => x.LastName).ThenBy(x => x.Id) .ToJsonPageByCursorAsync(page.NextCursor, 25); ``` When a page returns fewer than `pageSize` items, `NextCursor` is `null` — you have reached the end of the set. ### Rules * **You must `OrderBy`.** A query with no ordering is rejected. * **The terminal ordering key must be the document identity (`Id`).** This guarantees a *total order*, so pagination can never skip or duplicate rows across ties. End your ordering with `.ThenBy(x => x.Id)` (or order by `Id` alone). A non-identity terminal key is rejected. * Mixed ascending/descending orderings are supported; the seek predicate flips direction per key. ### How it works Polecat composes a keyset seek predicate `(k1 > v1) OR (k1 = v1 AND k2 > v2) OR …` (direction-flipped per ordering key), using the **same** SQL locators as the `ORDER BY` so the seek boundary lines up with the sort. The last row's sort-key values are carried in an opaque, versioned (`v1:`) base64-JSON cursor. Cursor values are **typed on decode** by the query's ordering key types (the cursor never dictates types) and enter the query as **bound parameters** — no injection. ::: tip SQL Server ordering String seeks use the column's collation and `uniqueidentifier` (`Guid`) seeks use SQL Server's byte-group ordering — the same ordering the `ORDER BY` uses, so the boundary always matches. (Note this differs from PostgreSQL `uuid` ordering, so cursors are not portable across stores.) ::: --- --- url: /documents/partial-updates-patching.md --- # Partial Updates / Patching Polecat provides a patching API for making targeted updates to documents without loading and re-saving the entire document. Under the hood, this uses SQL Server's `JSON_MODIFY()` function. ## Basic Usage ```cs // Patch a document by ID session.Patch(orderId) .Set(x => x.Status, "Shipped"); await session.SaveChangesAsync(); ``` ## Available Operations ### Set a Value ```cs session.Patch(orderId) .Set(x => x.Status, "Completed") .Set(x => x.ShippedDate, DateTimeOffset.UtcNow); ``` Set nested properties: ```cs session.Patch(orderId) .Set(x => x.Address.City, "New York"); ``` ### Increment a Numeric Value ```cs session.Patch(orderId) .Increment(x => x.ItemCount, 1); // Decrement by using a negative value session.Patch(orderId) .Increment(x => x.ItemCount, -1); ``` ### Append to a Collection ```cs session.Patch(orderId) .Append(x => x.Tags, "priority"); ``` Append only if the value doesn't already exist: ```cs session.Patch(orderId) .AppendIfNotExists(x => x.Tags, "priority"); ``` ### Insert at a Specific Position ```cs session.Patch(orderId) .Insert(x => x.Tags, "urgent", index: 0); ``` ### Remove from a Collection ```cs session.Patch(orderId) .Remove(x => x.Tags, "obsolete"); ``` ### Duplicate a Value Copy a value to multiple destinations: ```cs session.Patch(orderId) .Duplicate(x => x.BillingAddress, x => x.ShippingAddress); ``` ### Rename a Property ```cs session.Patch(orderId) .Rename("oldPropertyName", x => x.NewPropertyName); ``` ### Delete a Property ```cs session.Patch(orderId) .Delete("obsoleteField"); // Or via expression session.Patch(orderId) .Delete(x => x.ObsoleteField); ``` ## Patching with a Where Clause Apply patches to multiple documents matching a condition: ```cs session.Patch(x => x.Status == "Pending") .Set(x => x.Status, "Cancelled"); await session.SaveChangesAsync(); ``` ## How It Works Each patch operation generates a SQL `UPDATE` statement using `JSON_MODIFY()`: ```sql UPDATE pc_doc_order SET data = JSON_MODIFY(data, '$.status', 'Shipped') WHERE id = @id ``` For collection operations, Polecat uses `OPENJSON` and `STRING_AGG` to manipulate JSON arrays. ::: tip Patching is more efficient than loading, modifying, and re-saving a document because only the changed values are sent to the database, and it doesn't require reading the full document first. ::: --- --- url: /documents.md --- # Polecat as Document DB Polecat allows you to use SQL Server 2025 as a document database. Documents are stored as JSON using the native `json` data type, with each document type getting its own table (prefixed with `pc_doc_`). ## Key Concepts * **Documents** are plain .NET objects serialized to JSON * **Sessions** provide a unit of work pattern for batching changes * **LINQ queries** translate to SQL Server JSON path expressions * **Automatic schema management** creates and updates tables as needed ## Document Tables Each document type gets its own table: | Column | Type | Description | | :--- | :--- | :--- | | `id` | Varies | Primary key (Guid, string, int, long) | | `data` | `json` | Serialized document body | | `type` | `nvarchar(250)` | .NET type discriminator | | `last_modified` | `datetimeoffset` | Last modification timestamp | | `created` | `datetimeoffset` | Creation timestamp | | `dotnet_type` | `nvarchar(500)` | Full .NET type name | | `tenant_id` | `nvarchar(250)` | Tenant identifier (conjoined tenancy) | Additional columns are added for features like [soft deletes](/documents/deletes#soft-deletes), [versioning](/documents/concurrency), and [metadata](/documents/metadata). ## Quick Example ```cs // Define a document public class User { public Guid Id { get; set; } public string FirstName { get; set; } = ""; public string LastName { get; set; } = ""; public string Email { get; set; } = ""; } // Store a document await using var session = store.LightweightSession(); var user = new User { FirstName = "Jane", LastName = "Doe", Email = "jane@example.com" }; session.Store(user); await session.SaveChangesAsync(); // Load by ID var loaded = await session.LoadAsync(user.Id); // Query with LINQ var users = await session.Query() .Where(x => x.LastName == "Doe") .ToListAsync(); ``` --- --- url: /events.md --- # Polecat as Event Store Polecat provides a full-featured event store built on SQL Server 2025, following the same patterns as Marten's PostgreSQL-based event store. ## Key Concepts * **Event** -- An immutable record of something that happened in your domain * **Stream** -- A sequence of events related to a specific aggregate or entity * **Aggregate** -- A domain object whose state is derived by replaying events * **Projection** -- A read model built from events, either inline, live, or asynchronously ## Event Store Tables Polecat uses three core tables (all prefixed with `pc_`): | Table | Purpose | | :--- | :--- | | `pc_events` | All events with sequence IDs, stream references, JSON data | | `pc_streams` | Stream metadata: version, type, timestamps | | `pc_event_progression` | Async daemon progress tracking per projection | ## Stream Identity Streams can be identified by either `Guid` or `string`: ```cs var store = DocumentStore.For(opts => { opts.Connection("..."); // Default: Guid stream IDs opts.Events.StreamIdentity = StreamIdentity.AsGuid; // Alternative: String stream IDs opts.Events.StreamIdentity = StreamIdentity.AsString; }); ``` ## Quick Example ```cs // Define events public record InvoiceCreated(decimal Amount, string Customer); public record InvoicePaid(decimal AmountPaid, DateTimeOffset PaidAt); // Append events await using var session = store.LightweightSession(); var streamId = session.Events.StartStream( new InvoiceCreated(100m, "Acme Corp"), new InvoicePaid(100m, DateTimeOffset.UtcNow) ); await session.SaveChangesAsync(); // Replay to aggregate var invoice = await session.Events.AggregateStreamAsync(streamId); ``` See the [Quick Start](/events/quickstart) for a complete walkthrough. ## Projection Strategies | Strategy | When Applied | Use Case | | :--- | :--- | :--- | | [Inline](/events/projections/inline) | Same transaction as event append | Strong consistency requirements | | [Live](/events/projections/live-aggregates) | On-demand replay | Occasional reads, always current | | [Async](/events/projections/async-daemon) | Background daemon | Eventually consistent read models | ## Event Appending Polecat uses **QuickAppend** -- direct SQL `INSERT` statements with an `UPDATE...OUTPUT` pattern for version management. No stored procedures are used. See [Appending Events](/events/appending) for details. --- --- url: /documents/metadata.md --- # Polecat Metadata Polecat can automatically track metadata on your documents through several built-in interfaces. ## ITracked Implement `ITracked` to have correlation, causation, and user tracking automatically synced: ```cs public class Order : ITracked { public Guid Id { get; set; } public string Description { get; set; } = ""; // ITracked members - auto-synced from session public string? CorrelationId { get; set; } public string? CausationId { get; set; } public string? LastModifiedBy { get; set; } } ``` Set tracking values on the session: ```cs await using var session = store.LightweightSession(new SessionOptions { CorrelationId = "request-123", CausationId = "command-456", LastModifiedBy = "user@example.com" }); session.Store(new Order { Description = "New order" }); await session.SaveChangesAsync(); // Order.CorrelationId will be "request-123" ``` ## ITenanted Implement `ITenanted` to have the tenant ID automatically synced from the session: ```cs public class Order : ITenanted { public Guid Id { get; set; } public string TenantId { get; set; } = ""; } ``` See [Multi-Tenancy](/configuration/multitenancy) for more details. ## ISoftDeleted Implement `ISoftDeleted` for automatic soft delete tracking: ```cs public class Order : ISoftDeleted { public Guid Id { get; set; } public bool Deleted { get; set; } public DateTimeOffset? DeletedAt { get; set; } } ``` See [Deleting Documents](/documents/deletes#soft-deletes) for more details. ## IVersioned Implement `IVersioned` for Guid-based optimistic concurrency: ```cs public class Order : IVersioned { public Guid Id { get; set; } public Guid Version { get; set; } } ``` See [Optimistic Concurrency](/documents/concurrency) for more details. ## Built-in Metadata Columns Every document table includes these metadata columns automatically: | Column | Description | | :--- | :--- | | `last_modified` | Updated to `SYSDATETIMEOFFSET()` on every save | | `created` | Set to `SYSDATETIMEOFFSET()` on first insert | | `type` | Short type discriminator | | `dotnet_type` | Full .NET assembly-qualified type name | ## Metadata LINQ Extensions Query documents by their metadata: ```cs // Find documents modified since a given time var recent = await session.Query() .Where(x => x.ModifiedSince(DateTimeOffset.UtcNow.AddHours(-1))) .ToListAsync(); // Find documents modified before a given time var old = await session.Query() .Where(x => x.ModifiedBefore(DateTimeOffset.UtcNow.AddDays(-30))) .ToListAsync(); ``` --- --- url: /whitepaper.md --- # Polecat: A SQL Server-Native Event Store and Document Database *Building on a decade of Marten heritage — for the .NET teams that are already on SQL Server.* ## Executive summary Polecat is an event store and lightweight document database for .NET, backed by SQL Server 2025. It gives SQL Server-on-.NET shops the same productive development model that [Marten](https://martendb.io) brings to PostgreSQL — same `IDocumentStore` API, same projection model, same async daemon, same Critter Stack ecosystem — without a database migration. If you're a SQL Server shop considering event sourcing, CQRS, or a document-store productivity layer, Polecat lets you adopt those patterns inside your existing database, with infrastructure your operations team already runs. It's open source, MIT-licensed, shipped 1.0 in March 2026, and integrates natively with [Wolverine](https://wolverinefx.io) for end-to-end CQRS. *** ## The state of the .NET data layer Most production .NET applications run on SQL Server. That's not nostalgia — it's procurement, licensing, ops tooling, monitoring, backups, DR, DBA expertise, and ten years of internal applications already built around it. SQL Server is the path of least resistance for the majority of .NET teams, and it isn't going anywhere. Meanwhile, the productivity story for event sourcing and document-style persistence in .NET has been overwhelmingly PostgreSQL-shaped. Marten — the open-source library that turned PostgreSQL into a credible event store and document database for .NET — has been mature for years and shipped its 10-year line. EventStoreDB / KurrentDB is its own product. Building event sourcing or a document store on SQL Server has historically meant either (a) writing the plumbing yourself, (b) bolting on Cosmos DB or MongoDB, or (c) adopting PostgreSQL purely for the data-layer features. Each of those costs something real. Hand-rolled event sourcing skips the boring-but-important problems (projection rebuilds, async daemon coordination, optimistic concurrency, schema migrations, multi-tenancy). Adding a second database doubles ops complexity. Switching to PostgreSQL means a team retraining, ops re-tooling, and an extended migration window. **SQL Server 2025 changes the calculus.** It introduces a native `JSON` column type with `JSON_VALUE` / `JSON_QUERY` / `JSON_MODIFY` and modern T-SQL features (window functions, MERGE, partition functions, sequences) — closing the storage-engine gap that made PostgreSQL uniquely suited to this style of work. Polecat is the library that takes the Marten development model and points it at SQL Server 2025. If you'd adopt Marten on PostgreSQL, you can now adopt Polecat on SQL Server. *** ## What Polecat actually is Polecat is one library that gives you two things under a single `IDocumentStore` API: a **lightweight document database** and a **full event store**. ### As a document database Store and query .NET objects as JSON documents in SQL Server. Sessions are explicit (Lightweight or IdentityMap — no surprise dirty-tracking). LINQ is the query surface. Schema is managed for you. ```csharp var store = DocumentStore.For(opts => { opts.ConnectionString = connectionString; opts.UseNativeJsonType = true; // SQL Server 2025 JSON column type }); await using var session = store.LightweightSession(); session.Store(new Customer { Id = id, Name = "Acme", Region = "EMEA" }); await session.SaveChangesAsync(); await using var query = store.QuerySession(); var emea = await query.Query() .Where(x => x.Region == "EMEA") .OrderBy(x => x.Name) .ToListAsync(); ``` Polecat ships the productivity features you'd expect from a mature document layer: LINQ querying with paging and projections, optimistic concurrency (`IRevisioned` for numeric versions, `IVersioned` for GUID versions), soft deletes, document patching (`JSON_MODIFY` under the hood), bulk insert, HiLo and strongly-typed IDs, document metadata (created/modified/tenant), session listeners, multi-tenancy (conjoined, separate-database, or single-tenant), and an admin/diagnostics surface for migrations and dead-letter handling. ### As an event store Append events to streams, project them into read models, and run the projections async without leaving the same store: ```csharp session.Events.StartStream(orderId, new OrderPlaced(customerId, lineItems), new PaymentAuthorized(amount)); await session.SaveChangesAsync(); // Live aggregation (no read model required) var order = await query.Events.AggregateStreamAsync(orderId); // Or build a read model with a projection registered on the store opts.Projections.Add(ProjectionLifecycle.Async); ``` Under the hood: a single-table append (QuickAppend — direct `INSERT` with `OUTPUT` for version capture, no stored procedures); inline, async, or live projection lifecycles; `SingleStreamProjection`, `MultiStreamProjection`, `EventProjection`, `FlatTableProjection`, and composite projections; inline snapshots for fast aggregate fetch; per-tenant event partitioning for very large multi-tenant stores; an async projection daemon that coordinates across nodes with high-water-mark tracking, extended progression diagnostics, and a resilience pipeline on every command; dead-letter persistence under `SkipApplyErrors`; subscriptions; and `FetchForWriting` for exclusive-lock command handlers. ### Built on SQL Server 2025 The storage layer uses SQL Server 2025's native `JSON` data type for document bodies, event data, headers, and snapshots; `bigint IDENTITY` (or per-tenant `CREATE SEQUENCE` under the per-tenant flag) for the global event sequence; `MERGE` for upserts and progression; `sp_getapplock` for advisory locks during exclusive writes and sequence reservation; `datetimeoffset` + `SYSDATETIMEOFFSET()` for timestamps; and partition functions for archive-aware and per-tenant physical partitioning. Schema migrations are managed declaratively by `Weasel.SqlServer`. *** ## The Marten heritage The most important thing to know about Polecat is that it isn't a from-scratch reinvention. It's a SQL Server storage adapter for an architecture that has been deployed in production behind Marten for ten years. Marten launched in 2016 and has shipped continuously since. The non-storage logic — projection lifecycles, the async daemon coordinator, the source generators that emit aggregate appliers at compile time, the LINQ provider's compositional design, the dead-letter mechanism, the resilience pipeline contract, the extended progression-tracking surface, projection rebuilds, multi-tenancy, the strongly-typed-ID conventions, the `IDocumentStore` / `IDocumentSession` / `IQuerySession` shape — lives in shared upstream libraries: * **`JasperFx`** — core utilities, dependency-injection wiring (`IConfigureStore`), strongly-typed-ID detection, source-generator scaffolding. * **`JasperFx.Events`** — the storage-agnostic event-sourcing primitives. Projection types, the async daemon base (`JasperFxAsyncDaemon`), the projection coordinator, `ShardName` / `ShardState` / `HighWaterStatistics`, `IEventStoreInstrumentation`, `IMessageOutbox`, the resilience contract. * **`Weasel.SqlServer`** — declarative schema management, `DatabaseResource` integration, migration logic, partition support. Marten is one consumer of that stack. Polecat is another. The two stores intentionally share the same projection model, the same source generators, the same daemon coordinator, the same DI conventions, the same Critter Stack tooling integrations (CritterWatch monitoring, JasperFx aspirations). The delta is **purely the SQL storage layer** — the SQL dialect, the table layouts (`pc_streams` / `pc_events` / `pc_event_progression` instead of `mt_*`), the JSON storage choice (SQL Server 2025 native `JSON` instead of Postgres `jsonb`), the upsert syntax (`MERGE` instead of `ON CONFLICT`), and the locking primitives (`sp_getapplock` instead of `pg_advisory_lock`). What this means in practice: when a useful pattern lands in Marten — extended progression tracking with heartbeat/agent\_status/pause\_reason columns, per-tenant physical partitioning of the events table, the dedupe + lift wave that consolidated shared types into JasperFx, the `IEventStoreInstrumentation` DI mechanic for CritterWatch — Polecat picks it up too, because most of the work happened upstream of either store. The API surface is intentionally identical so Marten experience translates directly: a developer who knows `session.Events.StartStream` and `SingleStreamProjection` in Marten knows them in Polecat. You aren't betting on a new library. You're betting on the SQL Server adapter for an architecture that's been pressure-tested for a decade. *** ## CQRS with Polecat and Wolverine The canonical Critter Stack pattern is Wolverine on the front edge, Polecat on the storage edge, with the two wired together so command handlers stay small and explicit: * **Wolverine** — the message bus and HTTP/handler runtime from the same author. Handlers are plain methods. Code generation produces the wiring. * **Polecat** — the unit-of-work for state changes. Documents, events, and outgoing Wolverine messages all commit in one SQL Server transaction. `WolverineFx.Polecat` (the NuGet package — the namespace and library identity stays `Wolverine.Polecat`, historical naming quirk; structurally identical to `WolverineFx.Marten`) gives you: * **A transactional outbox** (`IPolecatOutbox`) — Wolverine messages emitted during a handler are written to the Polecat outbox in the same transaction as your document/event changes. Either everything commits, or nothing does. There's no "wrote to the database but the message wasn't published" failure mode. * **Aggregate handler codegen** — annotate a handler with `[WriteAggregate]` or `[ReadAggregate]` and Wolverine.Polecat generates the `FetchForWriting`/`SaveChangesAsync` plumbing. The handler stays a pure command-to-events function. * **Event subscription** — `SubscribeToEvents` publishes selected event types from the Polecat event store as Wolverine messages, so async projection-driven workflows are first-class. * **Concurrency model selection** — `[ConsistentAggregate]` picks optimistic vs revision-based concurrency without leaking SQL into the handler. A typical command handler: ```csharp public class PlaceOrderHandler { [AggregateHandler] public static OrderPlaced Handle(PlaceOrder cmd, [WriteAggregate] Order order) { if (order.Status != OrderStatus.Draft) throw new InvalidOperationException("Order is not in draft state."); return new OrderPlaced(cmd.CustomerId, cmd.LineItems); } } ``` There is no `IDocumentSession` parameter, no `session.SaveChangesAsync()`, no `Events.Append(...)`. Wolverine.Polecat's code generation handles the fetch-aggregate / apply-events / save-changes / publish-outgoing-messages dance. The handler is a function from command to event(s). Wired up at composition time, after `dotnet add package WolverineFx.Polecat`: ```csharp builder.Services.AddPolecat(opts => { opts.ConnectionString = connectionString; opts.Projections.LiveStreamAggregation(); }) .IntegrateWithWolverine(); // transactional outbox + handler codegen ``` The combined pattern — Wolverine for the entry-point and orchestration concerns, Polecat for state — is well-suited to **modular monolith** architectures: complex business workflows expressed as event-driven command handlers, all running in one process against one SQL Server, without the operational cost of premature microservices. *** ## SQL Server 2025 — the moment The features that made PostgreSQL the obvious choice for this style of library — `jsonb`, expressive indexing, partition functions, mature window functions, declarative-ish schema management — were the gap that made hand-rolling event sourcing on SQL Server painful. SQL Server 2025 closes the gap: * A **native `JSON` data type** with first-class `JSON_VALUE` / `JSON_QUERY` / `JSON_MODIFY` operators, suitable for document storage and event-data persistence. * Mature **`MERGE`**, **`OUTPUT`**, and **window function** support that the QuickAppend path and progression upserts depend on. * **Partition functions** and **`CREATE SEQUENCE`** — used by Polecat's archive-aware partitioning and per-tenant event sequencing. * **`sp_getapplock`** / **`sp_releaseapplock`** for advisory locks, used during exclusive writes and tenant-sequence reservation. Polecat targets SQL Server 2025 specifically because the JSON column type is non-negotiable for the projection performance and storage characteristics this kind of library needs. Older SQL Server versions can run the test matrix in `(edge)` mode (no native JSON) for compatibility checks, but production targets are 2025 forward. *** ## Honest limits A whitepaper that doesn't say "here's where we're younger than the alternative" isn't worth reading. So: * **SQL Server 2025 and .NET 10 required.** Older SQL Server versions are out of scope for production deployments — the native JSON type is core to how Polecat stores documents, event bodies, and snapshots. * **The community is younger than Marten's.** Marten has ten years of conference talks, blog posts, and field reports. Polecat is months into 1.0. The codebase is mature because the patterns are mature, but the community knowledge base is still catching up. * **Polecat is System.Text.Json only.** Marten supports both Newtonsoft.Json and STJ for historical reasons; Polecat skipped that fork. * **Polecat doesn't ship dynamic ancillary-store proxies.** `AddPolecatStore()` works for `T == IDocumentStore`; arbitrary marker interfaces are not yet supported the way Marten emits them. * **JasperFx Software offers paid support** for Marten today. Polecat is covered as part of the same ecosystem — the same team builds and supports both. Polecat is production-ready (1.0 shipped March 2026), but you should pick it because the SQL Server alignment matters to you, not because every Marten feature has a like-for-like Polecat copy on day one. *** ## Get started ```bash dotnet add package Polecat ``` Then: ```csharp builder.Services.AddPolecat(connectionString); ``` That's the minimum viable setup. From there the [Getting Started guide](/getting-started) walks you through documents, events, projections, and the async daemon in about an hour. * **Docs:** [polecat.jasperfx.net](https://polecat.jasperfx.net/) * **GitHub:** [JasperFx/polecat](https://github.com/JasperFx/polecat) * **Discord:** [Critter Stack chat](https://discord.gg/WMxrvegf8H) * **Wolverine integration:** [wolverinefx.io](https://wolverinefx.io) * **Support:** [JasperFx Software](https://jasperfx.net/support-plans/) If you're already running SQL Server in production and you've been watching the Critter Stack from across the PostgreSQL fence — this is your invitation. --- --- url: /documents/aspnetcore.md --- # Polecat.AspNetCore Polecat ships a small companion package, **Polecat.AspNetCore**, with helpers for ASP.NET Core development. The main feature is a set of typed `IResult` wrappers that let you return Polecat documents and event-sourced aggregates directly from Minimal API endpoints with correct status codes, content types, and OpenAPI metadata. Install the NuGet package: ```powershell PM> Install-Package Polecat.AspNetCore ``` ## Typed Streaming Result Types For Minimal API endpoints (and frameworks like [Wolverine.Http](https://wolverinefx.net/guide/http/) that dispatch any `IResult` return value), Polecat.AspNetCore ships three typed result wrappers: | Type | Source | Response shape | 404 on miss? | | --- | --- | --- | --- | | `StreamOne` | `IQueryable` — document query | Single `T` | yes | | `StreamMany` | `IQueryable` — document query | JSON array `T[]` | no (empty array = 200) | | `StreamPaged` | `IQueryable` + page number/size | Paged JSON envelope | no (empty page = 200) | | `StreamPagedByCursor` | `IQueryable` + cursor/page size | Keyset page envelope | no (empty page = 200) | | `StreamAggregate` | `IQuerySession` + stream id — event-sourced | Single `T` | yes | Each type implements both `IResult` (so ASP.NET dispatches it via `ExecuteAsync`) and `IEndpointMetadataProvider` (so OpenAPI generators see the right response shape). ### StreamOne — single document with 404 on miss ```csharp app.MapGet("/issues/{id:guid}", (Guid id, IQuerySession session) => new StreamOne(session.Query().Where(x => x.Id == id))); ``` Returns `200 application/json` with the document JSON on a hit, `404` on a miss. `Content-Length` and `Content-Type` are set automatically. ### StreamMany — JSON array ```csharp app.MapGet("/issues/open", (IQuerySession session) => new StreamMany(session.Query().Where(x => x.IsOpen))); ``` Returns `200 application/json` with a JSON array body. An empty result set yields `[]`, not a 404. ### StreamPaged — one page of documents plus paging metadata ```csharp app.MapGet("/issues/paged/{pageNumber:int}/{pageSize:int}", (int pageNumber, int pageSize, IQuerySession session) => new StreamPaged( session.Query().OrderBy(x => x.Number), pageNumber, pageSize)); ``` Streams a single page of documents **plus** paging metadata as one JSON envelope in a **single database round trip** — the total row count rides along on every row via `COUNT(*) OVER()`, so there is no separate `COUNT` query. Raw persisted document JSON is streamed straight into `items` with no deserialize/reserialize: ```json {"pageNumber":3,"pageSize":25,"totalItemCount":1207,"pageCount":49, "hasNextPage":true,"hasPreviousPage":true,"items":[...]} ``` The envelope key names and shape are byte-for-byte identical to Marten's `StreamPaged`, so clients are interchangeable across the two stores. Include an `OrderBy` on the query for a stable page order (SQL Server requires an `ORDER BY` for `OFFSET/FETCH` paging). `pageNumber` is 1-based; both `pageNumber` and `pageSize` must be `>= 1` (otherwise an `ArgumentOutOfRangeException` is thrown). An empty match — or paging past the end of the set — yields `totalItemCount: 0`, `pageCount: 0`, `items: []`. ::: warning Paging **past the end of a non-empty set** (e.g. 10 items, page 5 of size 3 → `OFFSET 12`) returns zero rows, so the window-function total is lost and the envelope reports `totalItemCount: 0` even though items exist. This matches `PagedList`/Marten behavior; page within range to get an accurate total. ::: The same one-round-trip envelope is available off any Polecat `IQueryable` without ASP.NET Core via `StreamPagedJsonArray`: ```csharp await session.Query().OrderBy(x => x.Number) .StreamPagedJsonArray(pageNumber, pageSize, destinationStream); ``` ### StreamPagedByCursor — keyset (cursor) pagination ```csharp app.MapGet("/issues/paged-cursor/{pageSize:int}", (int pageSize, string? cursor, IQuerySession session) => new StreamPagedByCursor( session.Query().OrderBy(x => x.Number).ThenBy(x => x.Id), cursor, pageSize)); ``` Streams one keyset (seek) page — constant cost at any depth — as a JSON envelope, and echoes the continuation cursor in a `Polecat-Continuation` response header: ```json {"items":[...],"nextCursor":"v1:..."} ``` `nextCursor` is `null` at the end of the set. The wrapped query must order so its terminal key is the document identity (e.g. `OrderBy(x => x.Number).ThenBy(x => x.Id)`); a query lacking an `OrderBy` or with a non-identity terminal key is rejected. See [Keyset (Cursor) Pagination](./querying/linq/paging.md#keyset-cursor-pagination) for the mechanics and rules. ### StreamAggregate — event-sourced aggregate (latest) ```csharp app.MapGet("/orders/{id:guid}", (Guid id, IQuerySession session) => new StreamAggregate(session, id)); ``` Returns `200 application/json` with the latest projected aggregate state, or `404` if no stream exists. A constructor overload accepts `string` ids for stores configured with string-keyed streams. ## Streaming Event Stream Metadata and Events `StreamEventState` and `StreamEvents` are the event-side siblings of `StreamAggregate`, backed by the `FetchStreamStatePlan` / `FetchStreamPlan` query plans — so the same plan can be batched through `IBatchedQuery.QueryByPlan()` and returned from an endpoint. ```csharp app.MapGet("/orders/{id:guid}/state", (Guid id, IQuerySession s) => new StreamEventState(s, id)); app.MapGet("/orders/{id:guid}/events", (Guid id, IQuerySession s) => new StreamEvents(s, id)); ``` Both take `Guid`, `string` and pre-built-plan constructors, and both implement `IResult` and `IEndpointMetadataProvider` so OpenAPI advertises the right `200` and `404` shapes. ### StreamEventState vs StreamAggregate * **`StreamEventState`** writes the stream's *metadata* — version, created/last timestamps, archived flag. * **`StreamAggregate`** writes the projected aggregate *state* built from the stream's events. ### The response DTOs Neither result writes the framework's own types to the wire, because neither can. `StreamState.AggregateType` and `IEvent.EventType` are `System.Type`, and System.Text.Json refuses to serialize those: ``` NotSupportedException: Serialization and deserialization of 'System.Type' instances is not supported. Path: $.AggregateType. ``` So the bodies are `StreamStateResponse` and `EventResponse`: the aggregate type reduces to its simple name, and `IEvent`'s assembly-qualified `DotNetTypeName` is deliberately kept off the wire — use `EventTypeName`, Polecat's stable event type alias, as the client-side discriminator. The property names match Marten's equivalents, so a client can move between the two stores unchanged. ### Empty streams are ambiguous `FetchStream` yields an empty list both for a stream that does not exist and for a filter that excludes every event, and the two cannot be told apart. `StreamEvents` exposes `OnEmptyStatus`, defaulting to `404` to match the other single-resource results. Set it to `200` to return an empty array instead — which is what you want when paging forward with `fromVersion` and running off the end is expected: ```csharp app.MapGet("/orders/{id:guid}/events", (Guid id, long fromVersion, IQuerySession s) => new StreamEvents(s, id, fromVersion: fromVersion) { OnEmptyStatus = StatusCodes.Status200OK }); ``` Both responses set `Content-Length`, and serialization buffers through an `ArrayBufferWriter` so the JSON never round-trips through a .NET string. ### StreamOne vs StreamAggregate * **`StreamOne`** is for regular documents — objects stored via `session.Store()` and queried with `session.Query()`. * **`StreamAggregate`** is for event-sourced aggregates — Polecat rebuilds (or reads the snapshot of) the latest aggregate state from events before writing the response. ### ETag / Conditional Requests `StreamOne` and `StreamAggregate` support HTTP conditional requests (`ETag` / `If-None-Match` → `304 Not Modified`) so polling clients skip re-downloading unchanged documents/aggregates. It is **on by default**. * On a normal hit, an `ETag` response header carrying the version is emitted. * An incoming `If-None-Match` that matches the current version yields `304 Not Modified` with an empty body and the `ETag` header — for `StreamAggregate` this skips the aggregation entirely (the stream version is read cheaply first). * The version source differs by type: * **`StreamOne`** uses the document's `version` column (read inline with the document JSON in a single round trip — no follow-up metadata query). * **`StreamAggregate`** uses the event stream's version (via `FetchStreamStateAsync`, read before folding). * `StreamMany` is intentionally out of scope (a cheap collection-wide ETag is hard to derive). Opt out per endpoint with `EmitETag = false`, which restores the exact pre-ETag behavior (no header, no conditional handling): ```csharp app.MapGet("/issues/{id:guid}", (Guid id, IQuerySession session) => new StreamOne(session.Query().Where(x => x.Id == id)) { EmitETag = false }); ``` ::: tip ETag values are opaque per RFC 7232. Polecat document tables always carry a `version` column, so a document ETag is always available when `EmitETag` is on; the only way to suppress it is `EmitETag = false`. `ETagHelpers` handles the `*` wildcard, comma-separated `If-None-Match` lists, and `W/` weak validators (weak comparison, the correct function for `If-None-Match`). ::: ### Customizing status code and content type All three types expose `init`-only properties: ```csharp app.MapPost("/issues", (CreateIssue cmd, IQuerySession session) => new StreamOne(session.Query().Where(x => x.Id == cmd.IssueId)) { OnFoundStatus = StatusCodes.Status201Created, ContentType = "application/vnd.myapi.issue+json" }); ``` ::: tip `StreamOne` and `StreamPaged` stream the raw persisted document JSON straight through (no deserialize/reserialize). `StreamMany` and `StreamAggregate` materialize via the regular query/projection path and serialize through `System.Text.Json`. All of them eliminate the endpoint boilerplate (null-check, status code, content type, OpenAPI metadata). ::: --- --- url: /projection-sg-audit-108.md --- # Projection source-generator dispatch audit (Polecat#108) > Pre-4.0 sweep verifying that every projection type in the Polecat test > libraries either gets a `[GeneratedEvolver]` dispatcher emitted by > `JasperFx.Events.SourceGenerator` or deliberately bypasses that path > (via an `Evolve` / `EvolveAsync` / `DetermineAction*` override, or via > its own non-SG dispatch like `FlatTableProjection`). > > **Headline count: 50 projection-shaped types audited. 41 generated, 9 > deliberate bypasses, 0 SG-discovery gaps filed.** > > Run against `JasperFx.Events 2.0.0-alpha.12` + `JasperFx.Events.SourceGenerator > 2.0.0-alpha.5` on Polecat `main` at `c24367b`. Reproducible with > `dotnet build polecat.slnx -c Release -p:EmitCompilerGeneratedFiles=true > -p:CompilerGeneratedFilesOutputPath=$(pwd)/_gensg` (then look under > `_gensg/**/JasperFx.Events.SourceGenerator/**`). ## How to read this report * Types marked *(shared: SomethingCompliance)* are declared in `JasperFx.Events.ComplianceTests` rather than in a Polecat test file. That package is source-only, so they compile inside `Polecat.Tests` and the source generator treats them exactly like local types. They moved there in compliance waves 1–3 (#393, #400, #399); everything else cites the file it lives in. * **✅ Generated**: SG emits a `[GeneratedEvolver]` partial / standalone evolver / `EventProjection` partial override; runtime registration finds the dispatcher; the post-#276 fail-fast does not fire. * **⚠ Bypass**: by design — the projection overrides one of the dispatch methods (`Evolve`, `EvolveAsync`, `DetermineAction`, `DetermineActionAsync`, or `ApplyAsync` on `EventProjection`) so `JasperFxAggregationProjectionBase. isOverridden(...)` returns `true` and the SG path is intentionally not exercised. `FlatTableProjection` and `EfCoreSingleStreamProjection` also fall in this column because they ship their own dispatch. * **❌ Gap**: SG should have emitted but didn't; filed against JasperFx#276 with a min-repro. **(None at this audit.)** The `partial` column reflects whether the **type itself** is declared `partial`. Self-aggregating doc types use the SG's `SelfAggregating` / `SelfAggregatingEvolve` modes, which emit a *separate* evolver class registered via `[assembly: GeneratedEvolver(...)]` and don't strictly require `partial` on the doc — but marking them `partial` is harmless and matches the broader Critter Stack convention. Projection subclasses with conventional `Apply` / `Create` / `Project` methods **do** require `partial` (the SG attaches a partial override to that class). ## Self-aggregating document types Registered via `opts.Projections.Snapshot()`, `opts.Projections.Add>(...)`, `session.Events.AggregateStreamAsync(...)`, etc. The SG inspects `T`'s `Apply` / `Create` / `ShouldDelete` / `Evolve` / `EvolveAsync` methods and emits a standalone `TEvolver` class. | # | Doc | TId | Methods | `partial`? | SG output | Status | |---|---|---|---|---|---|---| | 1 | `Bug4197Aggregate` *(Events/Bug\_4197\_fetch\_for\_writing\_natural\_key.cs)* | `Guid` | Apply + Create | ✓ | `Polecat_Tests_Events_Bug4197Aggregate_System_GuidEvolver.g.cs` | ✅ | | 2 | `DeletableAggregate` *(Events/aggregate\_stream\_to\_last\_known\_tests.cs)* | `Guid` | Apply + Create + ShouldDelete | ✓ | `..._DeletableAggregate_System_GuidEvolver.g.cs` | ✅ | | 3 | `StringQuestAggregate` *(Events/always\_enforce\_consistency\_tests.cs)* | `string` | Apply + Create | ✓ | `..._StringQuestAggregate_stringEvolver.g.cs` | ✅ | | 4 | `StudentCourseEnrollment` *(Events/dcb\_tag\_query\_and\_consistency\_tests.cs)* | `Guid` | Apply + Create | ✓ | `..._StudentCourseEnrollment_System_GuidEvolver.g.cs` | ✅ | | 5 | `QuestAggregate` *(Events/fetch\_for\_writing\_tests.cs)* | `Guid` | Apply + Create | ✓ | `..._QuestAggregate_System_GuidEvolver.g.cs` | ✅ | | 6 | `InlineSeAggregate` *(Events/inline\_projection\_side\_effects\_tests.cs — nested)* | `Guid` | Apply | ✓ | `..._inline_projection_side_effects_tests_InlineSeAggregate_System_GuidEvolver.g.cs` | ✅ | | 7 | `OrderAggregate` *(Events/natural\_key\_tests.cs)* | `Guid` | Apply + Create | ✓ | `..._OrderAggregate_System_GuidEvolver.g.cs` | ✅ | | 8 | `InvoiceAggregate` *(Events/natural\_key\_tests.cs)* | `Guid` | Apply + Create | ✓ | `..._InvoiceAggregate_System_GuidEvolver.g.cs` | ✅ | | 9 | `Report` *(Events/project\_latest\_tests.cs)* | `Guid` | Apply + Create | ✓ | `..._Report_System_GuidEvolver.g.cs` | ✅ | | 10 | `StringReport` *(Events/project\_latest\_tests.cs)* | `string` | Apply + Create | ✓ | `..._StringReport_stringEvolver.g.cs` | ✅ | | 11 | `ScenarioQuestParty` *(Events/projection\_scenario\_tests.cs)* | `Guid` | Apply + Create | ✓ | `..._ScenarioQuestParty_System_GuidEvolver.g.cs` | ✅ | | 12 | `MutableIEventEvolveAggregate` *(shared: SelfAggregatingEvolveCompliance)* | `Guid` | `Evolve(snapshot, IEvent)` | ✗ | `..._MutableIEventEvolveAggregate_System_GuidEvolveEvolver.g.cs` | ✅ | | 13 | `MutableObjectEvolveAggregate` *(shared: SelfAggregatingEvolveCompliance)* | `Guid` | `Evolve(snapshot, TEvent)` | ✗ | `..._MutableObjectEvolveAggregate_System_GuidEvolveEvolver.g.cs` | ✅ | | 14 | `ImmutableIEventEvolveAggregate` *(record; shared: SelfAggregatingEvolveCompliance)* | `Guid` | `Evolve(snapshot, IEvent)` returning new record | ✗ | `..._ImmutableIEventEvolveAggregate_System_GuidEvolveEvolver.g.cs` | ✅ | | 15 | `ImmutableObjectEvolveAggregate` *(record; same file)* | `Guid` | `Evolve(snapshot, TEvent)` returning new record | ✗ | `..._ImmutableObjectEvolveAggregate_System_GuidEvolveEvolver.g.cs` | ✅ | | 16 | `AsyncEvolveAggregate` *(shared: SelfAggregatingEvolveCompliance)* | `Guid` | `EvolveAsync` | ✗ | `..._AsyncEvolveAggregate_System_GuidEvolveEvolver.g.cs` | ✅ | | 17 | `ImmutableAsyncEvolveAggregate` *(record; same file)* | `Guid` | `EvolveAsync` returning new record | ✗ | `..._ImmutableAsyncEvolveAggregate_System_GuidEvolveEvolver.g.cs` | ✅ | | 18 | `CompositeQuestParty` *(Projections/composite\_projection\_tests.cs)* | `Guid` | Apply + Create | ✓ | `..._CompositeQuestParty_System_GuidEvolver.g.cs` | ✅ | | 19 | `QuestStats` *(Projections/composite\_projection\_tests.cs)* | `Guid` | Apply + Create | ✓ | `..._QuestStats_System_GuidEvolver.g.cs` | ✅ | | 20 | `QuestParty` *(Projections/inline\_projection\_tests.cs)* | `Guid` | Apply + Create + ShouldDelete | ✓ | `..._QuestParty_System_GuidEvolver.g.cs` | ✅ | | 21 | `SelfAggregatingStringQuest` *(shared: StringIdentitySingleStreamCompliance)* | `string` | Apply + Create | ✓ | `..._SelfAggregatingStringQuest_stringEvolver.g.cs` | ✅ | | 22 | `SnapshotParty` *(Projections/snapshot\_registration\_tests.cs)* | `Guid` | Apply + Create | ✓ | `..._SnapshotParty_System_GuidEvolver.g.cs` | ✅ | | 23 | `SnapshotPartyByString` *(Projections/snapshot\_registration\_tests.cs)* | `string` | Apply + Create | ✓ | `..._SnapshotPartyByString_stringEvolver.g.cs` | ✅ | | 24 | `Payment` *(Projections/using\_guid\_based\_strong\_typed\_id\_for\_aggregate\_identity.cs)* | `PaymentId` *(wraps Guid)* | Apply + Create | ✓ | `..._Payment_Polecat_Tests_Projections_PaymentIdEvolver.g.cs` | ✅ | | 25 | `Payment2` *(Projections/using\_string\_based\_strong\_typed\_id\_for\_aggregate\_identity.cs)* | `Payment2Id` *(wraps string)* | Apply + Create | ✓ | `..._Payment2_Polecat_Tests_Projections_Payment2IdEvolver.g.cs` | ✅ | | 26 | `Quest` *(AotSmoke/Program.cs)* | `Guid` | Create | ✓ | `Polecat_AotSmoke_Quest_System_GuidEvolver.g.cs` | ✅ | | 27 | `StreamingQuestParty` *(AspNetCore.Testing/Program.cs)* | `Guid` | Apply + Create | ✓ | `..._StreamingQuestParty_System_GuidEvolver.g.cs` | ✅ | > **27/27 self-aggregating doc types: ✅ all generated.** ## Projection subclasses (Polecat-side) Subclass of one of `EventProjection`, `SingleStreamProjection`, `MultiStreamProjection`, `PolecatCompositeProjection`, `FlatTableProjection`. The SG attaches a `[GeneratedEvolver]` partial method to the class (or, for `EventProjection`, an `ApplyAsync` partial override) — **`partial` required on the class** for these shapes. | # | Class | Base | Shape | `partial`? | SG output | Status | |---|---|---|---|---|---|---| | 1 | `QuestLogProjection` *(Projections/event\_projection\_tests.cs)* | `EventProjection` | conventional `Project(TEvent, IDocumentSession)` | ✓ | `..._QuestLogProjection.EventProjection.g.cs` | ✅ | | 2 | `MultiEventQuestLogProjection` *(Projections/event\_projection\_tests.cs)* | `EventProjection` | conventional `Project(TEvent, IDocumentSession)` × 2 | ✓ | `..._MultiEventQuestLogProjection.EventProjection.g.cs` | ✅ | | 3 | `SimpleEnrichmentProjection` *(shared: EventProjectionEnrichmentCompliance)* | `EventProjection` | `Project` + `EnrichEventsAsync` override | ✓ | `..._SimpleEnrichmentProjection.EventProjection.g.cs` | ✅ (EnrichEventsAsync is a side-channel; dispatch still through SG-emitted `Project`) | | 4 | `EnrichmentCallOrderProjection` *(shared: EventProjectionEnrichmentCompliance)* | `EventProjection` | `Project` + `EnrichEventsAsync` override | ✓ | `..._EnrichmentCallOrderProjection.EventProjection.g.cs` | ✅ | | 5 | `DbLookupEnrichmentProjection` *(shared: EventProjectionEnrichmentCompliance)* | `EventProjection` | `Project` + `EnrichEventsAsync` override | ✓ | `..._DbLookupEnrichmentProjection.EventProjection.g.cs` | ✅ | | 6 | `AuditRecordProjection` *(shared: EventProjectionRegistrationCompliance)* | `EventProjection` | overrides `ApplyAsync` directly | ✓ | `..._AuditRecordProjection.TypeRegistration.g.cs` *(TypeRegistration-only)* | ✅ — dispatch via `ApplyAsync` override; SG emits only the `RegisterPublishedType` constructor so `Schema.For` is discoverable (per [marten#4166](https://github.com/JasperFx/marten/issues/4166)) | | 7 | `AuditRecordCreatorProjection` *(shared: EventProjectionRegistrationCompliance)* | `EventProjection` | conventional `Create(AuditableEvent)` returning the doc type | ✓ | `..._AuditRecordCreatorProjection.EventProjection.g.cs` | ✅ | | 8 | `ImportSqlProjection` *(Projections/using\_event\_projection\_for\_flat\_tables.cs)* | `EventProjection` | conventional `Project(TEvent, IDocumentSession)` × 2 | ✓ | `..._ImportSqlProjection.EventProjection.g.cs` | ✅ | | 9 | `StringQuestPartyProjection` *(shared: StringIdentitySingleStreamCompliance)* | `SingleStreamProjection` | conventional Apply/Create + ShouldDelete on the **projection** (delegating to doc would be the alternative) | ✓ | `..._StringQuestPartyProjection.Evolver.g.cs` *(PartialProjection)* | ✅ | | 10 | `CustomerSummaryProjection` *(Projections/multi\_stream\_projection\_tests.cs)* | `MultiStreamProjection` | conventional Apply on the projection | ✓ | `..._CustomerSummaryProjection.Evolver.g.cs` | ✅ | | 11 | `MonthlyAccountActivityProjection` *(Projections/time\_based\_multi\_stream\_projection\_tests.cs)* | `MultiStreamProjection` | conventional Apply on the projection | ✓ | `..._MonthlyAccountActivityProjection.Evolver.g.cs` | ✅ | | 12 | `CompositeOrderProjection` *(Projections/composite\_try\_find\_upstream\_cache\_tests.cs)* | `SingleStreamProjection` | **overrides `Evolve` directly** | ✓ | — | ⚠ Deliberate bypass — override wins per #276 doctrine. Composite-projection test exercises upstream-cache lookup, not dispatch correctness. | | 13 | `OrderShippingNotificationProjection` *(Projections/composite\_try\_find\_upstream\_cache\_tests.cs)* | `MultiStreamProjection` | overrides `Evolve` + `EnrichEventsAsync` for the upstream-cache `TryFindUpstreamCache` pattern | ✓ | — | ⚠ Deliberate bypass — overrides win; the test specifically exercises the composite-projection upstream-cache plumbing. | | 14 | `InlineSeProjection` *(Events/inline\_projection\_side\_effects\_tests.cs — nested)* | `SingleStreamProjection` | overrides `RaiseSideEffects` only; **no Apply on the projection** | ✓ | — | ✅ Dispatch resolves via the SG-emitted `InlineSeAggregate` self-aggregating evolver (#6 in the doc table). The projection only adds outbox side effects; the empty-Apply-set on the projection class is intentional. | | 15 | `QuestMetricsProjection` *(Projections/flat\_table\_projection\_tests.cs)* | `FlatTableProjection` | own dispatch via `Project` / `Delete` registrations to `_handlers` dictionary | ✗ | — | ⚠ Deliberate bypass — `FlatTableProjection` is not part of the JasperFx.Events apply-method-discovery contract. It owns its own dispatch keyed on event Type in `_handlers`; `partial` would be a no-op. | | 16 | `PascalCaseFlatProjection` *(Projections/flat\_table\_projection\_tests.cs)* | `FlatTableProjection` | same | ✗ | — | ⚠ Same as above | > **11/16 projection subclasses: ✅ generated. 4/16: ⚠ deliberate bypass (Evolve override or FlatTable). 1/16: ✅ delegates to its self-aggregating doc evolver.** ## EF Core projections (Polecat.EntityFrameworkCore.Tests) | # | Class *(`TestProjections.cs`)* | Base | Dispatch | SG output | Status | |---|---|---|---|---|---| | 1 | `OrderAggregate` | `EfCoreSingleStreamProjection` | overrides `DetermineActionAsync` to extract `DbContext` from `EfCoreProjectionStorage` | — | ⚠ Deliberate bypass — EF Core integration shape; override wins per #276 doctrine. | | 2 | `OrderDetailProjection` | `EfCoreEventProjection` | extends `ProjectionBase`+`IProjection` directly (not `EventProjection`), wrapped in `ProjectionWrapper` for registration | — | ⚠ Deliberate bypass — different inheritance chain from the SG's `EventProjection` discovery; dispatch via the wrapper. | | 3 | `TenantedOrderAggregate` | `EfCoreSingleStreamProjection` | same as `OrderAggregate` | — | ⚠ Same as #1 | | 4 | `NonTenantedOrderAggregate` | `EfCoreSingleStreamProjection` | same | — | ⚠ Same as #1 | > **4/4 EF Core projections: ⚠ deliberate bypass.** EF Core integration owns its own dispatch path (extracts `DbContext` from `EfCoreProjectionStorage` inside an override) so SG emission was never expected for these. ## DcbLoadTest No projection types declared. (Load harness exercises the event store write path; no projections registered.) ## Summary | Category | Count | |---|---| | Self-aggregating doc types — ✅ generated | 27 | | Projection subclasses — ✅ generated | 11 | | Projection subclasses — ⚠ deliberate bypass (`Evolve` / `EnrichEventsAsync` override or `FlatTableProjection` / `EfCoreSingleStreamProjection`) | 9 | | Projection subclasses — ✅ delegates to a self-aggregating evolver | 1 | | Projection subclasses — ❌ SG-discovery gap filed against `JasperFx#276` | **0** | | **Total projection-shaped types inventoried** | **50** *(46 in Polecat.Tests + 4 in EFCore.Tests, plus AotSmoke + AspNetCore.Testing samples)* | **SG emission stats:** 38 `[GeneratedEvolver]` outputs from `JasperFx.Events.SourceGenerator` across the solution — 27 self-aggregating-doc evolvers, 8 `EventProjection` partial overrides, 3 `PartialProjection` `Evolver` overrides on `SingleStreamProjection<,>` / `MultiStreamProjection<,>` subclasses, and 1 TypeRegistration-only emit for `AuditRecordProjection`. ## Shape coverage (no silent-mis-execution check) Per chip §4, every shape with an SG-emitted dispatcher has at least one behavioral test in the existing Polecat test suite that fires events through the projection and asserts state evolves correctly: | Shape | Representative test | |---|---| | Self-aggregating sync Apply + Create | `fetch_for_writing_tests`, `Bug_4197_fetch_for_writing_natural_key` | | Self-aggregating Apply + Create + **ShouldDelete** | `aggregate_stream_to_last_known_tests` (DeletableAggregate), `inline_projection_tests` (QuestParty) | | Self-aggregating sync `Evolve(snapshot, IEvent)` (mutable + immutable record) | `SelfAggregatingEvolveCompliance` (×4 variants) | | Self-aggregating `EvolveAsync` | `SelfAggregatingEvolveCompliance` (×2 variants) | | String identity | `StringIdentitySingleStreamCompliance`, `project_latest_tests` (StringReport), `always_enforce_consistency_with_string_stream_id` | | Strong-typed-id identity (wrapper struct) | `using_guid_based_strong_typed_id_for_aggregate_identity`, `using_string_based_strong_typed_id_for_aggregate_identity` | | `EventProjection` with conventional `Project(TEvent, IDocumentSession)` | `event_projection_tests` | | `EventProjection` with `Create(TEvent)` returning a doc | `EventProjectionRegistrationCompliance` (AuditRecordCreatorProjection) | | `EventProjection` with `ApplyAsync` override | `EventProjectionRegistrationCompliance` (AuditRecordProjection) | | `EventProjection` with `EnrichEventsAsync` override | `EventProjectionEnrichmentCompliance` (×3 variants) | | `SingleStreamProjection` subclass with Apply on the projection | `StringIdentitySingleStreamCompliance` (StringQuestPartyProjection) | | `MultiStreamProjection` subclass with Apply on the projection | `multi_stream_projection_tests`, `time_based_multi_stream_projection_tests` | | `PolecatCompositeProjection` with `Snapshot` | `composite_projection_tests` | | `PolecatCompositeProjection` with `TryFindUpstreamCache` between stages | `composite_try_find_upstream_cache_tests` | | Inline projection with `RaiseSideEffects` outbox | `inline_projection_side_effects_tests` | | `FlatTableProjection` (non-SG dispatch) | `flat_table_projection_tests`, `using_event_projection_for_flat_tables` | | `EfCoreSingleStreamProjection` (non-SG dispatch) | `Polecat.EntityFrameworkCore.Tests` integration suite | | DCB tags + self-aggregating | `dcb_tag_query_and_consistency_tests` (StudentCourseEnrollment) | No shape is unrepresented; no new per-shape behavioral test was added by this audit. ## CI regression guard A focused dispatcher-resolution harness lives at `src/Polecat.Tests/Projections/projection_sg_dispatch_audit_tests.cs`. It registers every emit-expected test-library projection through a fresh `DocumentStore` and asserts construction completes without `InvalidProjectionException`. Future `JasperFx.Events.SourceGenerator` regressions (a shape the SG silently fails to emit a `[GeneratedEvolver]` for) trip this single test immediately, before the broader test suite's slower fixtures get a chance to fail with the same symptom across many files. ## SG gaps filed **None.** The post-#298 fix landed in `JasperFx.Events 2.0.0-alpha.12` unblocked the last shape that was tripping the runtime (self-aggregating docs with `ShouldDelete` going through `IGeneratedSyncDetermineAction`). Every projection in the inventory either generates correctly or falls into the deliberate-bypass column. --- --- url: /events/projections.md --- # Projections Overview Projections are the mechanism for building read models from events. Polecat supports several projection types and lifecycle strategies. ## Projection Lifecycle Every projection runs with one of three lifecycle strategies: ### Inline Projections run in the **same transaction** as the event append. This provides strong consistency -- the read model is always up to date: ```cs opts.Projections.Add>(ProjectionLifecycle.Inline); ``` ### Async Projections run in the **background** via the async daemon. The read model is eventually consistent: ```cs opts.Projections.Add>(ProjectionLifecycle.Async); ``` ### Live Projections are built **on demand** by replaying events each time. No read model is persisted: ```cs var order = await session.Events.AggregateStreamAsync(streamId); ``` ## Projection Types | Type | Description | Use Case | | :--- | :--- | :--- | | [Single Stream](/events/projections/single-stream-projections) | One aggregate per stream | Order, Invoice, Account | | [Multi Stream](/events/projections/multi-stream-projections) | Aggregate across multiple streams | Dashboard, Report | | [Event Projection](/events/projections/event-projections) | Per-event document creation | Audit log, Search index | | [Flat Table](/events/projections/flat) | Direct SQL table writes | Reporting, Analytics | | [Composite](/events/projections/composite) | Multi-stage orchestration | Complex pipelines | ## Conventional Projection Methods Polecat discovers projection methods by convention: ### Create Creates the initial aggregate from a stream-starting event: ```cs public static OrderSummary Create(OrderCreated e) => new() { Status = "Created", Amount = e.Amount }; ``` ### Apply Applies an event to an existing aggregate: ```cs public void Apply(OrderShipped e) { Status = "Shipped"; ShippedDate = e.ShippedAt; } ``` ### ShouldDelete Signals that the aggregate should be deleted: ```cs public bool ShouldDelete(OrderCancelled e) => true; ``` ## Registration ```cs var store = DocumentStore.For(opts => { opts.Connection("..."); // Single stream projection (inline) opts.Projections.Add>(ProjectionLifecycle.Inline); // Multi stream projection as async opts.Projections.Add(ProjectionLifecycle.Async); // Event projection opts.Projections.Add(ProjectionLifecycle.Inline); }); ``` --- --- url: /events/projections/project-latest.md --- # ProjectLatest — Include Pending Events `ProjectLatest()` returns the projected state of an aggregate including any events that have been appended in the current session but not yet committed. This eliminates the need for a forced `SaveChangesAsync()` + `FetchLatest()` round-trip when you need the projected result immediately after appending events. ## Motivation A common pattern in command handlers looks like this: ```csharp // Today's pattern: forced flush + re-read session.Events.StartStream(id, new ReportCreated("Q1")); await session.SaveChangesAsync(ct); // forced flush var report = await session.Events.FetchLatest(id, ct); // re-read return report; ``` With `ProjectLatest`, this becomes: ```csharp // Better: project locally including pending events session.Events.StartStream(id, new ReportCreated("Q1")); var report = await session.Events.ProjectLatest(id, ct); // SaveChangesAsync happens later (e.g., Wolverine AutoApplyTransactions) return report; ``` ## API ```csharp // On IDocumentSession.Events (IEventOperations) ValueTask ProjectLatest(Guid id, CancellationToken cancellation = default); ValueTask ProjectLatest(string key, CancellationToken cancellation = default); ``` ## Behavior 1. Fetches the current committed aggregate state from the database via `FetchLatest()` 2. Finds any pending (uncommitted) events for that stream in the current session 3. Applies the pending events on top using the aggregate's Apply/Create methods 4. Returns the result ### When No Pending Events Exist If there are no uncommitted events for the given stream in the session, `ProjectLatest` behaves identically to `FetchLatest` — it returns the current committed state. ## Example ```cs [Fact] public async Task includes_pending_events_from_start_stream() { var streamId = Guid.NewGuid(); await using var session = theStore.LightweightSession(); // Append events without committing session.Events.StartStream(streamId, new ReportCreated("Q1 Report"), new SectionAdded("Revenue"), new SectionAdded("Costs") ); // ProjectLatest includes the pending events above var report = await session.Events.ProjectLatest(streamId, TestContext.Current.CancellationToken); report.ShouldNotBeNull(); report.Title.ShouldBe("Q1 Report"); report.SectionCount.ShouldBe(2); // SaveChangesAsync can happen later await session.SaveChangesAsync(TestContext.Current.CancellationToken); } ``` snippet source | anchor ## Merging Committed and Pending Events `ProjectLatest` also works when the stream has previously committed events and new events are appended in the current session: ```cs [Fact] public async Task includes_pending_events_after_committed_events() { var streamId = Guid.NewGuid(); // First, commit some events await using (var session = theStore.LightweightSession()) { session.Events.StartStream(streamId, new ReportCreated("Q1 Report"), new SectionAdded("Revenue") ); await session.SaveChangesAsync(TestContext.Current.CancellationToken); } // In a new session, append more events without committing await using (var session = theStore.LightweightSession()) { session.Events.Append(streamId, new SectionAdded("Costs"), new SectionAdded("Outlook"), new ReportPublished() ); // ProjectLatest merges the committed state with pending events var report = await session.Events.ProjectLatest(streamId, TestContext.Current.CancellationToken); report.ShouldNotBeNull(); report.Title.ShouldBe("Q1 Report"); report.SectionCount.ShouldBe(3); // 1 committed + 2 pending report.IsPublished.ShouldBeTrue(); // from pending ReportPublished } } ``` snippet source | anchor ## String-Keyed Streams `ProjectLatest` supports string-keyed streams as well: ```csharp // For stores configured with StreamIdentity.AsString session.Events.StartStream("report-123", new ReportCreated("Annual Report"), new SectionAdded("Overview") ); var report = await session.Events.ProjectLatest("report-123"); ``` ## Limitations * **Read-only sessions**: `ProjectLatest` is only available on `IDocumentSession.Events` (not `IQuerySession.Events`) because it needs access to the session's pending work tracker. --- --- url: /documents/querying.md --- # Querying Documents Polecat provides several ways to query documents from SQL Server. ## Loading by ID The simplest way to retrieve a document: ```cs var user = await session.LoadAsync(userId); ``` See [Loading Documents by Id](/documents/querying/byid) for more details. ## LINQ Queries Full LINQ support for complex queries: ```cs var users = await session.Query() .Where(x => x.LastName == "Smith") .OrderBy(x => x.FirstName) .ToListAsync(); ``` See [Querying with LINQ](/documents/querying/linq/) for more details. ## Raw JSON Queries Load documents as raw JSON strings: ```cs var json = await session.LoadJsonAsync(userId); var jsonArray = await session.Query().ToJsonArrayAsync(); ``` See [Querying for Raw JSON](/documents/querying/query-json) for more details. ## Batched Queries Execute multiple queries in a single database round-trip: ```cs var batch = session.CreateBatchQuery(); var userTask = batch.Load(userId); var ordersTask = batch.Query().Where(x => x.Status == "Active").ToList(); await batch.Execute(); var user = await userTask; var orders = await ordersTask; ``` See [Batched Queries](/documents/querying/batched-queries) for more details. ## SQL Preview Preview the generated SQL for any LINQ query: ```cs var sql = session.Query() .Where(x => x.LastName == "Smith") .ToSql(); ``` --- --- url: /documents/querying/linq.md --- # Querying Documents with LINQ Polecat provides a custom LINQ provider that translates .NET LINQ queries into SQL Server queries against the JSON document data. ## Basic Queries ```cs // Simple filter var smiths = await session.Query() .Where(x => x.LastName == "Smith") .ToListAsync(); // With ordering var sorted = await session.Query() .OrderBy(x => x.LastName) .ThenBy(x => x.FirstName) .ToListAsync(); // First or default var first = await session.Query() .FirstOrDefaultAsync(x => x.Email == "alice@example.com"); ``` ## Aggregates ```cs var count = await session.Query().CountAsync(); var any = await session.Query().AnyAsync(x => x.Internal); ``` ## Paging ```cs var page = await session.Query() .OrderBy(x => x.LastName) .Skip(20) .Take(10) .ToListAsync(); ``` Or use the built-in paging support: ```cs var pagedList = await session.Query() .OrderBy(x => x.LastName) .ToPagedListAsync(pageNumber: 2, pageSize: 10); ``` See [Paging](/documents/querying/linq/paging) for more details. ## How It Works LINQ queries are translated to SQL using `JSON_VALUE()` to extract properties from the JSON document: ```sql SELECT data FROM pc_doc_user WHERE JSON_VALUE(data, '$.lastName') = @p0 ORDER BY JSON_VALUE(data, '$.firstName') ``` The LINQ provider supports: * Equality and comparison operators * String operations (Contains, StartsWith, EndsWith) * Boolean logic (And, Or, Not) * Null checks * Collection operations (Any, All, Contains) * Arithmetic operations * Nested property access See [Supported LINQ Operators](/documents/querying/linq/operators) for a complete list. --- --- url: /events/querying.md --- # Querying Events Polecat provides several ways to read events and aggregate state from streams. ## FetchStreamAsync Load all events for a stream: ```cs var events = await session.Events.FetchStreamAsync(streamId); foreach (var @event in events) { Console.WriteLine($"[{@event.Version}] {@event.EventTypeName}: {@event.Data}"); } ``` Events are returned in version order. Archived streams are automatically excluded. ## Stream Fetches as Query Plans `FetchStreamStatePlan` and `FetchStreamPlan` wrap the two raw stream fetches as reusable query plans. Both implement `IQueryPlan` **and** `IBatchQueryPlan`, so the same plan works standalone or inside a batched query, and both offer `Guid streamId` / `string streamKey` constructor overloads: ```cs // Standalone var state = await session.QueryByPlanAsync(new FetchStreamStatePlan(streamId)); var events = await session.QueryByPlanAsync(new FetchStreamPlan(streamId, version: 5)); // Batched — one round trip var batch = session.CreateBatchQuery(); var stateFetcher = batch.QueryByPlan(new FetchStreamStatePlan(streamId)); var eventsFetcher = batch.QueryByPlan(new FetchStreamPlan(streamId)); await batch.Execute(); ``` `FetchStreamStatePlan` yields `null` when the stream does not exist; `FetchStreamPlan` yields an empty list and carries `FetchStream`'s optional `version` / `timestamp` / `fromVersion` arguments. The underlying batched fetchers are also available directly as `batch.Events.FetchStreamState(...)` and `batch.Events.FetchStream(...)` — see [Batched Queries](/documents/querying/batched-queries#batched-event-store-fetches). To return either straight from an ASP.NET Core endpoint, see the `StreamEventState` and `StreamEvents` result types in [ASP.NET Core Integration](/documents/aspnetcore#streaming-event-stream-metadata-and-events). ## AggregateStreamAsync Replay events to build the current aggregate state: ```cs var party = await session.Events.AggregateStreamAsync(streamId); ``` Polecat replays all events in the stream through the aggregate's `Apply`/`Create` methods to build the current state. ### With Version Cap Replay only up to a specific version: ```cs var partyAtV3 = await session.Events.AggregateStreamAsync(streamId, version: 3); ``` ### With Timestamp Cap Replay only events before a specific timestamp: ```cs var partyAtTime = await session.Events.AggregateStreamAsync(streamId, timestamp: DateTimeOffset.Parse("2024-01-15")); ``` ## FetchForWriting Load an aggregate with its current version for optimistic concurrency: ```cs var stream = await session.Events.FetchForWriting(streamId); Console.WriteLine(stream.Aggregate.Name); // Current state Console.WriteLine(stream.CurrentVersion); // Current version stream.AppendOne(new MembersDeparted(...)); await session.SaveChangesAsync(); ``` ## FetchForExclusiveWriting Load with a pessimistic lock (SQL Server `UPDLOCK HOLDLOCK`): ```cs var stream = await session.Events.FetchForExclusiveWriting(streamId); // Row is locked until transaction completes ``` ## IEvent Interface Each event returned from `FetchStreamAsync` implements `IEvent`: | Property | Type | Description | | :--- | :--- | :--- | | `Id` | `Guid` | Unique event ID | | `Sequence` | `long` | Global sequence number | | `Version` | `int` | Position within the stream | | `Data` | `object` | Deserialized event body | | `EventTypeName` | `string` | Event type name (snake\_case) | | `Timestamp` | `DateTimeOffset` | When recorded | | `StreamId` / `StreamKey` | `Guid` / `string` | Stream identifier | | `TenantId` | `string` | Tenant identifier | | `CorrelationId` | `string?` | Correlation ID | | `CausationId` | `string?` | Causation ID | | `Headers` | `Dictionary` | Custom headers | ## Querying Directly Against Event Data ### QueryRawEventDataOnly You can issue LINQ queries against a specific event type's data. This searches the entire `pc_events` table filtered by event type, so it is primarily intended for diagnostics and troubleshooting: ```cs // Query all MembersJoined events var joinedEvents = await session.Events.QueryRawEventDataOnly() .ToListAsync(); // Count events of a specific type var count = await session.Events.QueryRawEventDataOnly() .CountAsync(); // Filter by event data properties var events = await session.Events.QueryRawEventDataOnly() .Where(x => x.Day == 1) .ToListAsync(); // Check if any events exist var any = await session.Events.QueryRawEventDataOnly() .AnyAsync(); ``` ### QueryAllRawEvents Query across all event types using the `IEvent` metadata properties: ```cs // Query all events for a specific stream var events = await session.Events.QueryAllRawEvents() .Where(x => x.StreamId == streamId) .OrderBy(x => x.Sequence) .ToListAsync(); // Filter by event metadata var recentEvents = await session.Events.QueryAllRawEvents() .Where(x => x.Timestamp > cutoffDate) .ToListAsync(); // Filter by event type name var joinedTypeName = store.Options.EventGraph .EventMappingFor(typeof(MembersJoined)).EventTypeName; var events = await session.Events.QueryAllRawEvents() .Where(x => x.EventTypeName == joinedTypeName) .ToListAsync(); // Count events matching a condition var count = await session.Events.QueryAllRawEvents() .CountAsync(x => x.Version == 1); // Select specific metadata columns var streamIds = await session.Events.QueryAllRawEvents() .Select(x => x.StreamId) .Distinct() .ToListAsync(); ``` The queryable `IEvent` properties available for filtering and projection are: | Property | SQL Column | Description | | :--- | :--- | :--- | | `Id` | `id` | Unique event ID | | `Sequence` | `seq_id` | Global sequence number | | `StreamId` | `stream_id` | Stream identifier (Guid) | | `Version` | `version` | Position within the stream | | `Timestamp` | `timestamp` | When recorded | | `EventTypeName` | `type` | Event type name | | `DotNetTypeName` | `dotnet_type` | .NET type name | | `IsArchived` | `is_archived` | Archive flag | | `TenantId` | `tenant_id` | Tenant identifier | | `CorrelationId` | `correlation_id` | Correlation ID | | `CausationId` | `causation_id` | Causation ID | ::: warning These queries search the entire event table and should be used judiciously. For routine application queries, prefer projected views or tag-based queries. ::: ### AggregateToAsync `AggregateToAsync()` is a LINQ terminal that folds **every** event matched by an event query into a single live aggregate of type `T`, regardless of which stream each event came from. It uses the same conventional `Create`/`Apply` aggregation that `AggregateStreamAsync` uses, but over an arbitrary event query instead of one stream: ```cs var questParty = await session.Events .QueryAllRawEvents() // You could of course chain all the Linq // Where()/OrderBy()/Take()/Skip() operators // you need here .AggregateToAsync(token: TestContext.Current.CancellationToken); ``` snippet source | anchor You can also seed the fold with initial state: ```cs var initial = new QuestParty { Members = ["Lan"] }; var questParty = await session.Events.QueryAllRawEvents() .Where(x => x.StreamId == streamId) .AggregateToAsync(initial); ``` The aggregate's identity is stamped from the last queried event's stream (`StreamId` or `StreamKey` depending on the store's `StreamIdentity`), and `null` is returned when the query matches no events. ### AggregateToManyAsync `AggregateToManyAsync()` is a LINQ terminal that runs the events matched by an event query through the **multi-stream projection** registered for `T` and returns the aggregate it produces for each resulting identity. It drives the projection's real slicer/grouper, `EnrichEventsAsync`, and per-slice build against the live query session — the same building blocks the projection step-through and the async daemon use — so custom groupers that read present-day reference data from the session work for free. Nothing is persisted. Given a multi-stream projection: ```cs public record MoneyDeposited(Guid AccountId, int Amount); public record AccountFrozen(Guid AccountId); public class Balance { public Guid Id { get; set; } public int Amount { get; set; } } public partial class BalanceProjection : MultiStreamProjection { public BalanceProjection() { Identity(e => e.AccountId); Identity(e => e.AccountId); } public void Apply(MoneyDeposited e, Balance b) => b.Amount += e.Amount; public bool ShouldDelete(AccountFrozen e) => true; } ``` snippet source | anchor query any slice of the event store and fan it out to one aggregate per identity: ```cs var aggregates = await session.Events.QueryAllRawEvents() .Where(e => e.StreamId == stream1 || e.StreamId == stream2) .AggregateToManyAsync(TestContext.Current.CancellationToken); ``` snippet source | anchor Each returned aggregate has its identity stamped from the projection's slice id. Aggregates whose event slice resolves to a delete (`ShouldDelete`) are omitted, an empty query returns an empty list, and calling it for an aggregate type with no registered projection throws `ArgumentException`. Contrast `AggregateToAsync`, which folds every queried event into a single aggregate; `AggregateToManyAsync` fans them out through the projection's slicer to one aggregate per identity. ## QueryForNonStaleData Wait for async projections to catch up before querying: ```cs var orders = await session.Query() .QueryForNonStaleData() .Where(x => x.Status == "Active") .ToListAsync(); ``` With a custom timeout: ```cs var orders = await session.Query() .QueryForNonStaleData(TimeSpan.FromSeconds(10)) .Where(x => x.Status == "Active") .ToListAsync(); ``` --- --- url: /documents/querying/query-json.md --- # Querying for Raw JSON Polecat can return documents as raw JSON strings without deserializing them into .NET objects. ## LoadJsonAsync Load a single document as JSON by ID: ```cs string? json = await session.LoadJsonAsync(userId); ``` Returns `null` if the document doesn't exist. ## ToJsonArrayAsync Convert a LINQ query result to a JSON array string: ```cs string jsonArray = await session.Query() .Where(x => x.Active) .ToJsonArrayAsync(); // Returns: [{"id":"...","firstName":"Alice",...},{"id":"...","firstName":"Bob",...}] ``` ### Projecting with Select() A **simple** `Select()` projection — an anonymous type or DTO composed only of (optionally nested) document member accesses — is translated to a server-side JSON object and streamed without hydrating the documents (SQL Server 2025 native JSON, via `JSON_OBJECT`): ```cs string jsonArray = await session.Query() .Where(x => x.Active) .Select(x => new { x.Name, City = x.Address.City }) .ToJsonArrayAsync(); // Returns: [{"name":"Alice","city":"Austin"}, ...] ``` Emitted keys honor the serializer naming policy and `[JsonPropertyName]`, exactly as System.Text.Json would serialize the projected shape. Safe/widening conversions (`int`→`long`, `T`→`object`, enum↔underlying, non-null→nullable) are transparent. ::: warning A projection that is **not** a simple member projection — a scalar select (`Select(x => x.Name)`), a method call (`x.Name.ToUpper()`), arithmetic (`x.Age * 2`), a conditional, etc. — **cannot** be streamed as raw JSON and throws `BadLinqExpressionException` rather than silently returning the raw documents. Materialize those with `ToListAsync()` (which runs the projection client-side) instead of a JSON-streaming call. The server-side JSON-object translation requires a native-JSON (SQL Server 2025) store; on other stores a `Select()` projection likewise throws when streaming. ::: ## Use Cases Raw JSON queries are useful when: * Serving JSON directly to HTTP responses without deserialization/serialization overhead * Building APIs where the response format matches the stored document * Streaming large result sets to clients * Debugging or inspecting stored data --- --- url: /documents/querying/linq/child-collections.md --- # Querying within Child Collections Polecat supports querying documents based on conditions within nested collections stored in the JSON data. ## Any Check if a child collection has any elements matching a condition: ```cs public class Order { public Guid Id { get; set; } public List Items { get; set; } = new(); } public class OrderItem { public string ProductName { get; set; } = ""; public int Quantity { get; set; } } // Find orders that have any item with quantity > 10 var orders = await session.Query() .Where(x => x.Items.Any(i => i.Quantity > 10)) .ToListAsync(); ``` ## Contains Check if a simple collection contains a value: ```cs public class User { public Guid Id { get; set; } public List Tags { get; set; } = new(); } var admins = await session.Query() .Where(x => x.Tags.Contains("admin")) .ToListAsync(); ``` ## Nested Property Access Query nested objects directly: ```cs public class Order { public Guid Id { get; set; } public Address ShippingAddress { get; set; } = new(); } var nyOrders = await session.Query() .Where(x => x.ShippingAddress.City == "New York") .ToListAsync(); ``` --- --- url: /configuration/retries.md --- # Resiliency Policies Polecat includes built-in resiliency for transient SQL Server failures using [Polly](https://github.com/App-vNext/Polly). ## Default Behavior By default, Polecat retries on transient SQL Server errors (deadlocks, timeouts, connection failures) with an exponential backoff strategy. ## Custom Polly Configuration ### Replace the Default Pipeline ```cs opts.ConfigurePolly(builder => { builder.AddRetry(new RetryStrategyOptions { MaxRetryAttempts = 5, BackoffType = DelayBackoffType.Exponential, Delay = TimeSpan.FromMilliseconds(200) }); }); ``` ### Extend the Default Pipeline ```cs opts.ExtendPolly(builder => { builder.AddTimeout(TimeSpan.FromSeconds(30)); }); ``` ## Circuit Breaker You can add a circuit breaker to prevent cascading failures: ```cs opts.ExtendPolly(builder => { builder.AddCircuitBreaker(new CircuitBreakerStrategyOptions { FailureRatio = 0.5, MinimumThroughput = 10, SamplingDuration = TimeSpan.FromSeconds(30), BreakDuration = TimeSpan.FromSeconds(15) }); }); ``` --- --- url: /schema/migrations.md --- # Schema Migrations Polecat uses Weasel.SqlServer for diff-based schema migrations. ## Auto-Create Behavior By default (`AutoCreate.CreateOrUpdate`), Polecat automatically creates and updates database schema on application startup: ```cs opts.AutoCreateSchemaObjects = AutoCreate.CreateOrUpdate; ``` ### What Auto-Create Does * Creates document tables on first use * Adds new columns when document configuration changes (e.g., enabling soft deletes) * Creates event store tables on startup * Creates the HiLo table on first use * **Never drops columns or tables** ### What Auto-Create Does NOT Do * Drop existing columns * Rename columns * Modify column types * Delete data ## Disabling Auto-Create For production environments: ```cs opts.AutoCreateSchemaObjects = AutoCreate.None; ``` When auto-create is disabled, you're responsible for ensuring the database schema matches your configuration. Use the [schema export](/schema/exporting) feature to generate DDL scripts. ## Migration Flow On startup, Polecat's migration flow: 1. Compare desired schema (from configuration) against actual database schema 2. Generate DDL for differences (new tables, new columns) 3. Execute DDL within a transaction 4. Run initial data seeding if configured ## Weasel.SqlServer All schema management is delegated to Weasel.SqlServer, which provides: * **Diff-based migrations** -- Only applies changes that are needed * **Safe operations** -- Never drops columns or tables * **Idempotent** -- Safe to run multiple times * **Transaction safety** -- Schema changes are transactional ## Command Line Schema Management With `RunJasperFxCommands(args)` on a host that calls `AddPolecat(...)`, both families of schema commands see the Polecat database: ```bash dotnet run -- db-apply ``` | Command | Does | |---|---| | `db-apply` | Apply every outstanding migration | | `db-assert` | Fail if the database is out of step with the configuration — useful as a deployment gate | | `db-dump` | Write the DDL to a file instead of executing it | | `resources setup` | Provision every registered resource, Polecat's schema among them | | `resources list` / `resources check` | List or verify registered resources | The `db-*` commands operate on Weasel databases specifically; `resources *` covers every JasperFx resource in the application, so on a Wolverine host it also provisions envelope storage and broker topology alongside the Polecat schema. Both work against a single database and against every tenant database under separate-database or master-table tenancy, and ancillary stores registered with `AddPolecatStore()` are included. --- --- url: /documents/querying/linq/strings.md --- # Searching on String Fields Polecat's LINQ provider translates .NET string methods into SQL Server JSON path queries. ## Contains ```cs var results = await session.Query() .Where(x => x.LastName.Contains("son")) .ToListAsync(); ``` ## StartsWith ```cs var results = await session.Query() .Where(x => x.LastName.StartsWith("Sm")) .ToListAsync(); ``` ## EndsWith ```cs var results = await session.Query() .Where(x => x.Email.EndsWith("@example.com")) .ToListAsync(); ``` ## Equals ```cs var results = await session.Query() .Where(x => x.Email.Equals("admin@example.com")) .ToListAsync(); ``` ## IsNullOrEmpty ```cs var results = await session.Query() .Where(x => !string.IsNullOrEmpty(x.Email)) .ToListAsync(); ``` ## Case Sensitivity SQL Server string comparisons follow the database collation by default. For case-insensitive searches, configure your database collation accordingly. --- --- url: /events/projections/side-effects.md --- # Side Effects ::: tip By default, side effects only fire during *continuous* asynchronous projection execution. They do not run during projection rebuilds, and they do not run during the *warm-up phase* of a new projection version that opts into the [blue/green side-effect gate](async-daemon.md#blue-green-deployments-with-projection-versioning). Inline projections can opt in via [`EnableSideEffectsOnInlineProjections`](#side-effects-in-inline-projections). ::: *Sometimes*, it can be valuable to emit new events during the processing of a projection when you first know the new state of the projected aggregate documents. Or maybe what you want to do is send a message reflecting the new state of an updated projection. Here are a few scenarios that might lead you here: * There's some kind of business logic that can be processed against an aggregate to "decide" what the system can do next * You need to send updates about the aggregated projection state to clients via web sockets * You need to replicate the Polecat projection data in a completely different database * There are business processes that can be kicked off for updates to the aggregated state To do any of this, you can override the `RaiseSideEffects()` method in any aggregated projection that uses one of the following base classes: 1. `SingleStreamProjection` 2. `MultiStreamProjection` Here's an example of that method overridden in a projection: ```cs public class TripProjection : SingleStreamProjection { public static Trip Create(IEvent @event) => new() { Id = @event.StreamId, Started = @event.Timestamp, Description = @event.Data.Description }; public void Apply(TripEnded ended, Trip trip, IEvent @event) { trip.Ended = @event.Timestamp; } // Other Apply / ShouldDelete methods... public override ValueTask RaiseSideEffects(IDocumentSession session, IEventSlice slice) { // Access to the current state as of the projection // event page being processed *right* now var currentTrip = slice.Snapshot; if (currentTrip.TotalMiles > 1000) { // Append a new event to this stream slice.AppendEvent(new PassedThousandMiles()); // Append a new event to a different event stream by // first specifying a different stream id slice.AppendEvent(currentTrip.InsuranceCompanyId, new IncrementThousandMileTrips()); // "Publish" outgoing messages when the event page is successfully committed slice.PublishMessage(new SendCongratulationsOnLongTrip(currentTrip.Id)); // And yep, you can make additional changes to Polecat session.Store(new CompletelyDifferentDocument { Name = "New Trip Segment", OriginalTripId = currentTrip.Id }); } return new ValueTask(); } } ``` A few important facts about this functionality: * The `RaiseSideEffects()` method is only called during *continuous* asynchronous projection execution. It is **not** called during projection rebuilds. For `Inline` projections, it is opt-in via [`EnableSideEffectsOnInlineProjections`](#side-effects-in-inline-projections). * Events emitted during the side effect method are *not* immediately applied to the current projected document value by Polecat * You *can* alter the aggregate value or replace it yourself in this side effect method to reflect new events, but the onus is on you the user to apply idempotent updates to the aggregate based on these new events in the actual handlers for the new events when those events are handled by the daemon in a later batch ## Routing Published Messages By default, calls to `slice.PublishMessage(...)` are dropped — Polecat ships a no-op `IMessageOutbox` so projections that do not need to emit messages incur zero overhead. To actually deliver published messages, register an `IMessageOutbox` implementation on the event store options: ```cs builder.Services.AddPolecat(opts => { opts.Connection(builder.Configuration.GetConnectionString("polecat")); // Replace the default no-op outbox with one that hands the messages // off to your messaging infrastructure opts.Events.MessageOutbox = new MyCustomMessageOutbox(); }); ``` Each implementation of `IMessageOutbox` vends a fresh `IMessageBatch` for every projection update. The batch is enlisted as a post-commit listener on the projection update — `BeforeCommitAsync` fires inside the projection's SQL transaction (right before `COMMIT`) and `AfterCommitAsync` fires once the projection's database changes are durably committed. This lets implementations choose between "at-least-once" patterns (persist the outgoing messages to a database table inside the same transaction) and "best-effort" patterns (flush to an external broker only after the projection write succeeds). A first-class [Wolverine](https://wolverinefx.net) integration is on the roadmap that will plug Wolverine's outbox in via `IntegrateWithWolverine()`, mirroring the existing Marten/Wolverine bridge. ### When Messages Do and Do Not Fire Published messages (and every other `RaiseSideEffects()` side effect) fire only when the daemon is processing *new* events in **continuous** execution. They are suppressed: * during a projection **rebuild** (`RebuildProjectionAsync`), and * during the **blue/green warm-up phase** of a new projection version that opts into `GateSideEffectsBehindPriorVersion` — while the new version replays the history the prior version already processed, side effects stay suppressed, and only start firing once the new version catches up to the prior version's progression mark and switches to continuous execution. See [Blue/Green Deployments with Projection Versioning](async-daemon.md#blue-green-deployments-with-projection-versioning) for the versioning mechanics. ## Side Effects in Inline Projections By default, Polecat only processes projection side effects during continuous asynchronous processing. To process them when running projections under the `Inline` lifecycle as well, flip the opt-in setting on the event store options: ```cs builder.Services.AddPolecat(opts => { opts.Connection(builder.Configuration.GetConnectionString("polecat")); // Run RaiseSideEffects() for inline projections too opts.EventGraph.EnableSideEffectsOnInlineProjections = true; }); ``` When the flag is on, `slice.PublishMessage(...)` from an inline projection's `RaiseSideEffects()` method enqueues the message into the configured `IMessageOutbox`'s batch on the active document session. `BeforeCommitAsync` fires inside the session's SQL transaction (right before `COMMIT`), and `AfterCommitAsync` fires once the session's database changes are durably committed. ::: warning Inline `RaiseSideEffects()` may **not** call `slice.AppendEvent(...)` — appending events back into the same session that's currently committing them is not supported. Doing so throws `InvalidOperationException`. Side effects from inline projections are limited to published messages. ::: --- --- url: /events/projections/single-stream-projections.md --- # Single Stream Projections Single stream projections build one aggregate document per event stream. This is the most common projection type. ## Defining a Projection Use conventional `Apply` methods: ```cs public class OrderSummary { public Guid Id { get; set; } public string Status { get; set; } = ""; public decimal TotalAmount { get; set; } public int ItemCount { get; set; } public DateTimeOffset? ShippedDate { get; set; } public static OrderSummary Create(OrderCreated e) => new() { Status = "Created", TotalAmount = e.Amount }; public void Apply(OrderItemAdded e) { TotalAmount += e.Price; ItemCount++; } public void Apply(OrderShipped e) { Status = "Shipped"; ShippedDate = e.ShippedAt; } public bool ShouldDelete(OrderCancelled e) => true; } ``` ## Registration ### Inline ```cs opts.Projections.Add>(ProjectionLifecycle.Inline); ``` The projection runs in the same transaction as event appending. The aggregate is stored in `pc_doc_ordersummary`. ### Async ```cs opts.Projections.Add>(ProjectionLifecycle.Async); ``` The async daemon processes events in the background. ## Using IEvent Metadata Access event metadata in your Apply methods: ```cs public void Apply(IEvent @event) { Status = "Created"; TotalAmount = @event.Data.Amount; CreatedAt = @event.Timestamp; CreatedBy = @event.Headers?["user"]?.ToString(); } ``` ## Identifying the Event Parameter Notice that the examples above name the event parameter `e` in some methods and `@event` in others — both work. Polecat identifies the event argument of a conventional `Create` / `Apply` / `ShouldDelete` method (and an [event projection](/events/projections/event-projections)'s `Project` / `Transform` methods) **by type, not by name**, using the same rule for every projection type: 1. A parameter typed `IEvent` is always the event, and `T` is the event type — use this when you need event metadata such as `Timestamp` or `Headers`. 2. Otherwise the single **concrete** parameter that is not an interface (`IQuerySession`, `IDocumentOperations`), not `IEvent`, not `CancellationToken`, and not the aggregate type is the event. So `Apply(OrderShipped e)` and `Apply(OrderShipped shipped)` are equivalent — the parameter name is incidental. A conventional event parameter **name** is only consulted to disambiguate an unusual signature in which more than one parameter could be the event; the recognized names are `@event`, `event`, `e`, and `ev`. ## Live Aggregation Use a single stream projection for on-demand replay without persisting: ```cs var order = await session.Events.AggregateStreamAsync(streamId); ``` This replays all events in the stream through the `Create` and `Apply` methods. ## Custom SingleStreamProjection Class For more control, extend `SingleStreamProjection`: ```cs public class OrderProjection : SingleStreamProjection { public OrderSummary Create(OrderCreated e) => new() { Status = "Created", TotalAmount = e.Amount }; public void Apply(OrderItemAdded e, OrderSummary current) { current.TotalAmount += e.Price; current.ItemCount++; } } // Register opts.Projections.Add(ProjectionLifecycle.Inline); ``` --- --- url: /events/snapshots.md --- # Snapshot Projections `Projections.Snapshot()` is a convenient shortcut for registering a self-aggregating projection: it builds and registers a [`SingleStreamProjection`](/events/projections/single-stream-projections) internally, with the identity type resolved automatically from the document's `Id` property. ::: tip How it works under the hood `Snapshot()` is purely a registration shortcut — it does **not** introduce a separate "snapshot" storage path. Internally it builds a `SingleStreamProjection` against the appropriate identity type and registers it through the same projection pipeline as any other projection. The aggregate is persisted to the document table for type `T` (the standard `pc_doc_{type}` table), not to a special snapshot column on `pc_streams`. This mirrors Marten's `Snapshot()` API exactly so the two ecosystems stay aligned. ::: ## Registration The aggregate type **must be self-aggregating** — i.e. it must have its own static `Create` and instance `Apply` methods (or implement the appropriate aggregation conventions). ```cs public class QuestParty { public Guid Id { get; set; } public string Name { get; set; } = ""; public List Members { get; set; } = new(); public static QuestParty Create(QuestStarted e) => new() { Name = e.Name }; public void Apply(MembersJoined e) => Members.AddRange(e.Members); } var store = DocumentStore.For(opts => { opts.Connection("..."); // Inline — snapshot updated in the same transaction as the events opts.Projections.Snapshot(SnapshotLifecycle.Inline); }); ``` If you need to subclass `SingleStreamProjection` to override behavior, register it directly via `opts.Projections.Add(ProjectionLifecycle.Inline)` instead. ## Lifecycles ```cs // Updated in the same transaction as the appended events opts.Projections.Snapshot(SnapshotLifecycle.Inline); // Updated asynchronously by the async projection daemon opts.Projections.Snapshot(SnapshotLifecycle.Async); ``` `SnapshotLifecycle.Live` is intentionally not supported — for live aggregation, use [`AggregateStreamAsync()`](/events/projections/live-aggregates) directly without registering a projection. ## Reading the Aggregate Because `Snapshot()` registers a regular `SingleStreamProjection`, you read the result the same way you would for any other projected document: ```cs await using var session = store.QuerySession(); // Loads from the projected document table (pc_doc_questparty) var party = await session.LoadAsync(streamId); // Or fetch the latest version from the event store var latest = await session.Events.FetchLatest(streamId); ``` For an aggregate registered with an `Inline` lifecycle, `FetchLatest()` reads the projected document rather than re-aggregating the stream — the same thing `LoadAsync()` does, and the same route Marten takes. It therefore returns `null` for a stream that exists but belongs to some other aggregate sharing the key space, because the inline projection never wrote a document for a stream it does not own. That makes `FetchLatest(id) is null` a reliable "does this aggregate exist?" probe, which is what code branching between `StartStream` and `Append` depends on. This matters most for an aggregate whose only handler is a catch-all `Evolve(IEvent)`: a catch-all accepts every event type at the method level, so live aggregation over a foreign stream would otherwise hand back a default-constructed instance — and default property values read as real state, not as absence. Aggregates on the `Live` and `Async` lifecycles still aggregate the stream, since there is no committed document to read. ## Snapshot in a Composite Projection Composite projections support the same shortcut via `composite.Snapshot()`: ```cs opts.Projections.CompositeProjectionFor("UserLifecycle", composite => { composite.Snapshot(); // stage 1 (default) composite.Snapshot(2); // stage 2 }); ``` Inside a composite projection, snapshots always run as `Async` — composite projections are themselves async-only. ## Choosing Between Snapshot and Add * Use **`Snapshot()`** when `T` is a self-aggregating aggregate type with conventional `Create`/`Apply` methods. This is the simplest registration. * Use **`Add(...)`** when you have a dedicated `SingleStreamProjection` subclass that overrides projection behavior, or when registering any other projection type (multi-stream, event projections, flat tables, etc). --- --- url: /documents/storing.md --- # Storing Documents Polecat provides several methods for persisting documents to SQL Server. ## Store (Upsert) The `Store()` method performs an upsert -- inserting new documents or updating existing ones: ```cs await using var session = store.LightweightSession(); var user = new User { FirstName = "Alice", LastName = "Smith" }; session.Store(user); // user.Id is now assigned (for Guid IDs) await session.SaveChangesAsync(); ``` Store multiple documents at once: ```cs session.Store(user1, user2, user3); ``` ## Insert `Insert()` will throw if a document with the same ID already exists: ```cs session.Insert(new User { FirstName = "Bob" }); await session.SaveChangesAsync(); ``` ## Update `Update()` will throw if the document does not already exist: ```cs var user = await session.LoadAsync(id); user.LastName = "Updated"; session.Update(user); await session.SaveChangesAsync(); ``` ## SaveChangesAsync All pending operations are flushed to the database in a single transaction: ```cs session.Store(user1); session.Insert(order1); session.Delete(invoiceId); // All three operations execute in one transaction await session.SaveChangesAsync(); ``` ::: tip `SaveChangesAsync()` processes event stream operations first, then document operations. This ensures inline projections can create documents from events in the same transaction. ::: ## ID Assignment ### Guid IDs Automatically assigned on `Store()` or `Insert()` if the ID is `Guid.Empty`: ```cs var user = new User(); // Id is Guid.Empty session.Store(user); // Id is now assigned ``` ### Numeric IDs (int/long) Automatically assigned via [HiLo sequences](/documents/identity#hilo-sequences): ```cs var invoice = new Invoice(); // Id is 0 session.Store(invoice); // Id is now assigned from HiLo ``` ### String IDs Must be assigned by the application before storing: ```cs var doc = new MyDoc { Id = "custom-id-123" }; session.Store(doc); ``` ### Strongly Typed IDs Wrapper types are automatically handled: ```cs public record struct OrderId(Guid Value); var order = new Order(); // Id.Value is Guid.Empty session.Store(order); // Id.Value is now assigned ``` --- --- url: /documents/querying/linq/operators.md --- # Supported LINQ Operators Polecat's LINQ provider supports the following operators and methods. ## Comparison Operators ```cs .Where(x => x.Age == 25) .Where(x => x.Age != 25) .Where(x => x.Age > 18) .Where(x => x.Age >= 18) .Where(x => x.Age < 65) .Where(x => x.Age <= 65) ``` ## Boolean Logic ```cs .Where(x => x.Active && x.Age > 18) .Where(x => x.Active || x.Admin) .Where(x => !x.Deleted) ``` ## String Operations ```cs .Where(x => x.Name.Contains("john")) .Where(x => x.Name.StartsWith("J")) .Where(x => x.Name.EndsWith("son")) .Where(x => x.Name == "John") .Where(x => string.IsNullOrEmpty(x.Name)) ``` See [Searching on String Fields](/documents/querying/linq/strings) for more details. ## Null Checks ```cs .Where(x => x.Email != null) .Where(x => x.Email == null) ``` ## Arithmetic ```cs .Where(x => x.Price * x.Quantity > 100) .Where(x => x.Total - x.Discount < 50) ``` ## Collection Operations ```cs // Check if a value is in a list var ids = new[] { id1, id2, id3 }; .Where(x => ids.Contains(x.Id)) // Check child collections .Where(x => x.Tags.Any()) .Where(x => x.Tags.Contains("priority")) ``` ## Ordering ```cs .OrderBy(x => x.LastName) .OrderByDescending(x => x.CreatedDate) .ThenBy(x => x.FirstName) .ThenByDescending(x => x.Age) ``` ## Projection ```cs .Select(x => new { x.FirstName, x.LastName }) .Select(x => x.Email) ``` ## Aggregation ```cs .CountAsync() .LongCountAsync() .AnyAsync() .AnyAsync(x => x.Active) .FirstAsync() .FirstOrDefaultAsync() .SingleAsync() .SingleOrDefaultAsync() .MinAsync(x => x.Age) .MaxAsync(x => x.Age) .SumAsync(x => x.Amount) .AverageAsync(x => x.Score) ``` ## GroupBy Polecat supports the `GroupBy()` LINQ operator for grouping documents by one or more keys and computing aggregate values. GroupBy translates to SQL `GROUP BY` with aggregate functions like `COUNT`, `SUM`, `MIN`, `MAX`, and `AVG`. ### Simple Key with Aggregates ```cs [Fact] public async Task group_by_simple_key_with_count() { await StoreSeedDataAsync(); await using var query = theStore.QuerySession(); var results = await query.Query() .GroupBy(x => x.Color) .Select(g => new { Color = g.Key, Count = g.Count() }) .ToListAsync(TestContext.Current.CancellationToken); results.Count.ShouldBe(3); results.Single(x => x.Color == TargetColor.Blue).Count.ShouldBe(2); results.Single(x => x.Color == TargetColor.Green).Count.ShouldBe(3); results.Single(x => x.Color == TargetColor.Red).Count.ShouldBe(1); } ``` snippet source | anchor ### Composite Key You can group by multiple properties using an anonymous type: ```cs var results = await session.Query() .GroupBy(x => new { x.Color, x.Name }) .Select(g => new { Color = g.Key.Color, Text = g.Key.Name, Count = g.Count() }) .ToListAsync(); ``` ### Where Before GroupBy Filter documents before grouping with a standard `Where()` clause: ```cs var results = await session.Query() .Where(x => x.Age > 20) .GroupBy(x => x.Color) .Select(g => new { Color = g.Key, Count = g.Count() }) .ToListAsync(); ``` ### HAVING (Where After GroupBy) Filter groups with a `Where()` clause after `GroupBy()` -- this translates to SQL `HAVING`: ```cs var results = await session.Query() .GroupBy(x => x.Color) .Where(g => g.Count() > 1) .Select(g => new { Color = g.Key, Count = g.Count() }) .ToListAsync(); ``` ### Supported Aggregates * `g.Count()` / `g.LongCount()` -- `COUNT(*)` * `g.Sum(x => x.Property)` -- `SUM(property)` * `g.Min(x => x.Property)` -- `MIN(property)` * `g.Max(x => x.Property)` -- `MAX(property)` * `g.Average(x => x.Property)` -- `AVG(property)` ## CountBy(), AggregateBy(), and Index() (.NET 9) The `CountBy`, `AggregateBy`, and `Index` operators added in .NET 9 are **not translated to SQL by Polecat**, because .NET only added them to `Enumerable` (there are no `IQueryable` overloads, so a query provider never sees them). Calling them on a Polecat query therefore **runs client-side**: the full result set is pulled into memory first and the operator is applied there with LINQ to Objects -- the same behavior you get with EF Core. For large tables, express a `CountBy` as `GroupBy(...).Select(...)`, which Polecat does translate to a SQL `GROUP BY`: ```cs // Translated to SQL: select count(*), color ... group by color var counts = await session.Query() .GroupBy(x => x.Color) .Select(g => new { Color = g.Key, Count = g.Count() }) .ToListAsync(); ``` There is no efficient SQL equivalent for `AggregateBy`'s arbitrary accumulator function; `Index` corresponds to a window `ROW_NUMBER()` but is likewise only an in-memory operator today. If/when .NET ships `Queryable` overloads for these, Polecat can revisit native translation. ## Paging ```cs .Skip(20) .Take(10) .ToPagedListAsync(pageNumber, pageSize) ``` ## Result Materialization ```cs .ToListAsync() .ToArrayAsync() .ToJsonArrayAsync() ``` --- --- url: /documents/partitioning.md --- # Table Partitioning Polecat can **RANGE-partition a document table** on a member you choose — the SQL Server companion to Marten's `PartitionOn`. The classic use is a time-series retention table partitioned by month, so that old data is reclaimed by retiring a partition instead of issuing a large `DELETE`. Three strategies are available: | Strategy | Who owns the partitions | | -------------------------------------------------------------- | ----------------------------------------------------------------------------- | | `ByRange(...)` / `PartitionByRange(...)` | you declare a fixed boundary list; Polecat rolls additions forward in place | | [`ByRollingRange(...)`](#rolling-time-windows) | Polecat, from a rolling-window policy — provisioning *and* retention | | [`ByExternallyManagedRange(...)`](#externally-managed-range-partitions) | something outside Polecat, which Polecat then never touches | ::: tip This is built on SQL Server partition **functions** and **schemes** rather than the child-table model PostgreSQL/Marten uses, so the migration story differs. Declarative range partitioning requires `Weasel.SqlServer` 9.3.0 or later; rolling time windows require 9.21.0 or later. ::: ## Partitioning by a date member Use `PartitionByRange` in `Schema.For()`, passing the member and the RANGE RIGHT boundary values (`N` boundaries produce `N + 1` partitions): ```csharp var store = DocumentStore.For(opts => { opts.Connection(connectionString); opts.Schema.For() .PartitionByRange(x => x.BucketEnd, new DateTimeOffset(2026, 1, 1, 0, 0, 0, TimeSpan.Zero), new DateTimeOffset(2026, 2, 1, 0, 0, 0, TimeSpan.Zero), new DateTimeOffset(2026, 3, 1, 0, 0, 0, TimeSpan.Zero)); }); ``` This creates a partition function and scheme for `pc_doc_metricssample` and places the table on the scheme. Supported member types are dates (`DateTimeOffset`, `DateTime`, `DateOnly`) and integers (`int`, `long`, `short`), plus `Guid`. ### The promoted partition column Unless you partition directly on the identity, the member's value is promoted into a real column (`bucket_end` for `BucketEnd`) that Polecat writes on every upsert. SQL Server requires the partitioning column to be part of the table's unique (clustered) index, so this column is **added to the primary key** — meaning the document `Id` is unique together with the partition value. For the typical time-series case the partition value is derived from immutable document data, so this is transparent. ## Rolling partitions forward Adding new boundaries over time is an in-place, online operation: Polecat (via Weasel) issues `ALTER PARTITION FUNCTION ... SPLIT RANGE` rather than rebuilding the table. Extend the boundary list and re-activate the schema (for example at application start-up): ```csharp opts.Schema.For() .PartitionByRange(x => x.BucketEnd, /* existing months... */ new DateTimeOffset(2026, 3, 1, 0, 0, 0, TimeSpan.Zero), new DateTimeOffset(2026, 4, 1, 0, 0, 0, TimeSpan.Zero)); // new ``` Schema migration adds the new partition with no data movement. Removing a boundary or changing the column/type is reported as a rebuild rather than performed silently. ## Rolling time windows Declaring every boundary up front only works while the set of partitions is *fixed*. Real time-series storage needs it to **move**: provision next month, retire last year. Rather than hand-writing that DDL on a schedule forever, describe the window and let Polecat own it: ```csharp var store = DocumentStore.For(opts => { opts.Connection(connectionString); // Keep 12 months of history, provision 3 months ahead. Polecat splits in the partitions at the // leading edge and retires the aged ones at the trailing edge — no application-authored DDL. opts.Schema.For() .PartitionOn(x => x.BucketEnd) .ByRollingRange(PartitionPeriod.Month, periodsAhead: 3, periodsBehind: 12); }); ``` `PartitionPeriod` (from `Weasel.Core.Partitioning`) supports `Hour`, `Day`, `Week`, `Month`, and `Year`, and the whole window is computed in UTC. The partition member must be a `DateTime` or `DateTimeOffset` — a rolling window is a function of the clock, so anything else is rejected at *configuration* time with a message that names the member, rather than surfacing as an opaque partition-function error during the first migration. There are also overloads taking a `RollingWindowPolicy` directly, or a pre-built `ManagedRangePartitions`. Pass the *same* manager instance to several document types to roll all of their tables forward in one pass: ```csharp using Weasel.SqlServer.Tables.Partitioning; // ManagedRangePartitions using Weasel.Core.Partitioning; // RollingWindowPolicy, PartitionPeriod var manager = new ManagedRangePartitions( RollingWindowPolicy.Monthly(periodsAhead: 3, periodsBehind: 12), column: "bucket_end", sqlDataType: "datetimeoffset"); opts.Schema.For().PartitionOn(x => x.BucketEnd).ByRollingRange(manager); opts.Schema.For().PartitionOn(x => x.BucketEnd).ByRollingRange(manager); ``` The manager is also where you set `Filegroup` if the partitions should not go to `PRIMARY`, and it takes a `TimeProvider` so the window can be rolled forward deterministically in tests instead of waiting on the calendar. ### How the two halves are driven | | Driven by | Why | | -------------------------------- | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | | **Provision** the leading edge | ordinary schema migration | with a rolling-window manager attached the partition delta is purely additive, so a rolled-forward window is a `SPLIT` | | **Retire** the trailing edge | startup pass + `Advanced.*` | migration never removes data, so the retention half has to be driven separately | The window is a pure function of the policy and the clock, which is what makes this safe: a window that has rolled forward differs from the database by exactly one new boundary at the leading edge and one aged boundary at the trailing edge. A boundary the declaration no longer names is a period that has **aged out** — the normal steady state of a rolling window, not drift — so `CreateDelta` reports `Additive` or `None`, never `Rebuild`. (A column or type change still rebuilds, as it must.) Polecat runs the maintenance pass — roll forward, then retire everything below the retention floor — at startup, alongside the schema changes it already applies: ```csharp builder.Services.AddPolecat(opts => { // ... the ByRollingRange() configuration above }).ApplyAllDatabaseChangesOnStartup(); ``` Applying changes on startup is how a host says "Polecat owns this schema", and retiring a partition is emphatically a schema change — so the pass is gated on the same opt-in as the migration itself. ### Retention is a partition operation, not a `DELETE` Retiring a period is `TRUNCATE TABLE ... WITH (PARTITIONS (n))` followed by `ALTER PARTITION FUNCTION ... MERGE RANGE`, **in that order**. `MERGE RANGE` on a partition that still holds rows does not reclaim anything — it *moves* those rows into the neighbouring partition, which is the opposite of the point. Truncating first deallocates the partition's pages in O(1), and the merge that follows is then metadata-only against an empty partition. If the truncate fails, the boundary is deliberately left in place. Only boundaries the policy itself would have produced are ever retired, so a hand-added boundary — or one left over from a different period size — is left strictly alone. ::: warning Retiring a period removes its rows. That is the point — it is what makes reclaim O(1) instead of a mass `DELETE` — but choose `periodsBehind` to match the retention policy you actually want. ::: If a process is long-lived enough to outrun the number of periods you provision ahead — an hourly window especially — run the pass yourself on whatever cadence the period size demands: ```csharp // Roll every rolling-window table forward to its current window and retire the partitions that have // aged past their retention floor. Idempotent, and safe to run from several nodes at once. await store.Advanced.ApplyRollingPartitionsAsync(token); // ...or run just one half await store.Advanced.RollPartitionsForwardAsync(token); // additive only, never removes data await store.Advanced.DropAgedRollingPartitionsAsync(token); // retention only ``` Each returns Weasel `TablePartitionStatus[]` — one entry per managed table, so a partial failure surfaces per table rather than taking the whole pass down. ::: tip A SQL Server RANGE function always spans `(-infinity, +infinity)`, so the outermost partitions absorb any row written outside the provisioned window. Unlike PostgreSQL there is no "no partition of relation" error to guard against and no `DEFAULT` overflow partition to declare. ::: ## Externally-managed range partitions If something genuinely outside Polecat owns the partitions, use the externally-managed variant. Polecat creates the partition function, scheme, and table once with the supplied initial boundaries and then never reconciles the partitioning again, so runtime `SPLIT`/`SWITCH`/`MERGE` from elsewhere survives a later schema apply: ```csharp opts.Schema.For() .PartitionOn(x => x.BucketEnd) .ByExternallyManagedRange( new DateTimeOffset(2026, 1, 1, 0, 0, 0, TimeSpan.Zero), new DateTimeOffset(2026, 2, 1, 0, 0, 0, TimeSpan.Zero)); ``` For an ordinary time-series retention table, prefer [rolling time windows](#rolling-time-windows). Opting out of Weasel means opting out of its **ordering and dependency management**, not just its DDL generation — a hand-rolled rebuild that re-creates a partitioned table can easily do so before a helper object its indexes depend on exists, and the resulting failure looks like a broken index rather than what it is. ## Limitations * Supported for **single-tenant** document tables only; combining member RANGE partitioning with conjoined multi-tenancy throws at start-up. Conjoined document tables can instead use the managed per-tenant partitioning below. * One partition scheme per table is a SQL Server constraint, so a document type cannot combine member RANGE partitioning (declared, rolling, or externally managed) with the store's managed per-tenant partitioning. ## Managed per-tenant partitioning (#335) Conjoined multi-tenanted document tables can be physically partitioned **per tenant** through the store's shared managed tenant partitioning — the SQL Server counterpart of Marten's `AllDocumentsAreMultiTenantedWithPartitioning` + `PartitionMultiTenantedDocumentsUsingMartenManagement`: ```csharp var store = DocumentStore.For(opts => { opts.Connection("..."); // Make every document conjoined multi-tenanted AND tenant-partitioned: opts.Policies.AllDocumentsAreMultiTenantedWithPartitioning(); // — or, when the store is already conjoined, enable just the partitioning: // opts.Events.TenancyStyle = TenancyStyle.Conjoined; // opts.Policies.PartitionMultiTenantedDocumentsUsingPolecatManagement(); // Per-type escape hatch (the [SingleTenanted]/DisablePartitioningIfAny analogue): opts.Policies.ForDocument(p => p.DisableTenantPartitioning = true); }); ``` Every document table then carries a `tenant_ordinal int` primary-key column and is `RANGE RIGHT` partitioned on it, driven by the **one `pc_tenant_partitions` registry per database** — the same registry, ordinals, and physical layout as [per-tenant event partitioning](/events/multitenancy#per-tenant-event-partitioning), so a store using both keeps a single coherent tenant → ordinal map across `pc_events`, `pc_streams`, and every document table. The ordinal is resolved server-side from the registry on every write (upsert, insert, update, bulk insert, and projection writes), so cross-process ordinal drift cannot mis-route rows. Tenants are onboarded **lazily on first write** (matching the event-append behavior), or explicitly with per-table status reporting: ```csharp // Onboard tenants up front — returns Weasel TablePartitionStatus[] per managed table: var statuses = await store.Advanced.AddPolecatManagedTenantsAsync(ct, "tenant-a", "tenant-b"); // Tenant bucketing (Weasel 9.18.0): map many small tenants onto one shared partition ordinal. // Requires ManagedTenantPartitions.AllowOrdinalSharing: await store.Advanced.AddPolecatManagedTenantsAsync( new Dictionary { ["small-1"] = 1, ["small-2"] = 1 }, ct); // Remove a tenant. SQL Server's MERGE RANGE alone would retain the rows — // TenantDropBehavior.DeleteData physically purges the tenant's rows from every managed // table first (PostgreSQL managed-drop parity): await store.Advanced.RemovePolecatManagedTenantsAsync( ["tenant-b"], TenantDropBehavior.DeleteData, ct); ``` Notes: * Requires `TenancyStyle.Conjoined` (asserted at store construction) and cannot be combined with member `PartitionByRange` on the same document type — a SQL Server table supports only one partition scheme. * The registry and partition function/scheme are database-global objects: one tenant-partitioned store per database. * `AddPolecatManagedTenantsAsync` splits the tables of document types **registered** with the store (`opts.Schema.For()` or prior use); a table created later bakes the full boundary set at creation. * The daemon's dead-letter document (`pc_doc_deadletterevent`) is always excluded, mirroring Marten. --- --- url: /schema/cleaning.md --- # Tearing Down Document Storage Polecat provides methods for cleaning up data, primarily useful during testing and development. ## Cleaning All Documents Delete all data from all document tables: ```cs await store.Advanced.CleanAllDocumentsAsync(); ``` This executes `DELETE FROM` on every `pc_doc_*` table. ## Cleaning a Specific Type Delete all documents of a specific type: ```cs await store.Advanced.CleanAsync(); ``` ## Cleaning All Event Data Delete all events, streams, and progression data: ```cs await store.Advanced.CleanAllEventDataAsync(); ``` This cleans: * `pc_events` -- All event records * `pc_streams` -- All stream metadata * `pc_event_progression` -- All daemon progression ## Usage in Tests Cleaning is most commonly used in integration test setup: ```cs public class MyTests : IAsyncLifetime { private IDocumentStore _store = null!; public async Task InitializeAsync() { _store = DocumentStore.For(opts => { ... }); // Clean slate for each test await _store.Advanced.CleanAllDocumentsAsync(); await _store.Advanced.CleanAllEventDataAsync(); } public async Task DisposeAsync() { await _store.DisposeAsync(); } } ``` ::: warning These methods permanently delete data. They should only be used in development and testing environments, never in production. ::: --- --- url: /events/projections/ancillary-stores.md --- # Using Ancillary Stores in Projections ## The Problem When building systems with multiple Polecat stores (using `AddPolecatStore()`), it's common to need projections in one store that reference data from another. For example, a billing projection in your primary store might need to look up tariff data from a separate `ITarievenStore`. Directly injecting an ancillary store via constructor can cause startup deadlocks because the DI container may attempt to resolve the ancillary store while the primary store is still being constructed. ## Solution: Inject `Lazy` `AddPolecatStore()` automatically registers `Lazy` in the DI container alongside the store itself. This lets you inject a lazy reference that defers resolution until the store is actually needed — safely past the startup phase: ```csharp public interface ITarievenStore : IDocumentStore; public class InvoiceProjection : SingleStreamProjection { private readonly Lazy _tarievenStore; public InvoiceProjection(Lazy tarievenStore) { _tarievenStore = tarievenStore; } public override async Task EnrichEventsAsync( SliceGroup group, IQuerySession querySession, CancellationToken cancellation) { // Safe - the store is fully constructed by the time // EnrichEventsAsync runs await using var session = _tarievenStore.Value.QuerySession(); var ids = group.Slices .SelectMany(s => s.Events().OfType>()) .Select(e => e.Data.TariefId) .Distinct().ToArray(); var tarieven = await session.LoadManyAsync(cancellation, ids); foreach (var slice in group.Slices) { foreach (var e in slice.Events().OfType>()) { if (tarieven.TryGetValue(e.Data.TariefId, out var tarief)) { e.Data.ResolvedPrice = tarief.Price; } } } } } ``` Register the stores and projection: ```csharp services.AddPolecat(opts => { opts.ConnectionString = "primary connection string"; }); services.AddPolecatStore(opts => { opts.ConnectionString = "tarieven connection string"; }); ``` ### Why `Lazy` Works The `Lazy` wrapper is constructed immediately (it's just a thin wrapper), but the inner `IDocumentStore` isn't resolved until `.Value` is accessed. By the time your projection's methods execute, all stores are fully constructed and the lazy resolution succeeds without deadlock. ### Multiple Ancillary Stores You can inject multiple lazy store references. Each `AddPolecatStore()` call automatically registers its own `Lazy`: ```csharp public class CrossStoreProjection : SingleStreamProjection { private readonly Lazy _tarieven; private readonly Lazy _debtors; public CrossStoreProjection( Lazy tarieven, Lazy debtors) { _tarieven = tarieven; _debtors = debtors; } // Use _tarieven.Value and _debtors.Value in your projection methods } ```