Asynchronous Projections

The async daemon is a background service that processes events and applies projections asynchronously, providing eventually consistent read models.

How It Works

1. The High Water Mark Detector monitors pc_events for new events using SQL LEAD() window functions to detect gaps
2. The Event Loader fetches batches of events for processing
3. Each projection processes its batch and updates its read model
4. Progress is tracked in pc_event_progression via atomic MERGE statements

Enabling the Async Daemon

Register projections with async lifecycle:

var store = DocumentStore.For(opts =>
{
    opts.Connection("...");

    opts.Projections.Snapshot<OrderSummary>(SnapshotLifecycle.Async);
    opts.Projections.Add<DashboardProjection>(ProjectionLifecycle.Async);
});

When wired up through AddPolecat(), opt the daemon into the host's lifetime explicitly with AddAsyncDaemon(DaemonMode):

builder.Services.AddPolecat(opts =>
{
    opts.Connection("...");

    opts.Projections.Snapshot<OrderSummary>(SnapshotLifecycle.Async);
    opts.Projections.Add<DashboardProjection>(ProjectionLifecycle.Async);
})
.AddAsyncDaemon(DaemonMode.Solo)        // start the daemon as IHostedService
.ApplyAllDatabaseChangesOnStartup();    // run schema migration at boot

TIP

Async projections do not run unless you call AddAsyncDaemon(...). Use DaemonMode.Solo for single-node deployments and DaemonMode.HotCold for multi-node deployments where only one host should own each projection shard.

Daemon Settings

Configure daemon behavior:

opts.DaemonSettings.StaleSequenceThreshold = 1000;

Polling

Unlike Marten's PostgreSQL LISTEN/NOTIFY, Polecat uses polling to detect new events:

// The daemon polls for new events at a configurable interval
// Default: 500ms

Waiting for Non-Stale Data

CatchUpAsync

Wait for all projections to catch up to the current high water mark:

await store.WaitForNonStaleProjectionDataAsync(TimeSpan.FromSeconds(30));

Per-Query

Wait for projections before a specific query:

var orders = await session.Query<OrderSummary>()
    .QueryForNonStaleData()
    .Where(x => x.Status == "Active")
    .ToListAsync();

Event Progression

Track daemon progress:

// The pc_event_progression table stores:
// - name: Projection/subscription name
// - last_seq_id: Last processed sequence ID
// - last_updated: When last updated

High Water Mark Detection

The high water mark detector uses SQL Server's LEAD() window function to detect sequence gaps in the event log. This prevents the daemon from processing events out of order when concurrent writers create gaps.

Listening for Daemon Commits

Sometimes you need to run a side effect after an aggregate has been durably updated by the async daemon — the canonical case is invalidating a cache key. Doing this before the commit would open a window where a concurrent read could repopulate the cache with stale state. Subscriptions, composite projections, and projection side effects all run before the batch is committed, so they can't close that window.

Register an IChangeListener on Projections.AsyncListeners to hook the commit boundary of each daemon projection batch:

public class CacheFlushingListener : IChangeListener
{
    // Runs AFTER the batch is committed → "at most once". Ideal for cache invalidation.
    public async Task AfterCommitAsync(IDocumentSession session, IChangeSet commit, CancellationToken token)
    {
        foreach (var party in commit.Updated.OfType<QuestParty>())
        {
            await _cache.RemoveAsync($"quest-party:{party.Id}", token);
        }
    }

    // Runs BEFORE the batch is committed → "at least once".
    public Task BeforeCommitAsync(IDocumentSession session, IChangeSet commit, CancellationToken token)
        => Task.CompletedTask;
}

var store = DocumentStore.For(opts =>
{
    opts.ConnectionString = connectionString;
    opts.Projections.Snapshot<QuestParty>(SnapshotLifecycle.Async);

    // Fires only within the async daemon, once per committed projection batch
    opts.Projections.AsyncListeners.Add(new CacheFlushingListener());
});

The commit parameter is an IChangeSet describing the projected documents written in that batch (Inserted / Updated / Deleted).

Delivery semantics mirror Marten:

• AfterCommitAsync runs once, after the transaction commits — at most once. A faulting after-commit listener is swallowed so the batch is not reprocessed (which would re-fire the side effect); the data is already durable.
• BeforeCommitAsync runs before the commit — at least once. A throw here aborts the batch before anything is committed.

TIP

Async listeners are suppressed during projection rebuilds. A full replay re-applies every event, so firing post-commit side effects for each historical batch is almost never what you want.

Error Handling

The daemon uses Polly resilience pipelines for error handling. See Resiliency Policies for configuration.

Architecture

pc_events
  │ (Polling)
  ▼
High Water Mark Detector
  │ (Sequence Range)
  ▼
Event Loader
  ├──► Projection A ──► pc_doc_summary
  ├──► Projection B ──► pc_doc_dashboard
  └──► Subscription C ──► External System
         │
         ▼
  pc_event_progression (tracks progress for all)