As-of dates is a calendar spine that controls which dates the downstream PITs and Bridges compute snapshots for. It is the simplest model in the pipeline — but one of the most important for performance.
How It Works
The model generates one row per day between two dates. On each pipeline run, it adds new dates using the Airflow ds and prev_ds variables:
- prev_ds — the start of the window (previous execution date)
- ds — the end of the window (current execution date)
This means only new dates since the last run are added — keeping the table compact and the pipeline fast.
Why Not Generate All Dates?
If you generated every date from project start to "far future," PITs would have to compute a snapshot for hundreds or thousands of dates on every run. By using a rolling window, PITs only process the new dates — making incremental runs fast.
What Happens Downstream
Every date in this table becomes a row in each PIT and a set of rows in the Bridge. A PIT for 4 satellites × 8 dates = 32 lookups. Controlling the date spine controls the cost of the entire business vault.
Idempotency Note
The as-of dates model is incremental. Re-running the same date range inserts no duplicates — dates already in the table are filtered by the incremental predicate.