Understanding Fill Rates and Panel Restatements

How to read changing Prism values, understand what is driving them, and build a consistent process around the data

1. Net/Net

Two mechanisms explain most day-to-day movement in panel data that powers Prisms. Fill determines how much of recent activity has arrived. Restatement rebuilds the panel and its history. Understanding the distinction matters, particularly when the data informs a forecast or a decision.

Mechanism	What it is	What it changes	When	Size
Fill rate (transaction lag)	Card transactions reach us over several days after they happen	The most recent ~6 days. Values rise as late transactions arrive	Continuous, every day	~73% of a day is in by T+3, ~97% by T+5, ~99.6% by T+6
Panel restatement	Once a month the panel is rebuilt and the full history recomputed	Any date in history, most visibly the most recent completed month	The week of the first Monday of each month	Often adds ~3 to 5 points to the latest month's YoY growth. It varies

What this means in practice

Every new query returns the current version of history. Platform, SDK, and MCP do not offer an “as of” control. If you want to know what the data said at a decision point, you need to capture and store it then (§5).
A purchased framework stays fixed. Its framework_id preserves that purchase. Later restatements appear in a new purchase with a new ID.
Recent card data is still filling. For the core card panel, T+6 is effectively complete for transaction arrival. You can wait or adjust using the fill curve (§3.4). Neither approach prevents later panel restatements.
Month end is one step in the process. A month is roughly 99% filled three days later and effectively fully filled by day six. It also goes through the monthly restatement cycle. The two schedules can overlap.
Early YoY readings need context. The prior-year month is fully filled and has already been restated. The current month may still be working through both. That mismatch can understate growth.

2. Fill rates: when the most recent days are still arriving

2.1 Example curve for Insight ID 626

Card transactions tend to arrive over several days depending on the data source. To understand a recent read, we first need to understand how much of that day is in the data. For example these are the average fill rates for the core card panel (insight 626):

Age of the day	Share of that day's transactions in our data
T+3	73%
T+4	88%
T+5	97%
T+6	99.6%
T+7 and later	~100%

These percentages are a measure of completeness. The curve is an average, and there is nuance around other factors such as weekends, and holidays.

2.2 Three ways to handle it

Approach	How	Best for	Trade-off
Cut off	Keep only days at T+6 or older	Backtests, models, reporting	You lose the freshest ~6 days
Scale up	Divide each recent day by its expected fill: `adjusted = observed / fill(age)`	Nowcasts	Adds error wherever the data's arrival pattern differs from the average
Fixed age	Always compare values read at the same age (for example, every month read at month end + 6 days)	Comparing across vintages	Needs your own snapshots (§5)

3. Panel restatements: what are they?

3.1 What happens with our CA0056 Panel for example:

During the week of the first Monday of each month:

The vendor restates its full panel. The vendor re-filters for the cards that best represent the U.S. consumer.
We reapply our panel filters (core or constant) and rebuild the full historical series.
Merchant-to-entity mappings are refined. A merchant may be mapped to a company, unmapped, or moved to a different one.
Methodology changes, if there are any, are released in the same cycle.

The result is a new version of history. Any historical date can change, even if the underlying transactions are unchanged. In practice, the most recent completed month tends to move the most.

3.2 Where to see which version you have

After a purchase, open the Information tab:

Last Reinstated: when this data was last rebuilt
Ontology Version: the mapping and framework version (for example v2026.9.2)

If these fields differ across purchases with the same definition, you are looking at different vintages. Establish that first before interpreting a change in the numbers.

4. Building a point-in-time history

4.1 An example process

Choose a schedule.
- Monthly first-release: pull once after month end. Pick a fixed age, for example month end + X days to account for fill, so data is near complete.
- Pre and post restatement: pull once before the first-Monday week and once after it, so you capture both versions.
- Daily: pull every day to see every value as it first appeared.
Purchase or reinstate the framework on that schedule.
Store each pull with: framework_id, order_id, pull timestamp (UTC), Last Reinstated, Ontology Version, requested and returned date range.
Over time, this builds a history of what you actually knew at each pull. You can then measure revisions and understand their impact on your decisions.

4.2 Sketch (Python SDK)

import json
from datetime import datetime, timezone
from pathlib import Path

VINTAGES = Path("vintages")


def snapshot(client, framework, label: str) -> Path:
    """Buy the current vintage and store it with the IDs needed to reproduce it."""
    order = client.explorer.buy_frameworks([framework])  # charges your account
    framework_id = order["frameworks"][0]
    df = client.explorer.get_framework_data(framework_id=framework_id)

    pulled_at = datetime.now(timezone.utc)
    out = VINTAGES / f"{label}_{pulled_at:%Y-%m-%d}"
    out.mkdir(parents=True, exist_ok=True)
    df.to_parquet(out / "data.parquet")
    (out / "meta.json").write_text(json.dumps({
        "label": label,
        "pulled_at_utc": pulled_at.isoformat(),
        "framework_id": framework_id,
        "order_id": order.get("order_id"),
        # Copy these from the purchase's Information tab:
        "last_reinstated": None,
        "ontology_version": None,
    }, indent=2))
    return out

To measure revisions, join two vintages on date and entity and compute (later − earlier) / earlier. Group the result by how old each date was at the first read (T+3, T+6, month end + 6, after restatement). That gives you your own revision curve for the entities you want to track.

5. Putting the framework to work

5.1 Forecasting and backtesting

Train and test on values as they would have looked at decision time, using your own vintages (§5). A backtest on current-vintage history uses information you would not have had: the fill and the restatements.
Pick a consistent read time, such as month end + 6 days or after restatement. Record whether each pull falls before or after that month’s restatement; a fixed day offset alone does not guarantee the same restatement status.
Expect the latest month to move up by a few points at restatement, and account for that in the error bands.

Questions or a specific read to work through? Reach out to support@carbonarc.co or your account team.