Web Content APIs

This guide covers the technical methods for accessing Web Content data on Carbon Arc: the Python SDK (Explorer API), the REST API (direct endpoints), and discovery tools for finding entities and insights programmatically.

Authentication

Python SDK

from carbonarc import CarbonArcClient
import os

client = CarbonArcClient(
    host=os.environ.get("CARBONARC_HOST", "https://api.carbonarc.co"),
    token=os.environ["CARBONARC_TOKEN"]
)

Generate your API token at User Portal → Profile in the Carbon Arc dashboard. Tokens expire after 12 months.

REST API

REST endpoints on api.carbonarc.co use the same API token as the SDK, passed as a Bearer token or via authenticated session.

curl -H "Authorization: Bearer $CARBONARC_TOKEN" \
  "https://api.carbonarc.co/v2/webcontent"

Python SDK (Explorer API)

The Explorer API follows a Build → Price → Buy → Get workflow. This is the primary programmatic access method for Web Content data.

Workflow Overview

build_framework()  →  check_framework_price()  →  buy_frameworks()  →  get_framework_data()

Step 1: Build a Framework

framework = client.explorer.build_framework(
    entities=[{"carc_id": 64231, "representation": "company"}],  # Chipotle
    insight=110864,  # Item Price
    filters={
        "location_resolution": "ww",
        "date_resolution": "month",
        "date_range": {"start_date": "2025-04-01", "end_date": "2026-04-01"},
        "category": "*",
        "type": "*"
    },
    aggregate="mean"
)

Step 2: Check Price

price = client.explorer.check_framework_price(framework)
print(f"Price: ${price:.2f}")

Step 3: Purchase

order = client.explorer.buy_frameworks([framework])
framework_id = order['frameworks'][0]

Step 4: Retrieve Data

import pandas as pd

data = client.explorer.get_framework_data(framework_id=framework_id)
df = pd.DataFrame(data['data'])
print(df.head())

Or return a DataFrame directly:

df = client.explorer.get_framework_data(
    framework_id=framework_id,
    data_type="dataframe"
)

Discover Available Filters

Some Web Content datasets have custom dimensions. Discover them before querying:

filter_keys = client.explorer.collect_framework_filters(framework)
print(filter_keys)
# ['location_resolution', 'date_resolution', 'category', 'type', 'is_available', ...]

options = client.explorer.collect_framework_filter_options(framework, "category")
print(options)
# ['Burrito', 'Bowl', 'Tacos', 'Salad', ...]

Complete Examples

from carbonarc import CarbonArcClient
import pandas as pd
import os

client = CarbonArcClient(
    host="https://api.carbonarc.co",
    token=os.environ["CARBONARC_TOKEN"]
)

framework = client.explorer.build_framework(
    entities=[{"carc_id": 64231, "representation": "company"}],
    insight=110864,
    filters={
        "location_resolution": "ww",
        "date_resolution": "month",
        "date_range": {"start_date": "2025-04-01", "end_date": "2026-04-01"},
        "category": "*"
    },
    aggregate="mean"
)

price = client.explorer.check_framework_price(framework)
print(f"Chipotle menu pricing: ${price:.2f}")

order = client.explorer.buy_frameworks([framework])
framework_id = order['frameworks'][0]

df = client.explorer.get_framework_data(
    framework_id=framework_id, data_type="dataframe"
)
print(df.head(10))

Carvana — Weekly Incentivized Vehicle Prices by Make

framework = client.explorer.build_framework(
    entities=[{"carc_id": 64256, "representation": "company"}],
    insight=110847,
    filters={
        "location_resolution": "ww",
        "date_resolution": "week",
        "date_range": {"start_date": "2025-10-01", "end_date": "2026-04-01"},
        "make": "*",
        "fuelType": "*",
        "bodyType": "*"
    },
    aggregate="mean"
)

price = client.explorer.check_framework_price(framework)
order = client.explorer.buy_frameworks([framework])
df = client.explorer.get_framework_data(
    framework_id=order['frameworks'][0], data_type="dataframe"
)
print(df.head(10))

Avis Budget Group — Daily Pay Now Amounts

framework = client.explorer.build_framework(
    entities=[{"carc_id": 64200, "representation": "company"}],
    insight=110822,
    filters={
        "location_resolution": "ww",
        "date_resolution": "day",
        "date_range": {"start_date": "2025-01-01", "end_date": "2026-04-01"},
        "carGroup": "*",
        "make": "*"
    },
    aggregate="sum"
)

price = client.explorer.check_framework_price(framework)
order = client.explorer.buy_frameworks([framework])
df = client.explorer.get_framework_data(
    framework_id=order['frameworks'][0], data_type="dataframe"
)
print(f"Rows: {len(df)}")
print(df.head())

Intuit — Daily Job Posting Count

framework = client.explorer.build_framework(
    entities=[{"carc_id": 64334, "representation": "company"}],
    insight=110937,
    filters={
        "location_resolution": "ww",
        "date_resolution": "day",
        "date_range": {"start_date": "2025-01-01", "end_date": "2026-04-01"}
    },
    aggregate="count_distinct"
)

price = client.explorer.check_framework_price(framework)
order = client.explorer.buy_frameworks([framework])
df = client.explorer.get_framework_data(
    framework_id=order['frameworks'][0], data_type="dataframe"
)
print(df.tail(10))

Booking Holdings — Weekly Car Rental Prices

framework = client.explorer.build_framework(
    entities=[{"carc_id": 64174, "representation": "company"}],
    insight=110946,
    filters={
        "location_resolution": "ww",
        "date_resolution": "week",
        "date_range": {"start_date": "2025-01-01", "end_date": "2026-04-01"},
        "carType": "*",
        "agency": "*"
    },
    aggregate="mean"
)

price = client.explorer.check_framework_price(framework)
order = client.explorer.buy_frameworks([framework])
df = client.explorer.get_framework_data(
    framework_id=order['frameworks'][0], data_type="dataframe"
)
print(df.head(10))

Paginated Retrieval (Large Datasets)

all_records = []
page = 1

while True:
    response = client.explorer.get_framework_data(
        framework_id=framework_id,
        fetch_all=False,
        page=page,
        size=1000
    )
    records = response.get('data', [])
    if not records:
        break
    all_records.extend(records)
    if len(all_records) >= response.get('total', 0):
        break
    page += 1

df = pd.DataFrame(all_records)
print(f"Total records: {len(df)}")

REST API (Direct Endpoints)

The REST API provides direct access to Web Content data. Base URL: https://api.carbonarc.co/v2

Web Content Endpoints

Method	Path	Description
`GET`	`/v2/webcontent`	List all Web Content feeds (paginated)
`GET`	`/v2/webcontent/subscribed`	List subscribed Web Content feeds
`GET`	`/v2/webcontent/{name}`	Get details for a specific feed
`GET`	`/v2/webcontent/{name}/dataframe`	Get feed data as DataFrame-ready JSON
`POST`	`/v2/webcontent/download`	Download Web Content data
`GET`	`/v2/webcontent/{id}/date`	Get available dates for a feed
`GET`	`/v2/webcontent/{id}/data`	Get data with date filtering
`GET`	`/v2/webcontent/{id}/code`	Get code snippet for the feed

Framework Endpoints (Builder / Order Flow)

Method	Path	Description
`POST`	`/v2/framework/order`	Build a framework order
`POST`	`/v2/framework/filters`	Get available filters
`POST`	`/v2/framework/metadata`	Get framework metadata
`POST`	`/v2/framework/buy`	Purchase a framework
`GET`	`/v2/framework/data`	Retrieve purchased framework data

Example: List Web Content Feeds

curl -H "Authorization: Bearer $CARBONARC_TOKEN" \
  "https://api.carbonarc.co/v2/webcontent?page=1&size=25"

Example: Get Feed Data with Date Filter

curl -H "Authorization: Bearer $CARBONARC_TOKEN" \
  "https://api.carbonarc.co/v2/webcontent/12345/data?webcontent_date_operator=gte&webcontent_date=2025-01-01&page=1&size=100"

Query Parameters

Parameter	Type	Description
`page`	int	Page number (default: 1)
`size`	int	Results per page (default: 25)
`webcontent_date_operator`	string	Date comparison: `gte`, `lte`, `eq`
`webcontent_date`	string	Date value in `YYYY-MM-DD`
`fetch_all`	bool	Return all data in one call

Discovery Tools

Before querying data, you need entity IDs (carc_id) and insight IDs. Use the Ontology API to discover them.

Search Entities

results = client.ontology.search_entities("Chipotle")

Get Insights for an Entity

insights = client.ontology.get_insights_for_entity(
    entity_id=64231,
    entity_representation="company"
)

Get Entities for an Insight

entities = client.ontology.get_entities_for_insight(insight_id=110864)

Discovery Workflow

client.ontology.search_entities("Chipotle")
    → carc_id: 64231, representation: "company"

client.ontology.get_insights_for_entity(64231, "company")
    → insight_id: 110864 (Item Price)
    → insight_id: 110865 (Menu Category Count)
    → ...

client.explorer.build_framework(entities=[{carc_id: 64231}], insight=110864, ...)
    → Framework object ready to price and purchase

Pricing

What Drives Cost?

Data is priced based on the volume and structure of the result set:

Date range — wider ranges increase cost
Date resolution — day costs more than week, month, or quarter
Number of entities — more entities increase cost
Location resolution — state / cbsa costs more than us or ww
Wildcard filters — "*" returns all values, increasing data volume

Checking Price Before Purchase

Always check the price before buying:

price = client.explorer.check_framework_price(framework)
print(f"Price: ${price:.2f}")

if price == 0:
    print("Free — already purchased or included in subscription")

Tips to Reduce Cost

Use month or quarter resolution instead of day
Narrow the date range to only what you need
Specify filter values instead of using "*" wildcards
Use ww location resolution unless you need geographic breakdowns

Parameter Reference

`entities[]`

Field	Type	Required	Description
`carc_id`	int	Yes	Carbon Arc entity identifier
`representation`	string	Yes	Entity type: `company`, `product`, `ticker`, `app`, `retailer`

`insight`

Field	Type	Required	Description
(int)	int	Yes	The insight ID (metric/KPI) to retrieve

`filters`

Field	Type	Required	Values
`location_resolution`	string	Yes	`ww`, `us`, `state`, `dma`, `cbsa`, `country`, `zip`
`date_resolution`	string	Recommended	`day`, `week`, `month`, `quarter`
`date_range`	object	Recommended	`{"start_date": "YYYY-MM-DD", "end_date": "YYYY-MM-DD"}`
`<custom_filter>`	string/array	No	Dataset-specific. Use `"*"` for wildcard

`aggregate`

Value	Description
`sum`	Sum of values across the dimension
`mean`	Average of values
`count_distinct`	Count of unique values

Dataset-Specific Filters

Dataset	`carc_id`	Insight ID	Custom Filters
Chipotle Menu	64231	110864 (Item Price)	`category`, `type`, `is_available`, `is_eligible_for_delivery`
Carvana Vehicles	64256	110847 (Incentivized Price)	`make`, `fuelType`, `bodyType`, `year`, `vehiclePurchaseType`, `saleStatus`
Carvana Reviews	64256	110855 (Review Count)	`Product`
Avis Rentals	64200	110822 (Pay Now Amount)	`carGroup`, `make`, `carAvailability`, `country`, `state`
Booking Rentals	64174	110946 (Rental Price)	`carType`, `agency`, `category`, `state`
Life Storage	32655	110952 (Available Units)	`size`, `hasAvailableInventory`, `isExactSize`
Tesla Inventory	64465	111042 (Total Price)	None
Eventbrite	64297	110896 (Total Events)	None
Live Nation (Sphere)	64356	110956 (Ticket Count)	None
Intuit Jobs	64334	110937 (Job Count)	None
Toast Jobs	64468	111044 (Job Count)	None
Extra Space Storage	64299	110897 (Review Count)	None
Sunbelt Rentals	64456	111035 (Inventory Count)	None
United Rentals	64474	111047 (Daily Rental Rate)	None

Error Handling

Common Errors

Error	Cause	Fix
`Invalid entity`	Wrong `carc_id`	Use `search_entities()` to find the correct ID
`Invalid insight`	Wrong insight ID	Use `get_insights_for_entity()` to find valid insights
`Invalid entity/insight combination`	Entity doesn't have data for that insight	Use `get_insights_for_entity()` or `get_entities_for_insight()` to verify compatibility
`Empty data returned`	Framework not purchased	Call `buy_frameworks()` first
`Price is $0`	Already purchased or free tier	Data should be available via `get_framework_data()`
`Timeout on large queries`	Too much data requested	Reduce date range or use pagination

SDK Error Handling

try:
    framework = client.explorer.build_framework(
        entities=[{"carc_id": 64231, "representation": "company"}],
        insight=110864,
        filters={
            "location_resolution": "ww",
            "date_resolution": "month",
            "date_range": {"start_date": "2025-01-01", "end_date": "2026-04-01"}
        },
        aggregate="mean"
    )
    price = client.explorer.check_framework_price(framework)

    if price > 100:
        print(f"Warning: query costs ${price:.2f}. Consider narrowing filters.")

    order = client.explorer.buy_frameworks([framework])
    df = client.explorer.get_framework_data(
        framework_id=order['frameworks'][0],
        data_type="dataframe"
    )
    print(f"Retrieved {len(df)} rows")

except Exception as e:
    print(f"Error: {e}")

Debugging Tips

import json

print(json.dumps(framework, indent=2))

data = client.explorer.get_framework_data(framework_id=framework_id)
print(f"Keys: {data.keys()}")
print(f"Records: {len(data.get('data', []))}")
print(f"Total available: {data.get('total', 'N/A')}")

SDK Method Reference

Method	Signature	Returns
`build_framework()`	`(entities, insight, filters, aggregate)`	Framework object
`collect_framework_filters()`	`(framework)`	List of filter keys
`collect_framework_filter_options()`	`(framework, filter_key)`	List of options
`check_framework_price()`	`(framework)`	Price (float)
`buy_frameworks()`	`([frameworks])`	Order object with framework IDs
`get_framework_metadata()`	`(framework_id)`	Metadata dict
`get_framework_data()`	`(framework_id, data_type?, page?, size?, fetch_all?)`	DataFrame, timeseries, or dict
`search_entities()`	`(query)`	Search results
`get_insights_for_entity()`	`(entity_id, entity_representation)`	List of insights
`get_entities_for_insight()`	`(insight_id)`	List of entities

Quick Reference

SDK WORKFLOW
═══════════
1. search_entities() / get_insights_for_entity()   → Discover IDs
2. build_framework()                                → Define query
3. collect_framework_filters()                      → (Optional) Discover filters
4. check_framework_price()                          → Verify cost
5. buy_frameworks()                                 → Purchase
6. get_framework_data()                             → Retrieve data

HOST: https://api.carbonarc.co
AUTH: API token (Bearer)