Data Retrieval

This guide demonstrates how to set up your environment, build a framework, estimate and place an order, and then retrieve the results using the Carbon Arc Python SDK. The below is an example showing how to return Walmart Credit Card Spend from our Card - US Detailed Panel dataset.

1. Load Environment Variables and Instantiate Client

First, load your environment variables and configure the SDK client. You can point the client to a development or production host as needed.

# Import required dependencies
import os
from dotenv import load_dotenv
from carbonarc import CarbonArcClient

load_dotenv()

## Read in environment variables
API_AUTH_TOKEN=os.getenv("API_AUTH_TOKEN")

# Create API Client
client = CarbonArcClient(API_AUTH_TOKEN)

print('Client initialized successfully')

Note

Never hardcode your tokens directly in code — use a .env file instead.

2. Build a Framework

Construct a framework to define what you wish to buy. In this example, we will retrieve a company-level insight with custom filters:

This example is Walmart Credit Card Spend in our Card - US Detailed Panel.

framework = client.explorer.build_framework(
    entities=[{"carc_id": 64781, "representation": "company"}],
    insight=347,
    filters={
        "location_resolution": "us",
        "date_resolution": "day",
        "date_range": {
            "start_date": "2022-06-20",
            "end_date": "2025-06-19"
        },
    },
    aggregate="mean"
)
print("Framework ID:", framework)

Check framework filters after creating your framework

framework_filters = client.explorer.collect_framework_filters(framework)

# additionally find filter options
filter_key = "platform"
filter_options = client.explorer.collect_framework_filter_options(framework, filter_key)

3. Check the Framework Filters

Run the following command to review the optional or dependent filters in your Framework.

framework_filters = client.explorer.collect_framework_filters(framework)
print("Available filters:")
print(framework_filters)

4. Check Framework Price and Order Information

price = client.explorer.check_framework_price(framework)
print(price)

order_information = client.explorer.buy_frameworks([framework])
print(order_information)

Estimate the token price of the data before purchasing:

Then place the order:

```python
price = client.explorer.check_framework_price(framework)

order_information = client.explorer.buy_frameworks([framework])

4. Retrieve Framework Metadata

Retrieve framework id from order information. Make sure to retrieve the Framework ID and use that below not the order ID. The order id willhelp you track billing.

print(framework)

framework_id = "INSERT FRAMEWORK ID HERE"
framework_metadata = client.explorer.get_framework_metadata(framework_id=framework_id)

5. Retrieve Data

You can retrieve data for the framework all at once or in pages.

Retrieve all results in one call (if small enough):

data = client.explorer.get_framework_data(framework_id=framework_id)

Or retrieve a single page:

page_data = client.explorer.get_framework_data(
    framework_id=framework_id,
    page=1,
    size=100
)
print(f"Retrieved {len(page_data)} records in page 1")

Use pretty print to inspect a sample:

from pprint import pprint
pprint(page_data)

6. Paginate Through All Results

To iterate over multiple pages, you can follow this pattern:

all_data = []
page = 1
while True:
    results = client.explorer.get_framework_data(
        framework_id=framework_id,
        page=page,
        page_size=100
    )
    if not results:
        break
    all_data.extend(results)
    print(f"Page {page}: {len(results)} records")
    page += 1

print(f"Total records retrieved: {len(all_data)}")

7. Save to CSV

Convert the results to a pandas DataFrame and save locally:

import pandas as pd

df = pd.DataFrame(all_data)
df.to_csv("framework_data_all.csv", index=False)
print("Saved data to framework_data_all.csv")

---

Next Steps:

• Integrate these retrieval patterns into a data pipeline
• Apply pandas transformations for advanced analytics
• Explore client.explorer capabilities to enrich your data purchases