CAISO Data Guide

Overview

The California Independent System Operator (CAISO) manages the bulk of California’s power grid and operates one of the most transparent electricity markets in the world. ISO-DART provides comprehensive access to CAISO’s OASIS API, covering pricing, load, generation, and market operations data.

Quick Reference

Data Category

Update Frequency

Historical Availability

Typical File Size

LMP (DAM)

Daily

2009-present

50-100 MB/month

LMP (RTM)

5-min intervals

2009-present

200-400 MB/month

Load Forecast

Hourly

2010-present

5-10 MB/month

Wind & Solar

5-min intervals

2015-present

20-30 MB/month

Fuel Prices

Daily

2014-present

<1 MB/month

CAISO Markets

CAISO operates several interconnected markets:

Day-Ahead Market (DAM)

  • Purpose: Schedule generation and transmission for next day

  • Timeline: Closes at 10 AM, results posted by 1 PM

  • Resolution: Hourly intervals

  • Use Cases: Price forecasting, scheduling, bilateral contracts

Hour-Ahead Scheduling Process (HASP)

  • Purpose: Finalize schedules for upcoming hour

  • Timeline: Runs 75 minutes before each operating hour

  • Resolution: Hourly intervals

  • Use Cases: Short-term adjustments, intraday trading

Real-Time Market (RTM)

  • Purpose: Balance supply and demand in real-time

  • Timeline: Continuous operation

  • Resolution: 5-minute intervals

  • Use Cases: Real-time balancing, regulation services

Real-Time Pre-Dispatch (RTPD)

  • Purpose: Provide 5-minute binding dispatch instructions

  • Timeline: Runs every 5 minutes

  • Resolution: 5-minute intervals

  • Use Cases: Real-time operations, frequency regulation

Residual Unit Commitment (RUC)

  • Purpose: Ensure sufficient capacity for next day

  • Timeline: Runs after DAM clears

  • Resolution: Hourly intervals

  • Use Cases: Capacity analysis, reliability assessment

Available Data Types

1. Pricing Data

Locational Marginal Prices (LMP)

LMP represents the cost of delivering electricity to a specific location, decomposed into:

  • Energy Component (LMP): Base energy cost

  • Congestion Component (MCC): Transmission constraint costs

  • Loss Component (MLC): Electrical loss costs

Available Markets: DAM, HASP, RTM, RTPD

Example - Download DAM LMP:

from datetime import date
from lib.iso.caiso import CAISOClient, Market

client = CAISOClient()
client.get_lmp(
    market=Market.DAM,
    start_date=date(2024, 1, 1),
    end_date=date(2024, 1, 31)
)
client.cleanup()

Output File: 20240101_to_20240131_PRC_LMP_{location}.csv

Key Columns:

  • OPR_DATE: Operating date (YYYY-MM-DD)

  • INTERVAL_NUM: Hour ending (1-24) or 5-min interval (1-288)

  • DATA_ITEM: Pricing node identifier (e.g., TH_NP15_GEN-APND)

  • VALUE: Price in $/MWh

  • MLC: Marginal Loss Component

  • MCC: Marginal Congestion Component

Common Pricing Nodes:

  • TH_NP15_GEN-APND: NP15 Generation Trading Hub

  • TH_SP15_GEN-APND: SP15 Generation Trading Hub

  • TH_ZP26_GEN-APND: ZP26 Generation Trading Hub

  • DLAP_LA_BASIN: LA Basin Load Aggregation Point

Ancillary Services Prices

Clearing prices for regulation, spinning reserves, and other AS products.

client.get_ancillary_services_prices(
    market=Market.DAM,
    start_date=date(2024, 1, 1),
    end_date=date(2024, 1, 31)
)

Service Types:

  • RU (Regulation Up): Frequency regulation - increase generation

  • RD (Regulation Down): Frequency regulation - decrease generation

  • SR (Spinning Reserve): 10-minute synchronized reserves

  • NR (Non-Spinning Reserve): 10-minute non-synchronized reserves

Fuel Prices

Natural gas and other fuel prices used in market clearing.

client.get_fuel_prices(
    start_date=date(2024, 1, 1),
    end_date=date(2024, 1, 31),
    region='ALL'  # or specific region
)

Fuel Regions: SFBAY, SOCAL, NP15, SP15

GHG Allowance Prices

Greenhouse gas allowance prices for resources participating in the California Cap-and-Trade program.

client.get_ghg_allowance_prices(
    start_date=date(2024, 1, 1),
    end_date=date(2024, 1, 31)
)

2. Load Data

System Load Forecasts

CAISO publishes multiple load forecasts with different time horizons:

Day-Ahead (DAM):

client.get_load_forecast(
    market=Market.DAM,
    start_date=date(2024, 1, 1),
    end_date=date(2024, 1, 31)
)

Two Day-Ahead (2DA):

client.get_load_forecast(
    market=Market.TWO_DA,
    start_date=date(2024, 1, 1),
    end_date=date(2024, 1, 31)
)

Seven Day-Ahead (7DA):

client.get_load_forecast(
    market=Market.SEVEN_DA,
    start_date=date(2024, 1, 1),
    end_date=date(2024, 1, 31)
)

Real-Time (RTM):

client.get_load_forecast(
    market=Market.RTM,
    start_date=date(2024, 1, 1),
    end_date=date(2024, 1, 31)
)

Advisory (RTPD):

client.get_advisory_demand_forecast(
    start_date=date(2024, 1, 1),
    end_date=date(2024, 1, 31)
)

Use Cases:

  • DA & 2DA: Next-day planning, unit commitment

  • 7DA: Weekly operations planning

  • RTM: Real-time balancing

  • Advisory: 4-hour ahead adjustments

3. Generation & Resources

Wind and Solar Summary

Real-time wind and solar generation across CAISO’s footprint.

client.get_wind_solar_summary(
    start_date=date(2024, 1, 1),
    end_date=date(2024, 1, 31)
)

Key Metrics:

  • Current wind/solar generation (MW)

  • Forecasted generation (MW)

  • Available capacity (MW)

  • Curtailment data

  • By resource type and location

Analysis Example:

import pandas as pd
import matplotlib.pyplot as plt

# Load wind/solar data
df = pd.read_csv('data/CAISO/20240101_to_20240131_ENE_WIND_SOLAR_SUMMARY_WIND_TOTAL_GEN_MW.csv')

# Plot daily generation pattern
df['OPR_DT'] = pd.to_datetime(df['OPR_DATE'])
daily = df.groupby(df['OPR_DT'].dt.date)['VALUE'].sum()

plt.figure(figsize=(14, 6))
plt.bar(range(len(daily)), daily.values)
plt.xlabel('Day of Month')
plt.ylabel('Total Generation (MWh)')
plt.title('January 2024 Wind Generation - CAISO')
plt.savefig('wind_generation_jan2024.png')

System Load and Resource Schedules

Scheduled generation, load, and interchange for each market.

client.get_system_load(
    market=Market.DAM,
    start_date=date(2024, 1, 1),
    end_date=date(2024, 1, 31)
)

Available for: DAM, RUC, HASP, RTM

4. Market Operations Data

Market Power Mitigation (MPM)

Shows which resources were mitigated due to local market power.

client.get_market_power_mitigation(
    market=Market.DAM,
    start_date=date(2024, 1, 1),
    end_date=date(2024, 1, 31)
)

Markets: DAM, HASP, RTPD

Mitigation Types:

  • Local Market Power Mitigation

  • Default Energy Bid (DEB) applied

  • Resource commitment status

Flexible Ramping

Flexible ramping products help CAISO manage uncertainty from renewables.

Requirements:

client.get_flex_ramp_requirements(
    start_date=date(2024, 1, 1),
    end_date=date(2024, 1, 31),
    baa_group='ALL'  # or specific BAA
)

Awards:

client.get_flex_ramp_awards(
    start_date=date(2024, 1, 1),
    end_date=date(2024, 1, 31)
)

Demand Curves:

client.get_flex_ramp_demand_curve(
    start_date=date(2024, 1, 1),
    end_date=date(2024, 1, 31)
)

Directions: Up (FRU), Down (FRD)

Energy Imbalance Market (EIM)

EIM transfers and limits across participating balancing authorities.

Transfer Data:

client.get_eim_transfer(
    start_date=date(2024, 1, 1),
    end_date=date(2024, 1, 31)
)

Transfer Limits:

client.get_eim_transfer_limits(
    start_date=date(2024, 1, 1),
    end_date=date(2024, 1, 31)
)

5. Ancillary Services

Requirements

System-wide AS requirements by product and region.

client.get_ancillary_services_requirements(
    market=Market.DAM,
    start_date=date(2024, 1, 1),
    end_date=date(2024, 1, 31),
    anc_type='ALL',      # or 'RU', 'RD', 'SR', 'NR'
    anc_region='ALL'     # or specific region
)

Markets: DAM, HASP, RTM

Awards/Results

Cleared quantities by product and resource.

client.get_ancillary_services_results(
    market=Market.DAM,
    start_date=date(2024, 1, 1),
    end_date=date(2024, 1, 31)
)

Operating Reserves

Actual deployed operating reserves.

client.get_operating_reserves(
    start_date=date(2024, 1, 1),
    end_date=date(2024, 1, 31)
)

Common Workflows

1. Daily Price Monitoring

from datetime import date, timedelta
from lib.iso.caiso import CAISOClient, Market

def daily_price_update():
    """Download yesterday's prices across all markets."""
    yesterday = date.today() - timedelta(days=1)

    client = CAISOClient()

    # Download DAM and RTM LMP
    client.get_lmp(Market.DAM, yesterday, yesterday)
    client.get_lmp(Market.RTM, yesterday, yesterday)

    # Download AS prices
    client.get_ancillary_services_prices(Market.DAM, yesterday, yesterday)

    client.cleanup()
    print(f"✓ Downloaded prices for {yesterday}")

if __name__ == '__main__':
    daily_price_update()

Schedule with cron:

0 8 * * * cd /path/to/ISO-DART && /path/to/venv/bin/python daily_price_update.py

2. Renewable Generation Analysis

import pandas as pd
import matplotlib.pyplot as plt
from datetime import date
from lib.iso.caiso import CAISOClient

# Download wind and solar data
client = CAISOClient()
client.get_wind_solar_summary(date(2024, 1, 1), date(2024, 1, 31))
client.cleanup()

# Load and analyze
wind_file = 'data/CAISO/20240101_to_20240131_ENE_WIND_SOLAR_SUMMARY_WIND_TOTAL_GEN_MW.csv'
solar_file = 'data/CAISO/20240101_to_20240131_ENE_WIND_SOLAR_SUMMARY_SOLAR_TOTAL_GEN_MW.csv'

wind_df = pd.read_csv(wind_file)
solar_df = pd.read_csv(solar_file)

# Convert to datetime
wind_df['datetime'] = pd.to_datetime(wind_df['OPR_DATE'])
solar_df['datetime'] = pd.to_datetime(solar_df['OPR_DATE'])

# Calculate daily totals
wind_daily = wind_df.groupby(wind_df['datetime'].dt.date)['VALUE'].sum()
solar_daily = solar_df.groupby(solar_df['datetime'].dt.date)['VALUE'].sum()

# Plot
fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(14, 10))

ax1.plot(wind_daily.index, wind_daily.values, 'b-', linewidth=2)
ax1.set_ylabel('Wind Generation (MWh)', fontsize=12)
ax1.set_title('January 2024 Renewable Generation - CAISO', fontsize=14)
ax1.grid(True, alpha=0.3)

ax2.plot(solar_daily.index, solar_daily.values, 'orange', linewidth=2)
ax2.set_ylabel('Solar Generation (MWh)', fontsize=12)
ax2.set_xlabel('Date', fontsize=12)
ax2.grid(True, alpha=0.3)

plt.tight_layout()
plt.savefig('caiso_renewables_jan2024.png', dpi=300)
print("✓ Analysis complete")

Data Quality & Limitations

Update Schedule

  • DAM LMP: Available by 1 PM PT for next operating day

  • RTM LMP: Published within 1 hour of operating interval

  • Load Forecasts: Updated multiple times daily

  • AS Prices: Available shortly after market clearing

Historical Data Availability

Data Type

Start Date

Notes

LMP (All markets)

April 2009

Complete history

AS Prices

April 2009

Complete history

Load Forecasts

January 2010

Earlier data may be sparse

Wind/Solar

March 2015

When tracking began

EIM Data

October 2014

When EIM launched

Flex Ramping

November 2016

Product launch date

Known Issues

  1. Holiday Data: Some holidays may have partial or delayed data

  2. Market Reruns: Occasionally markets are re-run; initial results may be revised

  3. Daylight Saving Time: Spring forward (23 hours) and fall back (25 hours) days handled automatically

  4. Location Changes: Some pricing nodes are renamed or retired; check CAISO’s Master File

Data Validation

import pandas as pd

def validate_caiso_data(file_path):
    """Basic validation checks for CAISO data."""
    df = pd.read_csv(file_path)

    # Check for required columns
    required_cols = ['OPR_DATE', 'INTERVAL_NUM', 'VALUE']
    missing = set(required_cols) - set(df.columns)
    if missing:
        print(f"❌ Missing columns: {missing}")
        return False

    # Check for nulls
    null_counts = df[required_cols].isnull().sum()
    if null_counts.any():
        print(f"⚠️  Null values found: {null_counts[null_counts > 0]}")

    # Check interval count
    intervals_per_day = df.groupby('OPR_DATE')['INTERVAL_NUM'].nunique()
    expected = 24  # or 288 for 5-min data
    irregular = intervals_per_day[intervals_per_day != expected]
    if len(irregular) > 0:
        print(f"⚠️  Irregular interval counts on: {irregular.index.tolist()}")

    # Check for negative prices (unusual but possible)
    if (df['VALUE'] < 0).any():
        neg_count = (df['VALUE'] < 0).sum()
        print(f"⚠️  {neg_count} negative prices found (may be valid)")

    print("✓ Validation complete")
    return True

# Example usage
validate_caiso_data('data/CAISO/20240101_to_20240131_PRC_LMP_TH_NP15_GEN-APND.csv')

Performance Tips

1. Optimize Date Ranges

Large date ranges can be slow. Break into chunks:

from datetime import date, timedelta
from lib.iso.caiso import CAISOClient, Market

def download_large_range(start, end, chunk_days=30):
    """Download in monthly chunks."""
    client = CAISOClient()

    current = start
    while current < end:
        chunk_end = min(current + timedelta(days=chunk_days), end)
        print(f"Downloading {current} to {chunk_end}")
        client.get_lmp(Market.DAM, current, chunk_end)
        current = chunk_end + timedelta(days=1)

    client.cleanup()

# Download full year in monthly chunks
download_large_range(date(2024, 1, 1), date(2024, 12, 31))

2. Adjust Step Size

Control requests per API call:

client = CAISOClient()

# Smaller step size for more stable downloads
client.get_lmp(
    market=Market.DAM,
    start_date=date(2024, 1, 1),
    end_date=date(2024, 12, 31),
    step_size=1  # 1 day per request (default)
)

3. Parallel Downloads

Download multiple markets simultaneously:

from concurrent.futures import ThreadPoolExecutor
from datetime import date
from lib.iso.caiso import CAISOClient, Market

def download_market(market, start, end):
    client = CAISOClient()
    result = client.get_lmp(market, start, end)
    client.cleanup()
    return market, result

markets = [Market.DAM, Market.HASP, Market.RTM]
start = date(2024, 1, 1)
end = date(2024, 1, 31)

with ThreadPoolExecutor(max_workers=3) as executor:
    futures = [executor.submit(download_market, m, start, end) for m in markets]
    results = [f.result() for f in futures]

for market, success in results:
    print(f"{market.value}: {'✓' if success else '✗'}")

Troubleshooting

Issue: “API Error 404: Data not found”

Possible Causes:

  • Date is too recent (data not yet published)

  • Date is before data collection began

  • Market was not operating on that date (holiday)

Solutions:

# Check date range is valid
from datetime import date, timedelta

# Use a date that's at least 2 days old
safe_date = date.today() - timedelta(days=2)
client.get_lmp(Market.DAM, safe_date, safe_date)

Issue: Slow Downloads

Solutions:

Issue: Missing Intervals

Some days may have 23 or 25 hours due to Daylight Saving Time.

Detection:

df = pd.read_csv('your_file.csv')
intervals_by_day = df.groupby('OPR_DATE')['INTERVAL_NUM'].nunique()
print(intervals_by_day[intervals_by_day != 24])  # Find irregular days

Handling:

# Fill missing intervals with NaN
df = df.set_index(['OPR_DATE', 'INTERVAL_NUM'])
df = df.reindex(pd.MultiIndex.from_product([
    df.index.levels[0],
    range(1, 25)  # Always 1-24
], names=['OPR_DATE', 'INTERVAL_NUM']))
df = df.reset_index()

Additional Resources

Next Steps

  • pricing - CAISO Pricing Data Deep Dive

  • load - Load Data Tutorial

  • generation - Generation Analysis Examples

  • ISO Coverage - Return to ISO Index