ContrailBench¶
Benchmarking forecasts for contrail avoidance.
See Introducing ContrailBench: Measuring contrail forecasts against reality for more background on the framework and the methodology behind the first release.
Overview¶
Re-routing flights to avoid persistent contrails can significantly reduce aviation’s climate impact. This is only possible with forecasts that support effective avoidance at a reasonable cost.
ContrailBench is an open framework for evaluating contrail forecasts against real-world observations using metrics that capture forecast effectiveness and cost. This site hosts reports evaluating participating forecasts alongside links to the ContrailBench evaluation code (on Github) and observation-based evaluation datasets (on a public cloud bucket).
ContrailBench is an evolving framework. File a GitHub issue or write to info@contrails.org to share ideas and suggestions with the team.
Reports¶
Contrail forecasts can be evaluated using many different metrics. ContrailBench targets metrics that capture the tradeoff between effectiveness (i.e., the fraction of persistent contrail kilometers or contrail radiative forcing eliminated) and cost (i.e., the increase in operating expenses or CO2 emissions) when using a particular forecast. For a detailed primer on metrics used in ContrailBench reports, see our notebook post.
Static benchmark reports are released on a roughly quarterly cadence. Frequent releases allow reports to add new forecasts and evaluation datasets, expand observation coverage, and incorporate improvements to evaluation methods.
All available reports are listed below.
Report |
Coverage |
|---|---|
January-December 2024 |
Participating forecasts¶
Forecast |
Type |
Added |
|---|---|---|
Deterministic, physics-based |
v1 |
|
Probabilistic, ML-physics hybrid |
v1 |
For examples showing how to access participating forecasts, see Participating Forecasts.
Have a forecast you’d like to see included? Submit a GitHub issue or write to info@contrails.org and we’ll work with you to include it in future reports.
Evaluation datasets¶
Observation |
Type |
Added |
|---|---|---|
Geostationary Satellite |
v1 |
|
In-situ (radiosonde) |
v1 |
|
In-situ (aircraft) |
v1 |
We also provide a public ADS-B dataset used to compute cost metrics.
For examples showing how to access and use evaluation datasets, see Evaluation Datasets.
Have an observational dataset you’d like to see included? Submit a GitHub issue or write to info@contrails.org and we’ll assess its suitability.