"Earthquakes cannot be predicted" states the title of a controversial Science article from 1997¹. Nearly 25 years later, seismologists and statisticians no longer argue about whether earthquakes are predictable in principle – but how predictable they are. Debates rage over: What is the predictive skill of the Coulomb stress transfer theory? Do b-value changes of the Gutenberg-Richter law herald the end of an earthquake sequence? Is there life beyond ETAS, the widely popular and statistical Epidemic Type Aftershock Sequence model of seismicity patterns, which has become the seismological equivalent of the ‘standard model’ of particle physics?

We—and many in the community—believe that answers to these questions require more than ‘conventional’ peer-reviewed earthquake science; they require carefully designed forecasting experiments that pitch quantitative expressions of competing scientific hypotheses against observations and data yet to be collected. Only prospective experiments, with zero degrees of freedom, pre-agreed authoritative data sources in specified testing regions, can truly quantify the predictive skill of a model. Other disciplines have shown that this type of confirmatory research (as opposed to exploratory research) also contains methodological defenses against irreproducible science².

Enter CSEP -  the Collaboratory for the Study of Earthquake Predictability. CSEP is an international effort to provide the cyberinfrastructure for conducting prospective and retrospective forecasting experiments in natural laboratories around the world, including California, Italy, Japan, and New Zealand^3,4. Supported by a range of funders, notably the W.M. Keck Foundation, the European Union’s Horizon 2020 program, and the U.S. Geological Survey, CSEP software development has been headquartered at SCEC since its birth in 2007 from the Regional Earthquake Likelihood Model (RELM) experiment⁵ in California. CSEP has made impressive accomplishments over the last decade⁶ and presents a community-endorsed framework for unbiased assessments of earthquake forecast models⁷.

The Collaboratory for the Study of Earthquake Predictability (CSEP) has supported the evaluation of over 400 different earthquake forecasting models worldwide since 2007, providing an unbiased assessment of models’ forecasting skill.

CSEP has supported the evaluation of over 400 different earthquake forecasting models worldwide since its inception. Enabling multi-region and global scale experiments is critical for advancing earthquake predictability, engaging a global community of earthquake scientists in the effort, and collecting sufficient data to allow meaningful testing of hypotheses about destructive earthquakes. 

PyCSEP Poster presented at SCEC2020, Poster #084

CSEP's current efforts are focused on evaluating candidate operational earthquake forecasting models of the USGS such as the Uniform California Earthquake Rupture Forecast with Epidemic-Type Aftershock Sequence (UCERF3-ETAS) model⁹ and supporting the Real-time earthquake risk reduction for a resilient Europe (RISE) project, which is developing a suite of new forecast models for Italy, Europe and the globe. As a true community effort, CSEP has embraced an open-source approach to code development.

We redeveloped the CSEP software as a modularized, open-source Python package freely available for anyone. This unified set of software tools not only allows earthquake forecasting experts to evaluate their forecasts. The toolkit also contributes to improving the open-source scientific software for the wider research community. The PyCSEP library was released on both PyPI and conda-forge, and is now available on GitHub (for those curious, PyCSEP documentation is available here). In the two months since its release, PyCSEP has received code contributions from developers from the international CSEP collaboration and is steadily growing an open-source software community. 

The new CSEP project website is under development to showcase research projects and participants involved with CSEP experiments, and provides announcements about the project. The website is actively being developed, so be sure to check it periodically for up-to-date information about new CSEP developments and activities. 

As CSEP looks to build upon the achievements over the last decade⁶, we aim to build a strong community of engaged forecasting researchers to advance the predictability of earthquakes. A strong research community is exemplified through the software development practices around PyCSEP, which have successfully introduced and engaged forecast modelers to the world of open-source software. We believe PyCSEP can help foster a community of seismologists that can accelerate predictability research through shared codes, benchmark datasets and reproducible model evaluations. After all, we will need trusted, community-vetted forecast models if we want to confidently exploit their predictive information for decision-making purposes under uncertainty.  

About the Authors

William Savran is a SCEC software engineer at the University of Southern California. He is the lead developer of the Collaboratory for the Study of Earthquake Predictability, and works with researchers around the world to develop and implement methods for unbiased evaluations of earthquake forecasting models in California and beyond.

Max Werner is Associate Professor (Reader) of Geophysics and Natural Hazards at the University of Bristol (UK), where he leads a diverse research group investigating earthquake processes, interactions, predictability and hazards. He leads the SCEC node of the global Collaboratory for the Study of Earthquake Predictability (CSEP), which provides tools, concepts and a software platform for testing earthquake forecasts and predictions.

Acknowledgements

This research was supported by the Southern California Earthquake Center, which is funded by NSF Cooperative Agreement EAR-1600087 and USGS Cooperative Agreement G17AC00047. The research also received funding from the European Union’s Horizon 2020 research and innovation program under Grant Agreement Number 821115, Real-Time Earthquake Risk Reduction for a Resilient Europe (RISE). We thank our international CSEP colleagues for collaborating in this research. 

References

1. Geller, R. J., Jackson, D. D., Kagan, Y. Y., and Mulargia, F. (1997). Earthquakes cannot be predicted. Science, 275(5306), 1616-1616. SCEC Contribution 404
2. National Academies of Sciences, Engineering, and Medicine. (2019). Reproducibility and replicability in science. National Academies Press. https://doi.org/10.17226/25303
3. Jordan, T. H. (2006). Earthquake predictability, brick by brick, Seismological Research Letters, 773-6. SCEC Contribution 10906
4. Michael, A. J., & Werner, M. J. (2018). Preface to the focus section on the Collaboratory for the Study of Earthquake Predictability (CSEP): New results and future directions. Seismological Research Letters, 89(4), 1226-1228. SCEC Contribution 8086
5. Field, E. H. (2007). Overview of the working group for the development of regional earthquake likelihood models (RELM). Seismological Research Letters, 78(1), 7-16. SCEC Contribution 10907
6. Schorlemmer, D., Werner, M. J., Marzocchi, W., Jordan, T. H., Ogata, Y., Jackson, D. D., Mak, S., Rhoades, D. A., Gerstenberger, M. C., Hirata, N., Liukis, M., Maechling, P. J., Strader, A., Taroni, M., Wiemer, S., Zechar, J. D., and Zhuang J. C. (2018). The Collaboratory for the Study of Earthquake Predictability: Achievements and Priorities, Seismological Research Letters 89 1305-1313. SCEC Contribution 8036
7. Zechar, J. D., Schorlemmer, D., Liukis, M., Yu, J., Euchner, F., Maechling, P. J., & Jordan, T. H. (2010). The Collaboratory for the Study of Earthquake Predictability perspective on computational earthquake science. Concurrency and Computation: Practice and Experience, 22(12), 1836-1847. SCEC Contribution 1226
8. Jordan, T. H., Chen, Y. T., Gasparini, P., Madariaga, R., Main, I., Marzocchi, W., and Papadopoulos, G. (2011). Operational earthquake forecasting. State of knowledge and guidelines for utilization, Annals of Geophysics, 54(4). SCEC Contribution 10908
9. Savran, W. H., Werner, M. J., Marzocchi, W., Rhoades, D. A., Jackson, D. D., Milner, K., Field, E., and Michael, A. (2020). Pseudoprospective Evaluation of UCERF3-ETAS Forecasts during the 2019 Ridgecrest Sequence, Bulletin of the Seismological Society of America 110 1799-1817. SCEC Contribution 10082