{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Calculate CSDI: Cold-spell duration index\n", "\n", "## Cold-spell duration index using the Copernicus Change Service (C3S)\n", "\n", "### About\n", "\n", "This tutorial will demonstrate how to calculate a climate index using a specific climate indices package. The example provided is of the Cold-spell duration index, but similar principles can be applied to many other types of single or multi-variable indices available in the icclim package.\n", "\n", "The data is provided by Copernicus Climate Change Service (C3S) and includes daily gridded meteorological data for Europe from 1950 to present derived from in-situ observations (E-OBS) of maximum temperature, minimum temperature, and precipitation.\n", "\n", "The tutorial will first show how to download the necessary data from the C3S Climate Data Store (CDS). It will then describe how to calculate the Cold-spell duration index, and finally plot a map of the Cold-spell duration index climatology over Europe.\n", "\n", "The steps shown in this tutorial can be applied to other climate datasets with the proper variable to calculate specific climate indices. For the current index, Cold-spell duration index, the Daily Minimum Temperature is needed." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### How to access the notebook\n", "\n", "This tutorial is in the form of a [Jupyter notebook](https://jupyter.org/)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "|NBViewer|\n", "|:-:|\n", "[![NBViewer](https://raw.githubusercontent.com/ecmwf-projects/copernicus-training/master/img/nbviewer_badge.svg)](https://nbviewer.org/github/cerfacs-globc/copernicus-training/blob/master/C3S_climate-indices_icclim.ipynb)|\n", "|(this will not run the notebook, only render it)|" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To run this notebook in your own environment, we suggest you install [Anaconda](https://docs.anaconda.com/anaconda/install/), which contains most of the libraries you will need.\n", "You also need at least python version 3.8 for this notebook to work because of some requirements from some packages. \n", "\n", "You will need to install [icclim](https://github.com/cerfacs-globc/icclim) (`%pip install icclim`) for calculating the climate indices, and the CDS API (`%pip install cdsapi`) for downloading data programatically from the CDS. You will also need to install matplotlib (`%pip install matplotlib`) and cartopy (`%conda install cartopy`) to enable plotting of the results. The % character is to ensure installation occurs in your environment. The installation of cartopy requires a conda environment, if not already installed." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Search, download and view data\n", "\n", "Before we begin we must prepare our environment. This includes installing the Application Programming Interface (API) of the CDS, and importing the various python libraries that we will need.\n", "\n", "### Install CDS API\n", "\n", "To install the CDS API, run the following command if not already installed in your environment." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Collecting cdsapi\n", " Downloading cdsapi-0.6.1.tar.gz (13 kB)\n", " Preparing metadata (setup.py) ... \u001b[?25ldone\n", "\u001b[?25hRequirement already satisfied: requests>=2.5.0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from cdsapi) (2.31.0)\n", "Collecting tqdm (from cdsapi)\n", " Downloading tqdm-4.66.1-py3-none-any.whl.metadata (57 kB)\n", "\u001b[2K \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m57.6/57.6 kB\u001b[0m \u001b[31m3.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: charset-normalizer<4,>=2 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from requests>=2.5.0->cdsapi) (3.3.2)\n", "Requirement already satisfied: idna<4,>=2.5 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from requests>=2.5.0->cdsapi) (3.6)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from requests>=2.5.0->cdsapi) (2.1.0)\n", "Requirement already satisfied: certifi>=2017.4.17 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from requests>=2.5.0->cdsapi) (2023.11.17)\n", "Downloading tqdm-4.66.1-py3-none-any.whl (78 kB)\n", "\u001b[2K \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m78.3/78.3 kB\u001b[0m \u001b[31m13.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hBuilding wheels for collected packages: cdsapi\n", " Building wheel for cdsapi (setup.py) ... \u001b[?25ldone\n", "\u001b[?25h Created wheel for cdsapi: filename=cdsapi-0.6.1-py2.py3-none-any.whl size=12008 sha256=e799f784cb58986da46ff3acf770b584483ef3236654524dbc1952d5839874bb\n", " Stored in directory: /home/bzah/.cache/pip/wheels/f9/26/57/a49a681496dea59363312ae87ffa8397100f8e8f6bab3591ea\n", "Successfully built cdsapi\n", "Installing collected packages: tqdm, cdsapi\n", "Successfully installed cdsapi-0.6.1 tqdm-4.66.1\n", "Note: you may need to restart the kernel to use updated packages.\n" ] } ], "source": [ "%pip install cdsapi" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Install icclim\n", "\n", "To install icclim, run the following command if not already installed in your environment." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: icclim in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (6.6.0)\n", "Requirement already satisfied: numpy>=1.16 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (1.26.2)\n", "Requirement already satisfied: xarray>=2022.6 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (2023.10.1)\n", "Requirement already satisfied: xclim<=0.47,>=0.45 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (0.47.0)\n", "Requirement already satisfied: cf_xarray>=0.7.4 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (0.8.6)\n", "Requirement already satisfied: cftime>=1.4.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (1.6.3)\n", "Requirement already satisfied: dask[array] in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (2023.12.1)\n", "Requirement already satisfied: netCDF4>=1.5.7 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (1.6.5)\n", "Requirement already satisfied: psutil in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (5.9.7)\n", "Requirement already satisfied: zarr in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (2.16.1)\n", "Requirement already satisfied: rechunker!=0.4,>=0.3 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (0.5.2)\n", "Requirement already satisfied: fsspec in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (2023.12.2)\n", "Requirement already satisfied: pandas>=1.3 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (2.1.4)\n", "Requirement already satisfied: dateparser in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (1.2.0)\n", "Requirement already satisfied: pint in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (0.19.2)\n", "Requirement already satisfied: jinja2 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (3.1.2)\n", "Requirement already satisfied: certifi in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from netCDF4>=1.5.7->icclim) (2023.11.17)\n", "Requirement already satisfied: python-dateutil>=2.8.2 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from pandas>=1.3->icclim) (2.8.2)\n", "Requirement already satisfied: pytz>=2020.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from pandas>=1.3->icclim) (2023.3.post1)\n", "Requirement already satisfied: tzdata>=2022.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from pandas>=1.3->icclim) (2023.3)\n", "Requirement already satisfied: mypy-extensions in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from rechunker!=0.4,>=0.3->icclim) (1.0.0)\n", "Requirement already satisfied: packaging>=21.3 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from xarray>=2022.6->icclim) (23.2)\n", "Requirement already satisfied: boltons>=20.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from xclim<=0.47,>=0.45->icclim) (23.0.0)\n", "Requirement already satisfied: bottleneck>=1.3.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from xclim<=0.47,>=0.45->icclim) (1.3.7)\n", "Requirement already satisfied: Click>=8.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from xclim<=0.47,>=0.45->icclim) (8.1.7)\n", "Requirement already satisfied: jsonpickle in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from xclim<=0.47,>=0.45->icclim) (3.0.2)\n", "Requirement already satisfied: lmoments3>=1.0.5 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from xclim<=0.47,>=0.45->icclim) (1.0.6)\n", "Requirement already satisfied: numba in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from xclim<=0.47,>=0.45->icclim) (0.58.1)\n", "Requirement already satisfied: pyyaml in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from xclim<=0.47,>=0.45->icclim) (6.0.1)\n", "Requirement already satisfied: scikit-learn>=0.21.3 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from xclim<=0.47,>=0.45->icclim) (1.3.2)\n", "Requirement already satisfied: scipy>=1.2 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from xclim<=0.47,>=0.45->icclim) (1.11.4)\n", "Requirement already satisfied: statsmodels in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from xclim<=0.47,>=0.45->icclim) (0.14.1)\n", "Requirement already satisfied: cloudpickle>=1.5.0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from dask[array]->icclim) (3.0.0)\n", "Requirement already satisfied: partd>=1.2.0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from dask[array]->icclim) (1.4.1)\n", "Requirement already satisfied: toolz>=0.10.0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from dask[array]->icclim) (0.12.0)\n", "Requirement already satisfied: importlib-metadata>=4.13.0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from dask[array]->icclim) (7.0.0)\n", "Requirement already satisfied: asciitree in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from zarr->icclim) (0.3.3)\n", "Requirement already satisfied: fasteners in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from zarr->icclim) (0.17.3)\n", "Requirement already satisfied: numcodecs>=0.10.0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from zarr->icclim) (0.12.1)\n", "Requirement already satisfied: regex!=2019.02.19,!=2021.8.27 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from dateparser->icclim) (2023.10.3)\n", "Requirement already satisfied: tzlocal in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from dateparser->icclim) (5.2)\n", "Requirement already satisfied: MarkupSafe>=2.0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from jinja2->icclim) (2.1.3)\n", "Requirement already satisfied: zipp>=0.5 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from importlib-metadata>=4.13.0->dask[array]->icclim) (3.17.0)\n", "Requirement already satisfied: locket in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from partd>=1.2.0->dask[array]->icclim) (1.0.0)\n", "Requirement already satisfied: six>=1.5 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas>=1.3->icclim) (1.16.0)\n", "Requirement already satisfied: joblib>=1.1.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from scikit-learn>=0.21.3->xclim<=0.47,>=0.45->icclim) (1.3.2)\n", "Requirement already satisfied: threadpoolctl>=2.0.0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from scikit-learn>=0.21.3->xclim<=0.47,>=0.45->icclim) (3.2.0)\n", "Requirement already satisfied: bokeh>=2.4.2 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from dask[array,diagnostics]->rechunker!=0.4,>=0.3->icclim) (3.3.2)\n", "Requirement already satisfied: llvmlite<0.42,>=0.41.0dev0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from numba->xclim<=0.47,>=0.45->icclim) (0.41.1)\n", "Requirement already satisfied: patsy>=0.5.4 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from statsmodels->xclim<=0.47,>=0.45->icclim) (0.5.4)\n", "Requirement already satisfied: contourpy>=1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from bokeh>=2.4.2->dask[array,diagnostics]->rechunker!=0.4,>=0.3->icclim) (1.2.0)\n", "Requirement already satisfied: pillow>=7.1.0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from bokeh>=2.4.2->dask[array,diagnostics]->rechunker!=0.4,>=0.3->icclim) (10.1.0)\n", "Requirement already satisfied: tornado>=5.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from bokeh>=2.4.2->dask[array,diagnostics]->rechunker!=0.4,>=0.3->icclim) (6.3.3)\n", "Requirement already satisfied: xyzservices>=2021.09.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from bokeh>=2.4.2->dask[array,diagnostics]->rechunker!=0.4,>=0.3->icclim) (2023.10.1)\n", "Note: you may need to restart the kernel to use updated packages.\n" ] } ], "source": [ "%pip install icclim" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Install matplotlib and cartopy\n", "\n", "To install matplotlib and cartopy to enable plotting, run the following command if not already installed in your environment.\n", "A conda environment is expected as cartopy cannot be correctly installed with pip." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: matplotlib in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (3.8.2)\n", "Requirement already satisfied: contourpy>=1.0.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from matplotlib) (1.2.0)\n", "Requirement already satisfied: cycler>=0.10 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from matplotlib) (0.12.1)\n", "Requirement already satisfied: fonttools>=4.22.0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from matplotlib) (4.46.0)\n", "Requirement already satisfied: kiwisolver>=1.3.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from matplotlib) (1.4.5)\n", "Requirement already satisfied: numpy<2,>=1.21 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from matplotlib) (1.26.2)\n", "Requirement already satisfied: packaging>=20.0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from matplotlib) (23.2)\n", "Requirement already satisfied: pillow>=8 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from matplotlib) (10.1.0)\n", "Requirement already satisfied: pyparsing>=2.3.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from matplotlib) (3.1.1)\n", "Requirement already satisfied: python-dateutil>=2.7 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from matplotlib) (2.8.2)\n", "Requirement already satisfied: six>=1.5 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from python-dateutil>=2.7->matplotlib) (1.16.0)\n", "Note: you may need to restart the kernel to use updated packages.\n" ] } ], "source": [ "%pip install matplotlib" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Collecting package metadata (current_repodata.json): done\n", "Solving environment: - \n", "The environment is inconsistent, please check the package plan carefully\n", "The following packages are causing the inconsistency:\n", "\n", " - conda-forge/linux-64::libcurl==8.5.0=hca28451_0\n", " - conda-forge/linux-64::libedit==3.1.20191231=he28a2e2_2\n", " - conda-forge/linux-64::netcdf4==1.6.5=nompi_py311he8ad708_100\n", " - conda-forge/linux-64::proj==9.3.1=h1d62c97_0\n", " - conda-forge/linux-64::libarrow-substrait==14.0.1=h61ff412_10_cpu\n", " - conda-forge/noarch::dask==2023.12.1=pyhd8ed1ab_0\n", " - conda-forge/linux-64::libarrow-flight==14.0.1=h120cb0d_10_cpu\n", " - conda-forge/linux-64::libarrow-dataset==14.0.1=h59595ed_10_cpu\n", " - conda-forge/linux-64::hdf5==1.14.3=nompi_h4f84152_100\n", " - conda-forge/linux-64::krb5==1.21.2=h659d440_0\n", " - conda-forge/linux-64::libarrow-gandiva==14.0.1=hacb8726_10_cpu\n", " - conda-forge/linux-64::pyarrow==14.0.1=py311h39c9aba_10_cpu\n", " - conda-forge/linux-64::libcups==2.3.3=h4637d8d_4\n", " - conda-forge/linux-64::libnetcdf==4.9.2=nompi_h9612171_113\n", " - conda-forge/linux-64::pyproj==3.6.1=py311hca0b8b9_5\n", " - conda-forge/linux-64::cartopy==0.22.0=py311h320fe9a_1\n", " - conda-forge/linux-64::libarrow-flight-sql==14.0.1=h61ff412_10_cpu\n", " - conda-forge/linux-64::libgoogle-cloud==2.12.0=h5206363_4\n", " - conda-forge/linux-64::xclim==0.47.0=py311h38be061_0\n", " - conda-forge/linux-64::libarrow==14.0.1=hfb4d3a9_10_cpu\n", " - conda-forge/linux-64::qt-main==5.15.8=h450f30e_18\n", " - conda-forge/noarch::pyarrow-hotfix==0.6=pyhd8ed1ab_0\n", " - conda-forge/linux-64::aws-sdk-cpp==1.11.210=h0853bfa_5\n", " - conda-forge/linux-64::libpq==16.1=h33b98f1_7\n", " - conda-forge/linux-64::libparquet==14.0.1=h352af49_10_cpu\n", " - conda-forge/linux-64::libarrow-acero==14.0.1=h59595ed_10_cpu\n", " - conda-forge/linux-64::matplotlib==3.8.2=py311h38be061_0\n", " - conda-forge/linux-64::pyqt==5.15.9=py311hf0fb5b6_5\n", "failed with initial frozen solve. Retrying with flexible solve.\n", "Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.\n", "Collecting package metadata (repodata.json): done\n", "Solving environment: | \n", "The environment is inconsistent, please check the package plan carefully\n", "The following packages are causing the inconsistency:\n", "\n", " - conda-forge/linux-64::libcurl==8.5.0=hca28451_0\n", " - conda-forge/linux-64::libedit==3.1.20191231=he28a2e2_2\n", " - conda-forge/linux-64::netcdf4==1.6.5=nompi_py311he8ad708_100\n", " - conda-forge/linux-64::proj==9.3.1=h1d62c97_0\n", " - conda-forge/linux-64::libarrow-substrait==14.0.1=h61ff412_10_cpu\n", " - conda-forge/noarch::dask==2023.12.1=pyhd8ed1ab_0\n", " - conda-forge/linux-64::libarrow-flight==14.0.1=h120cb0d_10_cpu\n", " - conda-forge/linux-64::libarrow-dataset==14.0.1=h59595ed_10_cpu\n", " - conda-forge/linux-64::hdf5==1.14.3=nompi_h4f84152_100\n", " - conda-forge/linux-64::krb5==1.21.2=h659d440_0\n", " - conda-forge/linux-64::libarrow-gandiva==14.0.1=hacb8726_10_cpu\n", " - conda-forge/linux-64::pyarrow==14.0.1=py311h39c9aba_10_cpu\n", " - conda-forge/linux-64::libcups==2.3.3=h4637d8d_4\n", " - conda-forge/linux-64::libnetcdf==4.9.2=nompi_h9612171_113\n", " - conda-forge/linux-64::pyproj==3.6.1=py311hca0b8b9_5\n", " - conda-forge/linux-64::cartopy==0.22.0=py311h320fe9a_1\n", " - conda-forge/linux-64::libarrow-flight-sql==14.0.1=h61ff412_10_cpu\n", " - conda-forge/linux-64::libgoogle-cloud==2.12.0=h5206363_4\n", " - conda-forge/linux-64::xclim==0.47.0=py311h38be061_0\n", " - conda-forge/linux-64::libarrow==14.0.1=hfb4d3a9_10_cpu\n", " - conda-forge/linux-64::qt-main==5.15.8=h450f30e_18\n", " - conda-forge/noarch::pyarrow-hotfix==0.6=pyhd8ed1ab_0\n", " - conda-forge/linux-64::aws-sdk-cpp==1.11.210=h0853bfa_5\n", " - conda-forge/linux-64::libpq==16.1=h33b98f1_7\n", " - conda-forge/linux-64::libparquet==14.0.1=h352af49_10_cpu\n", " - conda-forge/linux-64::libarrow-acero==14.0.1=h59595ed_10_cpu\n", " - conda-forge/linux-64::matplotlib==3.8.2=py311h38be061_0\n", " - conda-forge/linux-64::pyqt==5.15.9=py311hf0fb5b6_5\n", "failed with initial frozen solve. Retrying with flexible solve.\n", "Solving environment: - \n", "Found conflicts! Looking for incompatible packages.\n", "This can take several minutes. Press CTRL-C to abort.\n", "Examining conflict for cairo cached_property more-itertools tornado flake8-rst/ ^C\n", " failed\n", "\n", "CondaError: KeyboardInterrupt\n", "\n", "\n", "Note: you may need to restart the kernel to use updated packages.\n" ] } ], "source": [ "%conda install -y cartopy" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Import libraries\n", "\n", "We will be working with data in NetCDF format and calculating climate indices. We will use the icclim package and its dependencies for working with multidimensional arrays, in particular Xarray. We will also need libraries for plotting and viewing data, in this case we will use Matplotlib and Cartopy." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import sys\n", "\n", "import cartopy.crs as ccrs\n", "\n", "# CDS API\n", "import cdsapi\n", "\n", "# icclim package for calculating climate indices\n", "import icclim\n", "\n", "# Libraries for plotting and visualising data\n", "import matplotlib.pyplot as plt\n", "\n", "# Libraries for working with multidimensional arrays\n", "import numpy as np\n", "\n", "# Disable warnings for data download via API\n", "import urllib3\n", "import xarray as xr\n", "\n", "# To add specific units spelling\n", "\n", "urllib3.disable_warnings()\n", "\n", "# To unzip dataset files from the CDS\n", "from zipfile import ZipFile\n", "\n", "import dask\n", "\n", "# Dask" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "python: 3.11.7 | packaged by conda-forge | (main, Dec 15 2023, 08:38:37) [GCC 12.3.0]\n", "numpy: 1.26.2\n", "xarray: 2023.10.1\n", "icclim: 6.6.0\n", "dask: 2023.12.1\n" ] } ], "source": [ "print(\"python: \", sys.version)\n", "print(\"numpy: \", np.__version__)\n", "print(\"xarray: \", xr.__version__)\n", "print(\"icclim: \", icclim.__version__)\n", "print(\"dask: \", dask.__version__)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Enter your CDS API key\n", "\n", "We will request data from the Climate Data Store (CDS) programmatically with the help of the CDS API. Let us make use of the option to manually set the CDS API credentials. First, you have to define two variables: `URL` and `KEY` which build together your CDS API key. The string of characters that make up your KEY include your personal User ID and CDS API key. To obtain these, first register or login to the CDS (http://cds.climate.copernicus.eu), then visit https://cds.climate.copernicus.eu/api-how-to and copy the string of characters listed after \"key:\". Replace the `#########` below with this string." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "URL = \"https://cds.climate.copernicus.eu/api/v2\"\n", "KEY = \"118084:b3d0c1de-08ba-49d2-9f26-9bf464cd87ba\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here we specify a data directory in which we will download our data and all output files that we will generate:" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "DATADIR = \"./data\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Search for climate data to calculate the Cold-spell duration index\n", "\n", "The Cold-spell duration index we will calculate takes one parameter as input, it is the daily minimum temperature. Data for this parameter is available as part of the E-OBS daily gridded meteorological data for Europe from 1950 to present, but we will select a shorter period so that the download is faster: we will select the period from 2011 to 2021. We will search for this data on the CDS website: http://cds.climate.copernicus.eu. The specific dataset we will use is the `E-OBS daily gridded meteorological data for Europe from 1950 to present derived from in-situ observations`. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Having selected the dataset, we now need to specify what product type, variables, temporal and geographic coverage we are interested in. These can all be selected in the **\"Download data\"** tab. In this tab a form appears in which we will select the following parameters to download:\n", "\n", "- Product type: `Ensemble mean`\n", "- Variable: `Minimum temperature`\n", "- Grid resolution: `0.1deg`\n", "- Period: `2011_2021`\n", "- Version: `25.0e` (Latest version)\n", "- Format: `Zip file (.zip)`\n", "\n", "At the end of the download form, select **\"Show API request\"**. This will reveal a block of code, which you can simply copy and paste into a cell of your Jupyter Notebook (see cells below). You will do this once for Minimum temperature.\n", "\n", "### Download data\n", "\n", "... having copied the API request into the cell below, running this will retrieve and download the data you requested into your local directory. However, before you run the cell below, the **terms and conditions** of this particular dataset need to have been accepted in the CDS. The option to view and accept these conditions is given at the end of the download form, just above the **\"Show API request\"** option." ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2024-01-22 11:33:46,264 INFO Welcome to the CDS\n", "2024-01-22 11:33:46,264 Welcome to the CDS\n", "2024-01-22 11:33:46,267 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/insitu-gridded-observations-europe\n", "2024-01-22 11:33:46,267 Sending request to https://cds.climate.copernicus.eu/api/v2/resources/insitu-gridded-observations-europe\n", "2024-01-22 11:33:46,394 INFO Request is completed\n", "2024-01-22 11:33:46,394 Request is completed\n", "2024-01-22 11:33:46,396 INFO Downloading https://download-0011-clone.copernicus-climate.eu/cache-compute-0011/cache/data5/dataset-insitu-gridded-observations-europe-7854fb4f-0ba3-450f-ba4b-916089cafeba.zip to ./data/eobs_tasmin.zip (752.4M)\n", "2024-01-22 11:33:46,396 Downloading https://download-0011-clone.copernicus-climate.eu/cache-compute-0011/cache/data5/dataset-insitu-gridded-observations-europe-7854fb4f-0ba3-450f-ba4b-916089cafeba.zip to ./data/eobs_tasmin.zip (752.4M)\n", "2024-01-22 11:34:54,570 INFO Download rate 11M/s \n", "2024-01-22 11:34:54,570 Download rate 11M/s\n" ] }, { "data": { "text/plain": [ "Result(content_length=788898271,content_type=application/zip,location=https://download-0011-clone.copernicus-climate.eu/cache-compute-0011/cache/data5/dataset-insitu-gridded-observations-europe-7854fb4f-0ba3-450f-ba4b-916089cafeba.zip)" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "c = cdsapi.Client(url=URL, key=KEY)\n", "\n", "# For the full period, use period: 'full_period' but it takes a long time to download data\n", "# To download last decade use 'period': '2011_2021',\n", "\n", "c.retrieve(\n", " \"insitu-gridded-observations-europe\",\n", " {\n", " \"format\": \"zip\",\n", " \"product_type\": \"ensemble_mean\",\n", " \"variable\": \"minimum_temperature\",\n", " \"grid_resolution\": \"0.1deg\",\n", " \"period\": \"2011_2021\",\n", " \"version\": \"25.0e\",\n", " },\n", " f\"{DATADIR}/eobs_tasmin.zip\",\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Inspect Data\n", "\n", "We have requested the data in a zip archive. This zip archive contains a file in the NetCDF format. This is a commonly used format for array-oriented scientific data. To read and process this data we will make use of the underlying Xarray library that is used by the software to calculate the climate index. Xarray is an open source project and Python package that makes working with labelled multi-dimensional arrays simple and efficient. We will uncompress the archive and retrieve the filename(s). The archive could contain several files, but since we requested only one time period, we have a list of one file." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "['tn_ens_mean_0.1deg_reg_2011-2021_v25.0e.nc']\n" ] } ], "source": [ "# Create a ZipFile Object and load eobs_pr.zip in it\n", "with ZipFile(f\"{DATADIR}/eobs_tasmin.zip\", \"r\") as zipObj:\n", " # Get a list of all archived file names from the zip\n", " listOfFileNames = zipObj.namelist()\n", " # Extract all the contents of zip file in current directory\n", " zipObj.extractall()\n", "\n", "# List the NetCDF filenames of the dataset\n", "print(listOfFileNames)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Calculate Cold-spell duration index using icclim" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's calculate the Cold-spell duration index (CSDI) function using the [icclim](https://icclim.readthedocs.io/en/stable/) software. We retrieve the calculated values in an Xarray dataset, but alternatively we could also write the values automatically in an output NetCDF file using the keyword out_file when calling the icclim function." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Add out_file='out_icclim.nc' to also output data into a NetCDF file\n", "csdi = icclim.csdi(in_files=listOfFileNames[0], slice_mode=\"year\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Before we plot the results, we can query our newly created Xarray dataset ..." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
<xarray.Dataset>\n", "Dimensions: (time: 11, longitude: 705, latitude: 465, bounds: 2)\n", "Coordinates:\n", " * time (time) datetime64[ns] 2011-07-02 ... 2021-07-02\n", " * longitude (longitude) float64 -24.95 -24.85 -24.75 ... 45.25 45.35 45.45\n", " * latitude (latitude) float64 25.05 25.15 25.25 ... 71.25 71.35 71.45\n", " * bounds (bounds) int64 0 1\n", "Data variables:\n", " CSDI (time, latitude, longitude) float64 dask.array<chunksize=(1, 93, 141), meta=np.ndarray>\n", " time_bounds (time, bounds) datetime64[ns] 2011-01-01 ... 2021-12-31\n", "Attributes:\n", " title: cold index CSDI\n", " references: ATBD of the ECA&D indices calculation (https://knmi-ecad-as...\n", " institution: Climate impact portal (https://climate4impact.eu)\n", " history: Mon Mar 28 11:53:26 2022: ncks --no-abc -d time,22280,26297...\n", " source: \n", " Conventions: CF-1.6