{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "# Calculate percentage of days when Tmean > 90th percentile (TG90p)\n", "\n", "Example notebook that runs icclim.\n", "\n", "The example calculates the percentage of days when Tas > 90th percentile (TG90pindicator) for the dataset chosen by the user on C4I. \n", "We assume to have the **tas** variable in netCDF files in a `./data` folder for model `CMCC` and for one member `r1i1p1f1`. \n", "The data can be dowloaded using the [metalink](data/cmcc_tas.metalink) provided with this notebook.\n", "The data described in a `.metalink` file can be dowloaded with tools such as [aria2](https://aria2.github.io/) or a browser plugin such as [DownThemAll!](https://addons.mozilla.org/en-US/firefox/addon/downthemall/)\n", "If you wish to use a different dataset, you can use the [climate 4 impact portal](https://www.climate4impact.eu/c4i-frontend/) to search and select the data you wish to use and a metalink file to the [ESGF](https://esgf.llnl.gov/) data will be provided.\n", "\n", "The data is read using xarray and a plot of the time series over a specific region is generated, as well as an average spatial map. Several output types examples are shown.\n", "\n", "The dataset that is expected for this notebook are tas parameter (needed to calculate the TG90p indicator) for one specific climate model and experiment as well as one member. The time period should be continuous.\n", "\n", "The following time period is considered: 2081-01-01 to 2100-12-31 using the period 1981-01-01 to 2000-12-31 as a reference. Plots are shown over European region.\n", "\n", "### Install packages" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: icclim in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (6.6.0)\n", "Requirement already satisfied: matplotlib in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (3.8.2)\n", "Requirement already satisfied: nc_time_axis in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (1.4.1)\n", "Requirement already satisfied: numpy>=1.16 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (1.26.2)\n", "Requirement already satisfied: xarray>=2022.6 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (2023.10.1)\n", "Requirement already satisfied: xclim<=0.47,>=0.45 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (0.47.0)\n", "Requirement already satisfied: cf_xarray>=0.7.4 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (0.8.6)\n", "Requirement already satisfied: cftime>=1.4.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (1.6.3)\n", "Requirement already satisfied: dask[array] in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (2023.12.1)\n", "Requirement already satisfied: netCDF4>=1.5.7 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (1.6.5)\n", "Requirement already satisfied: psutil in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (5.9.7)\n", "Requirement already satisfied: zarr in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (2.16.1)\n", "Requirement already satisfied: rechunker!=0.4,>=0.3 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (0.5.2)\n", "Requirement already satisfied: fsspec in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (2023.12.2)\n", "Requirement already satisfied: pandas>=1.3 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (2.1.4)\n", "Requirement already satisfied: dateparser in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (1.2.0)\n", "Requirement already satisfied: pint in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (0.19.2)\n", "Requirement already satisfied: jinja2 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from icclim) (3.1.2)\n", "Requirement already satisfied: contourpy>=1.0.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from matplotlib) (1.2.0)\n", "Requirement already satisfied: cycler>=0.10 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from matplotlib) (0.12.1)\n", "Requirement already satisfied: fonttools>=4.22.0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from matplotlib) (4.46.0)\n", "Requirement already satisfied: kiwisolver>=1.3.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from matplotlib) (1.4.5)\n", "Requirement already satisfied: packaging>=20.0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from matplotlib) (23.2)\n", "Requirement already satisfied: pillow>=8 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from matplotlib) (10.1.0)\n", "Requirement already satisfied: pyparsing>=2.3.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from matplotlib) (3.1.1)\n", "Requirement already satisfied: python-dateutil>=2.7 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from matplotlib) (2.8.2)\n", "Requirement already satisfied: certifi in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from netCDF4>=1.5.7->icclim) (2023.11.17)\n", "Requirement already satisfied: pytz>=2020.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from pandas>=1.3->icclim) (2023.3.post1)\n", "Requirement already satisfied: tzdata>=2022.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from pandas>=1.3->icclim) (2023.3)\n", "Requirement already satisfied: six>=1.5 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from python-dateutil>=2.7->matplotlib) (1.16.0)\n", "Requirement already satisfied: mypy-extensions in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from rechunker!=0.4,>=0.3->icclim) (1.0.0)\n", "Requirement already satisfied: boltons>=20.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from xclim<=0.47,>=0.45->icclim) (23.0.0)\n", "Requirement already satisfied: bottleneck>=1.3.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from xclim<=0.47,>=0.45->icclim) (1.3.7)\n", "Requirement already satisfied: Click>=8.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from xclim<=0.47,>=0.45->icclim) (8.1.7)\n", "Requirement already satisfied: jsonpickle in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from xclim<=0.47,>=0.45->icclim) (3.0.2)\n", "Requirement already satisfied: lmoments3>=1.0.5 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from xclim<=0.47,>=0.45->icclim) (1.0.6)\n", "Requirement already satisfied: numba in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from xclim<=0.47,>=0.45->icclim) (0.58.1)\n", "Requirement already satisfied: pyyaml in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from xclim<=0.47,>=0.45->icclim) (6.0.1)\n", "Requirement already satisfied: scikit-learn>=0.21.3 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from xclim<=0.47,>=0.45->icclim) (1.3.2)\n", "Requirement already satisfied: scipy>=1.2 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from xclim<=0.47,>=0.45->icclim) (1.11.4)\n", "Requirement already satisfied: statsmodels in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from xclim<=0.47,>=0.45->icclim) (0.14.1)\n", "Requirement already satisfied: cloudpickle>=1.5.0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from dask[array]->icclim) (3.0.0)\n", "Requirement already satisfied: partd>=1.2.0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from dask[array]->icclim) (1.4.1)\n", "Requirement already satisfied: toolz>=0.10.0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from dask[array]->icclim) (0.12.0)\n", "Requirement already satisfied: importlib-metadata>=4.13.0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from dask[array]->icclim) (7.0.0)\n", "Requirement already satisfied: asciitree in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from zarr->icclim) (0.3.3)\n", "Requirement already satisfied: fasteners in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from zarr->icclim) (0.17.3)\n", "Requirement already satisfied: numcodecs>=0.10.0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from zarr->icclim) (0.12.1)\n", "Requirement already satisfied: regex!=2019.02.19,!=2021.8.27 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from dateparser->icclim) (2023.10.3)\n", "Requirement already satisfied: tzlocal in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from dateparser->icclim) (5.2)\n", "Requirement already satisfied: MarkupSafe>=2.0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from jinja2->icclim) (2.1.3)\n", "Requirement already satisfied: zipp>=0.5 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from importlib-metadata>=4.13.0->dask[array]->icclim) (3.17.0)\n", "Requirement already satisfied: locket in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from partd>=1.2.0->dask[array]->icclim) (1.0.0)\n", "Requirement already satisfied: joblib>=1.1.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from scikit-learn>=0.21.3->xclim<=0.47,>=0.45->icclim) (1.3.2)\n", "Requirement already satisfied: threadpoolctl>=2.0.0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from scikit-learn>=0.21.3->xclim<=0.47,>=0.45->icclim) (3.2.0)\n", "Requirement already satisfied: bokeh>=2.4.2 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from dask[array,diagnostics]->rechunker!=0.4,>=0.3->icclim) (3.3.2)\n", "Requirement already satisfied: llvmlite<0.42,>=0.41.0dev0 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from numba->xclim<=0.47,>=0.45->icclim) (0.41.1)\n", "Requirement already satisfied: patsy>=0.5.4 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from statsmodels->xclim<=0.47,>=0.45->icclim) (0.5.4)\n", "Requirement already satisfied: tornado>=5.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from bokeh>=2.4.2->dask[array,diagnostics]->rechunker!=0.4,>=0.3->icclim) (6.3.3)\n", "Requirement already satisfied: xyzservices>=2021.09.1 in /home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages (from bokeh>=2.4.2->dask[array,diagnostics]->rechunker!=0.4,>=0.3->icclim) (2023.10.1)\n", "Note: you may need to restart the kernel to use updated packages.\n" ] } ], "source": [ "%pip install icclim matplotlib nc_time_axis" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "python: 3.11.7 | packaged by conda-forge | (main, Dec 15 2023, 08:38:37) [GCC 12.3.0]\n", "numpy: 1.26.2\n", "pandas: 2.1.4\n", "xclim: 0.47.0\n", "dask: 2023.12.1\n", "xarray: 2023.10.1\n", "icclim: 6.6.0\n", "cftime: 1.6.3\n" ] } ], "source": [ "import datetime\n", "import sys\n", "from pathlib import Path\n", "\n", "import cartopy.crs as ccrs\n", "import cftime\n", "import dask\n", "import icclim\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import pandas as pd\n", "import xarray as xr\n", "\n", "print(\"python: \", sys.version)\n", "print(\"numpy: \", np.__version__)\n", "print(\"pandas: \", pd.__version__)\n", "\n", "\n", "import xclim\n", "\n", "print(\"xclim: \", xclim.__version__)\n", "print(\"dask: \", dask.__version__)\n", "print(\"xarray: \", xr.__version__)\n", "print(\"icclim: \", icclim.__version__)\n", "print(\"cftime: \", cftime.__version__)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Specification of index parameters" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['data/tas_day_CMCC-ESM2_historical_r1i1p1f1_gn_19250101-19491231.nc',\n", " 'data/tas_day_CMCC-ESM2_historical_r1i1p1f1_gn_19500101-19741231.nc',\n", " 'data/tas_day_CMCC-ESM2_historical_r1i1p1f1_gn_19750101-19991231.nc',\n", " 'data/tas_day_CMCC-ESM2_ssp585_r1i1p1f1_gn_20900101-21001231.nc',\n", " 'data/tas_day_CMCC-ESM2_historical_r1i1p1f1_gn_20000101-20141231.nc',\n", " 'data/tas_day_CMCC-ESM2_historical_r1i1p1f1_gn_19000101-19241231.nc',\n", " 'data/tas_day_CMCC-ESM2_ssp585_r1i1p1f1_gn_20400101-20641231.nc',\n", " 'data/tas_day_CMCC-ESM2_ssp585_r1i1p1f1_gn_20650101-20891231.nc',\n", " 'data/tas_day_CMCC-ESM2_historical_r1i1p1f1_gn_18500101-18741231.nc',\n", " 'data/tas_day_CMCC-ESM2_ssp585_r1i1p1f1_gn_20150101-20391231.nc',\n", " 'data/tas_day_CMCC-ESM2_historical_r1i1p1f1_gn_18750101-18991231.nc']" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# studied period\n", "dt1 = datetime.datetime(2081, 1, 1, tzinfo=datetime.timezone.utc)\n", "dt2 = datetime.datetime(2100, 12, 31, tzinfo=datetime.timezone.utc)\n", "\n", "# reference period\n", "dt1r = datetime.datetime(1981, 1, 1, tzinfo=datetime.timezone.utc)\n", "dt2r = datetime.datetime(2000, 12, 31, tzinfo=datetime.timezone.utc)\n", "\n", "out_f = \"tg90p_icclim.nc\"\n", "data_dir = Path(\"data\")\n", "filenames = [str(f) for f in data_dir.glob(\"tas_day_CMCC-ESM2*.nc\")]\n", "filenames" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2024-01-19 15:27:59,669 ********************************************************************************************\n", "2024-01-19 15:27:59,671 * *\n", "2024-01-19 15:27:59,672 * icclim 6.6.0 *\n", "2024-01-19 15:27:59,672 * *\n", "2024-01-19 15:27:59,673 * *\n", "2024-01-19 15:27:59,674 * Fri Jan 19 14:27:59 2024 *\n", "2024-01-19 15:27:59,674 * *\n", "2024-01-19 15:27:59,674 * BEGIN EXECUTION *\n", "2024-01-19 15:27:59,675 * *\n", "2024-01-19 15:27:59,675 ********************************************************************************************\n", "2024-01-19 15:27:59,676 Processing: 0%\n", "/home/bzah/workspace/cerfacs/icclim/src/icclim/generic_indices/generic_indicators.py:1319: UserWarning: Unable to infer the frequency of the time series. To mute this, set xclim's option data_validation='log'.\n", " check_freq(da, src_freq, strict=True)\n", "/home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages/xclim/core/cfchecks.py:41: UserWarning: Variable does not have a `cell_methods` attribute.\n", " _check_cell_methods(\n", "/home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages/xclim/core/cfchecks.py:45: UserWarning: Variable does not have a `standard_name` attribute.\n", " check_valid(vardata, \"standard_name\", data[\"standard_name\"])\n", "/home/bzah/micromamba/envs/icclim-dev/lib/python3.11/site-packages/distributed/client.py:3162: UserWarning: Sending large graph of size 108.75 MiB.\n", "This may cause some slowdown.\n", "Consider scattering data ahead of time and using futures.\n", " warnings.warn(\n", "2024-01-19 15:31:24,920 Processing: 100%\n", "2024-01-19 15:31:24,927 ********************************************************************************************\n", "2024-01-19 15:31:24,930 * *\n", "2024-01-19 15:31:24,931 * icclim 6.6.0 *\n", "2024-01-19 15:31:24,931 * *\n", "2024-01-19 15:31:24,932 * *\n", "2024-01-19 15:31:24,933 * Fri Jan 19 14:31:24 2024 *\n", "2024-01-19 15:31:24,934 * *\n", "2024-01-19 15:31:24,935 * END EXECUTION *\n", "2024-01-19 15:31:24,936 * *\n", "2024-01-19 15:31:24,936 * CP SECS = 63.326623434 *\n", "2024-01-19 15:31:24,937 * *\n", "2024-01-19 15:31:24,949 ********************************************************************************************\n" ] }, { "data": { "text/html": [ "
<xarray.Dataset>\n", "Dimensions: (lat: 192, lon: 288, time: 20, bounds: 2)\n", "Coordinates:\n", " * lat (lat) float64 -90.0 -89.06 -88.12 -87.17 ... 88.12 89.06 90.0\n", " * lon (lon) float64 0.0 1.25 2.5 3.75 5.0 ... 355.0 356.2 357.5 358.8\n", " height float64 2.0\n", " percentiles float64 90.0\n", " * time (time) object 2081-07-16 12:00:00 ... 2100-07-16 12:00:00\n", " * bounds (bounds) int64 0 1\n", "Data variables:\n", " TG90p (time, lat, lon) float64 dask.array<chunksize=(1, 192, 255), meta=np.ndarray>\n", " time_bounds (time, bounds) object 2081-06-01 00:00:00 ... 2100-08-31 00:...\n", "Attributes:\n", " title: number_of_days_when_average_air_temperature_is_greater_than...\n", " references: ATBD of the ECA&D indices calculation (https://knmi-ecad-as...\n", " institution: Climate impact portal (https://climate4impact.eu)\n", " history: 2020-12-21T16:22:42Z altered by CMOR: Treated scalar dimens...\n", " source: \n", " Conventions: CF-1.6
\n", " | lat | \n", "lon | \n", "height | \n", "percentiles | \n", "TG90p | \n", "
---|---|---|---|---|---|
time | \n", "\n", " | \n", " | \n", " | \n", " | \n", " |
2081-07-16 12:00:00 | \n", "43.82199 | \n", "3.75 | \n", "2.0 | \n", "90.0 | \n", "0.934783 | \n", "
2082-07-16 12:00:00 | \n", "43.82199 | \n", "3.75 | \n", "2.0 | \n", "90.0 | \n", "0.869565 | \n", "
2083-07-16 12:00:00 | \n", "43.82199 | \n", "3.75 | \n", "2.0 | \n", "90.0 | \n", "0.956522 | \n", "
2084-07-16 12:00:00 | \n", "43.82199 | \n", "3.75 | \n", "2.0 | \n", "90.0 | \n", "0.663043 | \n", "
2085-07-16 12:00:00 | \n", "43.82199 | \n", "3.75 | \n", "2.0 | \n", "90.0 | \n", "1.000000 | \n", "