ecmwf_models package

Subpackages

Submodules

ecmwf_models.grid module

Common grid definitions for ECMWF model reanalysis products (regular gridded)

ecmwf_models.grid.ERA5_RegularImgLandGrid(res_lat: float = 0.25, res_lon: float = 0.25, bbox: Optional[Tuple[float, float, float, float]] = None) CellGrid[source]

Uses the 0.25 DEG ERA5 land mask to create a land grid of the same size, which also excluded Antarctica.

Parameters
  • res_lat (float, optional (default: 0.25)) – Grid resolution (in degrees) in latitude direction.

  • res_lon (float, optional (default: 0.25)) – Grid resolution (in degrees) in longitude direction.

  • bbox (tuple, optional (default: None)) – (min_lon, min_lat, max_lon, max_lat) - wgs84 bbox to cut the global grid to.

ecmwf_models.grid.ERA_IrregularImgGrid(lons: ndarray, lats: ndarray, bbox: Optional[Tuple[float, float, float, float]] = None) CellGrid[source]

Create a irregular grid from the passed coordinates.

ecmwf_models.grid.ERA_RegularImgGrid(res_lat: float = 0.25, res_lon: float = 0.25, bbox: Optional[Tuple[float, float, float, float]] = None) CellGrid[source]

Create regular cell grid for bounding box with the selected resolution.

Parameters
  • res_lat (float, optional (default: 0.25)) – Grid resolution (in degrees) in latitude direction.

  • res_lon (float, optional (default: 0.25)) – Grid resolution (in degrees) in longitude direction.

  • bbox (tuple, optional (default: None)) – (min_lon, min_lat, max_lon, max_lat) - wgs84 bbox to cut the global grid to.

Returns

CellGrid – Regular, CellGrid with 5DEG*5DEG cells for the passed bounding box.

Return type

CellGrid

ecmwf_models.grid.get_grid_resolution(lats: ~numpy.ndarray, lons: ~numpy.ndarray) -> (<class 'float'>, <class 'float'>)[source]

try to derive the grid resolution from given coords.

ecmwf_models.interface module

class ecmwf_models.interface.ERAGrbDs(root_path, product, parameter=('swvl1', 'swvl2'), subgrid=None, mask_seapoints=False, h_steps=(0, 6, 12, 18), array_1D=True)[source]

Bases: MultiTemporalImageBase

Reader for a stack of ERA grib files.

Parameters
  • root_path (string) – Root path where the data is stored

  • product (str) – ERA5 or ERAINT

  • parameter (list or str, optional (default: ['swvl1', 'swvl2'])) – Parameter or list of parameters to read

  • expand_grid (bool, optional (default: True)) – If the reduced gaussian grid should be expanded to a full gaussian grid.

tstamps_for_daterange(start_date, end_date)[source]

Get datetimes in the correct sub-daily resolution between 2 dates

Parameters
  • start_date (datetime) – Start datetime

  • end_date (datetime) – End datetime

Returns

timestamps – List of datetimes

Return type

list

class ecmwf_models.interface.ERAGrbImg(filename, product, parameter=('swvl1', 'swvl2'), subgrid=None, mask_seapoints=False, array_1D=True, mode='r')[source]

Bases: ImageBase

Base class for reader for a single ERA Grib file.

Parameters
  • filename (str) – Path to the image file to read.

  • product (str) – ERA5 or ERAINT

  • parameter (list or str, optional (default: ['swvl1', 'swvl2'])) – Name of parameters to read from the image file.

  • subgrid (pygeogrids.CellGrid, optional (default:None)) – Read only data for points of this grid and not global values.

  • mask_seapoints (bool, optional (default: False)) – Read the land-sea mask to mask points over water and set them to nan. This option needs the ‘lsm’ parameter to be in the file!

  • array_1D (bool, optional (default: False)) – Read data as list, instead of 2D array, used for reshuffling.

  • mode (str, optional (default: 'r')) – Mode in which to open the file, changing this can cause data loss. This argument should not be changed!

close()[source]

Close file.

flush()[source]

Flush data.

read(timestamp=None)[source]

Read data from the loaded image file.

Parameters

timestamp (datetime, optional (default: None)) – Specific date (time) to read the data for.

write(data)[source]

Write data to an image file.

Parameters

image (object) – pygeobase.object_base.Image object

class ecmwf_models.interface.ERANcDs(root_path, product, parameter=('swvl1', 'swvl2'), subgrid=None, mask_seapoints=False, h_steps=(0, 6, 12, 18), array_1D=False)[source]

Bases: MultiTemporalImageBase

Class for reading ERA 5 images in nc format.

Parameters
  • root_path (str) – Root path where image data is stored.

  • parameter (list or str, optional (default: ['swvl1', 'swvl2'])) – Parameter or list of parameters to read from image files.

  • subgrid (pygeogrids.CellGrid, optional (default: None)) – Read only data for points of this grid and not global values.

  • mask_seapoints (bool, optional (default: False)) – Use the land-sea-mask parameter in the file to mask points over water.

  • h_steps (list, optional (default: [0,6,12,18])) – List of full hours for which images exist.

  • array_1D (bool, optional (default: False)) – Read data as list, instead of 2D array, used for reshuffling.

tstamps_for_daterange(start_date, end_date)[source]

Get datetimes in the correct sub-daily resolution between 2 dates

Parameters
  • start_date (datetime) – Start datetime

  • end_date (datetime) – End datetime

Returns

timestamps – List of datetimes

Return type

list

class ecmwf_models.interface.ERANcImg(filename, product, parameter=['swvl1', 'swvl2'], subgrid=None, mask_seapoints=False, array_1D=False, mode='r')[source]

Bases: ImageBase

Reader for a single ERA netcdf file.

Parameters
  • filename (str) – Path to the image file to read.

  • product (str) – ‘era5’ or ‘era5-land’ or ‘eraint’

  • parameter (list or str, optional (default: ['swvl1', 'swvl2'])) – Name of parameters to read from the image file.

  • subgrid (pygeogrids.CellGrid, optional (default: None)) – Read only data for points of this grid and not global values.

  • mask_seapoints (bool, optional (default: False)) – Read the land-sea mask to mask points over water and set them to nan. This option needs the ‘lsm’ parameter to be in the file!

  • array_1D (bool, optional (default: False)) – Read data as list, instead of 2D array, used for reshuffling.

  • mode (str, optional (default: 'r')) – Mode in which to open the file, changing this can cause data loss. This argument should not be changed!

close()[source]

Close file.

flush()[source]

Flush data.

read(timestamp=None)[source]

Read data from the loaded image file.

Parameters

timestamp (datetime, optional (default: None)) – Specific date (time) to read the data for.

write(data)[source]

Write data to an image file.

Parameters

image (object) – pygeobase.object_base.Image object

class ecmwf_models.interface.ERATs(ts_path, grid_path=None, **kwargs)[source]

Bases: GriddedNcOrthoMultiTs

Time series reader for all reshuffled ERA reanalysis products in time series format. Use the read_ts(lon, lat) resp. read_ts(gpi) function of this class to read data for locations!

Parameters
  • ts_path (str) – Directory where the netcdf time series files are stored

  • grid_path (str, optional (default: None)) – Path to grid file, that is used to organize the location of time series to read. If None is passed, grid.nc is searched for in the ts_path.

  • used (Optional keyword arguments that are passed to the Gridded Base when) –

  • ------------------------------------------------------------------------

    parameterslist, optional (default: None)

    Specific variable names to read, if None are selected, all are read.

    offsetsdict, optional (default: None)

    Offsets (values) that are added to the parameters (keys)

    scale_factorsdict, optional (default: None)

    Offset (value) that the parameters (key) is multiplied with

    ioclass_kws: dict, (optional)

    read_bulkboolean, optional (default: False)

    If set to True, the data of all locations is read into memory, and subsequent calls to read_ts then read from cache and not from disk. This makes reading complete files faster.

    read_datesboolean, optional (default: False)

    If false, dates will not be read automatically but only on specific request useable for bulk reading because currently the netCDF num2date routine is very slow for big datasets.

ecmwf_models.utils module

Utility functions for all data products in this package.

exception ecmwf_models.utils.CdoNotFoundError(msg=None)[source]

Bases: ModuleNotFoundError

ecmwf_models.utils.get_default_params(name='era5')[source]

Read only lines that are marked as default variable in the csv file

Parameters

name (str) – Name of the product to get the default parameters for

ecmwf_models.utils.load_var_table(name='era5', lut=False)[source]

Load the variables table for supported variables to download.

Parameters

lut (bool, optional (default: False)) – If set to true only names are loaded, so that they can be used for a LUT otherwise the full table is loaded

ecmwf_models.utils.lookup(name, variables)[source]

Search the passed elements in the lookup table, if one does not exists, raise a Warning

ecmwf_models.utils.make_era5_land_definition_file(data_file, out_file, data_file_y_res=0.25, ref_var='lsm', threshold=0.5, exclude_antarctica=True)[source]

Create a land grid definition file from a variable within a downloaded, regular (netcdf) era5 file.

Parameters
  • data_file (str) – Path to the downloaded file that cotains the image that is used as the reference for creating the land definition file.

  • out_file (str) – Full output path to the land defintion file to create.

  • data_file_y_res (float, optional (default: 0.25)) – The resolution of the data file in latitude direction.

  • ref_var (str, optional (default: 'lsm')) – A variable in the data_file that is the reference for the land definiton. By default, we use the land-sea-mask variable.

  • threshold (float, optional (default: 0.5)) – Threshold value below which a point is declared water, and above (or equal) which it is declared a land-point. If None is passed, then a point is declared a land point if it is not masked (numpy masked array) in the reference variable.

  • exclude_antarctica (bool, optional (default: True)) – Cut off the definition file at -60° Lat to exclude Land Points in Antarctica.

ecmwf_models.utils.mkdate(datestring)[source]

Turns a datetime string into a datetime object

Parameters

datestring (str) – Input datetime string

Returns

datetime – Converted string

Return type

datetime

ecmwf_models.utils.parse_filetype(inpath)[source]

Tries to find out the file type by searching for grib or nc files two subdirectories into the passed input path. If function fails, grib is assumed.

Parameters

inpath (str) – Input path where ERA data was downloaded to

Returns

filetype – File type string.

Return type

str

ecmwf_models.utils.parse_product(inpath: str) str[source]

Tries to find out what product is stored in the path. This is done based on the name of the first file in the path that is found.

Parameters

inpath (str) – Input path where ERA data was downloaded to

Returns

product – Product name

Return type

str

ecmwf_models.utils.save_gribs_from_grib(input_grib, output_path, product_name, filename_templ='{product}_AN_%Y%m%d_%H%M.grb', keep_original=True)[source]

Split the downloaded grib file into daily files and add to folder structure necessary for reshuffling.

Parameters
  • input_grib (str) – Filepath of the downloaded .grb file

  • output_path (str) – Where to save the resulting grib files

  • product_name (str) – Name of the ECMWF model (only for filename generation)

  • filename_templ (str, optional (default: product_OPER_0001_AN_date_time)) – Template for naming each separated grb file

ecmwf_models.utils.save_ncs_from_nc(input_nc, output_path, product_name, filename_templ='{product}_AN_%Y%m%d_%H%M.nc', grid=None, keep_original=True, remap_method='bil')[source]

Split the downloaded netcdf file into daily files and add to folder structure necessary for reshuffling.

Parameters
  • input_nc (str) – Filepath of the downloaded .nc file

  • output_path (str) – Where to save the resulting netcdf files

  • product_name (str) – Name of the ECMWF model (only for filename generation)

  • filename_templ (str, optional (default: "{product}_AN_%Y%m%d_%H%M.nc")) – Template for naming each separated nc file

  • keep_original (bool) – keep the original downloaded data

ecmwf_models.utils.str2bool(v)[source]

Parse a string to True/False

Parameters

v (str) – String to parse, must be part of the lists below.

Returns

str2bool – The parsed bool from the passed string

Return type

bool

Module contents