Download Satellite data
Preparations
One strength of wavy is to ease obtaining and using satellite altimetry data. To get a flavor of how this can be done illustrated with cmems level 3 altimeter data please execute wavy’s command line download script:
$ conda activate wavyopen
$ wavyDownload --help
The help-message displayed would give you, among other information, the following options:
cmems_L3_NRT:
s3a - Sentinel-3A
s3b - Sentinel-3B
j3 - Jason-3 (deprecated reference mission)
c2 - Cryosat-2
al - SARAL/AltiKa
cfo - CFOSAT
h2b - HaiYang-2B
s6a - Sentinel-6A Michael Freilich (reference mission)
swon - SWOT nadir altimeter
This means that for product cmems_L3_NRT you can choose among 9 satellite missions. In the world of wavy, cmems_L3_NRT is called the nID (name ID) and the individual missions are names. These are arbitrary names. Although they could be choosen freely by each user, it makes sense to keep them descriptive. As for cmems data, unfortunatly, most of other openly available satellite altimeter data is not accessible via thredds or similar options but needs to be downloaded from e.g. an FTP server. To do that you would need the credentials for CEDA, or for the AVISO cataloque as these are the main other sources that wavy currently exploits. Both CEDA and AVISO are possible to use as source in wavy but are not included in the help function for wavyDownload due to the large amount of choices.
wavy relies on the coprenicusmarine toolbox for CMEMS products. In case of remote access via FTP wavy needs you to store the respective usernames and passwords in your local .netrc file. This could look like:
machine ftp.ceda.ac.uk login {USER} password {PASSWORD}
machine ftp-access.aviso.altimetry.fr login {USER} password {PASSWORD}
In case of using the copernicusmarine toolbox the user needs to make sure that the CMEMS credentials are available by storing them in the .env file, e.g. located in the project directory:
COPERNICUSMARINE_SERVICE_USERNAME=YOUR_COPERNICUS_USERNAME
COPERNICUSMARINE_SERVICE_PASSWORD=YOUR_COPERNICUS_PASSWORD
Download using config files
Ammending config files
In a validation context, especially operational, download operation needs to be performed many times and it makes sense to adjust the satellite_cfg.yaml file to your needs. Assuming you established a project directory and therein config directory you first establish a minimal satellite_cfg.yaml file:
$ cd ~/my_wavy_project/config
$ conda activate wavyopen
$ wavyCFG --path ~/my_wavy_project/config/. --f satellite_cfg.yaml --t minimal
It looks like this:
cmems_L3_NRT:
name:
s3a: s3a
s3b: s3b
c2: c2
j3: j3
h2b: h2b
al: al
cfo: cfo
s6a: s6a
swon: swon
download:
copernicus:
dataset_id: cmems_obs-wave_glo_phy-swh_nrt_name-l3_PT1S
trgt_tmplt:
path_date_incr_unit: 'm'
path_date_incr: 1
strsub: ["name"]
server: "nrt.cmems-du.eu"
time_incr: 'h'
wavy_input:
src_tmplt:
fl_tmplt:
strsub: ["name"]
path_date_incr_unit: 'm'
path_date_incr: 1
reader: read_local_ncfiles
collector: get_remote_files_copernicusmarine
vardef:
Hs: VAVH
U: WIND_SPEED
coords:
misc:
processing_level:
provider:
obs_type:
Now, you can ammend it to your needs. Here is an explanation of the most important variables.
cmems_L3_NRT - this is the name ID (nID) which often refers to a product which has multiple subproducts that are called name
name - attributes names used in wavy (left hand side) to names used in the product (right hand side)
download - consists of two download types: copernicus marine toolbox or FTP. Both need specifications such as the target path (trgt_tmplt) and for dynamic paths a substituting list (strbsub). In the example above this list contains only ‘name’ which means that the string name in the dataset_id cmems_obs-wave_glo_phy-swh_nrt_name-l3_PT1S will be replaced by any of the names specified above. The same is valid for the trgt_tmplt.
wavy_input - specifies, among other things, the source path (src_tmplt) from where wavy should find the data. Typically this is the same as trgt_tmplt described above, but it can also be something different. The strbsub under, works in the same way as explained above.
vardef - here it is important to specify the exact netcdf name. The variable names on the left hand side are the names wavy is using internally and the right hand side are the names used in the files. Any variable name on the left hand side must be described in the variable_def.yaml file.
Since most things are already defined for this product, the only thing we need to change is the path to where we should download. An example is given below:
cmems_L3_NRT:
download:
copernicus:
dataset_id: cmems_obs-wave_glo_phy-swh_nrt_name-l3_PT1S
trgt_tmplt: /chosen/path/to/satellite/data/L3/name/%Y/%m
The str “name” in your path_template will be replaced by the satellite mission that you download because it was defined in the strsub list. So for Sentinel-3a the final path for your downloaded files will be automatically /chosen/path/to/satellite/data/L3/s3a with subfolders on year and month.
You can now proceed and download like:
$ wavyDownload --nID cmems_L3_NRT --name s3a --sd 20250101T00 --ed 20250110T00
You can find the downloaded files in your chosen download directory.
You can also download altimeter data directly in the python script with the following lines.
>>> from wavy.satellite_module import satellite_class as sc
>>> nID = 'cmems_L3_NRT'
>>> name = 's3a'
>>> sd = '2023-11-10 00'
>>> ed = '2023-11-10 10'
>>> # Initialize sc object
>>> sco = sc(sd=sd,ed=ed,nID=nID,name=name)
>>> # Download the data to a chosen directory
>>> path = '/chosen/path/to/satellite/data/L3/s3a'
>>> sco.download(path=path)
In case of ftp downloads the config setup is similar but you have to make the adjustments under the ftp section:
download:
ftp: # downloading method
src_tmplt: "/path/to/remote/dir/%Y/%m"
trgt_tmplt: /chosen/path/to/satellite/data/L3/name/%Y/%m
strsub: ['name']
Also the collector needs to be a suitable one like e.g. the one used for CCI files:
collector: get_remote_files_ftp
With ftp, parallel python can be used with a keyword specifying the number of processes, e.g.:
>>> sco.download(nproc=4, path=path)