Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
d237447
updated conda yml file that builds working env
Mar 3, 2020
a19e313
fixed CMEMS changes to code
Mar 4, 2020
9c152e4
added first stage unit testing code
Mar 6, 2020
0e79451
updated unit tests with rotate grid script
Mar 10, 2020
42e7c5a
updated rotate script
Mar 11, 2020
823b6fb
updated unit tests to include coord generation
Mar 11, 2020
7b09265
updated rotate coefficents so grids plot nicely
Mar 11, 2020
48cc079
updated unit test grid generation
Mar 13, 2020
d76b453
reogranised testing scripts - added rot grid output
Mar 13, 2020
1d09496
updated to build constant boundary file
Mar 13, 2020
b79e264
added bathy and mask generation tools and got running with orthoganal…
Mar 16, 2020
129c504
unit test script now generating data and testing constant temp
thopri Mar 18, 2020
71bb209
added salinity check
thopri Mar 18, 2020
4bf5c36
added multiple grid/bdy tests
thopri Mar 18, 2020
12043a0
updated grid size on test cases
thopri Mar 23, 2020
d35c258
fixed rotated grid test case
thopri Mar 24, 2020
eec6476
added U and V grid and constant current tests
thopri Mar 25, 2020
fae40b0
added log statements to show CMEMS behaviour more clearly
thopri Mar 25, 2020
118b1c6
fixed command line parsing issue
thopri Mar 26, 2020
8f6d0b5
added unit testing to setup.py script
thopri Mar 26, 2020
25555ab
added config file to remove hard coded paths
thopri Mar 26, 2020
ee8706f
moved unit test namelists to new folder
thopri Mar 27, 2020
8341942
added try except block to provide useful error for time counter discp…
thopri Mar 27, 2020
6069981
CMEMS download progress now prints to log file
thopri Mar 27, 2020
4f102ae
added skip existing files to CMEMS download functon
thopri Mar 30, 2020
f7cf069
small update to dl script
thopri Mar 31, 2020
a6d4437
added FES extract script
thopri Apr 2, 2020
14ffd7d
updated fes tide code
thopri Apr 2, 2020
6cf0fc2
updated FES scripts
thopri Apr 3, 2020
9bb4558
Implemented FES hc extraction
thopri Apr 6, 2020
518e6ca
multiple HC added to FES
thopri Apr 8, 2020
d5bdf5c
completed multiple HC for FES model
thopri Apr 15, 2020
84b3a17
nc write now working for tide HC boundary generation
thopri Apr 15, 2020
18ee6b9
implemented multiple rim width fix
thopri Apr 17, 2020
3d42e65
tested and working for both FES and TPXO
thopri Apr 17, 2020
654016d
added sphinx to environment yml file
thopri Apr 20, 2020
71d5242
fixed multiple HC crash bug
thopri Apr 20, 2020
fc0bb07
fix that generates list of FES harmonics automatically
thopri Apr 20, 2020
23523b0
added some more documentation files
thopri Apr 20, 2020
45a5386
updated documentation
thopri Apr 21, 2020
55587d9
added commandline spinner
thopri Apr 22, 2020
8168bdd
added try except to catch missing input files due to THREDDS server b…
thopri Apr 22, 2020
b8577c0
updated bdy_coords test script from python 2 to 3
thopri Apr 27, 2020
b5b10a4
fixed issues with U and V currents when ln_dyn3d is set to true
thopri Apr 27, 2020
a1d8070
fixed CMEMS download bug and updated troubleshooting
thopri Apr 28, 2020
1391537
fixed large spikes in FES cosine and sin output
thopri Apr 30, 2020
79a19a7
added tide test script comparing PyNEMO with reference model (FES only)
thopri May 5, 2020
c8d231d
added reference subset data to output of tide test
thopri May 6, 2020
cab2180
updated FES tide script
thopri May 6, 2020
20de1f1
fixed phase issue on FES tide boundary generation
thopri May 7, 2020
0e3ad30
added tide checker function to BDY file
thopri May 7, 2020
602269e
fixed units issue and tide test output bug
thopri May 11, 2020
d5d9f5c
updated testing function to cope wiht fill values in FES
thopri May 12, 2020
e243f7d
ready for release 0.1 (hopefuly)
thopri May 12, 2020
30b1faa
added update to documentation for tide checker
thopri May 14, 2020
eca2acc
meta_data test script
thopri May 20, 2020
ae85fb8
highlighting the importance of order in chk_list dicts
thopri May 21, 2020
0568353
removed CMEMS specifc tracer varable names, NCML remppaing now used
thopri May 22, 2020
a01a349
removed directory function calls from factory.py
thopri May 22, 2020
e8c66be
ammended CMEMS downloader to work with remapped variables
thopri Jun 16, 2020
9bc759f
updated documentation
thopri Jun 17, 2020
dc98306
Update index.rst
jdha Jul 16, 2020
f0c43ea
Update installation.rst
jdha Jul 16, 2020
e8d1815
updated bdy_mask.py so that it handles missing attributes
thopri Jul 17, 2020
a13ea7f
Merge branch 'IMMERSE' of https://github.com/NOC-MSM/PyNEMO into IMMERSE
thopri Jul 17, 2020
0eb2053
added exception to catch bathymetries with zeros along boundaries as …
thopri Jul 17, 2020
fd27464
added I/O defined by NCML
thopri Aug 6, 2020
2e34442
added xmltodict dependancy to yml
thopri Aug 6, 2020
8efb513
fixed handling of 2D varibles, e.g. SSH
thopri Aug 13, 2020
3dce3e6
updated time_counter to use base year in bdy file
thopri Aug 14, 2020
173865a
fixed start end date indexing
thopri Aug 17, 2020
21e9a43
added NCML templace to Tide output
thopri Aug 17, 2020
4566ada
renamed input NCML to reflect data/model source
Sep 15, 2020
83d3524
added FVCOM ncml
Nov 2, 2020
f677590
Update pynemo_37.yml
thopri Dec 17, 2020
1f30869
updated broken readme
thopri Dec 17, 2020
faadcc2
Merge branch 'IMMERSE' of https://github.com/NOC-MSM/PyNEMO into IMMERSE
thopri Dec 17, 2020
bb3e39b
update YML file to specify matplotlib and cftime versions
jpolton May 26, 2021
69e1b71
Merge pull request #69 from NOC-MSM/feature/yml_update
thopri May 27, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,11 @@ __pycache__/
*.so
*.nc
*.xml

# CMEMS creditial file
CMEMS_cred.py
motu_config.ini

# Distribution / packaging
.Python
build/
Expand All @@ -28,6 +33,8 @@ wheels/
MANIFEST
outputs
.DS_Store
DATA
SSW_FVCOM

# PyInstaller
# Usually these files are written by a python script from a template
Expand Down
72 changes: 48 additions & 24 deletions README.rst
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
PyNEMO
======

To be updated soon. This work springboards from the `PyNEMO Project <http://pynemo.readthedocs.io/en/latest/index.html/>`_.
Full documentation here.

What is this repository for?
----------------------------
PyNEMO boundary generation toolbox for NEMO ocean model.

How do I get set up?
--------------------
Expand All @@ -20,16 +21,16 @@ Steps to take to install PyNEMO, creating a specific conda virtual environment i
- Install Conda, either Anaconda or Miniconda (outside scope of this readme)
- Create conda environment for PyNEMO::

$ cd to/PyNEMO/directory
$ conda env create -f environment_pynemo.yml
$ cd PyNEMO
$ conda env create -f pynemo_37.yml

- Activate the new virtual environment::

$ source activate pynemo_env
$ source activate pynemo3

- Install Jave JRE (outside scope of this readme) and link libjvm.dylib to LD_LIBRARY_PATH variable::
- Install Jave JDK (outside scope of this readme) and link Java Home to conda environment::

$ export LD_LIBRARY_PATH=/path/to/java/library/folder/containing/libjvm.dylib:$LD_LIBARY_PATH # see notes below
$ export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk-13.0.2.jdk/Contents/Home # see notes below

- Install PyNEMO::

Expand All @@ -39,45 +40,54 @@ Steps to take to install PyNEMO, creating a specific conda virtual environment i

This should result in PyNEMO being installed in the virtual environment, and can be checked by entering::

$ pynemo -v
$ pynemo -h

Resulting in a help usage prompt::

$ usage: pynemo -g -s <namelist.bdy>
$ usage: pynemo [-g] -s -d <namelist.bdy>
-g (optional) will open settings editor before extracting the data
-s <bdy filename> file to use
-d (optional) will download CMEMS data using provided bdy file

The virtual environment can be deactivated to return you to the normal prompt by typing::

$ conda deactivate
To reactivate, the following needs to be typed::
$ conda deactivate

$ source activate pynemo_env
To reactivate, the following needs to be typed::

$ source activate pynemo3

To use PyNEMO, the following command is entered: (the example will run an benchmarking test)::

$ pynemo -s /path/to/namelist/file (e.g. PyNEMO/inputs/namelist_remote.bdy)

**Additional NOTES**
Other commands include -d which downloads the specified CMEMS data in the namelist bdy file.::

For Macbook Pro 2015, macOS Mojave and Java SDK 13 and JRE 8 the following path for the libjvm library should be correct::
$ pynemo -d /PyNEMO/inputs/namelist_cmems.bdy

/Library/Java/JavaVirtualMachines/jdk-13.0.1.jdk/Contents/Home/lib/server
To use the CMEMS download service an account needs to be created at http://marine.copernicus.eu/services-portfolio/access-to-products/
Once created the user name and password need to be added to PyNEMO. To do this a file with the name CMEMS_cred.py in the utils folder
needs to be created with two defined strings one called user and the other called pwd to define the user name and password.::

Resulting in the following command: (this will be different for different java versions and operating systems)::
$ touch pynemo/utils/CMEMS_cred.py
$ vim pynemo/utils/CMEMS_cred.py
press i
user='username'
pwd='password'
press esc and then :q

$ export LD_LIBRARY_PATH=/Library/Java/JavaVirtualMachines/jdk-13.0.1.jdk/Contents/Home/lib/server:$LD_LIBRARY_PATH
**IMPORTANT** This will create a py file in the right place with the parameters required to download CMEMS, the password is stored as plain text so please
do not reuse any existing password!

For an iMac 2013, macOS Catalina and JRE 8 only the followinng path was found to be correct::

/Library/Internet\ Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/lib/server

With the following command being required to set the environment variable::
PyNEMO creates a log file be default, this provided info, warning and error messages. By default this is called nrct.log and is saved in the directory where pynemo is run from. (usually /PyNEMO)
New runs are appended onto the end of the log file so it will periodically need to be delelted to reduce the size of the log.

$ export LD_LIBRARY_PATH=/Library/Internet\ Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/lib/server:$LD_LIBRARY_PATH
**Additional NOTES**

The conda environment creation command has not yet been tested. The yml document (can be opened using text editor) gives a list of all the modules and their versions that are required for PyNEMO so a environment can be constructed using this document as reference (or if you use pip!)
The above path for Java Home was valid for a Macbook Pro 2015 with macOS Catalina and Java SDK 13.0.2
however for different java versions, operating systems etc this may be different

**Update** conda environment yaml file has been tested (and works!) on a Macbook Pro 2015 and iMac 2013 running Anaconda 3.7 and Miniconda 3.7 respectively.
The conda environment yaml file has been tested with miniconda 3.7 and found to install the environment correctly.

Contribution guidelines
-----------------------
Expand All @@ -101,6 +111,19 @@ The PyNEMO module can be tested using the bench marking namelist bdy file in the
:width: 800
:alt: Example BDY coords output

Unit Tests
-------------------

To test operation of the PyNEMO module, running the PyTest script in the unit tests folder will perform a range of tests on different child grids,
e.g. checking the interpolation of the source data on to the child grid. To do this the following command is required::

$ pytest -v pynemo/pynemo_unit_test.py

The results of the test will show if all tests pass or the errors that result from failed tests.

Currently **(26/03/2020)** there are 7 tests that cover checking the interpolation results of different child grids. The input data is generated as part of the
test and is removed afterwards. The number of tests will be increased in the future to cover more PyNEMO functionality.

Who do I talk to?
-----------------

Expand All @@ -110,5 +133,6 @@ Who do I talk to?

* Other community or team contact

thopri

For more information regarding the use and development of PyNEMO see: [PyNEMO Wiki](https://github.com/jdha/PyNEMO/wiki)
104 changes: 0 additions & 104 deletions README_markdown.md

This file was deleted.

Loading