05. Run with open boundary conditions

1) Generate OBC

To generate OBC for this UK500 configuration we have used an AMM15 dataset generated as part of the CHAMFER project. the dataset is not public at the moment and is stored on Jasmin. Therefor, to generate the open boundary conditions (OBC) for UK500, we have installed pyBDY directly on JASMIN and generated the boundary there.

install pybdy on Jasmin

JASMIN has recently stopped supporting conda, however 'it is still possible to use conda environments on JASMIN, but for licensing reasons, please use miniforge (rather than Anaconda or miniconda) when creating these environments. Instructions that have been suitably modified to reflect this are at creating-and-using-miniforge-environments' [information from Jasmin helpdesk; 05/03/25]

log into JASMIN and then on one of the sci servers. Then follow the instructions on creating-and-using-miniforge-environments from the section obtaining-miniforge to activating-the-base-environment.

Now you can generate your own conda environments, make sure that the base environment that you have just created is activated and then follow instructions on pyBDY - how to get set up to generate the pybdy virtual environment and then install pyBDY.

Make sure your Java Runtime Environment is set with the correct path. On JASMIN (date 13/06/25) this is:

JVM="jre-17"
export JAVA_HOME=/usr/lib/jvm/$JVM/

Note: these JAVA paths are updated and change often, so you might want to double check what the latest one is.

generate OBC file

Once you have a working pyBDY on your JAMSIN space, download the genNEMO repository. Move into the directory below:

cd genNEMO/INPUT/CHAMFER_AMM15_to_UK500_clean/

make sure that the namelist.template contains the correct hardcoded paths to your input files:

!------------------------------------------------------------------------------
!  grid information
!------------------------------------------------------------------------------
   sn_src_hgr = '/gws/nopw/j04/chamfer/UK500_ocean/Domain/GEG_SF12_pybdy.nc'
   sn_src_zgr = '/gws/nopw/j04/chamfer/UK500_ocean/Domain/GEG_SF12_pybdy.nc'
   sn_dst_hgr = '/gws/nopw/j04/chamfer/UK500_ocean/Domain/domain_cfg_UK500_pybdy_gdept.nc'
   sn_dst_zgr = '/gws/nopw/j04/chamfer/UK500_ocean/Domain/domain_cfg_UK500_pybdy_gdept.nc'
   sn_src_msk =  '/gws/nopw/j04/jmmp/public/AMM15/DOMAIN_CFG/GEG_SF12.nc'
   sn_bathy   = '/gws/nopw/j04/chamfer/UK500_ocean/Domain/bathy_meter_UK500_pybdy.nc'

   sn_nme_map = 'grid_name_map.json'     ! json file mapping variable names to netcdf vars

and set you OBC output directory:

!------------------------------------------------------------------------------
!  I/O
!------------------------------------------------------------------------------
   sn_src_dir = 'src_data.ncml' ! src_files
   sn_dst_dir = '/gws/nopw/j04/chamfer/UK500_ocean/LBC/'

now move into cd /genNEMO/Scripts/ and add your conda environement to set_up.sh. Again check that the java path are correct:

source ~/miniforge3/bin/activate
conda activate /home/users/<user>/miniforge3/envs/pybdy
JVM="jre-17"
export JAVA_HOME=/usr/lib/jvm/$JVM/

fianlly open the iter_gen_bdy_UK500.sh file, and check that the path to genNEMO is correct, select the year that you want to generate the boundaires for. To run for a full year the file should look as below.

source set_up.sh

bdy_list=("CHAMFER_AMM15_to_UK500_clean")
year=1993

for bdy in ${bdy_list[@]}; do
    echo $bdy
    path="/home/users/<user>/genNEMO/INPUT/${bdy}/"
    cd $path
    for month in {01..12}; do
        echo "$year-$month"
        month_p1=$(date --date "$year-$month-01 +1 month" "+%m")
        year_p1=$(date --date "$year-$month-01 +1 month" "+%Y")
        echo "$year_p1-$month_p1"

        cat namelist.template \
            | sed "s,__MONTH0__,$month,g" \
            | sed "s,__MONTH1__,$month_p1,g" \
            | sed "s,__YEAR0__,$year,g" \
            | sed "s,__YEAR1__,$year_p1,g" \
            > namelist.bdy
        cat "src_data.template" \
            | sed "s,__YEAR__,$year,g" \
            | sed "s,__YEAR_p1__,$year_p1,g" \
            | sed "s,__MONTH__,$month,g" \
            | sed "s,__MONTH_p1__,$month_p1,g" \
            > src_data.ncml

        pybdy -s namelist.bdy
    done
done

then update the gen_bdy.slurm to your account, make sure it point to the correct JAVA_HOME paths and point to the shell script iter_gen_bdy_UK500.sh :

#!/bin/bash

#SBATCH --partition=standard
#SBATCH --qos=standard
#SBATCH --mem=500G
#SBATCH --time=03:00:00                    
#SBATCH --account=chamfer

#JVM="jre-17"
#JVM="jre-1.8.0-openjdk"
#export JAVA_HOME=/usr/lib/jvm/$JVM/
#ls /usr/lib/jvm

./iter_gen_bdy_UK500.sh

Run the script with : sbatch gen_bdy.slurm

Note: These scripts only work for running a full year. If you want to test for a shorter period, you can go in INPUT/CHAMFER_AMM15_to_UK500_clean/ , copy the namelist.template to a new file (eg namelist.testing) and in the test file change the period to a shorter run:

!------------------------------------------------------------------------------
!  Time information for output
!------------------------------------------------------------------------------
    sn_date_start   = '1993-07-01'    !  dst output date start YYYY-MM-DD
    sn_date_end     = '1993-08-01'    !  dst output date end YYYY-MM-DD

then make a copy of src_data.template as src_data_testing.ncml and hardcode the path to your input files, and run it using a slurm script that points pyBDY to you test namelist straight from that directory as below:

#!/bin/bash

#SBATCH --partition=standard
#SBATCH --qos=standard
#SBATCH --mem=128G
#SBATCH --time=03:00:00                    
#SBATCH --account=chamfer

source ~/miniforge3/bin/activate
conda activate /home/users/<user>/miniforge3/envs/pybdy
JVM="jre-17"
export JAVA_HOME=/usr/lib/jvm/$JVM/

pybdy -s namelist.testing

2) Run with Open boundaries

Work in progress