ESGF Publication Test

Prerequisites

Verify you have a test project in /esg/config/esgcet/esg.ini, eg.

project_options = 
    cmip5 | CMIP5 | 1
    ipcc4 | IPCC Fourth Assessment Report | 2
    test | Test Project | 3

the default dataroot is set up:

 thredds_dataset_roots =
     esg_dataroot |  /esg/data

and you have /esg/config/esgcet/esg.test.ini (if not you can fetch from github ESGF/config repo).

Make sure that in esg.ini you either comment/remove the Globus service line or replace the #DEFAULTENDPOINT# with your actual endpoint uuid. If you don't know yet, then best to comment/remove the line.

NEW to ESGF 2.5.1

Automated publication test only requires two steps

      (run as root for basic publishing test):

(1) source /usr/local/conda/bin/activate esgf-pub

(2) esgtest_publish # additonal instructions may follow

Detailed manual publication test

Download test dataset

mkdir /esg/data/test; cd /esg/data/test
wget -O sftlf.nc http://distrib-coffee.ipsl.jussieu.fr/pub/esgf/dist/externals/sftlf.nc

Execution user

Must be different from root. Must have recrursive write permission to /esg/content/thredds/esgcet

groupadd publication
useradd -G publication esg-user
chown -R tomcat.publication /esg/content/thredds/esgcet/
chmod -R g+rw /esg/content/thredds/esgcet/

Mapfile generation

 su esg-user
 source /etc/esg.env
 export ESGINI=/esg/config/esgcet/esg.ini

If you have a newer (>=3) of the publisher, use esgprep. Automatically names your mapfile to the dataset name + .map

 esgprep mapfile --dataset ipsl.fr.test.mytest --project test /esg/data/test
 mv ipsl.fr.test.mytest.map test_mapfile.txt

Older versions:

 esgscan_directory --dataset ipsl.fr.test.mytest --project test /esg/data/test > test_mapfile.txt 

Publication step 1 - Data Node Local Thredds Server

 esgpublish --service fileservice --map test_mapfile.txt --project test --thredds

Then check http://my-data-node/thredds/catalog/esgcet/catalog.html

Publication step 2 - Peer Index Node

Authenticate with an ESGF user which has publisher permissions (or rootAdmin). See the CoG site administration guide for managing user permissions: https://www.earthsystemcog.org/projects/cog/esgf_cog_data_access_control_setup

 myproxy-logon -T -s [idp_node] -p 7512 -l [login] -o ~/.globus/certificate-file -b

Publish

 esgpublish --service fileservice --map test_mapfile.txt --project test --publish

Then check the cog site - http://my-index-node/ If you are testing a site with both index and data for the first time, please refer to the CoG setup guide:

https://www.earthsystemcog.org/projects/cog/esgf_cog_data_search_configuration

Unpublish

 esgunpublish --map  test_mapfile.txt  --database-delete