ESGF_Search_Shards

ESGF Search Services Configuration

Currently, each ESGF Index Node administrator is responsible for configuring the list of other Index Nodes (a.k.a. shards) that are queried in a distributed Solr search. This list must be contained in the file _ /esg/config/esgf_shards_static.xml _ . Note that if this file is not found, the search services will default to a local-only search. The list will be re- loaded automatically every time the file is changed, or every 10 minutes.

This is a temporary solution available in only the latest release candidate version. The federation is designed to be dynamic, however, at this time we are posting this temporary solution. The release after this release candidate will once again support dynamic federating.

The following configuration are currently recommended:

To install any of these files on a running system, you may do:

Ex:
%> esg-node --fetch-static-shards-file cmip5

DO NOT ISSUE THIS COMMAND IF YOUR NODE IS "SUPER-NODING" OTHER INDEXES!!!!!

The argument of "cmip5" indicates which shards file is fetched and used. Other values currently are "cssef" and "test". Only one may be in place at a time using this flag. Otherwise the files must be manually aggregated if the situation calls.

** CMIP5 Production Group **

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<shards xmlns="http://www.esgf.org/whitelist">

    <!-- US -->
    <value>pcmdi9.llnl.gov:8983/solr</value>
    <value>pcmdi11.llnl.gov:8983/solr</value>
    <value>esg-datanode.jpl.nasa.gov:8983/solr</value>   
    <value>esg.ccs.ornl.gov:8983/solr</value>
    <value>esgf.nccs.nasa.gov:8983/solr</value>
    <value>esg01.nersc.gov:8983/solr</value>
    <value>esgdata.gfdl.noaa.gov:8983/solr</value>
    <value>hydra.fsl.noaa.gov:8983/solr</value>
    
    <!-- Europe -->
    <value>esgf-index1.ceda.ac.uk:8983/solr</value>
    <value>euclipse1.dkrz.de:8983/solr</value>
    <value>esgf-data.dkrz.de:8983/solr</value>
    <value>adm07.cmcc.it:8983/solr</value>
    <value>esgf-node.ipsl.fr:8983/solr</value>
    <value>noresg.norstore.uio.no:8983</value>
    <value>esg-dn1.nsc.liu.se:8983</value>
  
    <!-- Australia -->
    <value>esg2.nci.org.au:8983/solr</value>
 
    <!-- Asia -->
    <value>esg.bnu.edu.cn:8983/solr</value>

     
</shards>

** ESGF Test Group **

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<shards xmlns="http://www.esgf.org/whitelist">

        <!-- US -->
    <value>test-datanode.jpl.nasa.gov:8983/solr</value>   
    <value>esgf-node1.llnl.gov:8983/solr</value>
    <value>esgf-node2.llnl.gov:8983/solr</value>
    <value>esgf-node3.llnl.gov:8983/solr</value>
        
</shards>

** CCSEF project **

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<shards xmlns="http://www.esgf.org/whitelist">
    
    <!-- CSSEF -->
    <value>dev.esg.anl.gov:8983/solr</value>
    <!-- <value>esg1-gw.pnl.gov:8983/solr</value> -->
    
</shards>

See also idp static file