-
Notifications
You must be signed in to change notification settings - Fork 20
ESGFNode|InstallationProcedure
| Wiki Reorganisation |
|---|
| This page has been classified for reorganisation. It has been given the category REVISE. |
| This page contains useful content but needs revision. It may contain out of date or inaccurate content. |
The installation script is written in BASH and is targeted to run on *NIX operating systems. In particular the target platform is RH or CentOS, because of minor command idiosyncrasies, however, the script has been used successfully executed on other *NIX systems. The install script pulls down installation artifacts from our distribution server. All artifacts and even the scripts themselves are verified against posted checksums. If verification fails the script will abort. There is a single master script, however, there is actually a series of scripts that are used to install the ESGF Node stack. The script is able to run at start up via chkconfig, supporting [start|stop|restart|update] directives. The script is also self- updating and will alert you if it has been tampered with. There are many flags that may be used with the script (see usage by using --help|-h options), as it is the central mechanism for controlling the maintenance life cycle of the data node.
The script is the preferred way to install the ESGF Node stack. It is recommended to create an "esg-user" account that is able to issue the command "sudo -s" to become root.
Unix User
root or esg-user with sudo permissions
Prerequisites
Make sure epel repository is enabled and rpmforge is disabled, then run
yum install autoconf automake bison file flex gcc gcc-c++ gettext-devel libtool uuid-devel libuuid-devel libxml2 libxml2-devel libxslt libxslt-devel lsof make openssl-devel pam-devel pax readline-devel tk-devel wget zlib-devel perl-Archive-Tar perl-XML-Parser libX11-devel libtool-ltdl-devel e2fsprogs-devel.x86_64 gcc-gfortran libicu-devel.x86_64 libgtextutils-devel.x86_64 *ExtUtils*
Download the installation script
wget -O esg-bootstrap --no-check-certificate https://raw.github.com/ESGF/esgf-dist/master/installer/master/esg-bootstrap
esg-bootstrap [--devel]
Check ESGF version
esg-node --version
[VERIFIED]
Version: v1.6.2-bushwick_myrtle-release-master
Release: bushwick_myrtle
Earth Systems Grid Federation (http://esgf.org)
ESGF Node Installation Script
Start the installation process
script -a -c "esg-node --type data compute --install" /tmp/esgf_install.log
Use script instead of tee to log the installation. tee can cause configuration files corruption.
Interactive questionnaire 0
Are you ready to begin the installation? [Y/n] Y
Configured host IP address does not match available IPs...
Detected multiple IP addresses bound to this host...
Please select the IP address to use for this installation
-------------------------------------------
[0] : 134.157.XXX.XXX
[1] : 172.20.XXX.XXX
-------------------------------------------
select [] > 0
-------------------------------------------------------
Welcome to the ESGF Node installation program! :-)
What is the fully qualified domain name of this node? [my_node.xxx.xx]: my_node.xxx.xx
What is the admin password to use for this installation? (alpha-numeric only) []:
Please re-enter password:
What is the name of your organization? [IPSL]: IPSL
Please give this node a "short" name: []: VESGINT-DATA
Please give this node a more descriptive "long" name []: my_node.xxx.xx
What is the namespace to use for this node? (set to your reverse fqdn - Ex: "gov.llnl") [fr.jussieu.ipsl]: fr.jussieu.ipsl
What peer group(s) will this node participate in? (if not sure, use default) [esgf-test]: esgf-prod
What is the default peer to this node? [esgf-node1.llnl.gov]: some_index_node.xxx.xxx
What is the hostname of the node do you plan to publish to? [esgf-node1.llnl.gov]: some_index_node.xxx.xxx
What email address should notifications be sent as? []: [email protected]
Is the database external to this node? [y/N]: N
Please enter the database connection string...
(form: postgresql://[username]@[host]:[port]/esgcet)
What is the database connection string? [postgresql://dbsuper@localhost:5432/esgcet]: postgresql://
entered: postgresql://dbsuper@localhost:5432/esgcet
What is the (low priv) db account for publisher? [esgcet]: esgcet
What is the db password for publisher user (esgcet)? []:
Interactive questionnaire 1
/usr/local/pgsql/bin/createuser -U postgres -P -s -e dbsuper
Enter password for new role:
Enter it again:
Interactive questionnaire 2
[VERIFIED]
Please Enter PostgreSQL port number [5432]:> 5432
Interactive questionnaire 3
Would you like a "system" or "user" publisher configuration:
-------------------------------------------
*[1] : System
[2] : User
-------------------------------------------
[C] : (Custom)
-------------------------------------------
select [1] >
You have selected: 1
Publisher configuration file -> [/esg/config/esgcet/esg.ini]
Is this correct? [Y/n] Y
Your publisher configuration file will be: /esg/config/esgcet/esg.ini
What is your organization's id? [IPSL]: IPSL
Interactive questionnaire 4
Looking for keystore [/esg/config/tomcat/keystore-tomcat]... (don't see one)...
Keystore setup:
Launching Java's keytool:
store_password = ******
Would you like to use the DN: (OU=ESGF.ORG, O=ESGF) ? [Y/n]:
Using keystore DN = CN=my_node.xxx.xx, OU=ESGF.ORG, O=ESGF
Enter key password for <my_esgf_node>
(RETURN if same as keystore password):
Do you wish to generate a Certificate Signing Request at this time? [Y/n] Y
Interactive questionnaire 5
Oox:-)
Please enter the password for this keystore :
Please re-enter the password for this keystore:
Interactive questionnaire 6
Please enter username for tomcat [dnode_user]:
dnode_user
Please enter password for user, "dnode_user" [********]: 73769edbd97410aacfb3560ebb817f882d141517
Would you like to add another user? [y/N]: N
Please Enter the public (i.e. routable) IP address of this host [xxx.xxx.xxx.xxx]:> xxx.xxx.xxx.xxx
Interactive questionnaire 7
Please select the IDP Peer for this node:
-------------------------------------------
*[1] : ESGF-PCMDI-9 -> pcmdi9.llnl.gov
[2] : ESGF-PCMDI -> pcmdi3.llnl.gov
[3] : ESGF-JPL -> esg-gateway.jpl.nasa.gov
[4] : ESGF-ORNL -> esg2-gw.ccs.ornl.gov
[5] : ESGF-BADC -> cmip-gw.badc.rl.ac.uk
[6] : ESGF-DKRZ -> ipcc-ar5.dkrz.de
[7] : ESGF-PNNL -> esg1-gw.pnl.gov
[8] : ESGF-ANL -> dev.esg.anl.gov
[9] : ESGF-PCMDI-TEST3 -> esgf-node3.llnl.gov
-------------------------------------------
[C] : (Manual Entry)
-------------------------------------------
select [1] > C
Please enter the IDP Peer's name [ESGF-PCMDI-9] SOME-IDP-NODE
Please enter the IDP Peer's hostname [pcmdi9.llnl.gov] some_idp_node.xxx.xxx
You have selected: (Manual Entry)
SOME-IDP-NODE -> some_IDP_node.xxx.xxx
Is this correct? [Y/n] Y
Interactive questionnaire 8
Server sent 2 certificate(s):
1 Subject CN=some_idp_node.xxx.xxx, OU=simpleCA-some_idp_node.xxx.xxx, OU=GlobusTest, O=Grid
Issuer CN=Globus Simple CA, OU=simpleCA-some_idp_node.xxx.xxx, OU=GlobusTest, O=Grid
sha1 cf f9 20 2b ce a6 bc b0 5d b4 a7 bb 0c 08 18 99 14 47 a6 86
md5 bd 6d ab cb 0b 75 58 fb 54 52 89 60 8e 1b 44 b8
2 Subject CN=Globus Simple CA, OU=some_idp_node.xxx.xxx, OU=GlobusTest, O=Grid
Issuer CN=Globus Simple CA, OU=some_idp_node.xxx.xxx, OU=GlobusTest, O=Grid
sha1 06 09 9b cc b6 70 6f 3e 59 00 34 b9 fa 0a ba 87 0b f1 16 10
md5 0b b0 a3 56 f6 a7 c7 32 7e 35 b5 b9 e3 bb cd 26
Enter certificate to add to trusted keystore or 'q' to quit: [1] > 1
Interactive questionnaire 9
Creating directory /esg/content/thredds/esgcet
INFO 2013-12-19 17:35:51,812 Writing THREDDS ESG master catalog /esg/content/thredds/esgcet/catalog.xml
INFO 2013-12-19 17:35:51,848 Writing THREDDS root catalog /esg/content/thredds/catalog.xml
THREDDS dataset root directories (option=thredds_dataset_roots)
Each entry has the form 'path_identifier | absolute_directory_path':
Current value is:
esg_dataroot | /esg/data
Enter lines, or <RETURN> to end
Add new line:
Interactive questionnaire 10
*/15 * * * * sleep $(( $RANDOM\%900 )); /usr/sbin/puppetd -o --no-daemonize -l syslog >/dev/null 2>&1
# ESGF cronjob BEGIN ###
35 0,12 * * * ESG_USAGE_PARSER_CONF=/esg/config/gridftp/esg-bdm-usage-gridftp.conf /esg/tools/esg_usage_parser
# ESGF cronjob END ###
Is this ok ? [Y/n] Y
Interactive questionnaire 11
# ESGF cronjob BEGIN ###
35 0,12 * * * ESG_USAGE_PARSER_CONF=/esg/config/gridftp/esg-bdm-usage-gridftp.conf /esg/tools/esg_usage_parser
5 0,12 * * * ESG_USAGE_PARSER_CONF=/esg/config/gridftp/esg-server-usage-gridftp.conf /esg/tools/esg_usage_parser
# ESGF cronjob END ###
Is this ok ? [Y/n] Y
Interactive questionnaire 12
Server sent 2 certificate(s):
1 Subject CN=some_idp_node.xxx.xxx, OU=simpleCA-some_idp_node.xxx.xxx, OU=GlobusTest, O=Grid
Issuer CN=Globus Simple CA, OU=simpleCA-some_idp_node.xxx.xxx, OU=GlobusTest, O=Grid
sha1 cf f9 20 2b ce a6 bc b0 5d b4 a7 bb 0c 08 18 99 14 47 a6 86
md5 bd 6d ab cb 0b 75 58 fb 54 52 89 60 8e 1b 44 b8
2 Subject CN=Globus Simple CA, OU=simpleCA-some_idp_node.xxx.xxx, OU=GlobusTest, O=Grid
Issuer CN=Globus Simple CA, OU=simpleCA-some_idp_node.xxx.xxx, OU=GlobusTest, O=Grid
sha1 06 09 9b cc b6 70 6f 3e 59 00 34 b9 fa 0a ba 87 0b f1 16 10
md5 0b b0 a3 56 f6 a7 c7 32 7e 35 b5 b9 e3 bb cd 26
Enter certificate to add to trusted keystore or 'q' to quit: [1] > 1
Restart the node
esg-node restart
Web front-ends checks:
http://my_node.xxx/esgf-node-manager
http://my_node.xxx/thredds
http://my_node.xxx/esgf-desktop
http://my_node.xxx/esgf-dashboard
http://my_node.xxx/las
esg_root_dir = /esg
workdir = /usr/local/src/esgf
Location
Description
${esg_root_dir}
Top level directory location of the ESG configuration files and logs (default /esg).
${esg_root_dir}/backups
Application stack and database data archive location
${esg_root_dir}/config
ESGF configuration files
${esg_root_dir}/content
Thredds catalogs & LAS data files
${esg_root_dir}/data
Top level directory for data (.nc) files
${esg_root_dir}/data.replica
Top level directory for all replicated data from other nodes
${esg_root_dir}/data-index-*
Search index directories
${esg_root_dir}/gridftp_root
Chroot directory for gridftp access to data
${esg_root_dir}/log
ESGF log files
${esg_root_dir}/tools
ESGF tools (currently; esg_usage_parser)
${esg_root_dir}/config/esgcet/esg.ini or ~/.esgcet/esg.ini
esg publisher setup file. (system vs personal install)
%{esg_root_dir}/etc
Ancillary scripts and files
${esg_root_dir}/esgf-install-manifest
Log of all installed components of the application stack (date, name, location, version)
/etc/esg.env
Environment variables required by the script and used in node operation
${workdir}
Installation "scratch" directory for installation-time artifacts; source, helper scripts, et. al.
${workdir}/globus
globus sources
${workdir}/esg
publisher,thredds and other sources
The esg-node installation script is also the boot script
To stop/start or restart or check the status of the node...
%> esg-node stop
%> esg-node start
%> esg-node status
%> esg-node restart