StoRM Frontend installation and configuration guide

Introduction

The StoRM Frontend service provides a SRM interface for storage management and data access supporting VOMS authentication and authorization mechanisms.

Install the service package

Grab the latest package from the StoRM repository. See instructions here.

Install the metapackage:

yum install storm-frontend-mp

Service configuration

The Frontend component relies on a single configuration file that contains all the configurable parameters. This file is:

/etc/storm/frontend-server/storm-frontend-server.conf

containing a list of key-value pairs that can be used to configure the Frontend server. In case a parameter is modified, the Frontend service has to be restarted in order to read the new value.

Currently, the Frontend’s configuration parameters can be divided per section as follows.

Database settings

Service settings

Log settings

Monitoring settings

XML-RPC communication settings

REST communication settings

Blacklisting settings

Proxy settings

General settings

Configure the service with YAIM

StoRM Frontend can be configured with YAIM tool on CentOS 6 platform.

Read more about YAIM tool here and what are the general YAIM variables for a StoRM deployment.

Here is a minimal YAIM configuration example for a Frontend node:

## The site name
SITE_NAME="storm-testbed"

## BDII hostname
BDII_HOST="emitb-bdii-site.cern.ch"

## List of NTP hosts
NTP_HOSTS_IP="131.154.1.103 193.206.144.10"

## Users configuration
USERS_CONF="/etc/storm/siteinfo/storm-users.conf"

## Groups configuration
GROUPS_CONF="/etc/storm/siteinfo/storm-groups.conf"

## Supported VOs.
VOS="dteam"

## Backend hostname
STORM_BACKEND_HOST="storm-backend.example.org"

## Database hostname and password
STORM_DB_HOST=$STORM_BACKEND_HOST
STORM_DB_PWD="secret"

## Backend's XMLRPC secret token
STORM_BE_XMLRPC_TOKEN="secret"

To configure the service with yaim, run the following command:

/opt/glite/yaim/bin/yaim -c -s SITEINFO.def -n se_storm_frontend

Configure the service with Puppet

The StoRM puppet module can be used to configure the service on CentOS 7 platform.

The module contains the storm::frontend class that installs the metapackage storm-frontend-mp and allows site administrator to configure storm-frontend-server service by managing the following files:

• /etc/storm/frontend-server/storm-frontend-server.conf
• /etc/sysconfig/storm-frontend-server

The whole list of StoRM Frontend class parameters can be found here.

Example of StoRM Frontend configuration done through storm::frontend class:

class { 'storm::frontend':
  be_xmlrpc_host  => 'backend.test.example',
  be_xmlrpc_token => 'NS4kYAZuR65XJCq',
  db_host         => 'backend.test.example',
  db_user         => 'storm',
  db_passwd       => 'storm',
}

Logging  

The Frontend logs information on the service status and the SRM requests received and managed by the process. The Frontend’s log supports different level of logging (ERROR, WARNING, INFO, DEBUG, DEBUG2) that can be set from the dedicated parameter in storm-frontend-server.conf configuration file. The Frontend log file named storm-frontend-server.log is placed in the /var/log/storm directory. At start-up time, the FE prints here the whole set of configuration parameters, this can be useful to check desired values. When a new SRM request is managed, the FE logs information about the user (DN and FQANs) and the requested parameters. At each SRM request, the FE logs also this important information:

03/19 11:51:42 0x88d4ab8 main: AUDIT - Active tasks: 3
03/19 11:51:42 0x88d4ab8 main: AUDIT - Pending tasks: 0

about the status of the worker pool threads and the pending process queue. Active tasks is the number of worker threads actually running. Pending tasks is the number of SRM requests queued in the worker pool queue. These data gives important information about the Frontend load.

The monitoring.log file

Monitoring service, if enabled, provides information about the operations executed in a certain amount of time writing them on file /var/log/storm/monitoring.log. This amount of time (called Monitoring Round) is configurable via the configuration property monitoring.timeInterval; its default value is 1 minute. At each Monitoring Round, a single row is printed on log. This row reports both information about requests that have been performed in the last Monitoring Round and information considering the whole FE execution time (Aggregate Monitoring). Informations reported are generated from both Synchronous and Asynchronous requests and tell the user:

• how many requests have been performed in the last Monitoring Round,
• how many of them were successful,
• how many failed,
• how many produced an error,
• the average execution time,
• the minimum execution time,
• the maximum execution time.

This row reports the Monitoring Summary and this is the default behavior of the monitoring service.

Example:

03/20 14:19:11 : [# 22927 lifetime=95:33:18] S [OK:47,F:15,E:0,m:0.085,M:3.623,Avg:0.201] A [OK:16,F:0,E:0,m:0.082,M:0.415,Avg:0.136]
  Last:(S [OK:12,F:5,E:0,m:0.091,M:0.255] A [OK:6,F:0,E:0,m:0.121,M:0.415])

Furthermore it can be requested a more detailed Frontend Monitoring activity by setting the configuration property monitoring.detailed to true. Doing this, at each Monitoring Round for each kind of SRM operation performed in the Monitoring Round (srmls, srmPtp, srmRm, …) the following information are printed in a section with header “Last round details:”:

• how many request succeeded,
• how many failed,
• how many produced an error,
• the average execution time,
• the minimum execution time,
• the maximum execution time,
• the execution time standard deviation.

This is called the Detailed Monitoring Round. After this, the Monitoring Summary is printed. Then, considering the whole Frontend execution time, in a section with header “Details:”, a similar detailed summary is printed. This is called the Aggregate Detailed Monitoring.

Example:

03/20 14:19:11 : Last round details:
03/20 14:19:11 : [PTP] [OK:3,F:0,E:0,Avg:0.203,Std Dev:0.026,m:0.183,M:0.240]
03/20 14:19:11 : [Put done] [OK:2,F:0,E:0,Avg:0.155,Std Dev:0.018,m:0.136,M:0.173]
03/20 14:19:11 : [# 22927 lifetime=95:33:18] S [OK:47,F:15,E:0,m:0.085,M:3.623,Avg:0.201] A [OK:16,F:0,E:0,m:0.082,M:0.415,Avg:0.136]
  Last:(S [OK:12,F:5,E:0,m:0.091,M:0.255] A [OK:6,F:0,E:0,m:0.121,M:0.415])
03/20 14:19:11 : Details:
03/20 14:19:11 : [PTP] [OK:7,F:0,E:0,Avg:0.141,Std Dev:0.057,m:0.085,M:0.240]
03/20 14:19:11 : [Put done] [OK:5,F:0,E:0,Avg:0.152,Std Dev:0.027,m:0.110,M:0.185]
03/20 14:19:11 : [Release files] [OK:4,F:0,E:0,Avg:0.154,Std Dev:0.044,m:0.111,M:0.216]
03/20 14:19:11 : [Rm] [OK:3,F:0,E:0,Avg:0.116,Std Dev:0.004,m:0.111,M:0.122]

Note:

• Operations not performed in current Monitoring Round are not printed in Detailed Monitoring Round.
• Operations never performed are not printed in Aggregate Detailed Monitoring.
• Operation performed in current Monitoring Round are aggregated in Aggregate Detailed Monitoring.

gSOAP tracefile

If you have problem at gSOAP level, and you have already looked at the troubleshooting section of the StoRM site without finding a solution, and you are brave enough, you could try to find some useful information on the gSOAP log file. To enable gSOAP logging, set the following environment variables:

$ export CGSI_TRACE=1
$ export CGSI_TRACEFILE=/tmp/tracefile

and restart the Frontend daemon by calling directly the init script /etc/init.d/storm-frontend-server and see if the error messages contained in /tmp/tracefile could help. Please be very careful, it prints really a huge amount of information.