StoRM GridFTP DSI is a plugin of Globus GridFTP that computes the ADLER32 checksum on incoming file transfers and stores this value on an extended attribute of the file.
Grab the latest package from the StoRM repository. See instructions [here][download-page].
Install the metapackage:
yum install storm-globus-gridftp-mp
The Globus Gridftp server configuration relies on a single file that contains all the configurable parameters. This file is:
/etc/grid-security/gridftp.conf
It contains a list of key-value (space separated) pairs that can be used to configure the server and the loading of StoRM GriDFTP DSI plugin. The full list of supported parameters can be read from Globus GridFTP documentation here.
| Property Name | Description |
|---|---|
port |
Port on which a frontend will listen for client control channel connections, or on which a data node will listen for connections from a frontend. If not set a random port will be chosen and printed via the logging mechanism. |
connections_max |
Maximum concurrent connections allowed. Only applies when running in daemon mode. Unlimited if not set. |
port_range |
Port range to use for incoming connections. The format is “startport,endport”. This, along with -data-interface, can be used to enable operation behind a firewall and/or when NAT is involved. This is the same as setting the environment variable GLOBUS_TCP_PORT_RANGE. |
config_base_path |
Base path to use when config and log path options are not full paths. By default this is the current directory when the process is started. |
log_level |
A comma separated list of levels from: ERROR, WARN, INFO, TRANSFER, DUMP, ALL. TRANSFER includes the same statistics that are sent to the separate transfer log when -log-transfer is used. Example: error,warn,info. You may also specify a numeric level of 1-255. The default level is ERROR. |
log_single |
Path of a single file to log all activity to. If neither this option or log_unique is set, logs will be written to stderr unless the execution mode is detached or inetd, in which case logging will be disabled. |
log_transfer |
Log netlogger style info for each transfer into this file. You may also use the log-level of TRANSFER to include this info in the standard log. |
load_dsi_module |
With value “StoRM” it loads StoRM DSI module. |
allowed_modules |
With value “StoRM” it allows StoRM DSI module. |
At each transfer request, StoRM Globus GridFTP uses LCMAPS to get user mapping and start a new processes on behalf of the user to proceed with data transfer. GridFTP relies on a different db file to get the plugin to use. Obviously LCMAPS has to answer to GridFTP requests and StoRM requests in coeherent way. The GridFTP uses the LCMAPS configuration file located at:
/etc/lcmaps/lcmaps.db
The IPC channel is used between a Globus GridFTP server head node and its disk servers, e.g. for striped transfers (read more into the GridFTP System Administrator’s Guide). In the default behavior of StoRM deployment the IPC channel is not used. In fact, StoRM is mainly installed on a single host with one gridftp server which read/write directly on disk. In the cases it is a distributed deployment, there are usually n gridftp servers which read/write data directly on disk, behind a haproxy or a dns for example, so there are no separate frontends and one or more disk node servers. However, it’s important to know that the IPC channel must be kept firewalled for any hosts outside the SE system.
The IPC channel must be kept firewalled for any hosts outside the SE system.
StoRM Globus GridFTP server can be configured with YAIM tool on CentOS 6 platform.
Read more about YAIM tool here and what are the general YAIM variables for a StoRM deployment.
StoRM Globus GridFTP provides minimal support for YAIM. The following table summaries the general YAIM variables used for StoRM GridFTP component.
| Variable name | Description |
|---|---|
GRIDFTP_WITH_DSI |
Enable StoRM module to compute checksum. Default: false |
GLOBUS_TCP_PORT_RANGE |
GridFTP transfers port range. |
GRIDFTP_CONNECTIONS_MAX |
Set up maximum number of connections. |
The StoRM puppet module can be used to configure the service on CentOS 7 platform.
The module contains the storm::gridftp class that installs the metapackage storm-globus-gridftp-mp and allows site administrator to configure storm-globus-gridftp service by managing the following files:
/etc/grid-security/gridftp.conf, the main configuration file;/etc/sysconfig/storm-globus-gridftp, with some environment variables.The whole list of StoRM GridFTP class parameters can be found here.
Examples of StoRM Gridftp configuration:
class { 'storm::gridftp':
redirect_lcmaps_log => true,
llgt_log_file => '/var/log/storm/storm-gridftp-lcmaps.log',
}
GridFTP produce two separated log files:
The logging level can be specified by editing the configuration file:
/etc/globus-gridftp-server/gridftp.gfork
The supported logging levels are: ERROR, WARN, INFO, DUMP and ALL.
Administrators can redirect the LCMAPS logging to a different log file than the one used by syslog by setting the LLGT_LOG_FILE environment variable.
As example, consider the following setup for the gridftp service:
vim /etc/sysconfig/globus-gridftp
insert:
export LLGT_LOG_FILE="/var/log/storm/storm-gridftp-lcmaps.log"
After restarting the service, all LCMAPS calls will be logged to the new file.