Puppet Class: storm::backend

Defined in:

manifests/backend.pp

Summary

StoRM Backend puppet module

Overview

Examples:

Example of usage

class { 'storm::backend':
  hostname => 'backend.test.example',
  db_username => 'storm',
  db_password => 'bluemoon',
  srm_pool_members => [
    {
      'hostname' => 'frontend.test.example',
    }
  ],
  gsiftp_pool_members => [
    {
      'hostname' => 'gridftp.test.example',
    },
  ],
  webdav_pool_members => [
    {
      'hostname' => 'webdav.test.example',
    },
  ],
  storage_areas => [
    {
      'name'               => 'test.vo',
      'root_path'          => '/storage/test.vo',
      'access_points'      => ['/test.vo'],
      'vos'                => ['test.vo'],
      'storage_class'      => 'T0D1',
      'online_size'        => 4,
      'transfer_protocols' => ['file', 'gsiftp', 'https'],
    },
  ],
}

Parameters:

• hostname (String) (defaults to: $fqdn) —

  StoRM Backend Fully Qualified Domain Name

• db_hostname (String) (defaults to: $hostname) —

  Fully Qualified Domain Name of database hostname. Default value: hostname.

• db_username (String) —

  The name of user used to connect to local database. Default: storm

• db_password (String) —

  Password for the user in db_storm_username. Default: bluemoon

• xroot_hostname (String) (defaults to: $hostname) —

  Root server (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration.

• xroot_port (Integer) —

  Root server port (default value for all Storage Areas).

• gsiftp_pool_balance_strategy (Storm::Backend::BalanceStrategy) —

  Load balancing strategy for GridFTP server pool (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration. Available values: round-robin, smart-rr, random, weight. Default value: round-robin. See Storm::Backend::BalanceStrategy.

• gsiftp_pool_members (Array[Storm::Backend::GsiftpPoolMember]) —

  Array of Storm::Backend::GsiftpPoolMember. GridFTP servers pool list (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration.

• webdav_pool_balance_strategy (Storm::Backend::BalanceStrategy) —

  Load balancing strategy for WebDAV server pools (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration. Available values: round-robin, smart-rr, random, weight. Default value: round-robin. See Storm::Backend::BalanceStrategy.

• webdav_pool_members (Array[Storm::Backend::WebdavPoolMember]) —

  Array of Storm::Backend::WebdavPoolMember. WebDAV endpoints pool list (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration.

• srm_pool_members (Array[Storm::Backend::SrmPoolMember]) —

  Array of Storm::Backend::SrmPoolMember. Frontend endpoints pool list (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration.

• transfer_protocols (Array[Storm::Backend::TransferProtocol]) —

  List of supported (and published) transfer protocols (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration.

• fs_type (Storm::Backend::FsType) —

  File System Type (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration. Available values: posixfs, gpfs and test. Default value: posixfs See Storm::Backend::FsType.

• storage_areas (Array[Storm::Backend::StorageArea]) —

  List of supported Storage Areas. Array of Storm::Backend::StorageArea.

• frontend_public_host (String) (defaults to: $hostname) —

  StoRM Frontend service public host. It’s used by StoRM Info Provider to publish the SRM endpoint into the Resource BDII. Default value: hostname

• frontend_port (Integer) —

  StoRM Frontend service port. Default value: 8444

• directory_automatic_creation (Boolean) —

  Flag to enable automatic missing directory creation upon srmPrepareToPut requests. Default: false.

• directory_writeperm (Boolean) —

  Flag to enable directory write permission setting upon srmMkDir requests on created directories. Default: false.

• rest_services_port (Integer) —

  REST services port. Default value: 9998

• rest_services_max_threads (Integer) —

  REST services max active requests. Default: 100

• rest_services_max_queue_size (Integer) —

  REST services max queue size of accepted requests. Default: 1000

• xmlrpc_unsecure_server_port (Integer) —

  Port to listen on for incoming XML-RPC connections from Frontends(s). Default: 8080

• xmlrpc_maxthread (Integer) —

  Number of threads managing XML-RPC connection from Frontends(s). A well sized value for this parameter have to be at least equal to the sum of the number of working threads in all Frontends. Default: 100.

• xmlrpc_max_queue_size (Integer) —

  Max number of accepted and queued XML-RPC connection from Frontends(s). Default: 1000

• xmlrpc_security_enabled (Boolean) —

  Whether the backend will require a token to be present for accepting XML-RPC requests. Default: true.

• xmlrpc_security_token (String) —

  The token that the backend will require to be present for accepting XML-RPC requests. Mandatory if xmlrpc_security_enabled is true.

• ptg_skip_acl_setup (Boolean) —

  Skip ACL setup for PtG requests. Default: false.

• pinlifetime_default (Integer) —

  Default PinLifetime in seconds used for pinning files in case of srmPrepareToPut or srmPrepareToGet operation without any pinLifetime specified. Default: 259200.

• pinlifetime_maximum (Integer) —

  Maximum PinLifetime allowed in seconds. Default: 1814400.

• sanity_check_enabled (Boolean) —

  Enable/disable sanity checks during bootstrap phase. Default: true.

• service_du_enabled (Boolean) —

  Flag to enable disk usage service. Default: false.

• service_du_delay (Integer) —

  The initial delay before the service is started (seconds). Default: 60.

• service_du_interval (Integer) —

  The interval in seconds between successive run. Default: 360.

• max_ls_entries (Integer) —

  Maximum number of entries returned by an srmLs call. Since in case of recursive srmLs results can be in order of million, this prevent a server overload. Default: 500.

• gc_pinnedfiles_cleaning_delay (Integer) —

  Initial delay before starting the reserved space, JIT ACLs and pinned files garbage collection process, in seconds. Default: 10.

• gc_pinnedfiles_cleaning_interval (Integer) —

  Time interval in seconds between successive purging run. Default: 300.

• gc_purge_enabled (Boolean) —

  Enable the request garbage collector. Default: true.

• gc_purge_interval (Integer) —

  Time interval in seconds between successive purging run. Default: 600.

• gc_purge_size (Integer) —

  Number of requests picked up for cleaning from the requests garbage collector at each run. This value is use also by Tape Recall Garbage Collector. Default: 800

• gc_expired_request_time (Integer) —

  Time in seconds to consider a request expired after its submission. Default: 604800 seconds (1 week). From StoRM 1.11.13 it is used also to identify how much time is needed to consider a completed recall task as cleanable.

• gc_expired_inprogress_time (Integer) —

  Time in seconds to consider an in-progress ptp request as expired. Default: 2592000 seconds (1 month).

• gc_ptp_transit_interval (Integer) —

  Time interval in seconds between successive expired put requests agent run. Default: 3000.

• gc_ptp_transit_start_delay (Integer) —

  Initial delay before starting the expired put requests agent process, in seconds. Default: 60

• extraslashes_file (String) —

  Add extra slashes after the “authority” part of a TURL for file protocol. Defaul: ''

• extraslashes_root (String) —

  Add extra slashes after the “authority” part of a TURL for xroot protocol. Default: '/'

• extraslashes_gsiftp (String) —

  Add extra slashes after the “authority” part of a TURL for gsiftp protocol. Default: '/'

• db_connection_pool_enabled (Boolean) —

  Enable the database connection pool. Default: true

• db_connection_pool_max_active (Integer) —

  Database connection pool max active connections. Default: 10

• db_connection_pool_max_wait (Integer) —

  Database connection pool max wait time to provide a connection. Default: 50

• asynch_db_reconnect_period (Integer) —

  Database connection refresh time intervall in seconds. Default: 18000

• asynch_db_delay_period (Integer) —

  Database connection refresh initial delay in seconds. Default: 30.

• asynch_picking_initial_delay (Integer) —

  Initial delay before starting to pick requests from the DB, in seconds. Default: 1.

• asynch_picking_time_interval (Integer) —

  Polling interval in seconds to pick up new SRM requests. Default: 2.

• asynch_picking_max_batch_size (Integer) —

  Maximum number of requests picked up at each polling time. Default: 100.

• requests_scheduler_core_size (Integer) —

  Crusher Scheduler worker pool base size. Default: 50.

• requests_scheduler_max_size (Integer) —

  Crusher Schedule worker pool max size. Default: 200.

• requests_scheduler_queue_size (Integer) —

  Request queue maximum size. Default: 2000.

• ptp_requests_scheduler_core_size (Integer) —

  PrepareToPut worker pool base size. Default: 50.

• ptp_requests_scheduler_max_size (Integer) —

  PrepareToPut worker pool max size. Default: 200.

• ptp_requests_scheduler_queue_size (Integer) —

  PrepareToPut request queue maximum size. Default: 1000.

• ptg_requests_scheduler_core_size (Integer) —

  PrepareToGet worker pool base size. Default: 50.

• ptg_requests_scheduler_max_size (Integer) —

  PrepareToGet worker pool max size. Default: 200.

• ptg_requests_scheduler_queue_size (Integer) —

  PrepareToGet request queue maximum size. Default: 2000.

• bol_requests_scheduler_core_size (Integer) —

  BringOnline worker pool base size. Default: 50.

• bol_requests_scheduler_max_size (Integer) —

  BringOnline Worker pool max size. Default: 200.

• bol_requests_scheduler_queue_size (Integer) —

  BringOnline request queue maximum size. Default: 2000.

• info_config_file (String) —

  The key-value file used by StoRM Info Provider as its configuration file.

• info_sitename (String) —

  It’s the human-readable name of your site used to set the Glue-SiteName attribute.

• info_storage_default_root (String) —

  Default directory for Storage Areas.

• info_endpoint_quality_level (Integer) —

  Endpoint maturity level to be published by the StoRM info provider. Default value: 2.

• info_webdav_pool_list (Array[Storm::Backend::WebdavPoolMember]) —

  List of published webdav endpoints.

• info_frontend_host_list (Array[Storm::Backend::SrmPoolMember]) —

  List of published srm endpoints.

• jvm_options (String)
• jmx (Boolean)
• jmx_options (String)
• lcmaps_db_file (String)
• lcmaps_policy_name (String)
• lcmaps_log_file (String)
• lcmaps_debug_level (Integer)
• http_turl_prefix (String)
• storm_limit_nofile (Integer) —

  Sets LimitNOFILE value.

• manage_path_authz_db (Boolean) —

  If true, allows to set content of path-authz.db file.

• path_authz_db_file (String) —

  If manage_path_authz_db is true, set the content from this source path

# File 'manifests/backend.pp', line 304

class storm::backend (

  # Db connection
  String $db_username,
  String $db_password,

  ### Default values for Storage Areas
  # 1. xroot
  Integer $xroot_port,
  # 2. gridftp pool
  Storm::Backend::BalanceStrategy $gsiftp_pool_balance_strategy,
  Array[Storm::Backend::GsiftpPoolMember] $gsiftp_pool_members,
  # 3. webdav pool
  Storm::Backend::BalanceStrategy $webdav_pool_balance_strategy,
  Array[Storm::Backend::WebdavPoolMember] $webdav_pool_members,
  # 4. frontend pool
  Array[Storm::Backend::SrmPoolMember] $srm_pool_members,
  # 5. transfer protocols
  Array[Storm::Backend::TransferProtocol] $transfer_protocols,
  # 6. fs-type
  Storm::Backend::FsType $fs_type,

  # Storage Areas
  Array[Storm::Backend::StorageArea] $storage_areas,

  # Frontend public host and port
  Integer $frontend_port,

  # Directory options
  Boolean $directory_automatic_creation,
  Boolean $directory_writeperm,

  # REST server conf
  Integer $rest_services_port,
  Integer $rest_services_max_threads,
  Integer $rest_services_max_queue_size,

  # XMLRPC Server parameter
  Integer $xmlrpc_unsecure_server_port,
  Integer $xmlrpc_maxthread,
  Integer $xmlrpc_max_queue_size,
  Boolean $xmlrpc_security_enabled,
  String $xmlrpc_security_token,

  # Skip ACL setup for PTG requests
  Boolean $ptg_skip_acl_setup,

  # Pin lifetime
  Integer $pinlifetime_default,
  Integer $pinlifetime_maximum,

  # Sanity checks
  Boolean $sanity_check_enabled,

  # DU service
  Boolean $service_du_enabled,
  Integer $service_du_delay,
  Integer $service_du_interval,

  # Ls max entries
  Integer $max_ls_entries,

  # Pinned Files cleaning parameters
  Integer $gc_pinnedfiles_cleaning_delay,
  Integer $gc_pinnedfiles_cleaning_interval,

  # Garbage Collector
  Boolean $gc_purge_enabled,
  Integer $gc_purge_interval,
  Integer $gc_purge_size,
  Integer $gc_expired_request_time,
  Integer $gc_expired_inprogress_time,
  Integer $gc_ptp_transit_interval,
  Integer $gc_ptp_transit_start_delay,

  # Extraslashes
  String $extraslashes_file,
  String $extraslashes_root,
  String $extraslashes_gsiftp,

  # Db Connection Pool
  Boolean $db_connection_pool_enabled,
  Integer $db_connection_pool_max_active,
  Integer $db_connection_pool_max_wait,

  # Asynch Picker
  Integer $asynch_db_reconnect_period,
  Integer $asynch_db_delay_period,
  Integer $asynch_picking_initial_delay,
  Integer $asynch_picking_time_interval,
  Integer $asynch_picking_max_batch_size,

  # Requests schedulers
  Integer $requests_scheduler_core_size,
  Integer $requests_scheduler_max_size,
  Integer $requests_scheduler_queue_size,
  Integer $ptp_requests_scheduler_core_size,
  Integer $ptp_requests_scheduler_max_size,
  Integer $ptp_requests_scheduler_queue_size,
  Integer $ptg_requests_scheduler_core_size,
  Integer $ptg_requests_scheduler_max_size,
  Integer $ptg_requests_scheduler_queue_size,
  Integer $bol_requests_scheduler_core_size,
  Integer $bol_requests_scheduler_max_size,
  Integer $bol_requests_scheduler_queue_size,

  # Info Provider
  String $info_config_file,
  String $info_sitename,
  String $info_storage_default_root,
  Integer $info_endpoint_quality_level,
  Array[Storm::Backend::WebdavPoolMember] $info_webdav_pool_list,
  Array[Storm::Backend::SrmPoolMember] $info_frontend_host_list,

  # JVM options
  String $jvm_options,

  # JMX options
  Boolean $jmx,
  String $jmx_options,

  # LCMAPS
  String $lcmaps_db_file,
  String $lcmaps_policy_name,
  String $lcmaps_log_file,
  Integer $lcmaps_debug_level,

  # HTTP TURL prefix
  String $http_turl_prefix,

  # LimitNOFILE
  Integer $storm_limit_nofile,

  # manage path-authz.db
  Boolean $manage_path_authz_db,
  String $path_authz_db_file,

  # hostnames
  String $hostname = $fqdn,
  String $db_hostname = $hostname,
  String $xroot_hostname = $hostname,
  String $frontend_public_host = $hostname,

) {
  contain storm::backend::install
  contain storm::backend::configdb
  contain storm::backend::config
  contain storm::backend::service

  Class['storm::backend::install']
  -> Class['storm::backend::configdb']
  -> Class['storm::backend::config']
  -> Class['storm::backend::service']
}