Puppet Class: storm::backend

Defined in:
manifests/backend.pp

Summary

StoRM Backend puppet module

Overview

Examples:

Example of usage

class { 'storm::backend':
  hostname => 'backend.test.example',
  db_username => 'storm',
  db_password => 'bluemoon',
  srm_pool_members => [
    {
      'hostname' => 'frontend.test.example',
    }
  ],
  gsiftp_pool_members => [
    {
      'hostname' => 'gridftp.test.example',
    },
  ],
  webdav_pool_members => [
    {
      'hostname' => 'webdav.test.example',
    },
  ],
  storage_areas => [
    {
      'name'               => 'test.vo',
      'root_path'          => '/storage/test.vo',
      'access_points'      => ['/test.vo'],
      'vos'                => ['test.vo'],
      'storage_class'      => 'T0D1',
      'online_size'        => 4,
      'transfer_protocols' => ['file', 'gsiftp', 'https'],
    },
  ],
}

Parameters:

  • hostname (String) (defaults to: $::fqdn)

    StoRM Backend Fully Qualified Domain Name

  • install_native_libs_gpfs (Boolean)

    Set this if you need to install storm-native-libs-gpfs. Default: false.

  • db_hostname (String) (defaults to: $hostname)

    Fully Qualified Domain Name of database hostname. Default value: hostname.

  • db_username (String)

    The name of user used to connect to local database. Default: storm

  • db_password (String)

    Password for the user in db_storm_username. Default: bluemoon

  • xroot_hostname (String) (defaults to: $hostname)

    Root server (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration.

  • xroot_port (Integer)

    Root server port (default value for all Storage Areas).

  • gsiftp_pool_balance_strategy (Storm::Backend::BalanceStrategy)

    Load balancing strategy for GridFTP server pool (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration. Available values: round-robin, smart-rr, random, weight. Default value: round-robin. See Storm::Backend::BalanceStrategy.

  • gsiftp_pool_members (Array[Storm::Backend::GsiftpPoolMember])

    Array of Storm::Backend::GsiftpPoolMember. GridFTP servers pool list (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration.

  • webdav_pool_members (Array[Storm::Backend::WebdavPoolMember])

    Array of Storm::Backend::WebdavPoolMember. WebDAV endpoints pool list (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration.

  • srm_pool_members (Array[Storm::Backend::SrmPoolMember])

    Array of Storm::Backend::SrmPoolMember. Frontend endpoints pool list (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration.

  • transfer_protocols (Array[Storm::Backend::TransferProtocol])

    List of supported (and published) transfer protocols (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration.

  • fs_type (Storm::Backend::FsType)

    File System Type (default value for all Storage Areas). Note: you may change the settings for each SA acting on its configuration. Available values: posixfs, gpfs and test. Default value: posixfs See Storm::Backend::FsType.

  • storage_areas (Array[Storm::Backend::StorageArea])

    List of supported Storage Areas. Array of Storm::Backend::StorageArea.

  • frontend_public_host (String) (defaults to: $hostname)

    StoRM Frontend service public host. It’s used by StoRM Info Provider to publish the SRM endpoint into the Resource BDII. Default value: hostname

  • frontend_port (Integer)

    StoRM Frontend service port. Default value: 8444

  • directory_automatic_creation (Boolean)

    Flag to enable automatic missing directory creation upon srmPrepareToPut requests. Default: false.

  • directory_writeperm (Boolean)

    Flag to enable directory write permission setting upon srmMkDir requests on created directories. Default: false.

  • rest_services_port (Integer)

    REST services port. Default value: 9998

  • rest_services_max_threads (Integer)

    REST services max active requests. Default: 100

  • rest_services_max_queue_size (Integer)

    REST services max queue size of accepted requests. Default: 1000

  • xmlrpc_unsecure_server_port (Integer)

    Port to listen on for incoming XML-RPC connections from Frontends(s). Default: 8080

  • xmlrpc_maxthread (Integer)

    Number of threads managing XML-RPC connection from Frontends(s). A well sized value for this parameter have to be at least equal to the sum of the number of working threads in all Frontends. Default: 100.

  • xmlrpc_max_queue_size (Integer)

    Max number of accepted and queued XML-RPC connection from Frontends(s). Default: 1000

  • xmlrpc_security_enabled (Boolean)

    Whether the backend will require a token to be present for accepting XML-RPC requests. Default: true.

  • xmlrpc_security_token (String)

    The token that the backend will require to be present for accepting XML-RPC requests. Mandatory if xmlrpc_security_enabled is true.

  • ptg_skip_acl_setup (Boolean)

    Skip ACL setup for PtG requests. Default: false.

  • pinlifetime_default (Integer)

    Default PinLifetime in seconds used for pinning files in case of srmPrepareToPut or srmPrepareToGet operation without any pinLifetime specified. Default: 259200.

  • pinlifetime_maximum (Integer)

    Maximum PinLifetime allowed in seconds. Default: 1814400.

  • sanity_check_enabled (Boolean)

    Enable/disable sanity checks during bootstrap phase. Default: true.

  • service_du_enabled (Boolean)

    Flag to enable disk usage service. Default: false.

  • service_du_delay (Integer)

    The initial delay before the service is started (seconds). Default: 60.

  • service_du_interval (Integer)

    The interval in seconds between successive run. Default: 360.

  • max_ls_entries (Integer)

    Maximum number of entries returned by an srmLs call. Since in case of recursive srmLs results can be in order of million, this prevent a server overload. Default: 500.

  • gc_pinnedfiles_cleaning_delay (Integer)

    Initial delay before starting the reserved space, JIT ACLs and pinned files garbage collection process, in seconds. Default: 10.

  • gc_pinnedfiles_cleaning_interval (Integer)

    Time interval in seconds between successive purging run. Default: 300.

  • gc_purge_enabled (Boolean)

    Enable the request garbage collector. Default: true.

  • gc_purge_interval (Integer)

    Time interval in seconds between successive purging run. Default: 600.

  • gc_purge_size (Integer)

    Number of requests picked up for cleaning from the requests garbage collector at each run. This value is use also by Tape Recall Garbage Collector. Default: 800

  • gc_expired_request_time (Integer)

    Time in seconds to consider a request expired after its submission. Default: 604800 seconds (1 week). From StoRM 1.11.13 it is used also to identify how much time is needed to consider a completed recall task as cleanable.

  • gc_expired_inprogress_time (Integer)

    Time in seconds to consider an in-progress ptp request as expired. Default: 2592000 seconds (1 month).

  • gc_ptp_transit_interval (Integer)

    Time interval in seconds between successive expired put requests agent run. Default: 3000.

  • gc_ptp_transit_start_delay (Integer)

    Initial delay before starting the expired put requests agent process, in seconds. Default: 60

  • extraslashes_file (String)

    Add extra slashes after the “authority” part of a TURL for file protocol. Defaul: ''

  • extraslashes_root (String)

    Add extra slashes after the “authority” part of a TURL for xroot protocol. Default: '/'

  • extraslashes_gsiftp (String)

    Add extra slashes after the “authority” part of a TURL for gsiftp protocol. Default: '/'

  • db_connection_pool_enabled (Boolean)

    Enable the database connection pool. Default: true

  • db_connection_pool_max_active (Integer)

    Database connection pool max active connections. Default: 10

  • db_connection_pool_max_wait (Integer)

    Database connection pool max wait time to provide a connection. Default: 50

  • asynch_db_reconnect_period (Integer)

    Database connection refresh time intervall in seconds. Default: 18000

  • asynch_db_delay_period (Integer)

    Database connection refresh initial delay in seconds. Default: 30.

  • asynch_picking_initial_delay (Integer)

    Initial delay before starting to pick requests from the DB, in seconds. Default: 1.

  • asynch_picking_time_interval (Integer)

    Polling interval in seconds to pick up new SRM requests. Default: 2.

  • asynch_picking_max_batch_size (Integer)

    Maximum number of requests picked up at each polling time. Default: 100.

  • requests_scheduler_core_size (Integer)

    Crusher Scheduler worker pool base size. Default: 50.

  • requests_scheduler_max_size (Integer)

    Crusher Schedule worker pool max size. Default: 200.

  • requests_scheduler_queue_size (Integer)

    Request queue maximum size. Default: 2000.

  • ptp_requests_scheduler_core_size (Integer)

    PrepareToPut worker pool base size. Default: 50.

  • ptp_requests_scheduler_max_size (Integer)

    PrepareToPut worker pool max size. Default: 200.

  • ptp_requests_scheduler_queue_size (Integer)

    PrepareToPut request queue maximum size. Default: 1000.

  • ptg_requests_scheduler_core_size (Integer)

    PrepareToGet worker pool base size. Default: 50.

  • ptg_requests_scheduler_max_size (Integer)

    PrepareToGet worker pool max size. Default: 200.

  • ptg_requests_scheduler_queue_size (Integer)

    PrepareToGet request queue maximum size. Default: 2000.

  • bol_requests_scheduler_core_size (Integer)

    BringOnline worker pool base size. Default: 50.

  • bol_requests_scheduler_max_size (Integer)

    BringOnline Worker pool max size. Default: 200.

  • bol_requests_scheduler_queue_size (Integer)

    BringOnline request queue maximum size. Default: 2000.

  • info_sitename (String)

    It’s the human-readable name of your site used to set the Glue-SiteName attribute.

  • info_storage_default_root (String)

    Default directory for Storage Areas.

  • info_endpoint_quality_level (Integer)

    Endpoint maturity level to be published by the StoRM info provider. Default value: 2.

  • info_webdav_pool_list (Array[Storm::Backend::WebdavPoolMember])

    List of published webdav endpoints.

  • info_frontend_host_list (Array[Storm::Backend::SrmPoolMember])

    List of published srm endpoints.

  • jvm_options (String)
  • jmx (Boolean)
  • jmx_options (String)
  • debug (Boolean)
  • debug_port (Integer)
  • debug_suspend (Boolean)
  • lcmaps_db_file (String)
  • lcmaps_policy_name (String)
  • lcmaps_log_file (String)
  • lcmaps_debug_level (Integer)
  • http_turl_prefix (String)
  • storm_limit_nofile (Integer)

    Sets LimitNOFILE value.

  • manage_path_authz_db (Boolean)

    If true, allows to set content of path-authz.db file.

  • path_authz_db_file (String)

    If manage_path_authz_db is true, set the content from this source path

  • info_config_file (String)


304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
# File 'manifests/backend.pp', line 304

class storm::backend (

  # Install native libs gpfs
  Boolean $install_native_libs_gpfs,

  # Db connection
  String $db_username,
  String $db_password,

  ### Default values for Storage Areas
  # 1. xroot
  Integer $xroot_port,
  # 2. gridftp pool
  Storm::Backend::BalanceStrategy $gsiftp_pool_balance_strategy,
  Array[Storm::Backend::GsiftpPoolMember] $gsiftp_pool_members,
  # 3. webdav pool
  Array[Storm::Backend::WebdavPoolMember] $webdav_pool_members,
  # 4. frontend pool
  Array[Storm::Backend::SrmPoolMember] $srm_pool_members,
  # 5. transfer protocols
  Array[Storm::Backend::TransferProtocol] $transfer_protocols,
  # 6. fs-type
  Storm::Backend::FsType $fs_type,

  # Storage Areas
  Array[Storm::Backend::StorageArea] $storage_areas,

  # Frontend public host and port
  Integer $frontend_port,

  # Directory options
  Boolean $directory_automatic_creation,
  Boolean $directory_writeperm,

  # REST server conf
  Integer $rest_services_port,
  Integer $rest_services_max_threads,
  Integer $rest_services_max_queue_size,

  # XMLRPC Server parameter
  Integer $xmlrpc_unsecure_server_port,
  Integer $xmlrpc_maxthread,
  Integer $xmlrpc_max_queue_size,
  Boolean $xmlrpc_security_enabled,
  String $xmlrpc_security_token,

  # Skip ACL setup for PTG requests
  Boolean $ptg_skip_acl_setup,

  # Pin lifetime
  Integer $pinlifetime_default,
  Integer $pinlifetime_maximum,

  # Sanity checks
  Boolean $sanity_check_enabled,

  # DU service
  Boolean $service_du_enabled,
  Integer $service_du_delay,
  Integer $service_du_interval,

  # Ls max entries
  Integer $max_ls_entries,

  # Pinned Files cleaning parameters
  Integer $gc_pinnedfiles_cleaning_delay,
  Integer $gc_pinnedfiles_cleaning_interval,

  # Garbage Collector
  Boolean $gc_purge_enabled,
  Integer $gc_purge_interval,
  Integer $gc_purge_size,
  Integer $gc_expired_request_time,
  Integer $gc_expired_inprogress_time,
  Integer $gc_ptp_transit_interval,
  Integer $gc_ptp_transit_start_delay,

  # Extraslashes
  String $extraslashes_file,
  String $extraslashes_root,
  String $extraslashes_gsiftp,

  # Db Connection Pool
  Boolean $db_connection_pool_enabled,
  Integer $db_connection_pool_max_active,
  Integer $db_connection_pool_max_wait,

  # Asynch Picker
  Integer $asynch_db_reconnect_period,
  Integer $asynch_db_delay_period,
  Integer $asynch_picking_initial_delay,
  Integer $asynch_picking_time_interval,
  Integer $asynch_picking_max_batch_size,

  # Requests schedulers
  Integer $requests_scheduler_core_size,
  Integer $requests_scheduler_max_size,
  Integer $requests_scheduler_queue_size,
  Integer $ptp_requests_scheduler_core_size,
  Integer $ptp_requests_scheduler_max_size,
  Integer $ptp_requests_scheduler_queue_size,
  Integer $ptg_requests_scheduler_core_size,
  Integer $ptg_requests_scheduler_max_size,
  Integer $ptg_requests_scheduler_queue_size,
  Integer $bol_requests_scheduler_core_size,
  Integer $bol_requests_scheduler_max_size,
  Integer $bol_requests_scheduler_queue_size,

  # Info Provider
  String $info_config_file,
  String $info_sitename,
  String $info_storage_default_root,
  Integer $info_endpoint_quality_level,
  Array[Storm::Backend::WebdavPoolMember] $info_webdav_pool_list,
  Array[Storm::Backend::SrmPoolMember] $info_frontend_host_list,

  # JVM options
  String $jvm_options,

  # JMX options
  Boolean $jmx,
  String $jmx_options,

  # Debug options
  Boolean $debug,
  Integer $debug_port,
  Boolean $debug_suspend,

  # LCMAPS
  String $lcmaps_db_file,
  String $lcmaps_policy_name,
  String $lcmaps_log_file,
  Integer $lcmaps_debug_level,

  # HTTP TURL prefix
  String $http_turl_prefix,

  # LimitNOFILE
  Integer $storm_limit_nofile,

  # manage path-authz.db
  Boolean $manage_path_authz_db,
  String $path_authz_db_file,

  # hostnames
  String $hostname = $::fqdn,
  String $db_hostname = $hostname,
  String $xroot_hostname = $hostname,
  String $frontend_public_host = $hostname,

) {

  contain storm::backend::install
  contain storm::backend::configdb
  contain storm::backend::config
  contain storm::backend::service

  Class['storm::backend::install']
  -> Class['storm::backend::configdb']
  -> Class['storm::backend::config']
  -> Class['storm::backend::service']
}