

Contents

1   Introduction
2   Installation
3   Usage
3.1 Available Microservices
3.2 Using Proxy Certificates
3.3 Common Usage Scenarios
3.4 Some remarks to interoperabiltiy of SRM and dCache
3.5 Additional Tips
3.5.1   Proxy certificate validation
3.5.2   Possible adjustments in srm.h
4   Further Information
4.1 License
4.2 Contact
4.3 Links
A   Files
B   API Functions






1   Introduction

This is the documentation of the SRM module, developed for the iRODS Data Management System. The SRM module enables users to access files on Storage Resource Manager (SRM) servers, transfer them from/to iRODS, and manage files or directories on SRM servers. GSI proxy certificates or every iRODS user are supported, or the iRODS server can be run using a single certificate. The SRM microservice can be used with the irule command or it is possible to automate their usage with iRODS actions.
The software is developed within the German DGrid Integrationsprojekt 2 (DGI2) founded by the German Federal Ministry of Education and Research (BMBF). 



2   Installation

1. Copy the SRM module to <irods_dir>/modules.
2. Ensure item "Enabled" in info.txt is set to "yes".
3. The Globus GSI Libraries must be available on your system.
4. Adjust the include flags and library paths in Makefile, if necessary.
5. Run irodssetup.
6. Add Globus Toolkit libraries (<globus_dir>/lib) to your environment variable LD_LIBRARY_PATH.



3   Usage

The SRM microservices are build on working certificate validation. Therefore latest GSI certificate information, esp. CRLs in /etc/grid-security, must be available on iRODS host.

3.1 Available Microservices
msiSrmAbortFiles(*MAN,*SURLS,*TOKEN,*RES)
    Aborts an selective file request.

msiSrmAbortRequest(*MAN,*TOKEN)
    Aborts an existing request.

msiSrmBringOnline(*MAN,*SURLS,*TIME,*DESC,*PROTO,*RES)
    Transfers data online.

msiSrmGetBestSpaceToken(*MAN,*DESC,*SIZE,*RES)
    Returns the spacetoken with the biggest sufficient space.

msiSrmGetPermission(*MAN,*SURLS,*RES)
    Gets owner and group permissions for specified file.

msiSrmGetSpaceMD(*MAN,*TOKEN,*RES)
    Returns all space meta data for the requested space tokens.

msiSrmGetSpaceTokens(*MAN,*DESC,*RES)
    Returns all space tokens of the specified token descriptor.

msiSrmLs(*MAN,*SURLS,*LEV,*RES)
    Lists all files in specified directories.

msiSrmMkDir(*MAN,*SURL)
    Creates a new directory.

msiSrmPing(*MAN,*RES)
    Tests the reachability of a network host.

msiSrmPrepareToGet(*MAN,*SURLS,*DESC,*TIME,*PROTO,*RES)
    Prepares get data transfer operation.

msiSrmPrepareToPut(*MAN,*SIZES,*SURLS,*DESC,*TIME,*PROTO,*RES)
    Prepares 'put' data transfer operation.

msiSrmPutDone(*MAN,*TOKEN,*SURLS,*RES)
    Checks whether a prior put data transfer operation has finished.

msiSrmReleaseFiles(*MAN,*SURLS,*TOKEN,*RES)
    Release pins on previously requested copies of the SURL.

msiSrmRm(*MAN,*SURLS,*RES)
    Deletes specified files.

msiSrmRmDir(*MAN,*SURL,*REC,*RES)
    Deletes specified directory.

msiSrmAddGroupPermission(*MAN,*SURLS,*GRP,*PERM,*RES)
    Adds new Group permissions.

msiSrmChangeGroupPermission(*MAN,*SURLS,*GRP,*PERM,*RES)
    Replaces existing Group permissions with specified ones.

msiSrmRemoveGroupPermission(*MAN,*SURLS,*GRP,*PERM,*RES)
    Removes specified Group Permissions from already existing ones.

msiSrmAddOtherPermission(*MAN,*SURLS,*PERM,*RES)
    Adds new Other permissions.

msiSrmChangeOtherPermission(*MAN,*SURLS,*PERM,*RES)
    Replaces existing Other permissions with specified ones.

msiSrmRemoveOtherPermission(*MAN,*SURLS,*PERM,*RES)
    Removes specified Other permissions from already existing ones.

msiSrmAddOwnerPermission(*MAN,*SURL,*PERM,*RES)
    Adds new Owner permissions.

msiSrmChangeOwnerPermission(*MAN,*SURL,*PERM,*RES)
    Replaces existing Owner permissions with specified ones.

msiSrmRemoveOwnerPermission(*MAN,*SURL,*PERM,*RES)
    Removes specified Owner permissions from already existing ones.

msiSrmAddUserPermission(*MAN,*SURL,*UNAME,*PERM,*RES)
    Adds new User permissions.

msiSrmChangeUserPermission(*MAN,*SURL,*UNAME,*PERM,*RES)
    Replaces existing User permissions with specified ones.

msiSrmRemoveUserPermission(*MAN,*SURL,*UNAME,*PERM,*RES)
    Removes specified User permissions from already existing ones.


3.2 Using Proxy Certificates

The SRM microservices make use of user proxy certificates to authenticate the user on SRM servers. Therefore the user needs to upload a valid proxy certificate to iRODS. As an example the following commands load the proxy certificate of a user with UID 1000 to iRODS.

$ grid-proxy-init
Your identity: /C=DE/O=GridGermany/OU=Technische Universitaet Dresden/CN=Ronny Tschueter
Enter GRID pass phrase for this identity:
Creating proxy .............................. Done
Your proxy is valid until: Tue Nov 30 02:16:27 2010
$ iput -f /tmp/x509up_u1000 user-proxy
$ ils
/tempZone/home/rods:
  user-proxy

It is also possible to create proxy certificate using voms-proxy-init. The certificate must reside within the iRODS home directory of the user and be named "user-proxy". It can be renamed or a path could be added in the proxy_file variable within srm.h header file. See section 3.4.2 for further details. After changes to the header file the SRM module must be recompiled.

The SRM module is also able to work with a single server certificate. Therefore the iRODS server must be started with the userid of a valid certificate.


3.3 Common Usage Scenarios

This section presents a short usage scenario of SRM microservices in iRODS. The following irule command uses the msiSrmLs microservice to list all files of the specified SRM location.

$ irule -v "msiSrmLs(*MAN,*DIR,*LEV,*RES)" "*MAN=httpg://ophelia.zih.tu-dresden.de:8443/srm/managerv2%*DIR=srm://ophelia.zih.tu-dresden.de:8443/srm/managerv2?SFN=/pnfs/zih.tu-dresden.de/data/dgtest/testfile1,srm://ophelia.zih.tu-dresden.de:8443/srm/managerv2?SFN=/pnfs/zih.tu-dresden.de/data/dgtest/new_dir%*LEV=1" "*RES"
rcExecMyRule: msiSrmLs(*MAN,*DIR,*LEV,*RES)
outParamDesc: *RES
ExecMyRule completed successfully.    Output 

*RES: 
SURL:           /pnfs/zih.tu-dresden.de/data/dgtest/testfile1
Type:           File
Size:           2563
Permissions:    rw-r--r--
Locality:       Nearline
Checksum Type:  adler32
Checksum:       3a8a6abd
Space Token:    None
Nr of Subpaths: 0

SURL:           /pnfs/zih.tu-dresden.de/data/dgtest/new_dir
Type:           Directory
Size:           0
Permissions:    rwxrwxr-x
Locality:       Lost
Checksum Type:  No information
Checksum:       No information
Space Token:    None
Nr of Subpaths: 1

	SURL:           /pnfs/zih.tu-dresden.de/data/dgtest/new_dir/1
	Type:           Directory
	Size:           0
	Permissions:    rwxrwxr-x
	Locality:       Lost
	Checksum Type:  No information
	Checksum:       No information
	Space Token:    None
	Nr of Subpaths: 0

The msiSrmLs microservice has four parameters:
  MAN - URL of the SRM manager
  DIR - comma-separated list of Storage URLs (e.g. *DIR=surl1,surl2)
  LEV - sub-directories are processed up to the specified level
  RES - output parameter

The MAN parameter is used by all SRM microservices. Therefore it can be helpful to define an environment variable, which contains SRM manager address information.

$ export MY_SRM_MANAGER=httpg://ophelia.zih.tu-dresden.de:8443/srm/managerv2
$ irule -v "msiSrmLs(*MAN,*DIR,*LEV,*RES)" "*MAN=$MY_SRM_MANAGER%*DIR=...%*LEV=..." "*RES"

The *PERM parameter of SRM microservices to add, change or remove permissions supports following arguments: NONE, R, RW, RX, RWX, W, WX or X. The user can specifiy these arguments both in lower or upper case.

msiSrmPrepareToGet and msiSrmPrepareToGet have a parameter called *DESC to specify a SRM space descriptor. If you don't want to specify any space descriptor set *DESC to NULL.


3.4 Some remarks to interoperabiltiy of SRM and dCache

There some limitations with respect to interoperabiltiy of SRM and dCache. For example, user permissions are currently not supported by dCache and actions to manipulate group permissions has to use hyphen as group ID. If you work with the SRM door of dCache, set *GRP parameter of msiSrm[Add|Change|Remove]GroupPermission microservices to NULL and correct group ID will be set automatically.


3.5 Additional Tips

3.5.1   Proxy certificate validation
In this section it is shown how to use the grid-proxy-info tool to validate user proxy certificates. The grid-proxy-info tool comes with Globus Toolkit and is usually located in <globus_dir>/bin/. First, make a link from grid-proxy-info to <irods_dir>/server/bin/cmd/. In the following example both Globus Toolkit and iRODS are installed to the opt directory.

$ln -s /opt/globus/bin/grid-proxy-info /opt/iRODS/server/bin/cmd

As a result users gain a simple way to validate proxy certificates used in iRODS. To verify a proxy certificate the real path on the file-system to this certificate must be determined (e.g, <irods_dir>/Vault/home/<username>/user-proxy).

$ iexecmd "grid-proxy-info -f /opt/iRODS/Vault/home/rods/user-proxy"
subject  : /C=DE/O=GridGermany/OU=Technische Universitaet Dresden/CN=Ronny Tschueter/CN=1429353664
issuer   : /C=DE/O=GridGermany/OU=Technische Universitaet Dresden/CN=Ronny Tschueter
identity : /C=DE/O=GridGermany/OU=Technische Universitaet Dresden/CN=Ronny Tschueter
type     : RFC 3820 compliant impersonation proxy
strength : 512 bits
path     : /opt/iRODS/Vault/home/rods/user-proxy
timeleft : 11:17:38

Unfortunately the user has to specify the iRODS zone prefix (/opt/iRODS/Vault) everytime he wants to check a certificate. To permit paths relative to the zone prefix put the following script in <irods_dir>/server/bin/cmd.

#!/bin/sh
/opt/globus/bin/grid-proxy-info -f /opt/iRODS/Vault/$1

With the assistance of this script (named test-proxy-certificate in the following example) users can issue iexecmd with the irods path to their certificates, but without the zone prefix.

$ iexecmd "test-proxy-certificate /home/rods/user-proxy"


3.5.2   Possible adjustments in srm.h
Changes to srm.h take effect only after recompiling the SRM module.

- Enable debugging -
If the user defines SRM_DEBUG, additional information about microservice execution will be written to <irods_dir>/server/logs/rodsLog*.
  
- Set user proxy file location -
The variable char *proxy_file points to the user proxy file within iRODS user home directory. This file is used for GSI authentication. The default value is "/user-proxy". The user can change this value or add additional paths. However, don't forget the leading slash.



4.  Further Information

4.1 License

This software is released under BSD license.


4.2 Contact

You can contact the author and developer via email: ronny.tschueter@tu-dresden.de


4.3 Links

IRODS
    www.irods.org/
Globus Toolkit
    www.globus.org/
dCache
    www.d-cache.org/
DGI2 homepage
    http://dgi.d-grid.de/index.php?id=456&L=1
Project homepage
    http://tu-dresden.de/die_tu_dresden/zentrale_einrichtungen/zih/forschung/grid_computing/iRODS
