

Content

1	Installation
2	Usage
2.1	Available Microservices
2.2	Using Proxy Certificates
2.3	Common Usage Scenarios
2.4	Performing a module test
2.5	Additional Tips
2.5.1	How to enable a user to verify he is using a valid proxy certificate
2.5.2	Possible adjustments in gridftp.h file
3	Further Information
3.1	License
3.2	Contact
3.3	Links






1	Installation

1.	Copy the GridFTP module to <irods_dir>/modules.
2.	Ensure in info.txt Enabled is set to: yes.
3.	The Globus GridFTP and GSI Libraries must be available on your system.
4.	Adjust the include flags and library paths in Makefile, if necessary.
5.	Compile iRODS.
6.	Environment variable LD_LIBRARY_PATH must be set to Globus Toolkit libraries 
	(<globus_dir>/lib) to make iRODS starting properly.


2	Usage

GSI certificate information, esp. CRLs in /etc/grid-security must be available on 
iRODS host and up to date, that certificate validation works.


2.1	Available Microservices

msiGridftpTest()			A test microservice to verify the module is 
					integrated correctly.
msiGridftpFuncTest(*SRC,*DST,*RES)
					Test microservice that tests all the GridFTP 
					module functions.
msiGridftpLs(*SRC,*OUT)			Returns a list of the directory entries.
msiGridftpPut(*SRC,*DST)		Puts an iRODS file to a GridFTP server.
msiGridftpGet(*SRC,*DST)		Gets a file from a GridFTP server to iRODS.
msiGridftpCp(*SRC,*DST)			Copies a file on the server from SRC to DST
msiGridftpMv(*SRC,*DST)*		Moves a file on the server from SRC to DST
msiGridftpDel(*DST)			Deletes a file.
msiGridftpMkdir(*DST)			Creates a directory.
msiGridftpRmdir(*DST)			Deletes a directory.
msiGridftpChmod(*MODE,*DST)*		Changes permissions for a file. Unix like 
					permissions are used (600, 755, 644).
msiGridftpSize(*SRC,*OUT)		Returns the size of a file.
msiGridftpModtime(*DST,*OUT)		Returns the last modification time of a file.
msiGridftpCksm(*SRC,*OUT)*		Returns the checksum of a file.
msiGridftpExist(*DST,*OUT)		Checks whether a file exists.
*see 2.5.2.2


2.2	Using Proxy Certificates

The microservices make use of user proxy certificates to authenticate the user 
on GridFTP servers. Therefore the user needs to upload a valid proxy certificate 
to iRODS:

	~> grid-proxy-init
	Your identity: /C=DE/O=GridGermanyOU=Technische Universitaet Dresden/OU=ZIH/CN=Christian Loeschen
	Enter GRID pass phrase for this identity:
	Creating proxy ............................................ Done
	Your proxy is valid until: Wed May  5 02:58:46 2010
	~> iput /tmp/x509up_u1000 user-proxy
	~> ils
	/tempZone/home/rods:
		user-proxy
	
The proxy certificate can be created using voms-proxy-init, too. The certificate 
must reside within the iRODS home direcory of the user and be named "user-proxy". 
It can be renamed or a path could be added in the proxy_file variable within 
gridftp.h header file. The module must be compiled after that to take affect.
The GridFTP module is also able to work with a single server certificate. 
Therefore the iRODS server must be started with the userid of a valid certificate.


2.3	Common Usage Scenarios

Some common scenarios how to use GridFTP microservices with the irule command, 
combined within Rules are described within this section.
This one puts a single file from the iRODS user directory to a GridFTP server:

	~> irule -v "msiGridftpPut(*SRC,*DST)" "*SRC=/tempZone/home/rods/foo.bar%*DST=gsiftp://helena.zih.tu-dresden.de/pnfs/zih.tu-dresden.de/data/ghep/foo.bar" ""

This one changes the access rights for a file:

	~> irule -v "msiGridftpChmod(*MODE,*DST)" "*MODE=644%*DST=gsiftp://helena.zih.tu-dresden.de/pnfs/zih.tu-dresden.de/data/ghep/foo.bar" ""

But now let's do some more advanced stuff. In the next exampled, a directory is 
listed, and the output is processed in following microservices:

	~> irule -v "msiGridftpLs(*DIR, *FILES)##forEachExec(*FILES,writeLine(stdout,*FILES),nop)|nop" "*DIR=gsiftp://helena.zih.tu-dresden.de/pnfs/zih.tu-dresden.de/data/ghep/" "ruleExecOut"
	~> irule -v "msiGridftpLs(*DIR, *FILES)##writeLine(stdout,)##forEachExec(*FILES,msiGridftpSize(*FILES,*SIZE)##msiGridftpModtime(*FILES,*TIME)##writeLine(stdout,*TIME  *SIZE  *FILES),nop)|nop" "*DIR=gsiftp://helena.zih.tu-dresden.de/pnfs/zih.tu-dresden.de/data/ghep/" "ruleExecOut"
	~> irule -v "msiGridftpLs(*DIR, *FILES)##forEachExec(*FILES,msiGridftpDel(*FILES),nop)|nop" "*DIR=gsiftp://helena.zih.tu-dresden.de/pnfs/zih.tu-dresden.de/data/ghep/" ""
	~> irule -v "msiGridftpLs(*DIR, *FILES)##forEachExec(*FILES,msiGridftpGet(*FILES,*DEST),nop)|nop" "*DIR=gsiftp://helena.zih.tu-dresden.de/pnfs/zih.tu-dresden.de/data/ghep/%*DEST=/tempZone/home/rods/" ""

This is a simple check whether a file exists with some output:

	irule -v "msiGridftpExist(*DST,*OUT)##ifExec(*OUT == YES,writeLine(stdout,*DST exists),nop,writeLine(stdout,*DST not exists),nop)|nop" "*DST=gsiftp://helena.zih.tu-dresden.de/pnfs/zih.tu-dresden.de/data/ghep/foo.bar" "ruleExecOut"

It is also possible to add some actions in 
<irods_dir>/server/config/reConfigs/core.irb:

	acGridftpMv(*SRC,*DST)||msiGridftpCp(*SRC,*DST)##msiGridftpDel(*SRC)|nop

So one can use gridftp move, though msiGridftpMv won't work, as on my test machine:

	~> irule -v "acGridftpMv(*SRC,*DST)" "*SRC=gsiftp://helena.zih.tu-dresden.de/pnfs/zih.tu-dresden.de/data/ghep/foo%*DST=gsiftp://helena.zih.tu-dresden.de/pnfs/zih.tu-dresden.de/data/ghep/foo.bar" ""


2.4	Performing a module test

msiGridftpTest is just a test microservice, that indicates whether the GridFTP 
module is integrated correctly into iRODS. When it's output look like this, the 
GridFTP module is available:

	~> irule --test "msiGridftpTest" *A=0 *B
	Level 0: DEBUG:     msiGridftpTest log

With the msiGridftpFuncTest microservice a full test is run testing all GridFTP 
module functions. Two parameters are necessary:

- *SRC: an existing iRODS file to copy to the GridFTP server for testing
- *DST: an existing directory on the GridFTP server to write the test to
	
	~> irule -v "msiGridftpFuncTest(*SRC, *DST, *RES)" "*SRC=/tempZone/home/rods/foo.bar%*DST=gsiftp://helena.zih.tu-dresden.de/pnfs/zih.tu-dresden.de/data/ghep" "*RES"
	rcExecMyRule: msiGridftpFuncTest(*SRC, *DST, *RES)
	outParamDesc: *RES
	ExecMyRule completed successfully.    Output

	  output index: 0
	    label: *RES
	    type: STR_PI
	    str content:
	gridftp mkdir: ok
	gridftp put: ok
	gridftp cp: ok
	gridftp ls: ok
	gridftp exist: ok
	gridftp size: ok
	gridftp modtime: ok
	gridftp cksm: FAILED
	gridftp chmod: FAILED
	gridftp get: ok
	gridftp mv: FAILED
	gridftp del: ok
	gridftp rmdir: ok

As one can see, my GridFTP test server doesn't support checksum, chmod and move 
functions.


2.5	Additional Tips

2.5.1	How to enable a user to verify he is using a valid proxy certificate

For this purpose it is necessary to put a link from grid-proxy-info into 
<irods_dir>/server/bin/cmd/. grid-proxy-info comes with Globus Toolkit and is 
usually located in <globus_dir>/bin/. Now the user needs to know the real path 
on the filesystem to his proxy certificate. This is for example 
<irods_dir>/Vault/home/<username>/user-proxy. Using iexecmd the user can verify 
his proxy certificate:

	~> iexecmd "grid-proxy-info -f /opt/iRODS/Vault/home/rods/user-proxy"
	subject  : /C=DE/O=GridGermany/OU=Technische Universitaet Dresden/OU=ZIH/CN=Christian Loeschen/CN=203199236
	issuer   : /C=DE/O=GridGermany/OU=Technische Universitaet Dresden/OU=ZIH/CN=Christian Loeschen
	identity : /C=DE/O=GridGermany/OU=Technische Universitaet Dresden/OU=ZIH/CN=Christian Loeschen
	type     : RFC 3820 compliant impersonation proxy
	strength : 512 bits
	path     : /opt/iRODS/Vault/home/rods/user-proxy
	timeleft : 0:00:00

As one can see, my proxy certificate is expired.
Alternatively a small script can be put in <irods_dir>/server/bin/cmd:

	~> cat /opt/iRODS/server/bin/cmd/test-proxy
	#!/bin/sh
	/opt/globus/bin/grid-proxy-info -f /opt/iRODS/Vault$1

Now a user can issue iexecmd with the irods path to his certificate, but without 
the zone prefix:

	~> iexecmd "test-proxy /home/rods/user-proxy"


2.5.2	Possible adjustments in gridftp.h file

To take effect of changes of one of the following options the GridFTP module must 
be recompiled.


2.5.2.1	Enable debugging

Defining GRIDFTP_DEBUG makes the GridFTP microservices more verbose. Then in 
<rods_dir>/server/log/rodsLog* additional information about microservice execution 
can be found.


2.5.2.2	Disable feature check

Because some GridFTP server (like my dCache test machine) don't understand every 
GridFTP command, some of them won't work. Before executing a command, the GridFTP 
server is asked whether the command is supported. Because there seem to be 
different feature mappings within different libraries, this function may return 
wrong values. When there occure any problems, define #GRIDFTP_DISABLE_FEATURE_CHECK 
to prevent the feature check. Then the feature check always returns true.


2.5.2.3	Set user proxy file location

char *proxy_file points to the user proxy file for GSI authentication within the 
iRODS user home directory. It's default value is "/user-proxy". You can change 
this, and also add additional paths, but don't forget the leading shash.
Pay attentention to apply changes also to 2.5.1.


2.5.2.4	Build a GridFTP command line client

As a fall-out of the development of the GridFTP module, a simple command line GridFTP 
client is arised. If GRIDFTP_IRODS_MODULE is defined, the client is build. In the 
console_client/ directory the makefile for building the client is located. Change to 
this directory and issue a make. Therefore, no iRODS server is necessary. After building 
the binary is in this directory, too. To see a brief introduction about its functions 
you can simply type

	~> gridftp-client
	usage: gridftp
    	    get|put|cp|mv <src-url> <dst-url>
        	ls|del|rmdir|mkdir|exist|size|cksm|modtime|feat <url>
        	chmod <mode> <url>
        
Anything else works as usual:

	~> gridftp-client put /bin/sh gsiftp://helena.zih.tu-dresden.de/pnfs/zih.tu-dresden.de/data/ghep/foo.bar
	~> gridftp-client ls gsiftp://helena.zih.tu-dresden.de/pnfs/zih.tu-dresden.de/data/ghep/
	foo.bar
	~> gridftp-client size gsiftp://helena.zih.tu-dresden.de/pnfs/zih.tu-dresden.de/data/ghep/foo.bar
	606864
	~> gridftp-client modtime gsiftp://helena.zih.tu-dresden.de/pnfs/zih.tu-dresden.de/data/ghep/foo.bar
	Wed Jul  7 17:45:09 2010
	~> gridftp-client del gsiftp://helena.zih.tu-dresden.de/pnfs/zih.tu-dresden.de/data/ghep/foo.bar
	~> gridftp-client exist gsiftp://helena.zih.tu-dresden.de/pnfs/zih.tu-dresden.de/data/ghep/foo.bar
	file does not exist


3	Further Information

3.1	License

This software is released under BSD license.


3.2	Contact

You can contact the author and developer via email: christian.loeschen@tu-dresden.de


3.3	Links

IRODS				www.irods.org/
Globus Toolkit		www.globus.org/toolkit/
DGI2 homepage		http://dgi.d-grid.de/index.php?id=456&L=1
Project homepage	http://tu-dresden.de/die_tu_dresden/zentrale_einrichtungen/zih/forschung/grid_computing/iRODS
