18 KiB
Executable File
iRODS server setup
iRODS test server
This is a sample virtual machine with an iRODS server instance and S3 compatible storage.
VM test server
4 cpu 4GB RAM 40GB disk hostname irods.local user irods pass password ip 192.168.150.56 (should be a static ip to avoid hostname/domain/ssl type issues, millage may vary, virtualbox/hyperv NAT network maybe help) os Ubuntu 18.04
iRODS server installation
Server prep
sudo apt-get install wget ca-certificates
Netplan
Static ip required for minio s3 connectivity from irods.
sudo nano -cw /etc/netplan/01-netcfg.yaml
network:
version: 2
renderer: networkd
ethernets:
eth0:
addresses:
- 192.168.150.56/24
gateway4: 192.168.150.1
nameservers:
search: [local]
addresses: [192.168.150.1]
sudo netplan apply
Sudoers
Setup /etc/sudoers to include:
irods ALL=(ALL) NOPASSWD:ALL
Ubuntu software repo
wget -qO - https://packages.irods.org/irods-signing-key.asc | sudo apt-key add -
echo "deb [arch=amd64] https://packages.irods.org/apt/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/renci-irods.list
sudo apt-get update
Setup Postgresql database, install iRODS server
Install postgress packages.
sudo apt-get install postgresql-10 postgresql-client-10 postgresql-client-common postgresql-common postgresql-doc-10 postgresql-contrib
Create the iRODS user and database.
irods@iRODS:~$ sudo su - postgres
postgres@iRODS:~$ psql
psql (10.16 (Ubuntu 10.16-0ubuntu0.18.04.1))
Type "help" for help.
postgres=# CREATE USER irods WITH PASSWORD 'password';
CREATE ROLE
postgres=# CREATE DATABASE "ICAT";
CREATE DATABASE
postgres=# GRANT ALL PRIVILEGES ON DATABASE "ICAT" TO irods;
GRANT
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+-------------+-------------+-----------------------
ICAT | postgres | UTF8 | en_GB.UTF-8 | en_GB.UTF-8 | =Tc/postgres +
| | | | | postgres=CTc/postgres+
| | | | | irods=CTc/postgres
postgres | postgres | UTF8 | en_GB.UTF-8 | en_GB.UTF-8 |
template0 | postgres | UTF8 | en_GB.UTF-8 | en_GB.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_GB.UTF-8 | en_GB.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
(4 rows)
postgres=# \q
postgres@iRODS:~$ exit
logout
Install irods server, irods-dev is required for any msi* microservice (the built-in functions called by rules)
irods@iRODS:~$ sudo apt-get install irods-server irods-database-plugin-postgres irods-dev
Run the iRODS installer script, this will be a provider setup to ensure all roles and storage are provided locally. Most defaults are fine for this installation, enter the following at the installer prompts.
| ODBC driver | Unicode |
| postgres user | irods |
| postgres user password | password |
| salt | randomsalt |
| Zone name | OCF |
| iRODS server's zone key | randomzonekey |
| iRODS server's negotiation key | Is8710XeBKO26xlhCN5TcPfkE9qHJMDu |
| Control Plane key | ohyHLaej2s9CWMTUq5AiQ37DBYdIE6wG |
| iRODS server's administrator password | password |
irods@iRODS:~$ sudo python /var/lib/irods/scripts/setup_irods.py
Some of the script output that maybe useful.
iRODS server's role: 1. provider
Database Type: postgres
ODBC Driver: PostgreSQL Unicode
Database Host: localhost
Database Port: 5432
Database Name: ICAT
Database User: irods
Zone name: OCF
iRODS server port: 1247
iRODS port range (begin): 20000
iRODS port range (end): 20199
Control plane port: 1248
Schema validation base URI: file:///var/lib/irods/configuration_schemas
iRODS server administrator: rods
irods@iRODS:~$ ienv
irods_version - 4.2.8
irods_client_server_negotiation - request_server_negotiation
irods_server_control_plane_key - ohyHLaej2s9CWMTUq5AiQ37DBYdIE6wG
irods_server_control_plane_port - 1248
irods_client_server_policy - CS_NEG_REFUSE
irods_cwd - /tempZone/home/rods
irods_connection_pool_refresh_time_in_seconds - 300
irods_encryption_key_size - 32
irods_default_hash_scheme - SHA256
irods_environment_file - /home/irods/.irods/irods_environment.json
irods_default_number_of_transfer_threads - 4
irods_encryption_algorithm - AES-256-CBC
irods_encryption_salt_size - 8
schema_version - v3
irods_home - /tempZone/home/rods
irods_encryption_num_hash_rounds - 16
irods_default_resource - demoResc
irods_match_hash_policy - compatible
irods_maximum_size_for_single_buffer_in_megabytes - 32
irods_session_environment_file - /home/irods/.irods/irods_environment.json.1692
irods_port - 1247
irods_server_control_plane_encryption_algorithm - AES-256-CBC
schema_name - irods_environment
irods_server_control_plane_encryption_num_hash_rounds - 16
irods_user_name - rods
irods_zone_name - tempZone
irods_transfer_buffer_size_for_parallel_transfer_in_megabytes - 4
irods_host - iRODS
Change the iRODS administrator account password, the account 'rods' with password 'rods' with permission 'rodsadmin'. After the password change re-run iinit to login.
irods@iRODS:~$ iadmin moduser rods password password
irods@iRODS:~$ iinit
Enter your current iRODS password:
irods@iRODS:~$ ils
irods@iRODS:~$ ilsresc
Edit irods_environment.json
Whilst almost all commands will function correctly with the default irods_host target being the localhost name, change this to the FQDN/IP of the host to emulate external administration/configuration to avoid weird gotchas.
nano -cw ~/.irods/irods_environment.json
irods_host - 192.168.150.56
iinit
Change the Zone name
This doesnt work with the default zone only ones you subsequently create it woudl seem?
irods@iRODS:~$ iadmin lz tempZone irods@iRODS:~$ iadmin modzone tempZone name OCF
Add a new user
Add a new user (tseed) with password (password).
irods@iRODS:~$ iadmin mkuser tseed rodsuser
irods@irods:~$ iadmin lu
rods#OCF
tseed#OCF
irods@iRODS:~$ iadmin help moduser
irods@iRODS:~$ iadmin moduser tseed#OCF password password
Minio Install
Install docker
sudo apt-get remove docker docker-engine docker.io containerd runc
sudo apt-get update
sudo apt-get install \
apt-transport-https \
ca-certificates \
curl \
gnupg-agent \
software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo apt-key fingerprint 0EBFCD88
sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io
sudo systemctl start docker
sudo systemctl enable docker
sudo usermod -G sudo,docker irods
#logout/login to update access to docker socket
docker run hello-world
Install docker compose
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
Setup Minio S3 service
Create directories.
mkdir ~/minio
sudo mkdir /opt/minio_data
Create minio compose file and run.
cd ~/minio
sudo nano -cw ~/minio/docker-compose.yml
version: "3"
services:
minio:
image: minio/minio:latest
container_name: irods-minio
environment:
- MINIO_ACCESS_KEY=minio
- MINIO_SECRET_KEY=password
command: server /data1
volumes:
- /opt/minio_data:/data1
ports:
- "9000:9000/tcp"
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 30s
timeout: 20s
retries: 3
miniomc:
links:
- minio:minio-container
image: minio/mc:latest
container_name: irods-miniomc
volumes:
- ./createbucket.sh:/tmp/createbucket.sh
entrypoint: /bin/sh /tmp/createbucket.sh minio-container irods minio password
Create miniomc bucket create script.
nano -cw ~/minio/createbucket.sh
#!/bin/bash
# params
target=$1
directory=$2
directory_user=$2
user=$3
password=$4
# vars
mc=/usr/bin/mc
# now rhel8 container with new package manager, install netcat
microdnf install nc
# wait for minio container
until nc -z $target 9000
do
echo "waiting for minio"
sleep 1
done
sleep 2
# auth
$mc config host add minioinstance http://$target:9000 $user $password
sleep 2
setup () {
# create bucket
$mc mb minioinstance/$directory
# allow public read
$mc policy set download minioinstance/$directory
# add user
$mc admin user add minioinstance $directory_user $password
# define policy
cat > /tmp/bucketallaction.json << EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": ["s3:*"],
"Effect": "Allow",
"Resource": ["arn:aws:s3:::$directory_user/*"]
}
]
}
EOF
# create policy
$mc admin policy add minioinstance bucketallaction /tmp/bucketallaction.json
# attach policy to user
$mc admin policy set minioinstance bucketallaction user=$directory_user
}
# create bucket
if $mc ls minioinstance/$directory 2>/dev/null
then
echo "minioinstance/"$directory" exists"
else
setup
echo "minioinstance/"$directory" created"
fi
# destroy bucket
#/usr/bin/mc rm -r --force minioinstance/$directory
exit 0
Start the minio service, once the compose file is run the minio bucket and user has been created.
cd ~/minio
docker-compose up
docker-compose down
Create systemd unit files to start minio container on boot.
Find the docker run command from a running container started by compose.
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock assaflavie/runlike irods-minio
Take any parameters that have been specified in the compose file, effectively any parameter that isnt auto populated, this will look like the following:
docker stop irods-minio
docker rm irods-minio
# if the following command runs correctly populate the systemd unit file accordingly.
docker run --name=irods-minio --env=MINIO_ACCESS_KEY=minio --env=MINIO_SECRET_KEY=password --volume=/opt/OCF_s3:/data1:rw -p 9000:9000 --restart=unless-stopped --detach=true minio/minio:latest server /data1
NOTE: if the /opt/minio_data directory is wiped the minio config will also be wiped and the compose file will need to be rerun to create the irods bucket and user.
To ensure systemd can start the container ensure the --detach parameter is not used, the container restarts must be controlled by systemd and not the docker daemon.
sudo nano -cw /etc/systemd/system/minio.service
[Unit]
Description=minio
Wants=network-online.target
After=network.target network-online.target
[Service]
TimeoutStartSec=30
Restart=always
ExecStartPre=-/usr/bin/docker kill irods-minio
ExecStartPre=-/usr/bin/docker rm irods-minio
ExecStartPre=/usr/bin/docker pull docker.io/minio/minio:latest
ExecStart=/usr/bin/docker run \
--name=irods-minio \
--env=MINIO_ACCESS_KEY=minio \
--env=MINIO_SECRET_KEY=password \
--volume=/opt/minio_data:/data1:rw \
-p 9000:9000 \
minio/minio:latest server /data1
ExecStop=/usr/bin/docker stop irods-minio
[Install]
WantedBy=multi-user.target
sudo systemctl daemon-reload
systemctl list-unit-files | grep -i minio
sudo systemctl start minio
systemctl status minio
sudo systemctl enable minio
iRODS Resources
Log location
/var/lib/irods/log/
Add local filesystem Resource
There needs to be a default resource on the service, it shouldnt be the default demo resource.
NOTE: dont set the host as localhost, external users will have an issue resolving a remote host.
sudo mkdir /opt/OCFResc
sudo chown -R irods.irods /opt/OCFResc
iadmin mkresc OCFResc unixfilesystem $(hostname):/opt/OCFResc
ilsresc
#iadmin rmresc OCFResc
nano -cw ~/.irods/irods_environment.json
# change "irods_default_resource": "demoResc", to "irods_default_resource": "OCFResc",
iinit
Add new S3 Resource, this is a cached setup where files a local directory is sync'd with s3
https://docs.irods.org/4.2.0/plugins/composable_resources/ https://github.com/irods/irods_resource_plugin_s3
irods@iRODS:~$ sudo apt-get install irods-resource-plugin-s3
This is odd, unsure why this worked and previous didnt, maybe its because of using locally resolved hostname instead of ip, maybe it is due to /irods/Vault being one level deep vs two? S3_DEFAULT_HOSTNAME=$(hostname):9000 will work for a cached compound s3 resource but not for a cacheless resource?, S3_DEFAULT_HOSTNAME is probably best as the ip of the s3 compatible storage.
sudo mkdir /opt/iRODS_s3_cache /opt/iRODS_s3_cred
sudo nano -cw /opt/iRODS_s3_cred/minio.keypair
irods
password
iadmin mkresc compResc compound
iadmin mkresc cacheResc unixfilesystem $(hostname):/opt/iRODS_s3_cache
iadmin mkresc archiveResc s3 $(hostname):/irods/Vault "S3_DEFAULT_HOSTNAME=$(hostname):9000;S3_AUTH_FILE=/opt/iRODS_s3_cred/minio.keypair;S3_RETRY_COUNT=1;S3_WAIT_TIME_SEC=3;S3_PROTO=HTTP"
iadmin addchildtoresc compResc cacheResc cache
iadmin addchildtoresc compResc archiveResc archive
iput -R compResc docker-compose.yml
To remove this config:
irods@irods:~/minio$ iadmin rmchildfromresc compResc archiveResc
irods@irods:~/minio$ iadmin rmresc archiveResc
irods@irods:~/minio$ iadmin rmchildfromresc compResc cacheResc
irods@irods:~/minio$ iadmin rmresc cacheResc
irods@irods:~/minio$ iadmin rmresc compResc
irods@irods:~/minio$ ilsresc
demoResc:unixfilesystem
If there are files on the compound resource children you need to a combination of commands to clear it for deletion:
iadmin rmchildfromresc compResc archiveResc
iadmin rmresc archiveResc
error -835000 CAT_RESOURCE_NOT_EMPTY
iadmin rmchildfromresc compResc cacheResc
iadmin rmresc cacheResc
error -835000 CAT_RESOURCE_NOT_EMPTY
# re-add both children back to compound resource and clear out
iadmin addchildtoresc compResc cacheResc cache
iadmin addchildtoresc compResc archiveResc archive
nano -cw ~/.irods/irods_environment.json
# change "irods_default_resource": "demoResc", to "irods_default_resource": "compResc",
iinit
icd
ils
irm -r -f *
ils
itrim -vrMN 1 -S cacheResc /OCF
irmtrash -f
nano -cw ~/.irods/irods_environment.json
# change "irods_default_resource": "compResc", to "irods_default_resource": "demoResc",
iinit
iadmin rmchildfromresc compResc archiveResc
iadmin rmresc archiveResc
iadmin rmchildfromresc compResc cacheResc
iadmin rmresc cacheResc
iadmin rmresc compResc
Add new S3 Resource in cacheless mode, write direct to the bucket (great for local s3 service)
Create a path under the bucket to replicate a sub folder.
docker run -it --entrypoint=/bin/sh minio/mc
mc config host add minioinstance http://192.168.150.56:9000 irods password
mc mb minioinstance/irods/Data
exit
Create a resource, this could be created as a compound resource as in the previous example, or standalone and set a replica ruleset to duplictae files to it or even use the s3 resource as the sole resource. A cache directory is still required however this only holds transient data. An s3 resource cannot be mounted/bound at a certain location/collection in a directory structure.
sudo mkdir /opt/iRODS_s3_cred /opt/iRODS_s3_cache
sudo nano -cw /opt/iRODS_s3_cred/minio.keypair
irods
password
iadmin mkresc OCFs3Resc s3 $(hostname):/irods/Data "S3_DEFAULT_HOSTNAME=192.168.150.56:9000;S3_AUTH_FILE=/opt/iRODS_s3_cred/minio.keypair;S3_RETRY_COUNT=1;S3_WAIT_TIME_SEC=3;S3_PROTO=HTTP;ARCHIVE_NAMING_POLICY=consistent;HOST_MODE=cacheless_attached;S3_SIGNATURE_VERSION=4;S3_CACHE_DIR=/opt/iRODS_s3_cache"
ilsresc -l
ilsresc -l OCFs3Resc
iput -R OCFs3Resc docker-compose.yml
irm docker-compose.yml
# remove, might want to remove and recreate before any iput commands to ensure there is no directory structure
#irmtrash
#iadmin rmresc OCFs3Resc
Removing resources
list resources
irods@irods:~/minio$ ilsresc compResc:compound ├── archiveResc:s3 └── cacheResc:unixfilesystem demoResc:unixfilesystem
irods@irods:~/minio$ ilsresc compResc compResc:compound ├── archiveResc:s3 └── cacheResc:unixfilesystem
iadmin help
iadmin help
admin list resources
irods@irods:~/minio$ iadmin lr bundleResc demoResc compResc cacheResc archiveResc
irods@irods:~/minio$ iadmin lr demoResc resc_id: 10014 resc_name: demoResc zone_name: OCF resc_type_name: unixfilesystem resc_net: irods.local resc_def_path: /var/lib/irods/Vault free_space: free_space_ts: Never resc_info: r_comment: resc_status: create_ts: 2021-05-11.13:11:07 modify_ts: 2021-05-11.13:11:07 resc_children: resc_context: resc_parent: resc_objcount: 0 resc_parent_context:
irods@irods:~/minio$ iadmin lr compResc resc_id: 10019 resc_name: compResc zone_name: OCF resc_type_name: compound resc_net: EMPTY_RESC_HOST resc_def_path: EMPTY_RESC_PATH free_space: free_space_ts: Never resc_info: r_comment: resc_status: create_ts: 2021-05-11.13:49:51 modify_ts: 2021-05-11.13:49:51 resc_children: resc_context: resc_parent: resc_objcount: 0 resc_parent_context:
admin remove resource (from a compound)
Find any child resource.
irods@irods:~/minio$ ilsresc compResc compResc:compound ├── archiveResc:s3 └── cacheResc:unixfilesystem
Remove child resources.
iadmin rmchildfromresc compResc archiveResc iadmin rmchildfromresc compResc cacheResc
Remove resources that were used to create the compound resource.
iadmin rmresc archiveResc iadmin rmresc cacheResc # this threw an error iadmin rmresc compResc
to remove cacheResc, i had to switch the env file to use default resource cacheResc:
nano -cw ~/.irods/irods_environment.json # "irods_default_resource": "demoResc", to "irods_default_resource": "cacheResc", iinit
run a selection of commands to check if the resource is empty and clean it up if not, ready for deletion:
iquest "SELECT DATA_NAME WHERE DATA_RESC_NAME like 'cache%'" iquest "SELECT COLL_NAME WHERE DATA_RESC_NAME = 'cacheResc'" irmtrash -f itrim -vrMN 1 -S cacheResc /OCF irm -r -f /OCF
After this switched back to "irods_default_resource": "demoResc", and removed the resource:
iadmin rmresc cacheResc