irods_testbench_training_python_bash/documentation/1) iRODS server setup.md

iRODS server setup

# iRODS test server

This is a sample virtual machine with an iRODS server instance and S3 compatible storage.

## VM test server

4 cpu
4GB RAM
40GB disk
hostname irods.local
user irods
pass password
ip 192.168.150.56 (should be a static ip to avoid hostname/domain/ssl type issues, millage may vary, virtualbox/hyperv NAT network maybe help)
os Ubuntu 18.04

## iRODS server installation

> https://docs.irods.org/4.2.8/getting_started/installation/

### Server prep

```
sudo apt-get install wget ca-certificates
```

### Netplan

Static ip required for minio s3 connectivity from irods.

```
sudo nano -cw /etc/netplan/01-netcfg.yaml

network:
  version: 2
  renderer: networkd
  ethernets:
      eth0:
        addresses:
            - 192.168.150.56/24
        gateway4: 192.168.150.1
        nameservers:
            search: [local]
            addresses: [192.168.150.1]
            
sudo netplan apply
```

### Sudoers

Setup /etc/sudoers to include:

```
irods   ALL=(ALL) NOPASSWD:ALL
```

### Ubuntu software repo

```
wget -qO - https://packages.irods.org/irods-signing-key.asc | sudo apt-key add -
echo "deb [arch=amd64] https://packages.irods.org/apt/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/renci-irods.list
sudo apt-get update
```

### Setup Postgresql database, install iRODS server

Install postgress packages.

```
sudo apt-get install postgresql-10 postgresql-client-10 postgresql-client-common postgresql-common postgresql-doc-10 postgresql-contrib
```

Create the iRODS user and database.

```
irods@iRODS:~$ sudo su - postgres
postgres@iRODS:~$ psql
psql (10.16 (Ubuntu 10.16-0ubuntu0.18.04.1))
Type "help" for help.

postgres=# CREATE USER irods WITH PASSWORD 'password';
CREATE ROLE
postgres=# CREATE DATABASE "ICAT";
CREATE DATABASE
postgres=# GRANT ALL PRIVILEGES ON DATABASE "ICAT" TO irods;
GRANT
postgres=# \l
                                  List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges
-----------+----------+----------+-------------+-------------+-----------------------
 ICAT      | postgres | UTF8     | en_GB.UTF-8 | en_GB.UTF-8 | =Tc/postgres         +
           |          |          |             |             | postgres=CTc/postgres+
           |          |          |             |             | irods=CTc/postgres
 postgres  | postgres | UTF8     | en_GB.UTF-8 | en_GB.UTF-8 |
 template0 | postgres | UTF8     | en_GB.UTF-8 | en_GB.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_GB.UTF-8 | en_GB.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
(4 rows)

postgres=# \q
postgres@iRODS:~$ exit
logout
```

Install irods server, irods-dev is required for any msi* microservice (the built-in functions called by rules)

```
irods@iRODS:~$ sudo apt-get install irods-server irods-database-plugin-postgres irods-dev
```

Run the iRODS installer script, this will be a provider setup to ensure all roles and storage are provided locally.
Most defaults are fine for this installation, enter the following at the installer prompts.

|     |     |
| --- | --- |
| ODBC driver | Unicode |
| postgres user | irods |
| postgres user password | password |
| salt | randomsalt |
| Zone name | OCF |
| iRODS server's zone key | randomzonekey |
| iRODS server's negotiation key | Is8710XeBKO26xlhCN5TcPfkE9qHJMDu |
| Control Plane key | ohyHLaej2s9CWMTUq5AiQ37DBYdIE6wG |
| iRODS server's administrator password | password |

```
irods@iRODS:~$ sudo python /var/lib/irods/scripts/setup_irods.py
```

Some of the script output that maybe useful.

```
iRODS server's role: 1. provider
Database Type: postgres
ODBC Driver:   PostgreSQL Unicode
Database Host: localhost
Database Port: 5432
Database Name: ICAT
Database User: irods

Zone name:                  OCF
iRODS server port:          1247
iRODS port range (begin):   20000
iRODS port range (end):     20199
Control plane port:         1248
Schema validation base URI: file:///var/lib/irods/configuration_schemas
iRODS server administrator: rods
```

```
irods@iRODS:~$ ienv
irods_version - 4.2.8
irods_client_server_negotiation - request_server_negotiation
irods_server_control_plane_key - ohyHLaej2s9CWMTUq5AiQ37DBYdIE6wG
irods_server_control_plane_port - 1248
irods_client_server_policy - CS_NEG_REFUSE
irods_cwd - /tempZone/home/rods
irods_connection_pool_refresh_time_in_seconds - 300
irods_encryption_key_size - 32
irods_default_hash_scheme - SHA256
irods_environment_file - /home/irods/.irods/irods_environment.json
irods_default_number_of_transfer_threads - 4
irods_encryption_algorithm - AES-256-CBC
irods_encryption_salt_size - 8
schema_version - v3
irods_home - /tempZone/home/rods
irods_encryption_num_hash_rounds - 16
irods_default_resource - demoResc
irods_match_hash_policy - compatible
irods_maximum_size_for_single_buffer_in_megabytes - 32
irods_session_environment_file - /home/irods/.irods/irods_environment.json.1692
irods_port - 1247
irods_server_control_plane_encryption_algorithm - AES-256-CBC
schema_name - irods_environment
irods_server_control_plane_encryption_num_hash_rounds - 16
irods_user_name - rods
irods_zone_name - tempZone
irods_transfer_buffer_size_for_parallel_transfer_in_megabytes - 4
irods_host - iRODS
```

Change the iRODS administrator account password, the account 'rods' with password 'rods' with permission 'rodsadmin'.
After the password change re-run iinit to login.

```
irods@iRODS:~$ iadmin moduser rods password password
irods@iRODS:~$ iinit
Enter your current iRODS password:
irods@iRODS:~$ ils
irods@iRODS:~$ ilsresc
```

### Edit irods_environment.json

Whilst almost all commands will function correctly with the default irods_host target being the localhost name, change this to the FQDN/IP of the host to emulate external administration/configuration to avoid weird gotchas.

```
nano -cw ~/.irods/irods_environment.json

irods_host - 192.168.150.56

iinit
```

### Change the Zone name

This doesnt work with the default zone only ones you subsequently create it woudl seem?

irods@iRODS:~$ iadmin lz
tempZone
irods@iRODS:~$ iadmin modzone tempZone name OCF

### Add a new user

Add a new user (tseed) with password (password).

```
irods@iRODS:~$ iadmin mkuser tseed rodsuser
irods@irods:~$ iadmin lu
rods#OCF
tseed#OCF
irods@iRODS:~$ iadmin help moduser
irods@iRODS:~$ iadmin moduser tseed#OCF password password
```

## Minio Install

### Install docker

```
sudo apt-get remove docker docker-engine docker.io containerd runc
sudo apt-get update
sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo apt-key fingerprint 0EBFCD88
sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io
sudo systemctl start docker
sudo systemctl enable docker
sudo usermod -G sudo,docker irods
#logout/login to update access to docker socket
docker run hello-world
```

### Install docker compose

```
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
```

### Setup Minio S3 service

Create directories.

```
mkdir ~/minio
sudo mkdir /opt/minio_data
```

Create minio compose file and run.

```
cd ~/minio
sudo nano -cw ~/minio/docker-compose.yml

version: "3"
services:
  minio:
    image: minio/minio:latest
    container_name: irods-minio
    environment:
      - MINIO_ACCESS_KEY=minio
      - MINIO_SECRET_KEY=password
    command: server /data1
    volumes:
      - /opt/minio_data:/data1
    ports:
      - "9000:9000/tcp"
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3
  miniomc:
    links:
      - minio:minio-container
    image: minio/mc:latest
    container_name: irods-miniomc
    volumes:
      - ./createbucket.sh:/tmp/createbucket.sh
    entrypoint: /bin/sh /tmp/createbucket.sh minio-container irods minio password
```

Create miniomc bucket create script.

```
nano -cw ~/minio/createbucket.sh

#!/bin/bash

# params
target=$1
directory=$2
directory_user=$2
user=$3
password=$4

# vars
mc=/usr/bin/mc

# now rhel8 container with new package manager, install netcat
microdnf install nc

# wait for minio container
until nc -z $target 9000
do
  echo "waiting for minio"
  sleep 1
done
sleep 2

# auth
$mc config host add minioinstance http://$target:9000 $user $password
sleep 2

setup () {
  # create bucket
  $mc mb minioinstance/$directory
  # allow public read
  $mc policy set download minioinstance/$directory
  # add user
  $mc admin user add minioinstance $directory_user $password
  # define policy
cat > /tmp/bucketallaction.json << EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": ["s3:*"],
      "Effect": "Allow",
      "Resource": ["arn:aws:s3:::$directory_user/*"]
    }
  ]
}
EOF
  # create policy
  $mc admin policy add minioinstance bucketallaction /tmp/bucketallaction.json
  # attach policy to user
  $mc admin policy set minioinstance bucketallaction user=$directory_user
}

# create bucket
if $mc ls minioinstance/$directory 2>/dev/null
  then
    echo "minioinstance/"$directory" exists"
  else
    setup
    echo "minioinstance/"$directory" created"
fi

# destroy bucket
#/usr/bin/mc rm -r --force minioinstance/$directory

exit 0
```

Start the minio service, once the compose file is run the minio bucket and user has been created.

```
cd ~/minio
docker-compose up
docker-compose down
```

### Create systemd unit files to start minio container on boot.

Find the docker run command from a running container started by compose.

```
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock assaflavie/runlike irods-minio
```

Take any parameters that have been specified in the compose file, effectively any parameter that isnt auto populated, this will look like the following:

```
docker stop irods-minio
docker rm irods-minio

# if the following command runs correctly populate the systemd unit file accordingly.

docker run --name=irods-minio --env=MINIO_ACCESS_KEY=minio --env=MINIO_SECRET_KEY=password --volume=/opt/OCF_s3:/data1:rw -p 9000:9000 --restart=unless-stopped --detach=true minio/minio:latest server /data1
```

NOTE: if the /opt/minio_data directory is wiped the minio config will also be wiped and the compose file will need to be rerun to create the irods bucket and user.

To ensure systemd can start the container ensure the --detach parameter is not used, the container restarts must be controlled by systemd and not the docker daemon.

```
sudo nano -cw /etc/systemd/system/minio.service

[Unit]
Description=minio
Wants=network-online.target
After=network.target network-online.target
[Service]
TimeoutStartSec=30
Restart=always
ExecStartPre=-/usr/bin/docker kill irods-minio
ExecStartPre=-/usr/bin/docker rm irods-minio
ExecStartPre=/usr/bin/docker pull docker.io/minio/minio:latest
ExecStart=/usr/bin/docker run \
        --name=irods-minio \
        --env=MINIO_ACCESS_KEY=minio \
        --env=MINIO_SECRET_KEY=password \
        --volume=/opt/minio_data:/data1:rw \
        -p 9000:9000 \
        minio/minio:latest server /data1
ExecStop=/usr/bin/docker stop irods-minio
[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
systemctl list-unit-files | grep -i minio
sudo systemctl start minio
systemctl status minio
sudo systemctl enable minio
```

## iRODS Resources

### Log location

> /var/lib/irods/log/

### Add local filesystem Resource

There needs to be a default resource on the service, it shouldnt be the default demo resource.

NOTE: dont set the host as localhost, external users will have an issue resolving a remote host.

```
sudo mkdir /opt/OCFResc
sudo chown -R irods.irods /opt/OCFResc
iadmin mkresc OCFResc unixfilesystem $(hostname):/opt/OCFResc
ilsresc
#iadmin rmresc OCFResc
nano -cw ~/.irods/irods_environment.json
# change "irods_default_resource": "demoResc", to "irods_default_resource": "OCFResc",
iinit
```

### Add new S3 Resource, this is a cached setup where files a local directory is sync'd with s3

> https://docs.irods.org/4.2.0/plugins/composable_resources/
> [https://github.com/irods/irods\_resource\_plugin_s3](https://github.com/irods/irods_resource_plugin_s3)

```
irods@iRODS:~$ sudo apt-get install irods-resource-plugin-s3
```

This is odd, unsure why this worked and previous didnt, maybe its because of using locally resolved hostname instead of ip, maybe it is due to /irods/Vault being one level deep vs two?
S3\_DEFAULT\_HOSTNAME=$(hostname):9000 will work for a cached compound s3 resource but not for a cacheless resource?, S3\_DEFAULT\_HOSTNAME is probably best as the ip of the s3 compatible storage.

```
sudo mkdir /opt/iRODS_s3_cache /opt/iRODS_s3_cred
sudo nano -cw /opt/iRODS_s3_cred/minio.keypair

irods
password

iadmin mkresc compResc compound
iadmin mkresc cacheResc unixfilesystem $(hostname):/opt/iRODS_s3_cache
iadmin mkresc archiveResc s3 $(hostname):/irods/Vault "S3_DEFAULT_HOSTNAME=$(hostname):9000;S3_AUTH_FILE=/opt/iRODS_s3_cred/minio.keypair;S3_RETRY_COUNT=1;S3_WAIT_TIME_SEC=3;S3_PROTO=HTTP"
iadmin addchildtoresc compResc cacheResc cache
iadmin addchildtoresc compResc archiveResc archive
iput -R compResc docker-compose.yml
```

To remove this config:

```
irods@irods:~/minio$ iadmin rmchildfromresc compResc archiveResc
irods@irods:~/minio$ iadmin rmresc archiveResc
irods@irods:~/minio$ iadmin rmchildfromresc compResc  cacheResc
irods@irods:~/minio$ iadmin rmresc cacheResc
irods@irods:~/minio$ iadmin rmresc compResc
irods@irods:~/minio$ ilsresc
demoResc:unixfilesystem
```

If there are files on the compound resource children you need to a combination of commands to clear it for deletion:

```
iadmin rmchildfromresc compResc archiveResc
iadmin rmresc archiveResc
error -835000 CAT_RESOURCE_NOT_EMPTY
iadmin rmchildfromresc compResc  cacheResc
iadmin rmresc cacheResc
error -835000 CAT_RESOURCE_NOT_EMPTY
# re-add both children back to compound resource and clear out
iadmin addchildtoresc compResc cacheResc cache
iadmin addchildtoresc compResc archiveResc archive
nano -cw ~/.irods/irods_environment.json
# change "irods_default_resource": "demoResc", to "irods_default_resource": "compResc",
iinit
icd
ils
irm -r -f *
ils
itrim -vrMN 1 -S cacheResc /OCF
irmtrash -f
nano -cw ~/.irods/irods_environment.json
# change "irods_default_resource": "compResc", to "irods_default_resource": "demoResc",
iinit
iadmin rmchildfromresc compResc archiveResc
iadmin rmresc archiveResc
iadmin rmchildfromresc compResc  cacheResc
iadmin rmresc cacheResc
iadmin rmresc compResc
```

### Add new S3 Resource in cacheless mode, write direct to the bucket (great for local s3 service)

Create a path under the bucket to replicate a sub folder.

```
docker run -it --entrypoint=/bin/sh minio/mc
mc config host add minioinstance http://192.168.150.56:9000 irods password
mc mb minioinstance/irods/Data
exit
```

Create a resource, this could be created as a compound resource as in the previous example, or standalone and set a replica ruleset to duplictae files to it or even use the s3 resource as the sole resource. A cache directory is still required however this only holds transient data. An s3 resource cannot be mounted/bound at a certain location/collection in a directory structure.

```
sudo mkdir /opt/iRODS_s3_cred /opt/iRODS_s3_cache
sudo nano -cw /opt/iRODS_s3_cred/minio.keypair

irods
password

iadmin mkresc OCFs3Resc s3 $(hostname):/irods/Data "S3_DEFAULT_HOSTNAME=192.168.150.56:9000;S3_AUTH_FILE=/opt/iRODS_s3_cred/minio.keypair;S3_RETRY_COUNT=1;S3_WAIT_TIME_SEC=3;S3_PROTO=HTTP;ARCHIVE_NAMING_POLICY=consistent;HOST_MODE=cacheless_attached;S3_SIGNATURE_VERSION=4;S3_CACHE_DIR=/opt/iRODS_s3_cache"
ilsresc -l
ilsresc -l OCFs3Resc
iput -R OCFs3Resc docker-compose.yml
irm docker-compose.yml

# remove, might want to remove and recreate before any iput commands to ensure there is no directory structure
#irmtrash
#iadmin rmresc OCFs3Resc
```

## Removing resources

#### list resources

irods@irods:~/minio$ ilsresc
compResc:compound
├── archiveResc:s3
└── cacheResc:unixfilesystem
demoResc:unixfilesystem

irods@irods:~/minio$ ilsresc compResc
compResc:compound
├── archiveResc:s3
└── cacheResc:unixfilesystem

#### iadmin help

iadmin help

#### admin list resources

irods@irods:~/minio$ iadmin lr
bundleResc
demoResc
compResc
cacheResc
archiveResc

irods@irods:~/minio$ iadmin lr demoResc
resc_id: 10014
resc_name: demoResc
zone_name: OCF
resc\_type\_name: unixfilesystem
resc_net: irods.local
resc\_def\_path: /var/lib/irods/Vault
free_space:
free\_space\_ts: Never
resc_info:
r_comment:
resc_status:
create_ts: 2021-05-11.13:11:07
modify_ts: 2021-05-11.13:11:07
resc_children:
resc_context:
resc_parent:
resc_objcount: 0
resc\_parent\_context:

irods@irods:~/minio$ iadmin lr compResc
resc_id: 10019
resc_name: compResc
zone_name: OCF
resc\_type\_name: compound
resc\_net: EMPTY\_RESC_HOST
resc\_def\_path: EMPTY\_RESC\_PATH
free_space:
free\_space\_ts: Never
resc_info:
r_comment:
resc_status:
create_ts: 2021-05-11.13:49:51
modify_ts: 2021-05-11.13:49:51
resc_children:
resc_context:
resc_parent:
resc_objcount: 0
resc\_parent\_context:

#### admin remove resource (from a compound)

Find any child resource.

irods@irods:~/minio$ ilsresc compResc
compResc:compound
├── archiveResc:s3
└── cacheResc:unixfilesystem

Remove child resources.

iadmin rmchildfromresc compResc archiveResc
iadmin rmchildfromresc compResc cacheResc

Remove resources that were used to create the compound resource.

iadmin rmresc archiveResc
iadmin rmresc cacheResc # this threw an error
iadmin rmresc compResc

to remove cacheResc, i had to switch the env file to use default resource cacheResc:

nano -cw ~/.irods/irods\_environment.json # "irods\_default\_resource": "demoResc", to "irods\_default_resource": "cacheResc",
iinit

run a selection of commands to check if the resource is empty and clean it up if not, ready for deletion:

iquest "SELECT DATA\_NAME WHERE DATA\_RESC_NAME like 'cache%'"
iquest "SELECT COLL\_NAME WHERE DATA\_RESC_NAME = 'cacheResc'"
irmtrash -f
itrim -vrMN 1 -S cacheResc /OCF
irm -r -f /OCF

After this switched back to "irods\_default\_resource": "demoResc", and removed the resource:

iadmin rmresc cacheResc