irods_testbench_training_python_bash/documentation/4) Using the python server side rule engine.md

Using the python server side rule engine

Using the python server side rule engine

https://github.com/irods/irods_rule_engine_plugin_python

enable rules engine /etc/irods/server_config.json

To enable the engine, ensure the following definition is in the "rule_engines" list in /etc/irods/server_config.json:

        "rule_engines": [
            {
                "instance_name" : "irods_rule_engine_plugin-python-instance",
                "plugin_name" : "irods_rule_engine_plugin-python",
                "plugin_specific_configuration" : {}
            },
            {
                "instance_name": "irods_rule_engine_plugin-irods_rule_language-instance",

add a new rulebase /etc/irods.server_config.json

Add the new rule definition to the rulebase list in /etc/irods.server_config.json:

            "re_rulebase_set": [
                "ocf",
                "core"
            ],

create the new rulebase definition /etc/irods/ocf.re

Create the /etc/irods/ocf.re rules definition file:

irods@irods:~$ cat /etc/irods/ocf.re
# meta, object path, type -d
ocfTagDataObject(*str, *objpath, *objtype) {
    msiString2KeyValPair(*str, *kvp);
    msiAssociateKeyValuePairsToObj(*kvp, *objpath, *objtype);
    writeLine("serverLog", "added metadata to *objpath");

    #session variables also available here in native rules structure
    #
    #msiGetSessionVarValue("all", "server");
    #*get_session_var=msiGetSessionVarValue("objPath", "server");
    #*something = str(msiGetSessionVarValue("objPath", "client"));
    #writeLine("serverLog", "*something");
}

getSessionVar(*name,*output) {
    *output = eval("str($"++*name++")");
}

create the python rule entry script

The core.py entry script lists policy entry points (PEP) and contains logic to take actions typically based on the contents of session variables. When the rules engine matches an event it will call a rule.

/etc/core.py calls rule named ocfTagDataObject in rules definition file /etc/irods/ocf.re

The purpose of the following entry script is to illustrate the PEP acPostProcForPut being triggered and custom rule logic deciding on which rule(s) to invoke, this could be done purely in the native rule engine language at the cost of flexibility afforded by Python.

Rule 4 is calling to the native rule engine in ocf.re.

import os
import session_vars
import sys
import inspect
import time

def rule1(sv):
  print "this is rule1 unzip"

def rule2(sv):
  print "this is rule1 csv"

def rule3(sv):
  print "this is rule1 email"

def rule4(callback, sv):
  objpath = sv['data_object']['object_path']
  tag_ext = os.path.splitext(sv['data_object']['object_path'])[1]
  tag_upload_date = time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(int(sv['data_object']['modify_time'])))
  tag_upload_year = time.strftime('%Y', time.localtime(int(sv['data_object']['modify_time'])))
  tag_upload_month = time.strftime('%B', time.localtime(int(sv['data_object']['modify_time'])))
  tag_upload_day = time.strftime('%A', time.localtime(int(sv['data_object']['modify_time'])))
  tag_user = sv['client_user']['user_name']
  tag_dept = "OCF"
  tag_project = "OCF"
  metadata = []
  for tag in dir():
    if tag.startswith('tag_'):
      metadata.append(tag.split('tag_')[1] + "=" + eval(tag))
  metadata = '%'.join(metadata)
  callback.ocfTagDataObject(metadata, objpath, '-d')
  #call to the native rule in ocf.re

def acPostProcForPut(rule_args, callback, rei):
  callback.writeLine('serverLog', 'acPostProcForPut execrule triggered')

  sv = session_vars.get_map(rei)
  directory = os.path.dirname(sv['data_object']['object_path'])
  file = os.path.basename(sv['data_object']['object_path'])
  ext = os.path.splitext(sv['data_object']['object_path'])[1]
  zone = directory.split("/")[1]
  resource = sv['data_object']['resource_name']
  user = sv['client_user']['user_name']
  ruledict = {'rule1': {'directory': '/OCF/home/rods/unzip', 'ext': '.zip', 'zone': 'OCF', 'resource': 'OCFs3Resc', 'user': 'rods'}, 'rule2': {'directory': '/OCF/home/rods/csv', 'ext': '.csv'}, 'rule3': {'directory': '/OCF/home/tseed/email', 'user': 'tseed'}, 'rule4': {'directory': '/OCF/home/rods/jpg', 'ext': '.jpg', 'zone': 'OCF', 'resource': 'OCFResc', 'user': 'rods'}}
  ruleweight = {}

  for rule in ruledict:
    weight = 0
    rulesize = len(ruledict[rule])
    for attribute in ruledict[rule]:
      if eval(attribute): # check if sv returns empty for any variables, such as files without extension
        if eval(attribute) == ruledict[rule][attribute]: # if sv returned directory matches rule directory entry
          weight += 1
    if rulesize == weight: # if rule attributes are all matched set as possible rule
      ruleweight[rule] = weight

  weight = 0
  candidate = []
  for entry in ruleweight: # greatest rule weight
    if ruleweight[entry] > weight:
      weight = ruleweight[entry]
  for entry in ruleweight: # find rule(s) rules with greatest weight
    if ruleweight[entry] == weight:
      candidate.append(entry)
  if len(candidate) > 1:
    compete = ""
    for entry in candidate:
      compete = compete + entry + " "
    #print("competing rules: %s" % compete)
    message = ("core.py competing rules: %s" % compete)
    callback.writeLine('serverLog', message)
    return
  elif len(candidate) < 1:
    return # no matching rule
  else:
    execrule = candidate[0]
    try:
      inspect.isfunction(eval(execrule))
    except:
      #print("rule action missing: %s" % execrule)
      message = ("core.py rule action missing: %s" % execrule)
      callback.writeLine('serverLog', message)
      return
    else:
      #print("execute rule: %s" % execrule)
      message = ("core.py execute rule: %s" % execrule)
      callback.writeLine('serverLog', message)
      (eval(execrule))(callback, sv) # run function

TO DO

• example use of the native rule engine PEP triggering, this is an 'on' statement mechanism in core.re, the logic not as easy/flexible as python rule engine.
• purely python rule without callback to the native rule engine, this is a more flexible scenario but not well documented.