start portlet menu bar

HCLSoftware: Fueling the Digital+ Economy

Display portlet menu
end portlet menu bar
Close
Select Page
Event Rules are an extension of Workload Automation (WA) capabilities that enable events occurring external to the scheduling environment to trigger actions on scheduling objects within WA.  An ideal use of this capability is to detect the arrival of a file and then trigger an action to submit a Job Stream containing jobs to process the data contained in that file.  This capability has been available for a while and is widely used.  In this article, a hidden feature is explored where the name of the file and other properties related to the file are passed as variables to a Variable Table associated to the Job Stream being submitted as ac action where any Job within that Job Stream can retrieve those variables and process the data in the file. ​
 

Create an Event Rule with the Event, File Created, and Action, Submit Job Stream 

​Follow the steps below to create an Event Rule

  1. Login to the Dynamic Workload Console
  2. Navigate to Design à Create Event Rules
  3. Click on Create new à Event Rules
  4. Click on the field, <not_named> and the General Info panel is displayed on the right
  5. Click on the Name field, and enter the name, IF_FILE_EXISTS_PROCESS_FILE
  6. Click on Save as draft to disable it
  7. Click on Add Events à File Monitor à File created (click on + sign) à Close
  8. Click on the event just added, the Properties panel opens on the right
  9. Fill in the values in the fields as shown below:

​​​a. File name: /tmp/datafile.*.txt              An existing file with a matching name will not satisfy the condition, a new file with matching name must be created.
b. Sample interval: 300                           Indicates how often in seconds to check for the file
c. Workstation: EEL ​

event rule with the event
 
10. Click on Add Actions à IBM Workload Scheduler actions à Submit job stream
11. Click on the Action just added, the properties panel opens on the right
12. Fill in the values in the fields as shown below:
a. Job stream name:         PROCESS_FILEThe name of the Job Stream to be submitted that contains jobs to process the
​                                                contents of the file.
b. Workstation name:       EEL
 
job stream name
 
 
 c. ​ Custom parameter 1:
Click on the Variables icon on the right and choose fileCrtEvt1 à File name
FILE_NAME=%{fileCrtEvt1.FileName} will be entered
The value of File name will be passed to the Variable, FILE_NAME
 
custom parameter
 
 
d. Custom parameter 2:      Similarly, click on Variables icon on the right and choose any other variable that needs to be
passed as Variables, e.g.
e. Custom parameter 3:     WORKSTATION=%{fileCrtEvt1.Workstation}​
f. Custom parameter 4:      TIME_STAMP=%{fileCrtEvt1.TimeStamp}​​​
g. Custom parameter 5:     HOSTNAME=%{fileCrtEvt1.Hostname}​
h. Custom parameter 6:     IP_ADDRESS=%{fileCrtEvt1.IPAddress}
i. Custom parameter 7:      EVENT_RULE_ID=%{fileCrtEvt1.EventRuleId}
custom parameters
 
13. Click on the selection box next to the event Rule name and click on Save on the menu bar.
14. Click on the X to close the successful saved message.
15. The following is the Event Rule definition is composer format.
 
 
<?xml version=”1.0″?>
<eventRuleSet   xmlns:xsi=”https://www.w3.org/2001/XMLSchema-instance”
xmlns=”https://www.ibm.com/xmlns/prod/tws/1.0/event-management/rules”
xsi:schemaLocation=”https://www.ibm.com/xmlns/prod/tws/1.0/event-management/rules https://www.ibm.com/xmlns/prod/tws/1.0/event-management/rules/EventRules.xsd”>
<eventRule name=”IF_FILE_EXISTS_PROCESS_FILE” ruleType=”filter” isDraft=”no”>
<eventCondition name=”fileCrtEvt1″ eventProvider=”FileMonitor” eventType=”FileCreated”>
<scope>
/TMP/DATAFILE.*.TXT ON EEL
</scope>
<filteringPredicate>
<attributeFilter name=”FileName” operator=”eq”>
<value>/tmp/datafile.*.txt</value>
</attributeFilter>
<attributeFilter name=”SampleInterval” operator=”eq”>
<value>300</value>
</attributeFilter>
<attributeFilter name=”Workstation” operator=”eq”>
<value>EEL</value>
</attributeFilter>
</filteringPredicate>
</eventCondition>
<action actionProvider=”TWSAction” actionType=”sbs” responseType=”onDetection”>
<scope>
SBS EEL#PROCESS_FILE
</scope>
<parameter name=”JobStreamWorkstationName”>
<value>EEL</value>
</parameter>
<parameter name=”Parm_1″>
<value>FILE_NAME=%{fileCrtEvt1.FileName}</value>
</parameter>
<parameter name=”Parm_2″>
<value>SAMPLE_INTERVAL=%{fileCrtEvt1.SampleInterval}</value>
</parameter>
<parameter name=”Parm_7″>
<value>EVENT_RULE_ID=%{fileCrtEvt1.EventRuleId}</value>
</parameter>
<parameter name=”Parm_5″>
<value>HOSTNAME=%{fileCrtEvt1.Hostname}</value>
</parameter>
<parameter name=”Parm_6″>
<value>IP_ADDRESS=%{fileCrtEvt1.IPAddress}</value>
</parameter>
<parameter name=”Parm_3″>
<value>WORKSTATION=%{fileCrtEvt1.Workstation}</value>
</parameter>
<parameter name=”JobStreamName”>
<value>PROCESS_FILE</value>
</parameter>
<parameter name=”Parm_4″>
<value>TIME_STAMP=%{fileCrtEvt1.TimeStamp}</value>
</parameter>
</action>
</eventRule>
</eventRuleSet>
 
 

16. Select the Job Stream Name and choose Select an Action à Submit Job Stream into Current Plan

​Create a Variable Table, Job, and Job Stream with to process the file

Follow the steps below to create the objects

  1. Login to the Dynamic Workload Console
  2. Navigate to Design à Manage Workload Definitions
  3. Click on Create New à Variable Table
  4. In the General tab, enter in the Name field, PASS_VAR
  5. The variable to hold the properties of the file monitored are created and referenced internally, as such, no Variables need to be created.
  6. Click on the Save icon to save the Variable table.
  7. The following is the Variable Table definition is composer format.

 

VARTABLE PASS_VAR
MEMBERS
END

8. Click on Create New à Job Definition à UNIX
9. In General tab, enter in the Name filed, FILE_RECEIVED_N_PROCESSING
10. In the Task tab, Select Script and enter in the Script name field:

/bin/echo -e “\nThe properties of the file received are: \n\nFile Name=^FILE_NAME^ \nSample Interval=^SAMPLE_INTERVAL^ \nWorkstation=^WORKSTATION^ \nTime stamp=^TIME_STAMP^ \nHostname=^HOSTNAME^ \nIP Address=^IP_ADDRESS^ \nEvent Rule ID=^EVENT_RULE_ID^ \n\nThe next job in the Job Stream can now start processing the file.\n”
11. Notice that the same variable names are used to retrieve the values as they are assigned to the respective properties of the file monitored in the Event Rule, IF_FILE_EXISTS_PROCESS_FILE.

​12. Click on the save icon to save the Job
13. The following is the Variable Table definition is composer format.

 
EEL#FILE_RECEIVED_N_PROCESSING
SCRIPTNAME “/bin/echo -e \”\nThe properties of the file received are: \n\nFile Name=^FILE_NAME^ \nSample Interval=^SAMPLE_INTERVAL^ \nWorkstation=^WORKSTATION^ \nTime stamp=^TIME_STAMP^ \nHostname=^HOSTNAME^ \nIP Address=^IP_ADDRESS^ \nEvent Rule ID=^EVENT_RULE_ID^ \n\nThe next job in the Job Stream can now start processing the file.\n\””
STREAMLOGON iwadmin
DESCRIPTION “A Job to demo how to retrieve variables passed from an Event Rule”
TASKTYPE UNIX
RECOVERY STOP
 
 
14. Click on Create New à Job Stream
15. In General tab, enter in the Name filed, PROCESS_ FILE
16. Scroll down to the Variable table field and enter PASS_VAR to associate the variable table to the Job Stream.
17. In the Details view, right click on the Jobs and choose Add Jobs
18. Enter in the Job definition field, FILE_RECEIVED_N_PROCESSING, and click on Add
19. Click on the save icon to save the Job Stream
 
job stream process file
 
 
20. The following is the Variable Table definition is composer format.
 
SCHEDULE EEL#PROCESS_FILE
DESCRIPTION “A Job Stream to demo how to retrieve a variable passed from an Event Rule”
VARTABLE PASS_VAR
:
EEL#FILE_RECEIVED_N_PROCESSING
END
 
 
 

Monitor the execution of the Event Rule

​Follow the steps below to create the file, monitor the execution of the Event Rule, submission of the Job Stream, and the Job log to view the variable passed from the Event Rule to the Job Stream.

1. Login to server where the file is monitored and create a file to simulate the arrival of the file cd /tmp
touch datafile.100120201202.txt
2. On the MDM, monitor the contents of the messages.log cd /opt/ibm/iwa/iws/TWSDATA/stdlist/appserver/engineServer/logs
tail -f messages.log
3. Once messages are displayed about the Event Rule being executed, login to the Dynamic Workload Console and navigate to monitoring the Job stream.
4. The Job Stream, PROCESS_FILE, is submitted.
5. Go into Jobs view and monitor the status of the Job, FILE_RECEIVED_N_PROCESSING
​6. Once it is successful, view the Job Log.

 
 
file received and processing
 
 
7. Notice that the Variables are successfully passed from the Event Rule to the Job Stream and the Job is able to retrieve them from the Variable Table.
8. Also notice that the Variable Table has not been populated with the variables but were mapped to it internally.
 
VARTABLE PASS_VAR
MEMBERS
END
 
The same process can be used to create an Event Rule with Modification completed file monitor event.

Want to learn more? Write to me at Sajjad.Kabir@hcl.com or schedule a demo.
 
 
Comment wrap

Start a Conversation with Us

We’re here to help you find the right solutions and support you in achieving your business goals.

  |  August 2, 2023
HCL Workload Automation: How to drive your workload by business logic
The HCL Workload Automation, conditional dependencies feature, helps solve design problems where processes are not modelled as a straight pipeline and need branching.
  |  July 24, 2023
Workload Automation: More than 100 Plugins Lets You Automate More (and Better)
The Automation Hub is an innovative work-in-progress as we regularly add many more items to it. It showcases HCL Workload Automation’s ability to orchestrate IT and business workloads.
  |  February 10, 2023
Banking Case Study: New Client Adoption and Reverse Check: HWA+DRYiCE-iControl
We look at a banking case study wherein we are going through the “New Client Adoption and Reverse Check” business process.