I saw Dale’s Diagnosing Triggering Problems post, and thought I’d add a checklist I put together.

The idea was to give some help diagnosting common triggering issues when setting up a new triggered queue for an app-team, or diagnosing a queue-depth alert at 2am on a callout.

The Conditions for a Trigger Event section in the APG is the definitive reference, but this checklist is written more from a diagnosis point of view.

The document has a UNIX bias I’m afraid…

The Checklist

This checklist applies when messages are building up on a queue, and you would normally expect an application to be triggered by WMQ to process these messages.

There are a number of checks to perform (you are suggested go through each of these in order).
A possible action is provided for each check, if an issue is highlighted.

After performing the action, you may need to nudge WMQ to re-check the criteria for a trigger event to occur, and trigger the application if the issue is resolved.
To do this, alter the queue to disable and then re-enable triggering:

ALTER QL(TRIGGERED_QUEUE) NOTRIGGER
ALTER QL(TRIGGERED_QUEUE) TRIGGER

1. Triggering must be enabled, and of the correct type:

Check:

Display the TRIGGER TRIGTYPE and TRIGDPTH attributes of the queue to determine the type of triggering that is enabled:

DIS QL(TRIGGERED_QUEUE) TRIGGER TRIGTYPE TRIGDPTH

Sample output:

AMQ8409: Display Queue details. 
   QUEUE(TRIGGERED_QUEUE)                  TYPE(QLOCAL) 
   TRIGGER                                 TRIGDPTH(1) 
   TRIGTYPE(FIRST)

Action:

If NOTRIGGER is seen in the output, then triggering is disabled. Enable it as follows:
ALTER QL(TRIGGERED_QUEUE) TRIGGERIn most cases you can expect the type of triggering (TRIGTYPE) to be FIRST.
A summary of the three types is as follows:
- EVERY
  Trigger events occur every time a message is delivered to the queue. This occurs even if applications already have the queue open to retrieve messages.
- FIRST
  Trigger event occur when the depth of the queue transitions from zero to one.
- DEPTH
  Trigger events occur when the depth of the queue transitions from
  (TRIGDPTH-1) to TRIGDPTH.
  
  Triggering is automatically disabled on the queue, and must be re-enabled by the application (using MQSET).

2. No application can already have the queue open to get messages:

This only applies to FIRST or DEPTH triggering.
For EVERY triggering, a trigger event occurs every time a message arrives, regardless of how many applications are already actively retrieving messages from the queue.

Check:

Check the IPPROCS attribute in the QSTATUS for the initiation queue:

DIS QS(TRIGGERED_QUEUE) IPPROCS

Sample Output:

AMQ8450: Display queue status details. 
   QUEUE(TRIGGERED_QUEUE) 
   TYPE(QUEUE)                             CURDEPTH(0) 
   IPPROCS(0)

You can find which application has the queue open (IPPROCS>0) as follows:
DIS QS(TRIGGERED_QUEUE) TYPE(HANDLE) APPLTAG PIDSample Output:
```
AMQ8450: Display queue status details. 
   QUEUE(TRIGGERED_QUEUE) 
   TYPE(HANDLE) 
   APPLTAG(/home/myuser/bin/mycmd) PID(1234)
```

Action:

For FIRST or DEPTH triggering, if the IPPROCS number is any number greater than zero, then an application has the queue open to retrieve messages and triggering will not occur.
If the application is not the one expected to run against this queue, then it must end before the intended application can be triggered.
The name and PID of the application is shown in the QSTATUS output. If the name contains ‘amqcrsta’ or ‘amqrmppa’, then a remote client application has the queue open (this could be an administrator on another machine).

If an ‘amqrmppa’ process is terminated (kill -9 PID), this will shut down any other channels running within that process.
If the application connected to the queue is the triggered application for this queue, then the application team should be involved to investigate why the application is failing to process messages from the queue.

3. The initiation queue must be valid:

Check:

Display the INITQ attribute of the triggered queue:

DIS QL(TRIGGERED_QUEUE) INITQ

Sample Output:

AMQ8409: Display Queue details. 
  QUEUE(TRIGGERED_QUEUE)                    TYPE(QLOCAL) 
  INITQ(SYSTEM.DEFAULT.INITIATION.QUEUE)

Display the initiation queue to check it exists, is not an XMITQ, and is not inhibited for put or get:
DIS QL(SYSTEM.DEFAULT.INITIATION.QUEUE) USAGE PUT GETSample Output:
```
AMQ8409: Display Queue details. 
   QUEUE(SYSTEM.DEFAULT.INITIATION.QUEUE) 
   TYPE(QLOCAL)                      GET(ENABLED) 
   PUT(ENABLED)                      USAGE(NORMAL)
```
Remember to use single ticks for queues with lower case names:
DIS QL('inititaion.queue') USAGE PUT GET

Action:

If the INITQ attribute is blank, or incorrect, then change it to the name of the initiation queue against which the trigger monitor will run.
If the initiation queue is inhibited for put, then enable it as follows:
ALTER QL(SYSTEM.DEFAULT.INITIATION.QUEUE) PUT(ENABLED)

4. A trigger monitor must be active against the initiation queue:

Check:

Check the IPPROCS attribute in the QSTATUS for the initiation queue:
DIS QS(SYSTEM.DEFAULT.INITIATION.QUEUE) IPPROCSSample Output:
```
AMQ8450: Display queue status details. 
   QUEUE(SYSTEM.DEFAULT.INITIATION.QUEUE) 
   TYPE(QUEUE)                             CURDEPTH(0) 
   IPPROCS(1)
```
IPPROCS is the number of applications that have the queue open for input, or browse. This should be exactly 1.
Check the app which has the queue open is the Trigger Monitor:
DIS QS(SYSTEM.DEFAULT.INITIATION.QUEUE) TYPE(HANDLE) APPLTAGSample Output:
```
AMQ8450: Display queue status details. 
   QUEUE(SYSTEM.DEFAULT.INITIATION.QUEUE) 
   TYPE(HANDLE) 
   APPLTAG(/opt/mqm/bin/runmqtrm)
```

Action:

The trigger monitor can be started in the background as follows on UNIX:
nohup runmqtrm -m QMGR_NAME -q SYSTEM.DEFAULT.INITIATION.QUEUE >/dev/null 2>&1 &

5. The process definition must be valid

Check:

Display the PROCESS attribute of the triggered queue:

DIS QL(TRIGGERED_QUEUE) PROCESS

Sample Output:

AMQ8409: Display Queue details. 
   QUEUE(TRIGGERED_QUEUE)                  TYPE(QLOCAL) 
   PROCESS(TRIGGERED_PROCESS)

Ensure the process object exists, and validate the command which will be run by the trigger monitor when the trigger event occurs:
DIS PROCESS(TRIGGERED_PROCESS) APPLICIDSample Output:
```
AMQ8407: Display Process details. 
   PROCESS(TRIGGERED_PROCESS)     APPLICID(/home/myuser/bin/mycmd)
```

Action:

If the PROCESS attribute is blank, or incorrect, then change it to the name of a valid process object.
If the APPLICID attribute of the process object is incorrect, then alter it to correctly match the command which should be run.
Remember that on UNIX machines commands are case-sensitive, so must be entered in single ticks when defining/altering a process object in MQSC:
DEF PROCESS(TRIGGERED_PROCESS) APPLICID('/home/myusr/bin/mycmd')

6. The trigger monitor must be able to start the application

Check:

If the application fails to start an application, it places the trigger message on the dead-letter queue, with a feedback code in the ‘Reason’ field.
The Application Programming Reference manual describes the possible MQFB_* feedback codes
You can use the WebSphere MQ Explorer (or MO71) to view the message:
- Connect to the queue manager using the WebSphere MQ Explorer.
- Browse the contents of the dead-letter queue for the queue manager.
  The WMQ Explorer understands the format of the DLH header that is added to a message when it is placed on the DLQ.
Look at the ‘Reason’ field in the dead-letter header
If it is displayed in numeric form, the following link to the Constants manual should help to translate it to a MQFB_* code:

Action:

Investigate and resolve the problem starting the application.

7. The application must connect to the queue after it is started

Check:

From a WMQ administrator’s point of view, you can check whether the application is being started by the trigger monitor. Further investigation of the application’s operation would be performed by inspecting the application’s own logging/diagnostic info.

If you need to absolutely confirm that an application is being started by a trigger monitor, then the following process can be performed to restart the trigger monitor in the foreground (so you can see the output on your screen):

The best way to stop a trigger monitor is to inhibit the initiation queue for get:
ALTER QL(SYSTEM.DEFAULT.INITIATION.QUEUE) GET(DISABLED)
You can then immediately un-inhibit get on the queue:
ALTER QL(SYSTEM.DEFAULT.INITIATION.QUEUE) GET(ENABLED)
You can now run the trigger monitor as a foreground process, so you can monitor the output:
runmqtrm -m QMGR_NAME -q SYSTEM.DEFAULT.INITIATION.QUEUE
You will now need to open a new terminal (telnet/ssh session)
5. Disable and re-enable triggering on the queue to see the output
from the trigger monitor when a trigger message is processed:
ALTER QL(TRIGGERED_QUEUE) NOTRIGGER ALTER QL(TRIGGERED_QUEUE) TRIGGER

3 comments

Comments feed for this article

August 23, 2007 at 4:09 pm

Martha Princi

Hi. We have a trigger set up as FIRST and the used the browse option in the OPEN. What this APPEARS to have done is read blank MQ queues after the close. Have you ever seen this happen? We removed the browse in the OPEN and now it does not appear to be reaing blank queues

October 25, 2007 at 7:06 am

khangnt

I have MQ Server and one another app using VC++, but on different machines in the same LAN.

And I want the VC++ to get messages from a specific queue in MQ Server at realtime. I have 2 ways

1. VC++ will use polling method to read the queue frequently, 1s to check if there is a new message
2. Write a trigger on MQ Server, get a message then forward to VC++ using socket

Issue: new message does not always happen, method 1) is not good and polling will take many resource and time. method 2) is complex and requires a lot of coding.

Do you have any suggest?

October 25, 2007 at 7:37 am

Peter Broadhurst

The resource costs of your application waiting for messages to arrive on the WMQ queue is likely to be very low.
The application can simply contain a loop which issues a get method against the ImqQueue object, specifying MQGMO_WAIT (+ MQGMO_FAIL_IF_QUIESCING) and a wait interval (10 seconds for example, or MQWI_UNLIMITED).
WMQ deals with delivering a message to the application when one becomes available (business as normal for WMQ), and the application thread only wakes up every few seconds if required to check local state.
As always, the application should handle failed MQI method calls based on the reason codes. It may also require reconnection and retry logic – in case the queue manager is stopped for maintenance at some point or a network issue occurs.

Triggering Checklist

The Checklist

1. Triggering must be enabled, and of the correct type:

Check:

Action:

2. No application can already have the queue open to get messages:

Check:

Action:

3. The initiation queue must be valid:

Check:

Action:

4. A trigger monitor must be active against the initiation queue:

Check:

Action:

5. The process definition must be valid

Check:

Action:

6. The trigger monitor must be able to start the application

Check:

Action:

7. The application must connect to the queue after it is started

Check:

Categories

Archives

Meta

3 comments

Leave a reply to Peter Broadhurst Cancel reply

Triggering Checklist

The Checklist

1. Triggering must be enabled, and of the correct type:

Check:

Action:

2. No application can already have the queue open to get messages:

Check:

Action:

3. The initiation queue must be valid:

Check:

Action:

4. A trigger monitor must be active against the initiation queue:

Check:

Action:

5. The process definition must be valid

Check:

Action:

6. The trigger monitor must be able to start the application

Check:

Action:

7. The application must connect to the queue after it is started

Check:

Share this:

Related

Categories

Archives

Meta

3 comments

Leave a reply to Peter Broadhurst Cancel reply