A February post on listeners lead on to some discussion on the operation of channels, and diagnosing issues.
I thought this comment from John merited a post of its own.

There are a number of status indicators and other sources of information available on the operation of WMQ channels – with new ones for WMQ V6.0.

So here’s a summary of some things that you can use in diagnosing channel issues and links to more information. There’s lots out there on this subject, so do add links to other useful docs and sources of information.

  • Overall status
    DIS CHS(*) STATUS
    -> Intercomms Manual
    -> Redbooks: WMQ V6 Fundamentals
    This might be enough to look at (at both sides of a channel) to resolve an issue.
  • WMQ error logs
    -> Redbooks: WMQ V6 Fundamentals
    -> Technote 1172370
    The error logs are a key source of information. If any channel is failing to start, or has experienced a network error, information is logged in the qmgr error logs. For some listener failures, the failure is logged in the WMQ system error logs.
    Look at the logs from both sides of the channel.
    In WMQ V6.0 you can increase the size of the error logs, and block unwanted messages to reduce the chance of useful info wrapping out of the logs
    -> Sys Admin Manual.
  • In-doubt status
    DIS CHS(*) INDOUBT
    -> Intercomms Manual
    -> Redbooks: WMQ V6 Fundamentals
    In-doubt status (and the units of work involved) are resolved automatically between MCAs when a channel is restarted.
    There are some circumstances where you may want/need to resolve it manually using RESOLVE CHANNEL.
    Examples are where you want to force messages to return to the XMITQ – to transmit to a different queue manager down another channel (noting this can introduce duplicate messages). Or where the sequence numbers between queue managers cannot be reconciled (one queue manager is restored from backup for instance).
  • Sub-state
    DIS CHS(*) SUBSTATE
    -> MQSC Manual
    This is new for WMQ V6.0.
    If the above sources of information fail to explain why messages are not moving over a channel, sub-state can give you additional insight.
    It shows you instant-in-time detailed status for a channel.
    For example, if a channel is waiting for messages to arrive on the transmission queue, or if the channel is executing user code in an exit.
    You may need to display this status a number of times to distinguish between a channel remaining in one particular state and simply transitioning between states.
  • Events
    -> Monitoring manual
    -> MS0P: WebSphere MQ Events and Statistics Plug-in
    WMQ can generate messages when events occur on a channel – which you can build upon to be notified of that event (with an app that generates an e-mail in response to particular conditions for example), or as an alternative to the error logs.
  • Statistics & real-time monitoring
    -> Monitoring manual – Accounting and Statistics
    -> Monitoring manual – Real-time monitoring – Channels
    -> MS0P: WebSphere MQ Events and Statistics Plug-in
    WMQ V6.0 can give you even more information, including point-in-time averages from MQSC/PCF and periodic statistics reports on your channels and transmission queues.
    The possible uses for this information are numerous, and you can analyse the data using the amqsmon sample, the MS0P SupportPac, or roll your own more sophisticated monitoring application.
  • End-to-end troubleshooting
    Often the key to diagnosing channel issues is to logically work through the end-to-end operation of the channel, checking each point.
    The following guide may help you in doing this: Redbooks: WMQ V6 Fundamentals
  • Search for known solutions
    The fact that you are reading this blog, means you are aware of the large community of people working with the WMQ product.
    The size of this community means you may well not be the first person to work through a particular issue.
    The following doc has some suggestions on where how you can find out whether others have hit the same issue, and find a solution:
    -> Redbooks: WMQ V6 Fundamentals
  • MustGather documentation for IBM Service
    If you are unable to explain why a channel is operating unexpectedly, and are raising an issue with IBM Service, then you can help IBM resolve your issue by gathering detailed information – at the time the issue is occurring.
    IBM have put together some documents to help you do this:
    -> MustGather: Read first for all WebSphere MQ v5.3, v5.3.1, and v6.0 products
    -> MustGather: Documentation required by the WebSphere MQ support team for a channel problem
    If your investigation has determined a channel is ‘stuck’ in an unexpected state, and are going to restart your queue manager to try resolve the issue, then you might even consider gathering extra information for IBM Service – before you take this workaround action.
    -> MustGather: Documentation required by the WebSphere MQ support team for a wait or hang
Advertisements