Queue file storage system is full in websphere mq

Question

We are faced with a scenario where disk space was used for empty queues in a linux environment.

Our queue manager unexpectedly finished working with the file system, and we need to clear the q file in order to return the queue manager.

But in fact, we don’t have any messages in the queue at all. This shows a specific queue.

Why is disk space stored here? What is the main reason?

Answer 1

WMQ does not shrink queue files in real time. For example, you have 100 messages in the queue, and you consume the first. After that, WMQ does not compress the file and does not move all messages to one position. If he tried to do this for each message, you can never get the bandwidth that you now see in the product.

What happens is that WMQ will shrink queue files at certain points in the processing life cycle. Between the queue it becomes empty, and the file below it is shortened, but this latency is usually so small as to be invisible.

The event you are describing could theoretically exist in some very specific conditions, but that would be extremely rare. In fact, for 15 years when I was working with WMQ, I saw only a couple of cases where the delay in the compression of the queue file was even noticeable. I would suggest that what is actually happening here is that one of your assumptions or observations is wrong. For instance:

Was the line really empty?

• After you blew the file, the queue was most definitely empty. How do you know that it was empty before you blew the file?
• If any queue contains inconsistent messages, after QMgr reboot the queue will be empty. This is another case where the queue may seem empty after restarting QMgr, but was not at the time of the failure.
• If a message is received from a queue under synchronization, the depth of the queue is reduced, but the message is still active in the queue file. If the queue is omitted in one transaction, it retains full depth until a COMMIT occurs. This can cause the queue to be empty if it is not.

Is this a queue file populating the file system?

• Log instances can populate the file system even with circular logs. For example, with a large value for secondary extents, the log files can expand significantly and then disappear just as quickly.
• FDC files may populate the file system, depending on how the allocation was made.

Was there even an MQ?

• If QMgr shares the file system space with other users or applications, temporary files can fill the gap.

One of the problems that we see very often is that the application will try to place more than 5000 messages in the queue and get a QFULL error. The very first thing that most people do is install MAXDEPTH (999999999) to make sure that this will NEVER happen again. The problem is that QFULL is a soft error from which the application can recover, but filling the file system is a hard error that can reduce the whole of QMgr. The MAXDEPTH setting (999999999) trades a manageable soft error for a fatal error. The MQ administrator is responsible for ensuring that the MAXDEPTH and MAXMSGL in the queues are set so that the underlying file system is not populated. In most stores, all file systems have additional monitoring installed to raise warnings long before they are filled.

So, in most cases, WMQ does a very good job of reducing queue files. In particular, when a queue is emptied, this is a natural synchronization point at which the file can be shortened, and this usually happens within a few seconds after the queue is empty. You either fell into a rare race condition in which the file was not quickly reduced, or something else happens here, which is not so obvious in your initial analysis. In any case, control MAXDEPTH and MAXMSGL so that no queue can fill the file system and write code to handle QFULL conditions.