| Date: | February 17, 2005 / year-entry #41 |
| Tags: | code |
| Orig Link: | https://blogs.msdn.microsoft.com/oldnewthing/20050217-00/?p=36423 |
| Comments: | 37 |
| Summary: | One danger of the MsgWaitForMultipleObjects function is calling it when there are already messages waiting to be processed, because MsgWaitForMultipleObjects returns only when there is a new event in the queue. In other words, consider the following scenario: PeekMessage(&msg, NULL, 0, 0, PM_NOREMOVE) returns TRUE indicating that there is a message. Instead of processing the... |
One danger of
the In other words, consider the following scenario:
This wait will not return immediately, even though
there is a message in the queue. That's because the call to
A common variation on this is the following:
If it so happens that there were two messages
in your queue, the
When Note, however, that this sequence is not a problem:
This wait does return immediately, because the incoming posted
message sets the "There is a new message in the queue that nobody
knows about" flag, which
The Armed with this knowledge, explain why the observed behavior with the following code is "Sometimes my program gets stuck and reports one fewer record than it should. I have to jiggle the mouse to get the value to update. After a while longer, it falls two behind, then three..."
// Assume that there is a worker thread that processes records and
// posts a WM_NEWRECORD message for each new record.
BOOL WaitForNRecords(HANDLE h, UINT cRecordsExpected)
{
MSG msg;
UINT cRecords = 0;
while (true) {
switch (MsgWaitForMultipleObjects(1, &h,
FALSE, INFINITE, QS_ALLINPUT)) {
case WAIT_OBJECT_0:
DoSomethingWith(h); // event has been signalled
break;
case WAIT_OBJECT_1:
// we have a message - peek and dispatch it
if (PeekMessage(&msg, NULL, 0, 0, PM_REMOVE)) {
TranslateMessage(&msg);
DispatchMessage(&msg);
}
if (SendMessage(hwndNotify, WM_GETRECORDCOUNT,
0, 0) >= cRecordsExpected) {
return TRUE; // we got enough records
}
break;
default:
return FALSE; // unexpected failure
}
}
}
|
Comments (37)
Comments are closed. |
Shouldn’t that be:
while (PeekMessage …
No, that was Raymond’s whole point! If you peek, then wait, it fails as Raymond pointed out. While it is possible that you could write it as a PeekMessage loop, just writing while(PeekMessage()) isn’t enough.
If while’s used to replace the current if, that should be enough, I think…
Raymond, in your example, is there actually any work done in the windows proc for WM_NEWRECORD? I currently don’t understand why the falling behind would increase unless the counter was actually increased inside the window proc.
I guess I wasn’t explicit enough. I assumed people would have figured out that the window procedure goes something like this:
case WM_NEWRECORD:
process the new record;
records++;
display new record count in window;
break;
case WM_GETRECORDCOUNT:
return records;
When we have a message notification, we should process all available messages and check if the event is signalled .
case WAIT_OBJECT_1:
// we have message(s) – peek and dispatch
while (PeekMessage(&msg, NULL, 0, 0, PM_REMOVE)) {
if(WaitForSingleObject(h,0) == WAIT_OBJECT_0)
DoSomethingWith(h); // event has been signalled
TranslateMessage(&msg);
DispatchMessage(&msg);
}
if (SendMessage(hwndNotify, WM_GETRECORDCOUNT,
0, 0) >= cRecordsExpected) {
return TRUE; // we got enough records
}
break;
Thanks for posting this entry. It, looks like I was doing it the wrong way in my code but never noticed any ill effects. There goes 95 and NT 4.0 support.
The WaitMessage function works the same way. It only returns on NEW messages.
Can somebody post what is the correct code? I’ve spent some time reading everything, but I don’t want to be left guessing whether I understood it all correctly.
Thx!
Very helpful topic, BTW.
I would rather have had the ability to open a read-only handle to a live event object created by the message queue. It’d be way more flexible than trying to kludge MsgWaitForMultipleObjects into whatever scheme or framework I’m working with.
Ray: reentrancy is a general problem in code that uses Windows message queues. This is just something you have to deal with – the system can’t protect you from all possible scenarios.
Joshua: simply exposing an event would be very inefficient if you are only interested in a certain type of messages. You’d need some mechanism to tell the system when to signal the event. And then you’d have to deal with cases where two or more threads are waiting on the same event (should this be illegal? or should each thread get its own event object?)
Ray: reentrancy is a general problem in code that uses Windows message queues. This is just something you have to deal with – the system can’t protect you from all possible scenarios.
Joshua: simply exposing an event would be very inefficient if you are only interested in a certain type of messages. You’d need some mechanism to tell the system when to signal the event. And then you’d have to deal with cases where two or more threads are waiting on the same event (should this be illegal? or should each thread get its own event object?)
Jiggling the mouse to get a response is a common occurence in the Windows 2000 Start menu and Windows XP Classic version Start menu. It is so common that it doesn’t even take thinking. But this base note does make me wonder. You know, click on the Start button, move up to Programs, move to the right and locate the folder containing the link you really want to click on, but that folder doesn’t expand. You have to move the mouse to hover over another folder and then move back to the one you really wanted to expand. So does Start menu processing contain the bug described here?
No that’s caused by a global foreground idle hook. Remember? You complained about this last year and I debugged it for you.
ATL includes a function "AtlWaitWithMessageLoop". Back a while ago on the ATL mailing lists, there was discussion of some deadlock possibilities with it (in part due to the issue you raise here), and how it might be improved. Here’s the version that I came up with and currently use. I’ve often wondered if the approach I use has any possible problems. What improvements could be made to this version? (the formatting might get messed up)
BOOL WaitWithMessageLoop(
HANDLE hHandleToWaitOn,
DWORD dwInitialTimeOutMilliseconds = 0,
DWORD dwIterateTimeOutMilliseconds = 500)
{
DWORD dwRet=0;
MSG msg={0};
dwRet = ::WaitForSingleObject(hHandleToWaitOn, dwInitialTimeOutMilliseconds);
if(dwRet == WAIT_OBJECT_0)
{
// The object is already signalled.
return TRUE;
}
while(true)
{
// There are one or more window message available. Dispatch them.
while(::PeekMessage(&msg,NULL,NULL,NULL,PM_REMOVE))
{
::TranslateMessage(&msg);
::DispatchMessage(&msg);
dwRet = ::WaitForSingleObject(hHandleToWaitOn, 0);
if(dwRet == WAIT_OBJECT_0)
{
// The object is already signalled.
return TRUE;
}
}
// Now we’ve dispatched all the messages in the queue.
// Use MsgWaitForMultipleObjects to either tell us there are
// more messages to dispatch, or that our object has been signalled.
dwRet = ::MsgWaitForMultipleObjects(1, &hHandleToWaitOn, FALSE,
dwIterateTimeOutMilliseconds, QS_ALLINPUT);
if(dwRet == WAIT_OBJECT_0)
{
// The event was signaled.
return TRUE;
}
else if(dwRet == WAIT_OBJECT_0 + 1)
{
// New messages have come that need to be dispatched.
continue;
}
else if(dwRet == WAIT_TIMEOUT)
{
// We hit our time limit, continue with the loop.
continue;
}
else
{
// Something else happened.
return FALSE;
}
}
return FALSE;
}
Ah… This is exactly one problem that I’ve encountered during my serial port application development. I was totally puzzled at the random lost of records. We (the development team) did tried to figure out the pattern of problem but never success, so the problem hang there until a new collegue told us about this…
Gads… this function is so florked up that several articles have appeared in Windows Developer/MSDN magazine about it (I wrote one of them, so it stuck in my mind).
A much more fundamental danger of this function is that it can break the synchronization of the mutex if you’re not careful how you use it (recursive waits in particular are problematic).
Consider the case where you have a Lock() function that properly first calls MsgWaitForMultipleObjects, then processes the pending messages in a loop as Raymond suggests.
Now consider two different message handlers that need to access the protected resource:
case WM_PAINT:
Lock();
if (pProtected) {
// Call some function that calls some other
// function that does the following:
Lock(); // This recursive Lock is broken
pProtected->ProcessPaint();
Unlock();
}
Unlock();
case WM_KILLFOCUS:
Lock();
if (pProtected) {
delete pProtected;
pProtected = NULL;
}
Unlock();
The Lock above that’s labelled "This recursive lock is broken" can process a WM_KILLFOCUS message which will successfully delete the protected resource (the thread already holds the mutex, so the wait will suceed). This, of course, causes a crash in the next line.
This all seems to be due to the fact that MsgWaitForMultipleObjects checks the message queue *first* rather than checking the state of the waited-for object first.
So, effectively, the necessity for processing the messages with a Peek() loop has generated a little preemptive operating system of its own, and since it’s in the same thread that has the locked resource, the mutex doesn’t protect.
The only solution I was able to come up with is to always put a "if (WaitForSingleObject(hMutex, 0) != WAIT_OBJECT_0)" before you try to use MsgWaitForMultipleObjects.
It should have been implemented this way in the OS, though.
What happens if a message is enqueued between the call to MsgWaitForMultipleObjects and the call to PeekMessage…
E.G:
<- (worker thread) Message 1 Added
(processing thread) MsgWaitForMultipleObjects
<- (worker thread) Message 2 Added
(processing thread) PeekMessage
(processing thread) Message 2 processed
Since PeekMessage is only looking at the top message on the queue would not the first message be ‘lost’ since it will be considered ‘old’ as far as MsgWaitForMultipleObjects is concerned?
Of course, I could be barking up the wrong tree, I don’t have much experience with windows messaging and I suck at concurrency.
2/17/2005 4:17 PM Raymond Chen
> No that’s caused by a global foreground idle
> hook. Remember? You complained about this
Sorry, I don’t remember. Thank you for pointing it out. Perhaps I should keep notes :-)
> last year and I debugged it for you.
Thank you again. Do you know if it will be in Windows XP SP3 and Windows 2000 SP5? (I don’t think I need to ask if you have the power to put it there, but can still ask if you know.)
Can someone tell me if the following would be acceptable in the message loop for a window thread, or if I am still using the MsgWaitForMultipleObjects the wrong way. Sorry, in advance, if the code shows up incorrectly, I have never posted here before.
Thanks.
/* Start message loop for this thread. */
HANDLE[] hArray = { GetCurrentThread() };
while (TRUE)
{
MSG msg;
DWORD dwWait;
while (PeekMessage(&msg,NULL,0,0,PM_REMOVE))
{
if (msg.message == WM_QUIT)
{
ExitThread(0);
return 1;
}
TranslateMessage(&msg);
DispatchMessage(&msg);
}
dwWait = MsgWaitForMultipleObjectsEx (1, hArray, INFINITE, QS_ALLINPUT, 0);
if (dwWait == (WAIT_OBJECT_0 + 1))
{
continue;
}else{
// Do something..
}
}
It was some program you installed that was messing with foreground idle. A translation program or something as I recall. Perhaps you could complain to the vendor of that program.
Don’t you have to do something like while(::PeekMessage(&msg,NULL,NULL,NULL,PM_REMOVE))
{
::TranslateMessage(&msg);
::DispatchMessage(&msg);
}
on WAIT_OBJECT_0 too, or else you can forget about those messages as well?
> The MsgWaitForMultipleObjectsEx function lets you pass the MWMO_INPUTAVAILABLE flag to indicate that it should check for previously-ignored input.
But this flag is not supported on NT.
> You call MsgWaitForMultipleObjects and include the QS_ALLPOSTMESSAGE flag
What is the difference between
QS_ALLPOSTMESSAGE = $0100;
and
QS_POSTMESSAGE = $0008;
?
The MSDN docs says the same for both:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/msgwaitformultipleobjectsex.asp
"QS_POSTMESSAGE A posted message (other than those just listed) is in the queue."
"QS_ALLPOSTMESSAGE A posted message (other than those listed here) is in the queue"
IMO, this is a really messy API, but I think we are using it correctly – we use an object oriented wrapper (in Delphi) around the raw API calls, but it really boils down to calling
MsgWaitForMultipleObjects with the flag:
QS_ALLINPUT = (QS_INPUT or QS_POSTMESSAGE or QS_TIMER or QS_PAINT or QS_HOTKEY or QS_SENDMESSAGE);
and when it returns WAIT_OBJECT_0 + FObjCount we basically return to Delphi VCL’s TApplication.ProcessMessages routine that will empty the message queue.
The very bottom of the MsgWaitForMultipleObjectsEx page says: "The QS_ALLPOSTMESSAGE and QS_POSTMESSAGE flags differ in when they are cleared" and some more detail after that.
I sure hope an answer key is forthcoming…
In my opinion, situations that justify MsgWaitForMultipleObjects are quite rare nowadays. In the past thread were expensive and developers needed to scratch their heads how to cram in gazillion things in one thread. Today it’s just doesn’t worth it. Besides, this is annoying when you have message loops splattered here and there in the code. If I need to process some data which involves synchronization issues, then I don’t even want to think about messing it with GUI. Just stash it in worker thread, do the job, then notify GUI in some flexible and lazy way (say, with PostMessage).
2/17/2005 10:05 PM Raymond Chen
> It was some program you installed that was
> messing with foreground idle.
That is a completely unrelated issue. The matter of the Start menu, at random times not expanding folders unless the mouse is moved to a different folder and moved back, occurs even when all applications come from Microsoft. It is intermittently reproducible both on real PCs and virtual PCs. Now I resume thinking that your base note here looks like a likely explanation. The symptom really resembles the example you gave.
Hm well I’ve never experienced it. But I doubt it’s this problem. Menus don’t mix user and kernel waits.
The dichotomy breaks in the other direction, too.
Is there a way to tell Windows to "from now on pretend i don’t know about any messages in the queue" function? Basically undo the strangeness that PeekMessage does.
Does MsgWaitForMultipleObjects also tag queued messages as "read" like PeekMessage. i.e. Once i "realize" there is a message in the queue with MsgWaitForMultipleObjects, am i then forced to process all messages in the queue?
"In my opinion, situations that justify MsgWaitForMultipleObjects are quite rare nowadays. In the past thread were expensive and developers needed to scratch their heads how to cram in gazillion things in one thread."
Actually MsgWaitForMultipleObjects turns out to be very useful in multithread situations.
It’s the only way for you to add a user-defined window message that works like WM_PAINT — a very low priority message, and there’s only one of them in the queue no matter how many times you post it.
I used a message loop based on MsgWaitForMultipleObjects when I had a program with a worker thread that did computation and wrote output text. There was also a main UI thread which controlled the window, so the worker thread should not be allowed to change the window contents itself.
When the worker thread (which you can imagine as printing a million digits of Pi) has some text to output, it must post a message to the UI thread to get the text displayed. But if you used ordinary user-defined messages, you could get thousands of tiny WM_USERs clogging up the message queue.
The solution is to have a text buffer that holds the new output. Any time new text is added to the buffer, the worker thread posts a WM_PAINT-type message to the UI thread. There’s only one WM_PAINT at any time, and it’s always handled after more critical interactive messages (like scroll-bar manipulation). When the UI thread services the message, it outputs all the text in the buffer at once, just as a WM_PAINT handler updates the whole invalidated region of the window, regardless of how many smaller invalidations composed it.
But since you can’t define your own WM_PAINT-type message, you can get the same effect by using MsgWaitForMultipleObjects and replacing "post a message" with "flip on the event object." If you write the loop correctly (as described by Raymond) you get the same behavior.
(Raymond and others: Are there any potential gotchas to this approach?)
daev wrote:
"Actually MsgWaitForMultipleObjects turns out to be very useful in multithread situations.
It’s the only way for you to add a user-defined window message that works like WM_PAINT — a very low priority message, and there’s only one of them in the queue no matter how many times you post it."
I agree. That’s the example of MsgWaitForMultipleObjects where it’s handy. However, with two reservations:
1. You use it to "improve" regular main message loop, not to implement bastard message loop somewhere in program.
2. While less elegant, simple WM_TIMER with polling of text buffer is much less confusing than MsgWaitForMultipleObjects. Unless you really need to update GUI ASAP, updating it 3-5 times in second will be indistinguishable from user’s point of view. Consider how significantly less explanation requires WM_TIMER to maintenace programmer than esoteric gotchas of MsgWaitForMultipleObjects.
daev wrote:
"Actually MsgWaitForMultipleObjects turns out to be very useful in multithread situations.
It’s the only way for you to add a user-defined window message that works like WM_PAINT — a very low priority message, and there’s only one of them in the queue no matter how many times you post it."
I agree. That’s the example of MsgWaitForMultipleObjects where it’s handy. However, with two reservations:
1. You use it to "improve" regular main message loop, not to implement bastard message loop somewhere in program.
2. While less elegant, simple WM_TIMER with polling of text buffer is much less confusing than MsgWaitForMultipleObjects. Unless you really need to update GUI ASAP, updating it 3-5 times in second will be indistinguishable from user’s point of view. Consider how significantly less explanation requires WM_TIMER to maintenace programmer than esoteric gotchas of MsgWaitForMultipleObjects.
Putting together pieces you already know.
Disclaimer: I hesitated posting this because this is a topic that is extremely complicated and deep,…
PingBack from http://sarathc.wordpress.com/2008/09/19/when-and-how-should-we-use-msgwaitformultipleobjects/
PingBack from http://www.cromis.net/blog/2008/12/active-sleep-procedure-aftermath/