| Date: | January 21, 2005 / year-entry #20 |
| Tags: | code |
| Orig Link: | https://blogs.msdn.microsoft.com/oldnewthing/20050121-00/?p=36633 |
| Comments: | 27 |
| Summary: | Not very well known is that the bottom two bits of kernel HANDLEs are always zero; in other words, their numeric value is always a multiple of 4. Note that this applies only to kernel HANDLEs; it does not apply to pseudo-handles or to any other type of handle (USER handles, GDI handles, multimedia handles...)... |
Not very well known is that the bottom two bits of kernel HANDLEs are always zero; in other words, their numeric value is always a multiple of 4. Note that this applies only to kernel HANDLEs; it does not apply to pseudo-handles or to any other type of handle (USER handles, GDI handles, multimedia handles...) Kernel handles are things you can pass to the CloseHandle function.
The availability of the bottom two bits is buried in the
// // Low order two bits of a handle are ignored by the system and available // for use by application code as tag bits. The remaining bits are opaque // and used to store a serial number and table index. // #define OBJ_HANDLE_TAGBITS 0x00000003L
That at least the bottom bit of kernel HANDLEs is always zero is implied by
the This information is not useful for most application writers, which should continue to treat HANDLEs as opaque values. The people who would be interested in tag bits are those who are implementing low-level class libraries or are wrapping kernel objects inside a larger framework. |
Comments (27)
Comments are closed. |
Wow, that’s ugly.
Why not add an extra parameter "bool suppresCompletionNotification" to the functions that use this mechanism ?
Because it would have to be added to every API that could possibly cause an event to be set to the signalled state.
This includes the ReadFile and WriteFile API, as well as all the synchronization APIs.
That means that literally dozens of APIs would have had to be updated.
Remember: completion ports were added in NT 3.5, it would have been difficult (to say the least) to rev all the Win32 APIs to handle this.
Instead, they encoded the "suppress notifications" functionality into the handle itself.
I like how it says, "For compatibility with future operating systems, it is best to call GetCurrentProcess instead of hard-coding this constant" but you know that in 10 years, the App. Compat. team is going to have to figure out a way to make at least 3 dozen applications work that all used the constant value.
The ability to avoid a completion in a completion port for a particular operation is useful for perf. Consider:
A) I post an operation (read, write, what have you)
B) The file handle is already attached to a completion port
C) I know it’s going to take a while. I don’t want to make this call synchronous since I can reuse the thread for other things in the mean time.
D) I don’t want another thread to preempt this thread when this operation completes. For some other operations on this file handle, it’s ok but for this particular one, it isn’t.
E) In other words, I wan’t to go verify on my own terms with my current thread whether the operation has completed. I’ll probably do that when I have some spare cycles.
Thus it’s useful to be able to disable completion for a particular operation.
Someone correct me if I screwed up…
Why not just make the completion notification another attribute of the underlaying kernel object? Then it could have been settable with:
SetCompletionNotification(handle, TRUE|FALSE);
This just kind of smells as a hacked on interface. I wonder what Cutler thought of this hack?
You didn’t actually answer the question in the title: Why are the low two bits always zero? Because it’s really a pointer to a pointer, which is always aligned on a four-byte boundry. (Which is actually readable from user-space, but would I emphatically not suggest doing so on any code that you’re planning on shipping or want to be stable across any sort of OS upgrade, or you’ll give Raymond more backwards-compatability nightmares. Read: debugging tools and the fun of it only.)
It’s true that this information is not very interesting for people writing applications, but knowing what kernel handles look like is very useful for debugging.
For example, if you see a printf call stuck in WriteFile with a handle like 0x5fc you know that your stdout has probably been redirected. Regular standard handles look totally different (on my system stdin/out/err are 3, 7 and 11 respectively).
(Um, James, try dereferencing 0x5fc and see what that gets you… Kernel HANDLEs are not pointers to anything.)
Larry, well, instead of adding bool parameter to a bunch of functions, they could simply add a EnableNotifications(HANDLE,BOOL) API instead…
See my previous explanation of why EnableNotifications wouldn’t work: http://weblogs.asp.net/oldnewthing/archive/2005/01/21/358109.aspx#358334
What did Cutler think of this hack?
I mean, given that kernel handles are always multiples of four, I imagine that he was one of the people that thought this up. Unless ntdef.h has changed much since NT3.51?
Again, that affects all I/O on that handle, not just the one I/O call you’re making – see "cola"s example above. If you did a MakeHandleNotifyable(hfile, FALSE), then you just screwed up your other threads which are doing async I/O on that handle simultaneously and expecting completion notifications.
I think people fail to realize that the completion suppression flag is a >per-I/O-request< flag, not per-file or per-handle.
Because someone in the API team wanted to use a filehandle with non-APC asynchronous notification without spamming the completion port, and they knew a programmer in the OS team, and the OS manager didn’t want to change the interface just for one programmer, so they hacked it in and it got documented?
(Also the binary and source have to be backwards compatible and C compatible, so: no changing the arguments to DeviceIoControl in the DLL; *or* with a #define in the source; *or* with C++ overloading either! There has to be an entirely new function name.)
Let’s see if we can figure out why anyone would want to register a completion port and then to suppress completion port notifications.
How about a synchronous write? Like:
BOOL FakeSyncWrite(HANDLE hfile, int offset_, char *buf, int *length) {
BOOL success = 0;
waitobj = CreateEvent(name=NULL, security=NULL, manual=TRUE, signaled=FALSE);
OVERLAPPED async = { offset: offset_, hEvent: waitobj };
*(int*)&async->hEvent |= 1; // suppress completion ports, if any
success = WriteFile(hfile, buf, length, &async)
&& WaitForSingleObject(waitobj, delay=INFINITE);
if(CloseHandle(waitobj)) return success;
else return 0;
}
I think Waleri was requesting a function like:
HANDLE MakeHandleNotifieable(HANDLE x)
{
return x | 1;
}
so that he doesn’t have to hard-code 1 in his program.
But that would change the behavior for *all* I/O against the file, not just for the one I/O request. And if you wanted to disable completion notification for all I/O against a file, then don’t associate it with a completion port to begin with!
Ops, sorry missed that one…
Good point, but still, its better be via API call rather than set a bit directly.. something like
HANDLE MakeHandleNotifieable(HANDLE);
This would be much better than
*(int*)&async->hEvent |= 1;
Oh, well, from now on, our handles will be always round of four :)
Console handles have something different in the bottom
2 bits as well since they aren’t kernel objects but
get passed to ReadFile, WriteFile, etc APIs.
When using the same ‘handle’ between threads I always thought you were supposed to use DuplicateHandle when giving it up to other threads? Or am I missing something here.
I can see the use of the ‘spare cycle’ example above. But the function to set something inside the handle struct sounds much cleaner? Then clear on next io compleation? When using other threads? Or does DuplicateHandle ‘cheat’ and reuse some of the same stuff underneath?
Course it is a BIT (heh) late now…
Again people, remember, there can be multiple outstanding overlapped I/O’s on a single handle. (Hence the whole "overlapped" thing.) You can’t set a flag on the handle struct because the same handle is being used for multiple I/O’s. After all, if you could have only one outstanding I/O on a handle at a time, there would be no need for the event in the first place! You could just wait on the file handle.
Clearly there is widespread misunderstanding of how overlapped I/O works. Most of the "suggestions" on how to "fix" the completion port "problem" completely miss the point of overlapped I/O.
lazybones » Dirty trick
Well it was just an example :)
I think people are misunderstanding the original article, and thinking that the event handle itself needs to have its LSB set to prevent completion port notification.
What Raymond (and the docs) are saying is that the hEvent _member_ of the OVERLAPPED structure must have the LSB set to prevent completion port notification, eg:
OVERLAPPED o;
o.hEvent = hEvent;
*(DWORD*)&o.hEvent | = 1;
Or something like this:
#define EventNoNotify(e) (HANDLE)(((DWORD)e)|1)
OVERLAPPED o;
o.hEvent = EventNoNotify(hEvent);
You aren’t modifying the event handle, you’re just modifying the member of the OVERLAPPED struct when you pass the event handle to the function in question.
@Jon Potter: note that you need to typecast handles to ULONG_PTR/DWORD_PTR, not DWORD. You want to write 64bit compatible code, don’t you?
OVERLAPPED o;
o.hEvent = hEvent;
*(ULONG_PTR*)&o.hEvent | = 1;
or
#define EventNoNotify(e) (HANDLE)(((ULONG_PTR)e)|1)
While coding, I sometime ask myself, jokingly: What Would Cutler Do? (WWCD?)
Heh, my "interesting" code for modifying the bits of a void pointer seems to have caught on. I think it’s actually clearer than the same code with parentheses added, and the precedence happens to be correct. (Although I shouldn’t have used int.)
Although ov->hEvent is the value you are modifying, what you are doing is setting a tag bit on the OVERLAPPED structure. It has no effect elsewhere, probably. So the macro should apply to the structure:
#define SetCPortSuppression(overlapped) (*(LONG_PTR*)&(overlapped).hEvent))|=1
Hi NTDebuggers, this week’s puzzler just so happens to match its number: 0x000000006 = ERROR_INVALID_HANDLE.