| Date: | August 29, 2007 / year-entry #320 |
| Tags: | code |
| Orig Link: | https://blogs.msdn.microsoft.com/oldnewthing/20070829-00/?p=25363 |
| Comments: | 16 |
| Summary: | Here's a question that floated past some time ago: In my code, I have multiple objects that want to talk to the same handle (via DeviceIoControl). Each time I create an object, I use DuplicateHandle to increment the reference count on the handle. That way, when each object calls CloseHandle, only the last one actually... |
Here's a question that floated past some time ago:
In other words, the code went something like this:
// h is the handle that we want to share with a new CFred object
CFred *MakeFred(HANDLE h)
{
// "Duplicate the handle to bump the reference count"
// This code is wrong - see discussion
// All error checking removed for expository purposes
HANDLE hDup;
DuplicateHandle(GetCurrentProcess(), h,
GetCurrentProcess(), &hDup,
0, FALSE, DUPLICATE_SAME_ACCESS);
return new CFred(h);
}
Kernel handles aren't reference-counted.
When you call
From the original problem statement, we know that
the CFred *pfred1 = MakeFred(h); CFred *pfred2 = MakeFred(h); delete pfred1; delete pfred2; What actually happens when you run this fragment?
The first time we call
When
When
Meanwhile, the calling code's copy of
What we really want to do here is duplicate the handle
and pass the duplicate to each object.
The
// h is the handle that we want to share with a new CFred object
CFred *MakeFred(HANDLE h)
{
// Create another handle that refers to the same object as "h"
// All error checking removed for expository purposes
HANDLE hDup;
DuplicateHandle(GetCurrentProcess(), h,
GetCurrentProcess(), &hDup,
0, FALSE, DUPLICATE_SAME_ACCESS);
return new CFred(hDup);
}
The fix is one word, highlighted in blue.
We give the duplicated handle to the
You can think of In summary, a handle is not a reference-counted object. When you close a handle, it's gone. When you duplicate a handle, you gain a new obligation to close the duplicate, in addition to the existing obligation to close the original handle. The duplicate handle refers to the same object as the original handle, and it is the underlying object that is reference-counted. (Note that kernel objects can have reference from things that aren't handles. For example, an executing thread maintains a reference to the underlying thread object. Closing the last handle to a thread will not destroy the thread object because the thread keeps a reference to itself as long as it's running.) |
Comments (16)
Comments are closed. |
I love the DuplicateHandle() function since the very first time I managed to make an "big" Interprocess Communication work (excluding sockets).
It also helped me learn that a file object has only one file pointer, regardless of the number of handles. In fact, I can’t really think of any good reason to call DuplicateHandle() on a file handle. But I did know a dirty trick on duplicated console input buffer handles…
so many people get it wrong. even ones who understand pointers.
@CornedBee: the handle is not just a reference to the kernel object, but also a token that signifies that a process has access to the object and is allowed to decrease the internal reference count. The kernel must be able to enforce that such a token is genuine, to make sure the internal books remain balanced.
Suppose a handle was just a pointer to a kernel object that has a single reference count. Then processes would be able to delete references they didn’t create (maybe that would do for Windows 3.11, but not in any secure system) and you wouldn’t be able to remove outstanding references on process exit.
So, you’ll have to do some per-process bookkeeping, which means that you’ll effectively have multiple handles to the same kernel object anyway (because handles differ between processes). As Raymond points out, POSIX file descriptors work in the same way and that is not surprising since the scenario is similar: different processes (which must be isolated) share access to a single object.
Given that you have to keep track of references, it’s easier to keep track of simple ones than having a two-level scheme of reference counting per process, and per kernel object. Ofcourse, if you want to have only one kernel reference and do per-process reference counting too (which is arguably somewhat more space-efficient) you can implement that in user mode just fine. (But don’t bother unless you’re creating hundreds of thousand handles.)
Or you could use a funny thing called a reference counted pointer! std::tr1::shared_ptr
The fact is, this stuff is not really that well documented. In an effort to abstract the internals of the system (taken to the extreme), many of the nuiances are left inadequately explained, leading to many hours of developer frustration.
Even a simple picture paints a thousand words, so here’s the data structures from the Windows kernel (from WinDbg 6.6 running under Windows XP). Each (executive) process in the kernel is represented by the EPROCESS structure. The ObjectTable field points to a HANDLE_TABLE structure that maintains the per-process handle table. Each entry in this table is represented by the HANDLE_TABLE_ENTRY structure, whose Object field points to the object, and indirect, the OBJECT_HEADER, which contains the actual reference (and handle) counts. The Body field is the first byte of the type-specific object body…
0:000> dt _EPROCESS
—snip—
+0x0c4 ObjectTable : Ptr32 _HANDLE_TABLE
—snip—
0:004> dt _HANDLE_TABLE
+0x000 TableCode : Uint4B
+0x004 QuotaProcess : Ptr32 _EPROCESS
+0x008 UniqueProcessId : Ptr32 Void
+0x00c HandleTableLock : [4] _EX_PUSH_LOCK
+0x01c HandleTableList : _LIST_ENTRY
+0x024 HandleContentionEvent : _EX_PUSH_LOCK
+0x028 DebugInfo : Ptr32 _HANDLE_TRACE_DEBUG_INFO
+0x02c ExtraInfoPages : Int4B
+0x030 FirstFree : Uint4B
+0x034 LastFree : Uint4B
+0x038 NextHandleNeedingPool : Uint4B
+0x03c HandleCount : Int4B
+0x040 Flags : Uint4B
+0x040 StrictFIFO : Pos 0, 1 Bit
0:004> dt _HANDLE_TABLE_ENTRY
+0x000 Object : Ptr32 Void
+0x000 ObAttributes : Uint4B
+0x000 InfoTable : Ptr32 _HANDLE_TABLE_ENTRY_INFO
+0x000 Value : Uint4B
+0x004 GrantedAccess : Uint4B
+0x004 GrantedAccessIndex : Uint2B
+0x006 CreatorBackTraceIndex : Uint2B
+0x004 NextFreeTableEntry : Int4B
0:004> dt _HANDLE_TABLE_ENTRY_INFO
+0x000 AuditMask : Uint4B
0:004> dt _OBJECT_HEADER
+0x000 PointerCount : Int4B
+0x004 HandleCount : Int4B
+0x004 NextToFree : Ptr32 Void
+0x008 Type : Ptr32 _OBJECT_TYPE
+0x00c NameInfoOffset : UChar
+0x00d HandleInfoOffset : UChar
+0x00e QuotaInfoOffset : UChar
+0x00f Flags : UChar
+0x010 ObjectCreateInfo : Ptr32 _OBJECT_CREATE_INFORMATION
+0x010 QuotaBlockCharged : Ptr32 Void
+0x014 SecurityDescriptor : Ptr32 Void
+0x018 Body : _QUAD
Hope that clears things up.
So let me get this straight.
There are kernel objects.
There are handles to kernel objects. When all handles to an object have been closed, the kernel object is destroyed.
With DuplicateHandle, you can create a new handle to a given kernel object. The new handle has a different value, but references the same object.
Hmm …
It seems to me like this would be more complicated to implement than really doing it the way this guy thought it worked: by having all handles to the same object be the same, and just incrementing/decrementing reference counts. Any specific reason it wasn’t done this way?
Offhand, I can only come up with one: defense against multiple calls of CloseHandle on the same handle invalidating other handles. Given that several processes would share the same handle values (after all, what’s the point of simplifying the system when you still need the complicated version for some use cases), that’s probably a very valid reason.
dgr: That’s hardly surprising. After all, the pointers themselves aren’t reference-counted, it can only be the underlying memory that is. The problem here is that handles do *not* behave like simple, reference-counted pointers to kernel objects.
OK, came up with another reason: there is handle-specific information to be stored, like open mode.
I hope this isn’t offtopic…
I have always wondered about the accounting associated with DuplicateHandle when creating handles for other processes.
For example, two processes A and B. B is just an unwitting process standing around minding it’s own business. Process A is nasty and wants Process B to either go away or at least look extremely bad.
Process A starts Duplicating handles with Process B as the target process. Process B never knows about these handles as there is never any IPC between A and B.
Unix solves this problem by limiting handle duplication to an IPC mechanism (the kernel can then know each process is in agreement and any failure can be cleanly dealt with – either the IPC op worked or it didn’t).
Just wondering what other people think…
Luke: to duplicate a handle into Process B, you need a handle to Process B with PROCESS_DUP_HANDLE access. Which, as http://msdn2.microsoft.com/ms684880.aspx notes, allows you to just as easily duplicate (HANDLE)-1, and do what you like with the target process.
Those who do not understand unix, are doomed to reinvent it. Poorly.
@HS
And those who do understand UNIX long for something better.
Perhaps a more interesting question is why, when something is working, people tend to assume it’s because they’ve not include the correct magic incantation of flags in their system calls.
It’s a perplexing question, actually. A quick scan of the CreateFile documentation shows that there isn’t a FILE_ALLOW_CLOSE_ORIGINAL_HANDLE_INSTEAD_OF_DUPLICATE that could plausibly fix this problem, so why didn’t the poster realise this wasn’t the problem?
nksingh> Time for Plan 9, then? ;)
Jules wrote:
"Perhaps a more interesting question is why, when something is working, people tend to assume it’s because they’ve not include the correct magic incantation of flags in their system calls."
Perhaps because so many attributes are not documented and must be discovered :-).
Pat O
Sohail: "a reference counted pointer! std::tr1::shared_ptr"
So you’re OK with requiring user programs to be written in C++? And you’re OK with forcing all programs to use a specific C++ standard library (that includes the ref-counted pointer), and if they use some other C++ standard library, they’d be incompatible with the kernel?
And, after all that, you’re happy having the library do all the same work that the kernel does right now *anyway*, to give you the *sole* benefit of having this particular situation work?
If so, that’s fine. But that seems like a massively, massively over-engineered solution to a nonexistant problem to me. Just realize that objects have refcounts instead of handles, and each additional reference can have a different handle value. Now it’s obvious that the lpTargetHandle parameter’s out value could be different, and poof, the problem disappears.
(Granted, I don’t know whether it’s documented that objects have refcounts instead of handles. But I must have picked it up *somewhere*.)
"Granted, I don’t know whether it’s documented that objects have refcounts instead of handles. But I must have picked it up *somewhere*."
It’s made pretty clear by the documentation on CloseHandle:
"CloseHandle invalidates the specified object handle, decrements the object’s handle count, and performs object retention checks. After the last handle to an object is closed, the object is removed from the system."
It didn’t do quite as much back then.