| Date: | July 16, 2007 / year-entry #256 |
| Tags: | history |
| Orig Link: | https://blogs.msdn.microsoft.com/oldnewthing/20070716-00/?p=26003 |
| Comments: | 18 |
| Summary: | (Prefatory remark: The following information is of the "behind the scenes" nature and does not constitute formal documentation. Actually, nothing I write counts as formal documentation, so I shouldn't have needed to write that, but people sometimes intentionally play stupid and interpret all forms of the future tense as if I were making some sort... |
(Prefatory remark: The following information is of the "behind the scenes" nature and does not constitute formal documentation. Actually, nothing I write counts as formal documentation, so I shouldn't have needed to write that, but people sometimes intentionally play stupid and interpret all forms of the future tense as if I were making some sort of "guarantee" on behalf of Microsoft Corporation. I assure you that I have no such authority! It's times like that that I'm tempted to just give up writing.) Let's start with 16-bit window handles. Those are simple: They are just pointers into the window manager's data segment, cast to the In Windows 95, the window manager moved several categories of objects out of the default data segment into their own custom heaps. (And those were 32-bit heaps so they could be bigger than 64KB.) Window classes, menus, and windows each got their own "big" heap. There may have been other categories of objects that moved out of the default data segment, but those are the ones I remember. But since Windows 95 still had to support 16-bit programs, it needed a way to return 16-bit window handles back to those programs. To do this, the window manager allocated the memory in the 32-bit heap as "movable", which as we learned some time ago isn't actually movable. The purpose of allocating it as movable memory was to get that local memory handle, the No, wait, but that doesn't actually solve the problem, because a local handle in a 32-bit heap is still a 32-bit value. How do we get a 16-bit value out of that? When the window manager created the 32-bit heap, it asked the 32-bit heap manager very nicely if it could give back 16-bit handles instead of 32-bit handles. The heap manager did this by pre-allocating a 64KB block of memory and allocating its handles out of that memory block, using the offset into the block as the handle. Since each entry in the handle table is four bytes (a 32-bit pointer), the 64KB handle table can hold up to 16384 entries. This is why the documentation for
Actually, it was a little bit less than that because some of the entries were lost to bookkeeping overhead. For example, the handle value of zero could not be used because that would be confused with Now, you may have asked, "Well, if all the window handles are multiples of four, why not divide by four and then you can get the full range of 65535 window handles?" Well, remember that Windows 3.1 could handle only around 700 windows. Increasing this to 16,384 was enormous progress already. I mean, it's more than 23 times as much as what you had before. A hundred windows was already considered excessive at the time, so the window manager already could accommodate 163 abusive, badly-written programs. There's really no reason to bump that up to 655 badly-written programs. That'd just be encouraging programs to behave badly. Both the 16-bit Windows technique and the Windows 95 technique did suffer from the problem of handle re-use. When a window is destroyed, its memory is freed (as well as its handle on Windows 95). When a new window is created, there's a good chance that the memory or handle will get re-used, and consequently the numerical value of the window handle once again becomes valid, but refers to a different window. It so happens that boatloads of programs (and "boatloads" is a technical term) contain bugs where they use window handles after the window has been destroyed. When a window handle is re-used, that program sends a message to the window it thinks is still there, but instead it sends the message to a completely unrelated window. This doesn't bode well for the program, and it usually doesn't bode well for the new window that received the message by mistake either. Next time, we'll look at how the Windows NT folks addressed this problem of window handle re-use. Nitpicker's corner "Boatloads" is not a technical term. That was a joke. The initial version of this article accidentally omitted the word "not" from the opening sentence. Kudos to the people who were able to exercise their brain and figure this out from context instead of robotically taking everything at face value. There may be hope for the world yet. |
Comments (18)
Comments are closed. |
I think there’s a ‘not’ missing in the first line.
Nah, I reckon its overloaded by the "Nothing" in bold.
But that "Nothing" is in the second sentence. The first sentence says the exact opposite of the second.
Oh, am I nitpicking now, by pointing out a problem in an anti-nitpick paragraph?…
> It’s times like that that I’m tempted to just give up writing.
Raymond, just put a disclaimer in the side bar.
Well, Raymond, I agree with you on the effect of idiots on one’s ability to tolerate it.
However, having just read your book from cover to cover, which is something I very rarely do with anything technical, there is a LOT of value in what you write. Thank you.
For what it’s worth, if anyone has an old copy of my "Windows 95 System Programming Secrets" book, the subject of window handles and the USER heaps is discused therein.
Wow, I wrote that chapter 13 years ago!
Wow, 700 windows. I’ve got about 900 windows open on my system right now (so Spy++ tells me anyway) and most of those are inside background processes as I don’t have many "applications" (i.e. top-level windows I’m interacting with) open at the moment.
I would say that I’m glad the limit has increased since 3.11 but I guess people would have written things differently if the low limit was still there (e.g. not creating windows for background processes until they needed to be shown).
I’m looking forward to the next part about handle re-use. I have some ideas about how it might work but I’ll keep my mouth shut for now!
I tracked a problem on a Windows CE device recently to window handle reuse. A window was being destroyed but the handle was left dangling and then destroyed again which caused an Av in the process. The same code was running on Windows XP without issues for many years.
Window handle reuse is a major operating system flaw. To get around it, I make sure to never close my handles.
Raymond, I read your blog because I’m fascinated by little Windows design details like this. (I had speculated that window handles were just memory pointers and interesting to have that confirmed.) I’m also really amused by your disclaimers, but I hope you’ll ignore the idiots and keep writing.
I agree, in fact I would take it further: any numeric identifier that gets reused is a flaw. All window handles, kernel handles, memory pointers, file numbers, sockets and so on should represented by 512-bit numbers that we can safely gaurantee won’t be reused for the next 10,000 years. And if we need more than 10,000 years, we have plenty of time to switch to 1024-bit pointers…
"any numeric identifier that gets reused is a flaw"
Indeed it is. If a program tries to send message to a window and the handle is no longer valid because it destroyed it seconds ago it should crash and burn with a big red fat’n’ugly blinking exception.
Of course, if handle was refcounted on a per thread basis then it would be possible to detect reuse and crash the bastard. But I am sure Raymond will tell us how they fixed it. I can’t wait to hear the solution.
Er, how would that work exactly? How would you add a ref-count to the handle while still keeping it a 16-bit value?
Do you also propose that memory pointers returned from malloc() be ref-counted in the same way? What about inodes in a Posix-style filesystem? Or any of the other values which are represented by simple integers?
Window handles aren’t scoped to threads or processes so I don’t think that would work on its own.
If thread A destroys handle H then, without some other protection mechanism, it’s possible and legal for thread B (which could be in another process) to create a new window that happens to also have handle H. B could then pass H to A and A could, legitimately, send messages to H.
Let’s wait to see what Raymond says is the real answer.
Um, Dean, inodes already are refcounted. That’s how hard-links work (two directory entries pointing to the same inode means the inode’s refcount (aka link count) is 2). And actually, processes with the file open bump the refcount too. (When you unlink the last link to a file, the inode number isn’t reused until the file is really removed from disk — i.e. when all processes close their handle(s) to it. Actually the number may not be reused at all, but I think it can be.)
Now it’s true that the inode *number* doesn’t have a refcount in it, but the inode structure does. I’m not sure on the internal details of window handles, etc., but it may be possible to do something similar with them.
Or it may cause all kinds of other headaches.
(And now that I re-read what was being discussed, I see that this doesn’t really even relate to the handle-reuse problem. I don’t think there are any, but if any syscalls operate on an inode number (as opposed to a file handle or pathname), then they’d be susceptible to the same type of "process X gets an inode number, then the file gets deleted, then process X uses that number" type of problem. So never mind.)
Ok, maybe not exactly refcounting but something like this:
Assume you have process P with Thread A and Thread B.
Thread A creates window with handle 0x124. Thread B can send the messages to that window.
Then Thread A destroys the window and handle 0x124 is now free to be reused, but not for any thread that belongs to process P.
Handle is a void pointer so it points to some structure. In that structure you can have an entry where you store process ID of the last process which has used that particular handle.
Since process IDs are not reused but monotonically increasing, you can assume that all processes with ID <= to that one should not be able to reuse the handle.
But I am sure that there are other smarter ways. This is just a 2 minute hack so don’t expect miracles.
The correct technical term is, as you point out, not boatload. It’s shipload.
Igor, you are forgetting that HWNDs are like randomly assigned filenames. Any process in a desktop is allowed to send any message to any window on that desktop, just like any process can access any file owned by the uid of that process.
Just because Process P doesn’t need to use its own window 0x124 any longer doesn’t mean that P doesn’t have a legitimate need to talk to Process Q’s new window 0x124.