| Date: | July 17, 2007 / year-entry #258 |
| Tags: | history |
| Orig Link: | https://blogs.msdn.microsoft.com/oldnewthing/20070717-00/?p=25983 |
| Comments: | 29 |
| Summary: | We left off our story last time by raising the problem of programs that send messages to windows that have already been destroyed and how window handle re-use exacerbates the problem. Although this is clearly a bug in the programs that use window handles after destroying the window, the problem is so widespread that the... |
We left off our story last time by raising the problem of programs that send messages to windows that have already been destroyed and how window handle re-use exacerbates the problem. Although this is clearly a bug in the programs that use window handles after destroying the window, the problem is so widespread that the window manager folks in Windows NT decided to take a more proactive approach. (Reiterating in case you couldn't figure it out: This entry goes "behind the scenes". Behavior described here falls into the category of "implementation detail" and is subject to change at any time.) In Windows NT, the 32-bit For example, the first time entry 0x0124 in the table was used, the handle corresponding to that entry was 0x00010124. After that window is destroyed and a new one created in slot 0x0124, the handle for the new entry is 0x00020124. Each time the entry is re-used, the uniquifier increments by one. But what about those 16-bit programs? When a 16-bit program used a window handle, the window manager would just use that value as the index into the table. There was no uniquifier, so the window manager just ran with whatever was in that slot. After all, the uniquifier is wasn't essential to correct operation of the window manager; it was added merely to cover for buggy programs. Consequently, 16-bit programs were just as susceptible to the "handle re-use problem" as they were in Windows 3.1 and Windows 95. Only 32-bit programs got the extra protection. But since the window handle was an index, the full 16-bit range of window handles became available (minus special values like Next time, we'll look at the magical 10,000 per-process handle limit. |
Comments (29)
Comments are closed. |
I’ve always wonder how common that trick is. I’ve seen it used to avoid the reuse problem in VMS’s process IDs. I’ve used it myself to avoid the same.
NTFS does a similar thing. File reference numbers (like inode numbers in Unix) are 64 bits. The first 48 bits are the index into a table and the next 16 bits are the sequence number (incremented every time that index is reused). That way if you have something that references file 12345, but file 12345 is deleted and gets reused, you know that your reference to file 12345 is no longer valid.
Assuming that they’re even lets you use the last bit as an internal flag of sorts. You just need to AND it with FFFE before handing it back to the OS.
(Not that I advocate doing such a thing. But I’ve seen it done.)
Was the last bit used as a flag in older versions of Windows the same way it is in Windows Mobile?
(Calling SetForegroundWindow on WM acts differently if you add 1 to your HWND)
This is the part about Windows that I just don’t understand. I know backwards compatibility is important, but at what cost? Wouldn’t it be better for everyone in the long run if you forced people to do things the right way? With all these hacks to support bad practices it’s amazing Windows even works at all. It would not surprise me if there are still ancient hacks from the Windows 3.1 (or even DOS) days embedded deep in the kernel.
On the other hand, if a million programs do something the wrong way does that make it the right way? Personally I don’t think so, but I could see the argument for it.
It’s good to know I’ll still be able to run Lotus 1-2-3 on Windows 2045.
"Wouldn’t it be better for everyone in the long run if you forced people to do things the right way?"
Raymond repeats it a lot :
If a new version of Windows removed those hacks and application X ceased working, most users would charge the OS and say "I can’t use app X on Windows Y, so Windows Y sucks"
I guess the problem is when the general media tells everyone a new OS like Vista "doesn’t work with older hardware", that just lets vendors off the hook. (as if they didn’t write new drivers for XP or 2000).
After over a year of opportunity to beta test and seven months of release Hewlett Packard has decided to make their custmers buy new printers (to replace fairly new printers). I guess they need an influx of cash.
Such an unethical practice is going to get me looking elsewhere. I’m glad someone tries to make the upgrade to a new OS compatible, and that of course is MSFT.
I’m constantly amazed at the amount of work MSFT does to retain backwards compatibility and make the OS work better around buggy programs.
In some sense, it could hurt theoretical maximums of the system, like in this example, but I think in each case a reasonable bargain is struck between compatibility and having an extra 30,000 window handles or so that go unused.
Speculating wildly, I would assume that programs assumed HWNDs were even because they have historically been classed with pointers in the 16-bit eras.
My guess is that people used the least significant bit to overload the meaning of an HWND/HANDLE. Some Win32 functions do something similar as I recall.
@CGomez
Not wishing to drag things too far off topic, but such a tactic by HP seems incredibly self defeating – as soon as you force someone to purchase a new product you’re giving them free license to jump ship to a competitor.
(Not to mention the fact that HP make their money on consumables rather than new machines)
Uh, you won’t. You can’t even run it on Vista. MS has dropped 16-bit support going forward.
As far as the never-ending backwards compatibility, I see the move to .NET as an attempt to break with the past. It gives developers a new API and, hopefully, corrects the flaws and assumption of the past. Of course, I expect that in 10-20 years we’ll be hearing stories of the broken bits of .NET that are still around because of all the programs that misused the API.
@kbiel: untrue. 16-bit support is dropped on x64 because the processor doesn’t support 16-bit and 64-bit programs simultaneously, and the effort of providing an emulator was considered too great. (One does appear in Virtual Server, though, you can still run DOS programs in a VM running a 32-bit OS.)
I think it’s unfair to blame applications for using HWNDs after they’re destroyed. For a start, the application might be in a separate process, or the DestroyWindow might be in asynchronous code which the application vendors don’t have access to. This sort of race condition is therefore inevitable (especially if it is in practice harmless)…
Rich:
That sounds something like "it’s unfair to blame applications for using pointers to memory that’s been free()’d". Window handles are resources, and require the same primacy of management that memory allocations do. If you’re using a window handle, you’d better be able to control both ends of its lifecycle, or you shouldn’t be using that handle. There may be some esoteric edge condition where that’s not possible, but none springs to mind at the moment.
As for "in practice harmless", it’s all fun and games until someone loses an eye.
Just my personal opinion – at all costs. Microsoft is in business of providing operating systems that makes all existing software simply work. Because of this companies and individuals pay billions of dollars to Microsoft every year. And that’s why there is no other business like Microsoft. Other folks inevitable try to "force people to do things the right way" and they soon learn what the outcome is – irrelevance and/or bankruptcy.
I got an impression that with .NET folks at MS tried to avoid this backward-compatibility "curse" by allowing to have multiple .NETs running side by side. But this approach seemed to have too many limitation to work, such as only one runtime is allowed per process. So eventually it appears they had to make it backward compatible to allow previous components to run seamlessly.
Certainly, backward compatibility is a very hard thing to do, but those succeeding at it will be paid off handsomely.
Rich: the problem isn’t that another program is using that same number later, but that the same program is using an HWND that has already been freed, and then may have been assigned to a different program, and thus sending messages to a window that won’t be expecting them.
An example of why it matters: WM_USER. Maybe in App X WM_USER+1 means "update the whoozit in the corner of the window" but in App Y it means "Reload widget file" or something else. If App X is randomly sending messages to a window in App Y, the outcome could be innocuous, or disastrous.
Mike: I stand corrected. There is much misinformation on this point throughout the intarweb and I fell for it.
BTW, running it in Virtual Server (Virtual PC, VMWare, etc.) is not exactly the same. Implementing compatibility through a VM would probably stop the whining about backwards compatibility preventing forward progress. On the other hand, it hasn’t been until recently, in my opinion, that using VMs to run older software on older versions of Windows/DOS has become practical (or at least acceptable by business users).
Gabe: IIRC the commonality goes one step further, too – you can always use zero for the sequence number when requesting FILE records, and the first 16 entries have zero there because they never get reused. (It’s a while since I dug around in any of that, though!)
I’m with John on backwards compatibility. In this case, the cost is trivial anyway – a limit of "only" 30,000 or so windows instead of 60,000 in exchange for a few customers able to use your product without jumping through an extra hoop or two. In a similar vein, you’ll find malloc normally aligns blocks of memory so that you can safely put almost anything in them, even at the expense of losing a few bytes of memory. Do you really think the "feature" of having thousands more potential windows going unused is worth breaking existing code? I don’t.
It’s different with features which actually offer something useful – supporting files over 2Gb in size by breaking the ability to treat file sizes as a signed 32 bit integer, for example – but how often have you found yourself needing the extra 30,000 windows?
"In a similar vein, you’ll find malloc normally aligns blocks of memory so that you can safely put almost anything in them, even at the expense of losing a few bytes of memory."
malloc is REQUIRED to give you memory that’s properly aligned to hold anything. (C89 4.10.3; C99 7.20.3)
"Why did we lose half of the window handles? Because there are programs out there that assume that window handles are always even numbers!"
*boggle*
That’s just amazing! I wonder what they actually used this assumption for? Did they perhaps divide all their handles by 2 so they could save space by fitting them in … a 15-bit int?!?
I thought before I read the final sentence that it was going to be that some programs stored handles in a signed 16-bit int, but assumed that only +ve values were valid. That kind of makes sense in a weird kind of way. Not much sense, I’ll grant, but surely more sense than assuming they’re even!
But malloc() still doesn’t give you 16-byte aligned memory for SIMD register access.
I think it was probably more of a case that as the platform matures, the need for a complete break becomes less and less. It’s obviously "nice" to be able to completely break compatibilty when needed, but it’s probably better not to if you can avoid it.
Just to stick up for Rich: these were 16-bit programs. They didn’t have pre-emption to worry about, and the main way for communicating was through HWNDs. Sure, use them defensively, but you can never get exclusive control of one.
HWNDs are a shared resource even now (per-winstation), and my guess is that 32-bit programs are still at risk of some lunatic creating and destroying a billion windows.
"Raymond repeats it a lot :
If a new version of Windows removed those hacks and application X ceased working, most users would charge the OS and say "I can’t use app X on Windows Y, so Windows Y sucks" "
I know this goes against Raymond’s policy of avoiding naming examples, but Vista made an important change to the permissions required for normal users to make changes to Program Files, which also impacted programs. And we saw exactly what happened when programs weren’t allowed to do that, and how those faults were laid at Vista’s feet, instead of the programs that changed Program Files to suit itself instead of putting its configuration settings in the user profile where it belongs.
"I know this goes against Raymond’s policy of avoiding naming examples, but Vista made an important change to the permissions required for normal users to make changes to Program Files, which also impacted programs…."
In the name of backwards compatibility, Vista redirects writes to Program Files and HKLMSoftware into a user’s profile.
I was floored when I learned about this. This isn’t some little appcompat hack… it’s practically a filesystem filter inspecting each file open and moving it around. Granted, not a new idea, but not exactly something that was widely deployed. (It was amusing to read the KB article explaining how files can appear in a app’s File Open dialog but not in Explorer.)
Back to window handles: what did NT do when it detected somebody using an obsolete HWND? Can I set a flag in the OS so the Window Manager kills apps that try to use obsolete HWNDs? :=)
DriverDude: You would probably find that most of your favorite programs stopped functioning if you could do that.
There’s always the brain-tickling option of reference-counting window handles (open/close).
BTW, I remember reading that Windows NT 3.51 and older automatically filled out the uniquifier part of a window handle (the high 16-bit) if it was zeroed. e.g. casting a window handle to a short, like in the Excel XLL function xlGetHwnd before Excel 2007. NT 4.0 stopped doing that.
PingBack from http://weseetips.com/2008/08/24/how-to-check-whether-the-window-handle-is-valid/