| Date: | December 18, 2003 / year-entry #168 |
| Tags: | code |
| Orig Link: | https://blogs.msdn.microsoft.com/oldnewthing/20031218-00/?p=41443 |
| Comments: | 21 |
| Summary: | MSDN just published an article telling people that it's okay to use reserved fields in an internal structure. Anybody who does this has just voided their warranty. Please put a "This program has a high likelihood of crashing after you install the next Service Pack or upgrade your OS" in your license agreement. And you... |
MSDN just published an article telling people that it's okay to use reserved fields in an internal structure. Anybody who does this has just voided their warranty. Please put a "This program has a high likelihood of crashing after you install the next Service Pack or upgrade your OS" in your license agreement. And you wonder why app compat is so hard. |
Comments (21)
Comments are closed. |
There are a high number of .NET newbies that come here looking for information. I’m not a newbie and I still don;t have any idea what you’re talking about. Care to elaborate?
Sorry. This is not a .NET blog. This is a Win32 blog. If you’re looking for .NET stuff, you should look elsewhere.
My point was that this article explores the internals of how critical sections work (good for debugging your program when its critical sections start acting weird), but it also decides to run around MODIFYING internal data structures, which is bad. The Spare fields are "reserved for future use". Someday, Windows will use them and your code that modifies an "unused" field is now corrupting a field that contains important information.
The article doesn’t mention that the internals of critical section behavior can change at any time. In fact, I know for a fact that it has already changed in Longhorn in a manner that will make the provided program crash under certain (rare but legal) conditions.
So any information you get from that article, use it only for diagnostic purposes. Do not ship a program that relies on it, because I already know that it will crash on Longhorn.
A field marked "reserved" means "This is here because we expect to add features and use this field in the future. Don’t touch it."
If you use that field for yourself, but a future version of the OS starts using it, one or the other will break. And when your code breaks, you can’t claim "M$ put code in Windoze to break my program! waaah!"
Thanks for the pointer to the article. I don’t intend to shoot myself in the foot by using reserved data fields, but anything by Matt Pietrek is typically worth reading.
So if this article exists on a Microsoft site, why don’t you guys correct it?
I think you’re exaggerating Raymond. Everyone knows that "under the hood" articles like this aren’t reliable from OS to OS. They just want to see an implementation on a fixed platform and mess around with it, I highly doubt anyone would ship code with this stuff enabled (well except for the Office team).
As for fixing articles on MS’s site, I’ve used the links on MSDN to contact MS on many occasions to fix winapi docs to no avail, why would they start changing stuff now? Especially when they want to make it appear that C/C++/winapi is outdated.
MSJ is a magazine and they can print what they want. Sometimes they send an article to the product teams for comment or review but at the end of the day, it’s their magazine and they decide what goes into it.
The article is actually a good one. Like he says in the opening: "A solid understanding of critical sections in Windows can really come in handy when you need to track down multithreading performance issues in your code."
Just stay away from the part where he recommends modifying the system critical sections or messing with the reserved fields. And remember of course that any implementation details are subject to change, so don’t write code that relies on them.
I’ve used the links on MSDN to report doc bugs too, and I’ve gotten replies. Maybe I’m just more famous.
I had a bad dream involving critical sections two nights ago, but not at Pietrek and Osterlund’s level of detail. Fortunately vacation is nigh.
asdf, I reported a SDK bug yesterday via the link and got a reply today ("fixed for Jan. ’04") and I am not as famous, in some circles, as Raymond.
This is the API team’s fault, really. A forward-thinking programmer would have labeled the fields "BOOL ShouldFormatHardDisk" and "DWORD KernelObjectIndexHint" instead of the tempting "DWORD Spare[2]."
Raymond, you said “And remember of course that any implementation details are subject to change, so don’t write code that relies on them.”
After reading this article I wrote some CS code that relies on the fields in the CRITICAL_SECTION struct. The reason is that I’m using SEH, and have some __try/__except blocks in which the code enters and leaves critical sections. If an exception occurs, I don’t know if I’m currently in the CS or not. Even wrapping the code in __try/__finally wouldn’t solve my problems.
So I was forced to write some code that examines the contents of the CRITICAL_SECTION struct to see if the current thread has ownership of the CS. If so, I leave. My question is, is this a bad idea? Thanks…
Seth.
There are so many ways of doing this that don’t involve relying on undocumented behavior. I’ll make a separate post on it.
Surely naming the fields "Spare" _invites_ people to co-opt them for their own use. Why weren’t they named "Reserved", which does have the desirable "keep your dirty hands off" implications?
Thank you for the good observation, and sorry to go off on a tangent, but the title is just asking for it.
"How to void your warranty"? I thought that the purchase of any Microsoft product was a way to void its warranty. I first learned this with Windows 95, attaching a new SCSI disk via a new PCMCIA-SCSI adapter and using the fdisk command as documented. I had problems immediately, and problems continued both within the first 90 days and after the first 90 days. Microsoft’s web site said go to the PC’s vendor for support. When I’ve gone to vendors they’ve sometimes had bug fixes for their own stuff (drivers etc.) but vendors have never supplied fixes for Microsoft’s stuff. After several months I figured out that fdisk, used as directed, was creating overlapping logical drives. The workaround was to not use fdisk. If the PCMCIA-SCSI card vendor supplied a utility to run under Windows 3.1 to partition hard disks, that would run under Windows 95 and create non-overlapping logical drives, but of course no FAT32.
In addition to the sentences buried in the above paragraph, there has been since been additional refusal of warranty service. Originally I lost 1 GB of data (which was non-trivial in those days) and spent several months and 11,000 yen in train fares going to vendors and the retail store where I bought the stuff, trying to track down the problem. Of course after tracking it down, it only takes 10 minutes to reproduce. One time Microsoft accidentally let me report this bug without charging me the customary US$35 or 4,000 yen fee for reporting bugs. I explained how to reproduce it in 10 minutes. Then Microsoft did yet again refuse warranty service.
You say it is possible for a customer to void a Microsoft warranty, meaning that the warranty wasn’t really void at the time of purchase? I do not believe you.
Funny that Windows 98’s fdisk doesn’t have that bug, but Microsoft has still never fixed Windows 95’s fdisk. If a SCSI disk was ever fdisked under Windows 95 but did not subsequently have its MBR zeroed, then it’s still a time bomb. It doesn’t matter what OS you upgrade to, the partitions are still sitting there, still overlapping, and will eventually lose all of their contents.
The title was a joke. The point is that messing with internal data structures is a pretty certain way of ensuring that nobody will even try to support your system. If you call developer support because your grovelling into undocumented system data structures isn’t working, nobody will try to help. And any system instability will be blamed on said grovelling.
First off Raymond, I love your blog. It explains a lot of stuff in Windows over the years that I’d wondered about.
In any event, as one of the authors of the article, its purposes were twofold:
1) Explain how critical sections work, and give some insight how you might track down issues in the debugger.
2) Provide some sample code that automates some things that are painful to do manually during development process. No where (and correct me if I’m wrong) did Russ or I suggest that you ship your code using the critical section library. It’s a *development* tool.
It’s good to know that the "spare" fields will change in future OS builds. However, there are developers *now* who are having problems debugging code in shipping apps. Should we never implement a valuable technique simply because it might not work some day?
Truth be told, many innovative products over the years have "broken the rules". Anybody remember that TSR’s originally weren’t documented? The whole TSR industry got started because a tiny little company called Borland started using an undocumented OS feature.
Over the years, I’ve written countless amounts of version specific and/or undocumented code. I’ve gone through hell and high water to first make sure there’s not a documented/supported way first, however. When people come asking me if some undocumented technique will work, I first grill them to make sure there’s not a supported way.
When I have gone the unsupported route, I don’t think I’ve ever complained that Microsoft "broke me". I simply made whatever changes were necessary to accomodate the newer way of doing things and moved on. It’s a fact of live when writing advanced software that lives on the edge.
So there’s my two cents, and keep up the great posts!
If you weren’t able to find a documented way of doing something, shouldn’t that that be a clue that you probably shouldn’t be doing it?
My concern is that people are going to write code that assumes that critical sections always work as the article describes, since there wasn’t (in my opinion) a strong enough warning that the information is for diagnostic purposes only. The information is valuable and good to have, but developers need to make sure to use the powers only for good.
If you do something that’s unsupported, make sure your customers understand it. "This program makes assumptions about Windows that may become invalid at any service pack or security update."
I’ll write another blog entry on some of the "grovelling into internal data structures" that apps have done and that the Windows team have had to "accomodate" in one way or another.
Each time an app relies on undocumented behavior, the OS ships a little later as developers are taken off task to hack around yet another app. To take your example: In the DOS source code there are chunks of "junk DNA" – code fragments that do absolutely nothing, just sitting around wasting space. They exist because various TSRs do memory scans looking for certain byte sequences, so DOS has to carry these "magic byte sequences" around to keep those TSRs from crashing.
> If you weren’t able to find a documented way of doing something, shouldn’t that that be a clue that you probably shouldn’t be doing it?
This is exactly the sentiment that’s frustrated non-Microsofties for decades. It’s an attitude that seems to say "We’ve provided you developers with everything you could possibly need." Obviously, this isn’t the case as people are still hacking around in the OS.
My favorite story here is when an NT team member told me that it wasn’t possible to write a single machine kernel debugger. It was all I could do to not tell him I was using a very early build of SoftIce/NT. The point is, some really great products were only possible by doing undocumented things.
I’m not one of those open source zealots at all, but this is one area where they have an advantage. A good idea can come from anybody, not just within Microsoft.
As for junk DNA, I try to avoid searching for byte strings if at all possible. When left with no other choice, I’ve always made my code degrade gracefully. Developers that don’t code defensively frustrate the hell out of me as well.
I certainly understand your points about apps hacking around the OS causing problems for MS. I’ve heard and seen many examples where the OS goes to great lengths to not break existing apps, and I applaud Microsoft’s effort in this area.
However, saying that people shouldn’t innovate because the OS didn’t plan for it is just wrong. There’s a long history of undocumented APIs becoming documented because there was a need that MS didn’t originally see.
So to sum up, sometimes there’s no way to avoid "breaking the rules". That said, if you’re going to break the rules, do it responsibly.