| Date: | August 11, 2004 / year-entry #304 |
| Tags: | other |
| Orig Link: | https://blogs.msdn.microsoft.com/oldnewthing/20040811-00/?p=38193 |
| Comments: | 38 |
| Summary: | Physical memory is not virtual address space. In my opinion, this is another non sequitur. I'm not sure what logical process led to this myth. It can't be a misapprehension of a 1-1 mapping between physical memory and virtual memory, because that mapping is blatantly not one-to-one. You typically have far more virtual memory than... |
Physical memory is not virtual address space. In my opinion, this is another non sequitur. I'm not sure what logical process led to this myth. It can't be a misapprehension of a 1-1 mapping between physical memory and virtual memory, because that mapping is blatantly not one-to-one. You typically have far more virtual memory than physical memory. Free physical memory doesn't have any manifestation in any virtual address space. And shared memory has manifestations in multiple virtual address spaces yet correspond to the same physical page. Though this brings up a historical note. In Windows/386, the kernel just so happened to map all physical memory into the kernel-mode virtual address space. There was a function _MapPhysToLinear(p, x) = _MapPhysToLinear(0, x) + p In Windows 95, the memory manager was completely rewritten and the above coincidence was no longer true. To conserve kernel-mode virtual address space, physical memory was now mapped linearly only as necessary. Of course, the drivers that relied on the old behavior were now broken because the undocumented behavior they relied upon was no longer present. As a result, when it starts up, Windows 95 looks around to see if any drivers known to rely on this undocumented behavior are loaded. (Windows 3.1 didn't support dynamically-loaded kernel drivers so looking at boot time was sufficient.) If so, then it went ahead and mapped all of physical memory into the kernel-mode virtual address space to keep those driver happy. This wasted virtual address space but kept your machine running. I can already hear people saying, "Microsoft shouldn't have made those buggy drivers work. They should have just let the computer crash in order to put pressure on the authors of those drivers to fix their bugs." This assumes, of course, that the cause of the crash could be traced back to the buggy driver in the first place. A very common manifestation of a stray pointer in kernel mode is memory corruption, which means that the component that crashes is rarely the one that caused the problem in the first place. For example, nearly all Windows 95 bluescreen crashes in |
Comments (38)
Comments are closed. |
Based on your last few posts, and the comments, I’d say it’s pretty obvious that quite a lot of people just don’t ‘get’ virtual memory at all. I suppose this means that it’s doing its job – after all, it’s supposed to be transparent to a user-mode process and to a user-mode programmer.
I can’t remember if I grokked it the first time through my Operating Systems course at University. The second time through (I had to retake a number of courses after changing programmes) I recall drawing diagrams of how x86 differed from the system the lecturer was demonstration – never identified, but probably IBM hardware.
If you think the problem is bad with programmers, you should try to explain it to users.
Usually it’s no problem when virtual address space is much larger than the practical physical memory that someone can put in a computer. We are in the odd part of the CPUs life cycle when it’s feasible to have more physical memory than virtual. Once we move to 64bit address space, the problem will be pushed back for another decade or so and we won’t need to explain all this to users.
By the way, I remember hearing a lecture from an Intel guy when I was in college talking about the 4GB virtual address space for the upcoming 286 (or was it 386 I can’t remember). This was at a time when hard disks held less than 40 meg and a typical PC had 256K. I thought that 4GB ought to be enough for anyone ;-)
Where is your diagram explaining the relation between "virtual address space" and "virtual memory"? Highly anticipated item!
vince:
1) You can’t change the past. Hindsight it 20/20
3) Pack the number of users of Windows (and developers) onto Linux and you’ll find plenty of shit programs being written.
I have multiple 8+ year old programs that work fine on Windows now too. Big deal.
I already described this in the article. Windows 95 did not auto-detect that a driver was buggy. It consulted a list of "known buggy drivers". The list of known buggy drivers (and the workaround for each one) was developed only after years of debugging in test labs.
What would be the point of putting up the warning? It’s not actionable. There’s nothing the user can do short of uninstalling whatever driver it was (if they could even figure out what the "BGMEM386" driver was and how to uninstall it).
"Aren’t there Microsoft Certification things for device drivers to somehow keep this from happening?"
How do you prove that a driver is not making invalid assumptions?
"I don’t understand Microsoft’s obsession with backwards bug compatibility."
Did you see the story where IBM telling its employees not to install XP SP2 because of backwards compatibility problems?
"Did you see the story where IBM telling its employees not to install XP SP2 because of backwards compatibility problems?"
Yes, I did, and it’s a perfect example of why you should do the right thing as you move forward. By not making everything backward compatible, XP SP2 is not compromising the security enhancements that help everyone, and it’s forcing IBM to fix their (internal) apps, which only hurts them in the short term.
A few object lessons like this, and I think you’d find a lot fewer problems going forward. APIs that do a better job validating proper usage (when possible) could also help avoid letting these broken apps from getting into the wild in the first place.
Besides, I’ve experienced many breakages from Windows updates, even when the application was coded to the API. Things like printing and user-mode communication through serial ports have changed more times that I can could since Windows 3.1.
[Mike Dimmick]
>>Based on your last few posts, and the comments, I’d say it’s pretty obvious that quite a lot of people just don’t ‘get’ virtual memory at all. I suppose this means that it’s doing its job – after all, it’s supposed to be transparent to a user-mode process and to a user-mode programmer.<<
Is it? Why? If you’re writing in C# I guess it’s mostly not an issue. When you start using say memory mapped files and shared memory you do begin to get confronted with the issues. I notice many questions in Forums which indicate that the poster didn’t have a basic idea of what an address space is, or that other programs’ storage is not even directly addressable (and let’s not mention shared data sections!) Even with modern hardware and software some knowledge of storage access patterns to optimise paging activity for example might be of value in certain applications. I write a lot of code in assembler and deal with things like cross memory accesses to other address spaces. Lots of this stuff runs in "user" mode, some of it in some system mode, but that’s only to do with needing to be authorised for certain things, it doesn’t alter the amount of understanding necessary (admittedly, it’s not for Windows).
The only people who really need to know the ins and outs of virtual storage in depth are probably the guys who write real and virtual memory managers and invent paging algorithms, but there’s still a lot to be said for having a basic understanding of what’s going on.
I notice for example (just a bit off topic, perhaps, but not completely) that VB.Net 2005 has now got something called "My", which (among quite a few other things) will solve the problems of all those ex-VB6 programmers who can’t understand how to access a second VB Form in VB.Net, because they just don’t have a basic (pun unintentional) understanding of what an object is. Good for them maybe, whether it will add to the average quality of future programs is disputable.
"I suppose this means that it’s doing its job – after all, it’s supposed to be transparent "
Yes, I completely agree with this. You can get an "A" in your Intro to Operating Systems class, but I don’t think it really sinks in till you spend lots of time with MapViewOfFile or the PE file format (file pos -> RVA -> VA, what fun).
Since this is quickly becoming a virtual mem myths series. How about this one: "Putting your swap file on a RAM drive is faster than having no swap at all!"
I only see your explanation of the relation between "virtual address" and "physical address", something I have concluded in my previous comment. So what’s the relation between "virtual address space" and "virtual memory"? Oh and what does it mean when two ptrs have two different values and when you say "They both map the same virtual memory"?
8-)
Raymond wrote: "In Windows/386, the kernel just so happened to map all physical memory into the kernel-mode virtual address space. There was a function _MapPhysToLinear. … the kernel mapped all of physical memory and just handed out pointers into that single mapping."
This is exactly what Linux normally does, only it uses a macro so there’s no point in bypassing it for speed. (Of course this means that drivers have to be rebuilt if the kernel is reconfigured to allow for larger amounts of virtual and/or physical memory, but the kernel developers have never pretended to maintain binary compatibility of drivers.) I’m surprised that Windows 95 didn’t continue to do the same, given that it was meant for quite small systems.
"A few object lessons like this, and I think you’d find a lot fewer problems going forward. "
Adrian, that’s one of those statements that’s very easy to make when you haven’t suffered through the consequences of a decision like that.
If we made a change that was "right", but caused the system to crash whenever X-product (that sold 10 million copies) is installed, no one would want to hear it. I’ve lived through this (one that slipped through) and it’s not pretty.
This is why you don’t let idealists/purists run massive engineering enterprises, IMO. Imagine if they tried to build the Apollo lander making no comprimises. Sure, it’d be reliable and safe, but they’d still be engineering today.
Remember what life was like in 1994. NT was only barely in the world, and nobody used it. Windows 3.1 was common to see, but was still mostly viewed as an add-on to DOS (you had to type ‘win’ to get there). While Microsoft was obviously dominant with regard to DOS, it didn’t have nearly the lock on the GUI OS market that it has today, and there were other DOS extenders, multitaskers, and GUIs.
They needed, more than anything else, to get Windows 95 to be accepted, so that Win32 programs would start being written, so that Windows NT would eventually become viable in the market. It’s easy to see why Microsoft was so eager to make Windows 95 as backward-compatible as possible, in order to encourage a high adoption rate.
-sd
I agree that refusing to load a driver from the list of known-to-be-buggy drivers would have been the most elegant solution.
What to do about non-standards compliant code/data has always been a big issue. If you let the non-compliant stuff by then it never gets fixed and propagates (but your customers are happy) and if you are stringent about it, things eventually get fixed but the developers and customers will blame you until that happens.
To a certain extent the blame rests with authors of the standards, who should really have provided a way to check that code/data does in fact comply. Microsoft did a good job with the driver verifier so that the issue is addressed in the future, but I don’t think a verifier was even possible in the days of Windows/386 and 3.1.
I am having my own experience with all this at the moment. It turns out that with the exception of my own program, there isn’t a single program out there on Windows, Linux or Mac that I can find that actually correctly implements the vcard standard. In most cases it seems like the developers only checked importing their own vcards that they exported.
And the vcard standards non-compliance isn’t over deeply technical things. It is stuff like getting field names wrong, or not decoded quoted printable fields.
What to do about non-standards compliant code/data has always been a big issue. If you let the non-compliant stuff by then it never gets fixed and propagates (but your customers are happy) and if you are stringent about it, things eventually get fixed but the developers and customers will blame you until that happens.
To a certain extent the blame rests with authors of the standards, who should really have provided a way to check that code/data does in fact comply. Microsoft did a good job with the driver verifier so that the issue is addressed in the future, but I don’t think a verifier was even possible in the days of Windows/386 and 3.1.
I am having my own experience with all this at the moment. It turns out that with the exception of my own program, there isn’t a single program out there on Windows, Linux or Mac that I can find that actually correctly implements the vcard standard. In most cases it seems like the developers only checked importing their own vcards that they exported.
And the vcard standards non-compliance isn’t over deeply technical things. It is stuff like getting field names wrong, or not decoded quoted printable fields.
"Microsoft shouldn’t have made those buggy drivers work. They should have just let the computer crash in order to put pressure on the authors of those drivers to fix their bugs."
What about slapping said developers? There’s nothing against Bill and Microsoft laying an open fisted slap right across the face of those that use undocumented functions and bitch about it when they’re broken in the next release. If it’s undocumented, it’s bound to break eventually so why rely on it?
If I was developing Windows I would have let them get pissed I think. Sure I would have lost Application X but only for 10 minutes. They would have figured out that it’s way too much work to redo everything for Apple and Linux and stuck with making a couple of changes to their existing code base.
It makes me also wonder if any vulnerabilities were caused from this very mindset. I’m sure it’s possible that while fixing Windows to make driver X work, you opened a hole in the process. I would love to see the ratios to see if vulnerabilities were caused by new code or these types of "patches". If more vulnerabilities came out of patches versus new code then I would stop doing those kinds of patches. Application X may suffer a little bit but I’d rather have a more secure OS than let Application X open a hole in my system. Application X wouldn’t be blamed for the hole because Microsoft is the one who put it there in the first place.
I like hearing about how you have to cater to lazy developers even if it makes me want to slap the living crap out of them. I’m glad that virtually everything you talk about you give those little caveats. It’ll frustrate me at some point since I’m so anal about standards and practices but I want to hear about stuff like this. I think it should be widely known so we can ridicule those developers. Microsoft should have PR releases every time it makes some of these changes and give the exact reason why MS had to do it. I want to know when Application X isn’t following your specifications so that I can stay clear from their product. I’m not about to shell out money to developer’s who can’t program correctly in the first place. The application may work now but if it breaks in a service pack release I don’t want it.
Why couldn’t the buggy behavior be traced back to the faulty driver and reported to the user? Just a paragraph earlier you said the work around detected the behavior and worked around it, surely it could have warned you that your machine was being crippled performancewise.
User confusion I bet you’d say. *sigh*
Aren’t there Microsoft Certification things for device drivers to somehow keep this from happening?
I don’t understand Microsoft’s obsession with backwards bug compatibility. It just rewards lazy programmers and hurts everyone else.
I am glad I only use Linux where usally the "right thing" is done even if it causes some pain. It makes things so much easier.
And before you go off on backwards compatibility, I have multiple 8+ year old Linux programs that work perfectly on modern distributions. Linux is excellent with backwards compatibility as long as you stick to *published* interfaces.
And even Microsoft compatibility isn’t that great. There are DOS games I have that play much better under dosbox on Linux than they do on win98.
Soem of the issues raised by this (and Raymond’s entire series of 3GB posts) remind me of Apple’s old distinction between 24 and 32 bit addressing.
If you don’t remember (or never knew), the Apple Macintosh started out using the Motorola 68000 CPU, a chip designed for 32-bit wide flat addresses, but implemented with a 24-bit wide address bus. Apparantly, the way this was done is that when a pointer was dereferenced, the upper eight bits (of 32) got discarded. This made it pretty easy for developers to stick flag bits (for Lisp-like tagged pointers, etc.) into the discarded upper eight bits.
Now, fast-forward three years and Apple switches to the 68020 Microprocessor, which fully realizes the 32-bit potential of the 68K architecture. Now, rather than discarding the upper eight bits of addresses, the chip actually drives them out onto the address bus… even if they’re tag bits. Oops.
The way Apple handles this was actually pretty novel. On the Mac II motherboard was a socket for an optional 68851 PMMU, a device that handles memory paging of the sort that this series of articles talks about (and was used to support Apple’s A/UX Unix and Virtual Memory under Systems >=7). If the PMMU was not installed, Apple shipped AN alternative chip, the AMU. The AMU was mainly able to optionally mask off the upper eight bits of CPU addresses to allow software written (poorly) for the older 68000 to run unmodified. For a long time, this older software included the OS ROM’s on the system motherboard. It wasn’t until the early 90’s (with the 68040 based Centris 660AV, IIRC) that Apple finally discarded the ability to put their hardware in 24-bit compatibility mode.
They did, however, eventually stop supporting the brokenness inherent in these old 24-bit only applications. I realize that Apple isn’t exactly the model of software industry success, but IMO, this is perhaps the way to do it. Handle the broken software for a couple years, make it clear that this support will be going away soon to allow companies to transition, and then drop support in a well documented manner. That seems far preferable to me than dragging support for all the old broken cruft the industry can produce through the ages into successive versions of an OS.
(Alternatively, and like so many others on this blog have proposed, now that VM technology is more mature, as long as old software can run in a VM on the older OS, and stay connected to its environment, who really cares…)
x86 computers had a similar thing when they moved to the 286 where they put an AND gate on the 20th address line.
Also, logging these things somewhere for developers to realise that their program is broken, and to inform users (that understand enough to read the logs) that the application they are running is making bad assumptions and get an update. Normal users aren’t irritated, but developers get the message and the problem gets fixed quickly.
For instance, the 2.6 kernel under Linux reports something like "tcpdump uses obsolete (PF_INET,SOCK_PACKET)". It is still supported by the kernel via a backwards compatibility layer, but everyone (other than users who don’t care) is informed about the problem, and in the next major release it can be removed.
"This is why you don’t let idealists/purists run massive engineering enterprises, IMO."
On the contrary, eschewing backward compatibility hacks is the pragmatic approach. In the long term, back-hacks build upon each other to create intractable problems.
I *have* worked on software that sells millions of copies. And I’ve been bit by a wildly popular printer with a buggy driver come out just after we shipped a few million copies. Yes, we provided a shim for customers with this printer, but we did not compromise performance and functionality for everyone else (which is what many of Raymond’s stories say Windows did).
Now, in this case, Windows only supported the old behavior for users with known-bad drivers, but it puts the entire system into the backward compatibility mode, leading to further conflicts. Nothing else on such a machine gets the benefit of the enhancement. (I assume there was value to tha change, otherwise they would just undo the change rather than support two modes of operation.)
Consider now, a company that tests the heck out of their new (unrelated) drivers on a machines that are not in this backward compatibility mode. They probably don’t even know the second mode exists. What if their driver breaks in the other mode? If they do learn of the other mode, they have to double their testing. Next thing you know, there are four modes, then eight, and so one. Eventually, the problem becomes intractable, all because you went too far trying to support a buggy driver that nobody has today.
Was it worth it? Does Microsoft have a good reputation for maintaining backward compatibility? Maybe the stats are good, but the perception isn’t. DLL Hell, drivers that break anyway. Heck, I can’t even successfully install the .NET framework on my vanilla Windows 98 box (fails without explanation).
Back to the problem at hand. What should Microsoft do on machines that have some (buggy) drivers that only work in backward compatibility mode and some (also buggy) others that only work in normal mode? You’ll eventually end up with contradictions like this.
This isn’t idealism, is pragmatism.
Perry,
One very big difference: On the 286 machines, the A20 line was disabled by default.
And when A20 support was added, you had to explicitly enable the A20 line, do your thing, and disable the A20 line. That was to prevent the myriad known applications from breaking. There was a device driver known as himem.sys that was added to arbitrate this process.
If you followed the guidelines, the only things that broke were T&SR’s that depended on address wrap. There were some (I had to work around some of them in the DOS LM redirector when I added himem support to it) but very few.
Carmen Crincoli wrote: "This is why you don’t let idealists/purists run massive engineering enterprises, IMO."
On the contrary, eschewing backward compatibility hacks is the pragmatic approach. In the long term, back-hacks build upon each other to create intractable problems.
I *have* worked on software that sells millions of copies. And I’ve been bit by a wildly popular printer with a buggy driver come out just after we shipped a few million copies. Yes, we provided a shim for customers with this printer, but we did not compromise performance and functionality for everyone else (which is what many of Raymond’s stories say Windows did).
Now, in this case, Windows only supported the old behavior for users with known-bad drivers, but it puts the entire system into the backward compatibility mode, leading to further conflicts. Nothing else on such a machine gets the benefit of the enhancement. (I assume there was value to tha change, otherwise they would just undo the change rather than support two modes of operation.)
Consider now, a company that tests the heck out of their new (unrelated) drivers on a machines that are not in this backward compatibility mode. They probably don’t even know the second mode exists. What if their driver breaks in the other mode? If they do learn of the other mode, they have to double their testing. Next thing you know, there are four modes, then eight, and so one. Eventually, the problem becomes intractable, all because you went too far trying to support a buggy driver that nobody has today.
Was it worth it? Does Microsoft have a good reputation for maintaining backward compatibility? Maybe the stats are good, but the perception isn’t. DLL Hell, drivers that break anyway. Heck, I can’t even successfully install the .NET framework on my vanilla Windows 98 box (fails without explanation).
Back to the problem at hand. What should Microsoft do on machines that have some (buggy) drivers that only work in backward compatibility mode and some (also buggy) others that only work in normal mode? You’ll eventually end up with contradictions like this.
This isn’t idealism, is pragmatism.
I think one of the reasons people think that it’s better to just break stuff and "slap" developers into making better code is that everybody assumes that the bad developers are always other people.
Just like almost everybody thinks that they are an above average driver, almost every developer thinks that when we’re talking about non-compliant and dubious behavior we must be talking about somebody else.
Trust me, before I joined MS, I had a pretty similar attitude. Because obviously my code was great, and it was only those *other* developers that made dumb mistakes or took stupid shortcuts.
Then I came across one of Raymond’s internal rants about bad applications he had to fix for Windows 95 compat. And guess what, on the list were apps that I had worked on. Now, it turned out that the particular piece of offending code wasn’t mine, but it easily could have been – it wasn’t like my team was full of bad developers, it was mostly full of developers just like me, developers who I considered to be pretty good at what they did.
One more thing… how would MS expect ppl NOT to use undocumented features, if in fact MS supports applications that uses them. If I am a developer and I know "all of the above" – that MS will keep makes things compatible, I definetly won’t hesitate to use them – if these things MY life easier, why should I bother – let’s make MS life harder :)
I see a couple of recurring themes in the complaints about MS going the extra mile to support bad applications.
i/ assumption is that it is lazy developers, rather than stressed developers in under-resourced departments being pushed by non-technical managers that don’t care about the details, causing the problems.
ii/ Assumption is that the company that wrote the code is still around to fix it. With economic fluctuations over the last decade or so, that is a mighty big assumption. While you might not depend on decade-old software, other do, sometimes for ‘mission critical’ tasks. Should MS discard these customers, just to make our lives as developers easier?
I would really like to see some sort of ‘forced deprecation’ process, where misbahaving programs aer shamed in one release, and forced to run in some kind of compatibility sandbox in the next (much like DOS apps today) to keep hacked-support out of the OS mainline. I am also free to dream, as I don’t actually have to implement/support this stuff ;¬ )
Just remember the press MS gets on this stuff today. Imagine how much worse it would be if they actualy DID allow large numbers of installations to break!
Wasn’t there something weird in Windows 3.1 that meant it was faster to have a swap file in a RAM drive than no swap file at all?
(Or maybe it was Windows 3.0.)
Re: ASDF – as far as I know the Windows API has never changed incompatibly (ie ABI breaks).
Basically I think people would have fewer issues with these sorts of hacks if in userland the APIs were versioned properly. Windows API policy has always been versioning on the function call/flag level, so you have CreateWindowEx but never a new version of the actual windowing infrastructure.
Obviously maintaining backwards compatibility as long as possible is crucial for performance: if you have 1 old app using USER32 v1 and the rest use v2, because some backwards compat hacks that were removed in v2, that’s still double the memory usage that could have been avoided if you’d just kept USER32 compatible.
Now, that policy must have made a great deal of sense back in the day (like the non-blinking clock) but is it still true today?
I don’t know. All I know is that reading Raymonds blog and the comments posted herein have changed my mind about OS backwards compatibility dramatically.
"I would really like to see some sort of ‘forced deprecation’ process, where misbahaving programs aer shamed in one release, and forced to run in some kind of compatibility sandbox in the next (much like DOS apps today) to keep hacked-support out of the OS mainline."
"Re: ASDF – as far as I know the Windows API has never changed incompatibly (ie ABI breaks). "
Historically, Microsoft has actually done this (sort of) before. Windows 3.0’s big innovation was the ability to run Windows applications (those dependant on USER/GDI/KERNEL) in Protected Mode on >=286 chips. Prior to 3.0, Windows applications always ran in real mode, and could do real mode things with segment/offset arithmetic, etc. Of course, for applications that took advantage of the leniency of x86 Real mode, the switch to Protected mode would cause serious problems. As a result, applications had to be tagged with a bit that marked them as safe to run in Windows 3.0 Protected Mode. If you attempted to launch an app that wasn’t so tagged, the OS would pop up a dialog box explaining the situation. If this was an application you just had to have, Windows 3.0 could be restarted explicitly in real mode, with all of the costs that implies. More intrepid users could actually use a utility/hex-editor to set the protected mode bit in the .EXE and attempt to run their application in protected mode anyway. A year or two later, Windows 3.1 discontinued real mode operation, so you either had to stay with 3.0, tag your old executables, or find replacements/upgrades.
"Wasn’t there something weird in Windows 3.1 that meant it was faster to have a swap file in a RAM drive than no swap file at all? "
I don’t know about that, but for a long time it was better to run Windows in Standard mode versus 386 Enhanced Mode. 386 Enhanced mode added to standard mode a V86 DOS multitasker that could run multiple DOS apps on the Windows desktop, preemptively multitasked, and virtual memory. It was also slower. IIRC, the first really compelling use of 32 bit code that benefited Windows (USER/GDI/KERNEL) apps was 32-Bit File Access in Windows for Workgroups 3.11, a reimplementation of the FAT file system that ran in the 32-bit layer of the OS. 32BFA offered significant performance benefits. (It also was the foundation for the Long File Name file system in Windows 95, just as the Win32s infrastructure laid the foundation for Windows 95’s 32-bit API’s).
"Has anyone considered emulation as a solution to these problems? Not a full-blown x86 emulator, just a layer between the OS and the hardware like VMWare. "
Windows XP introduced a compatibility layer:
http://support.microsoft.com/default.aspx?scid=kb;en-us;286705&sd=tech
"Has anyone considered emulation as a solution to these problems? Not a full-blown x86 emulator, just a layer between the OS and the hardware like VMWare. "
Windows XP introduced a compatibility layer:
http://support.microsoft.com/default.aspx?scid=kb;en-us;286705&sd=tech
I think backward compatibility should have a limit… I think memory manager wasn’t rewritten just for fun, so make it works as before doesn’t make much sense. *If* windows can detect such old drivers, it should simply refuse to load them and display proper message… Otherwise, it is same as check whether user has 16bit applications and said "OK, lets’ work in 16bit mode, since user has old applications"..
What’s the point to improve things, if they still keep working as before?
While I can understand (but do not agree with) the reasoning that leads to the massive accumulation of compatibility hacks that is Windows, a question remains.
If it is true that in the history of Windows, the behaviour of officially published and documented APIs was changed (I don’t know, it’s just the impression I get from various posts on this forum), all the efforts that Raymond describes to keep buggy programs working seems hypocritical.
Maybe programmers can trust on Windows bugs they rely on to be emulated in future OS versions *if* their applications are popular enough to appear on the Microsoft compatibility radar. If your application is unimportant, however, Microsoft can change the *public API* so that your properly written application stops working.
As I said, I don’t know if this is true. If it is, I would like to hear the reasons for a change of the documented Windows API. Security? (Honestly, I wouldn’t believe it.) Bugs in the previous implementation? (Nah, Raymond describes how these are typically emulated.) Preserving compatibility with other, very popular applications?
"x86 computers had a similar thing when they moved to the 286 where they put an AND gate on the 20th address line. "
Oh yeah. I had forgotten, but now I remember. *shudder*
At the time, I had an old ALR Powerflex machine that drove the A20 gate with its (marginal) keyboard controller. The net result of this was that any memory manager that rapidly toggled the A20 gate to access the UMA would screw up the keyboard. Apparantly, the ALR’s poor keyboard controller couldn’t keep up.
I guess one of the issues people have with compat work is that they apply not only to the old applications they were meant for, but to newly written ones as well. There needs to be a strategy so that applications compiled after (say) January 1 2005, or linked with –don’t-use-pre-Windows-XP-compatibility-hacks get an OS that behaves as if it were rewritten from scratch today. (I know that *actually* rewriting Windows from scratch would be a very bad idea.)
Has anyone considered emulation as a solution to these problems? Not a full-blown x86 emulator, just a layer between the OS and the hardware like VMWare.
That way, we could run old programs on their native OSs with all their bugs. In the new OS, these bugs would be fixed (not emulated).
Maybe the performance hit is too large for applications from the last OS generation. So maybe Longhorn could only emulate software written for up to Windows 98.
The benefits are still apparent. All the compatibility codepaths for these old OSs would be rigorously cut out. In software development in general (at Microsoft in particular, it seems =)), fewer codepaths ==> fewer security related bugs. Not just because compat hacks are potentially buggy, but because a smaller kernel means more security reviews per line of code.
Windows Superdick (release date 2012) would then emulate Win 2000 software and so on. In 2020, we will get Windows/2 Warp, which is virtually hack-free.
Users will install old versions of applications on new operating systems. They won’t necessarily download patches to make the applications work. The vendor may not decide to produce and support such patches (especially if the vendor has gone bankrupt…) The user expects the same application, installed directly from the original media, to continue working. Corporate users will have in-house applications developed by third parties, and won’t want to spend anything on getting those applications updated if they’re still performing their tasks. So we as platform developers have to, as far as possible, be binary compatible.
Once you’ve developed a binary compatibility layer, the safest thing to do is to leave it there for all time. Why? Because any change to software is risky. Modifying a module to remove backward compatibility has the possibility of introducing new bugs, which could break existing software that doesn’t use the obsolete APIs. Modifying the module for no other purpose still means performing the whole cycle of regression testing. Larry Osterman wrote an article on Friday – http://blogs.msdn.com/larryosterman/archive/2004/08/13/214338.aspx – about how a piece of code from the Windows 3.1 beta ended up in RTM even though it wasn’t actually used.
Since we’re on a virtual memory topic, I’ll remind you that Windows is demand paged – it only loads code into memory that’s referenced. If no code calls the obsolete API, it won’t be loaded, won’t affect the process’s working set and won’t use – even temporarily – any physical memory. The bits take a little disk space, that’s all. Will the user prefer to pay $0.01 for the disk space, or an extra $10 per copy for the additional testing?
"If no code calls the obsolete API, it won’t be loaded, won’t affect the process’s working set and won’t use – even temporarily – any physical memory. The bits take a little disk space, that’s all. Will the user prefer to pay $0.01 for the disk space, or an extra $10 per copy for the additional testing? "
I agree with what you’re saying, but doesn’t that sentence ignore testing costs for the compatibility layer? Presumably, it costs Microsoft some amount of money to keep things working and tested.
As Evan already mentioned on his blog, Raymond Chen has a great series on /3GB switch on his blog. What is really cool is that Raymond takes on some myths about the /3GB switch and the fact that he…