| Date: | August 22, 2004 / year-entry #314 |
| Tags: | other |
| Orig Link: | https://blogs.msdn.microsoft.com/oldnewthing/20040822-00/?p=38093 |
| Comments: | 36 |
| Summary: | A table of contents now that the whole thing is over. I hope. The oft-misunderstood /3GB switch. It's simple to explain what it does, but people often misunderstand. Kernel address space consequences of the /3GB switch. An adverse consequence of the /3GB switch. Myth: Without /3GB the total amount of memory that can be allocated... |
A table of contents now that the whole thing is over. I hope.
I'm not sure how successful this series has been, though, for it appears that even people who have read the articles continue to confuse virtual address space with physical address space. (Or maybe this person is merely mocking a faulty argument? I can't tell for sure.) |
Comments (36)
Comments are closed. |
The confusion on some comments just highlights what we all know: memory management is hard to grasp and is easier to use when handled by the OS or the language.
I think that the wording now reflects what I was thinking. I didn’t think that I was confused but now I’m not so sure. Help?
For what it’s worth, I really liked this series – I always enjoy your blog, and it was particularly good to have such thorough and deep coverage of a topic.
If some people have misunderstood certain fundamental points like the distinction between virtual and physical address space, then I don’t think that’s necessarily indicative of a problem with the series. I would say simply that it indicates that those readers weren’t really ready to read the articles.
I suppose you could have assumed a different level of knowledge from your readers, but that would simply have made the articles useful to a different set of readers. It wouldn’t necessarily have made them better as such.
I have to say I’ve enjoyed these articles too. To top the series off, do you have any tidbits about the x86-64 Windows that you can share, especially memory related?
Like somebody else commented in one of the articles – it’s a revolution waiting to happen. Once x86-64 Windows is out, there’s no going back.
Myth:Virtual memory is system memory that is simulated by the hard drive.
This definition comes from googling "what is virtual memory":
http://www.google.com/search?hl=en&ie=UTF-8&oe=UTF-8&q=what+is+virtual+memory
Surely developers aren’t this clueless?
When interviewing I used to ask some questions about virtual memory. I never got good answers – the interviewees would have no idea or they would give me something similar to the definition above. Virtual address space was a mystery.
I’m not even slightly surprised that people have trouble with /3GB, PAE, etc.
The articles are excellent in themselves, but I think that the basic misunderstandings fuel the myths, so an introductory article would have helped.
Well, what the hardware does with virtual memory isn’t the entirety of the meaning of virtual memory. Sure all the rest of the baggage associated with virtual memory has part of the CPU’s hardware as its foundation, but the baggage is what users usally have to deal with. Plus in Windows the settings for page file sizes often say virtual memory instead of page file. When the abstraction doesn’t leak, no one has to remember that it’s founded on address translation units and stuff like that.
Meanwhile, virtual memory isn’t alone in confusing people. Recently I had a run-in with someone who didn’t understand the possible benefit of increasing the physical memory in his notebook from 64MB to 128MB. He designs electronic circuits, and he uses computers (using both hardware and software), but I guess the design and meaning of computer hardware just never entered into his experience.
I’m not surprise that your average computer user has trouble with the virtual memory / physical memory / address space distinction. But your average computer user also doesn’t go spouting off about the /3GB switch.
This is a very difficult subject for most people to understand, even programmers. I mean, this:
>>Virtual memory is system memory that is simulated by the hard drive.
This isn’t a bad explanation for a non-programmer. I assume Carlos is taking the position that this is wrong, but — conceptually — it works for laymen.
If virtual memory is simulated by the hard drive, what are virtual disks simulated by?
Hmmm…. :)
I sincerely thank Raymond for covering this topic. I don’t think I was alone in the confusion. My main incorrect notion was about the distiction between Virtual Memory and Virtual Address Space. (I had no idea there even was a distiction.)
However, it is really unfortunate that on msdn, the functions that were mentioned here (CreateFileMapping,(Un)MapViewOfFile) aren’t mentioned anywhere in the "memory management" section:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/memory/base/virtualalloc.asp
That the "virutal memory" functions listed there actually handle both the allocation AND the mapping is surely the cause of much of this confusion (it was for me).
It seems to me the confusion is per machine vs per process. Raymond is talking per process and people are thinking per machine and interpreting it from that perspective.
Larry O was talking about VM from the machine perspective (page file size + physical memory + (I presume) code segments in exe and dlls) which is called commit charge.
http://weblogs.asp.net/larryosterman/archive/2004/03/18/92010.aspx (and look at surrounding days).
With raymond, he is talking about things that are normally done for system configuration (editing boot.ini) but is talking about the effect on a process not the box.
The person you linked too is also thinking you are talking about system configuration (eg setting the page file size).
Although in 3.1 enhanced mode it was said that a swap file made the system run quicker, even if it wasn’t used as it wqas designed to swap and was slower for ????????????? reason if it couldn’t.
Still changing LRU sweep frequency et al never made any detectable difference.
Thank you for the jump table.
I’ve been quite interested in this stuff because I rarely encounter these kinds of problems in my day-to-day programming and so I’ve never encounted this 3GB switch nor had to worry about distinctions between memory types.
Of course, as a programmer I can’t help but know what virtual address space is. However, I don’t recall anyone ever saying specifically what virtual *memory* is, as a concrete definition. I have some hazy ideas in my head about it, but I don’t feel confident that I could actually give an accurate definition in English. Can anyone offer one?
David,
In general, having a paging file makes systems run faster – that’s a corrolary to my post (btw, I wish you’d also included a reference to the correction: http://weblogs.asp.net/larryosterman/archive/2004/05/05/126532.aspx – while my post was roughly correct, there were certain critical details I missed).
Raymond,
Is this series a personal rant? (An early entry mentioned something about having to live with it for six years.)
I am puzzled as to how these posts relates to your other programming posts in your blog. Well there were those few discussion about lock free approaches.
Hmm. Maybe you are just leading up to details of 64-bit pointers issues and other memory management issues and design issues with regard to 64-bit hardware and drivers.
Haven’t read the each entry in the series yet. Been catching up. Good to see a little coverage of these poorly understood things.
Will the series continue to cover other aspects of memory management such as the secured memory concepts of .Net and Non-executable memory regions.
See:
http://www.sonic.net/~ezzell/Error%20Messages/Its_Virtual_Memory.htm
At work, there’s been some debate on how best to structure our J2EE servers to maximise utilisation of a scarce resource (per-CPU licenses!). The crux of the debate centered, for some reason, on how we can allocate more JVM instances…
Back when Windows Server 2003 SP1 shipped, HTTP.sys hit an important milestone, WOW64 support.What is…
PingBack from http://forums.macnn.com/104/alternative-operating-systems/322800/windows-not-recognizing-total-ram-upgrade/#post3264015
PingBack from http://poundbang.in/2007/03/29/interesting-links-from-delicious-9/
PingBack from http://www.ditii.com/blog/2007/04/14/memory-management-demystifying-3gb/
32bit Windows is Not Limited to 2GB/4GB
PingBack from http://www.mcseboard.de/windows-forum-ms-backoffice-31/citrix-8-gb-32-bit-betreiben-121295.html#post748073
PingBack from http://tphinney.wordpress.com/2008/12/19/no-font-install/
PingBack from http://www.thomasphinney.com/2008/12/19/no-font-instal/
I started programming in the old times of 16-bit MS-DOS, where you had only 640 KB of memory, and you
I started programming on x86 machines during a period of large and rapid change in the memory management