| Date: | December 22, 2008 / year-entry #420 |
| Tags: | other |
| Orig Link: | https://blogs.msdn.microsoft.com/oldnewthing/20081222-00/?p=19763 |
| Comments: | 43 |
| Summary: | On 64-bit Windows, 32-bit programs run in an emulation layer. This emulation layer simulates the x86 architecture, virtualizing the CPU, the file system, the registry, the environment variables, the system information functions, all that stuff. If a 32-bit program tries to look at the system, it will see a 32-bit system. For example, if the... |
On 64-bit Windows, 32-bit programs run in an emulation layer. This emulation layer simulates the x86 architecture, virtualizing the CPU, the file system, the registry, the environment variables, the system information functions, all that stuff. If a 32-bit program tries to look at the system, it will see a 32-bit system. For example, if the program calls the And that's the point of the emulation: To keep the 32-bit program happy by simulating a 32-bit execution environment. is writing an installer in the form of a 32-bit program that detects that it's running on a 64-bit system and wants to copy files (and presumably set registry entries and do other installery things) into the 64-bit directories, but the emulation layer redirects the operations into the 32-bit locations. The question is "What is the way of finding the x64 Program Files directory from a 32-bit application?" The answer is "It is better to work with the system than against it." If you're a 32-bit program, then you're going to be fighting against the emulator each time you try to interact with the outside world. Instead, just recompile your installer as a 64-bit program. Have the 32-bit installer detect that it's running on a 64-bit system and launch the 64-bit installer instead. The 64-bit installer will not run in the 32-bit emulation layer, so when it tries to copy a file or update a registry key, it will see the real 64-bit file system and the real 64-bit registry. |
Comments (43)
Comments are closed. |
From reading Koro’s question, it sounds to me like he wasn’t TRYING to fight the emulator — he just didn’t realize that he needed to recompile his installer as a 64-bit program.
Let’s hope that your answer pointed him in the right direction.
Since you bring it up, consider this the obligatory tired old joke about System32 versus SysWoW64.
Nitpickers Corner- I know you didn’t code it. I know it makes perfect sense from a certain point of view. I’ll even admit this might be the best way of doing it. I’m not asking it to be changed. It’s still amusing to me.
Mark Russinovich talked a little about this a few years ago; he eventually went with disabling the file system redirection, because he didn’t like the added complexity of having a separate installer for 64-bit. I guess if you are installing a small, simple program, it makes more sense to do that, whereas a dedicated 64-bit installer may make more sense for larger and more complicated installs.
http://blogs.technet.com/markrussinovich/archive/2005/05/07/running-everyday-on-64-bit-windows.aspx
I still do not understand why system32 is redirected to SysWOW64 for x86 apps while really contains x64 binaries. Why you didn’t simple rename system32 to system64 like you did in win16->win32 times? Old bad 32bit apps would be happy cause all files in their hardcoded %windir%System32, new 64 bit apps should be written so they know already that system directory is System64 for them. And there no need for that hacky FS redirection at all.
@Killer: In a perfect world, this would have been the solution. However, common sense isn’t, especially when it comes to application development.
The requirement for a separate 64-bit installer is a real pain – especially for managed code for which there literally is no difference between the 32-bit and 64-bit binaries. Hopefully "the system" will improve at some point, as this is become an increasingly common scenario.
[quote]If a 32-bit program tries to look at the system, it will see a 32-bit system. For example, if the program calls the GetSystemInfo function to see what processor is running, it will be told that it’s running on a 32-bit processor[/quote]
[quote]Have the 32-bit installer detect that it’s running on a 64-bit system and launch the 64-bit installer instead.[/quote]
Except… the 32-bit installer doesn’t have a way to detect that it’s running on a 64-bit system, because that’s the point of the emulation layer? What’s the correct way for a 32-bit app on windows to realise that it’s running a on 64-bit system?
Gary (-;
Chunky, search engines are your friend…
I searched (you could even say I googled, but I’ll never tell ;-) "How to detect if you’re running on 64bit windows from a 32bit application" and the first result was http://www.berezniker.com/content/pages/visual-foxpro/how-detect-64-bit-os which gives exactly what’s needed to determine if a 32bit app is running on 64bit windows..
Unfortunately, 32-bit program’s IDT and GDT bases in WoW64 are at addresses higher that 4GB so that instructions SIDT and SGDT are broken – they return only the low 32 bits of the base, which is useless.
You can use this to determine if a process is wow64:
http://msdn.microsoft.com/en-us/library/ms684139(VS.85).aspx
@Chunky
This article should help.
http://msdn.microsoft.com/en-us/library/ms684139(VS.85).aspx
A 32-bit app can check if the IsWow64Process function is exported by kernel32.dll
http://msdn.microsoft.com/en-us/library/ms684139(VS.85).aspx
Thanks,
Gabriel Topala
http://www.gtopala.com
Gabriel,
You can’t just check to see if the IsWow64Process is there, you have to actually call it.
Old, 32 bit Windows- IsWow64Process doesn’t exists
New, 32 bit- IsWow64Process does exists, will return 0.
New, 64 bit- IsWow64Process does exists, will return 1.
I think there is a MSDN Article detailing this for you, wonder if someone could link it?
Why go through the trouble of writing an installation program when everyone should be using Windows Installer and MSI files?
Just use WiX. It is free and easy.
I wonder if the System32/SysWOW64 redirection could have been avoided if Windows had locked down the system folders – like Vista did – before 64-bit support was rolled out.
Something I’ve always wondered about XP and Vista x64: does running 64-bit and 32-bit processes at the same time incur a more pronounced performance penalty? That is, would both types of apps running incur both the cost of a context switch plus some overhead for adjusting registers/memory allocation differences/whatever?
What kind of performance penalty, if any, is incurred when running a 32-bit program on a 64-bit OS, such an WinXP 32-bit? If I buy a new PC and OS, and money’s no object, and gaming’s my intent, which should I go for – a 32-bit or a 64-bit OS?
If money is no object, then you’ll likely be getting 8GB of ram (or higher if your mobo supports it) so you’ll want to get Vista 64-bit ;)
for the guy with the original question; NSIS has a header file called x64.nsh with helper functions to detect and disable 64bit and redirection, the registry commands also support 64bit
"In a perfect world, this would have been the solution. However, common sense isn’t, especially when it comes to application development."
Maybe it’s finally time for MS to stop catering to bad ignorant developers that skip on reading the documentation? I suppose that’s what big A did.
Dorfl – some games (I’m thinking of Crysis here) have started to include 64-bit binaries to take advantage of some faster floating point calculations. Also, on 32-bit, you’re stuck with 4.0-x gigs of RAM (where x is probably between 0.5 and 1.0 for a gaming machine), which basically means you get 2x1GB RAM, which means a 32-bit process using all of its virtual memory can’t all fit into RAM at once. Whereas on 64-bit (I have 8 gigs of RAM), not only is your process all in RAM at once, but you get a hefty amount left over for J.Random apps you might have open (IRC, voice comms etc.) plus, more significantly, disk cache.
8 gigs is probably overkill for that (I also use a lot of VMs) but 4 gigs is definately useful.
The only downside of 64-bit, in my opinion, is slightly worse driver support. This is nowhere near as bad as it used to be, and it’ll be usb consumer printers and the like that you might have problems with (getting drivers for my USB bluetooth dongle was waaay harder than it should have been). As long as you check your hardware and peripherals for support beforehand, you’re fine
Some MFs always want to ice skate uphill…
PingBack from http://brandonlive.com/2008/12/22/why-does-windows-put-64-bit-binaries-in-system32/
Alexander said:
"Maybe it’s finally time for MS to stop catering to bad ignorant developers that skip on reading the documentation? I suppose that’s what big A did."
You’re making some big assumptions there that I don’t think are correct. For more, check the link above to my post on that subject.
Please don’t tell people to chainload installers! It makes silent and scripted installs hard to make work if you it can be made to work at all. The are many of us out here in real world land trying to implement this stuff and telling developers to hack around the stupid redirection routines just makes our lives harder.
@Killer{R}:
When compiling old 32 bits projects to 64 bits Windows, programmers will fix the visible bugs, but, will probably overlook changing "system32" to "system64", especially when refering to rarely-installed DLLs, since it will seem to work… until somebody installs a 32 bits applications using the same DLL.
Because often it’s a p.i.t.a. (for example it restarts the installer if you overwrite a DLL making a patching process more difficult, etc.).
Of course other free alternative exists so I agree that either WI or one of the others (for example NSIS) should fit any needs.
"in a world with a 32-bit sky and 32-bit birds in the 32-bit trees."
One of the funnier phrases I’ve heard this year, thanks Raymond
@Andrew Fidel
I hear you man. However, properly supporting command line arguements (And properly documenting them! And making the documentation availible and easy to find!) goes a long way towards helping this issue.
Any program that wants to sell to Corporate Customers should, at very least, support silent installs with the option to turn off all the "Corporation Repelling Additional Programs". EULA’s, Product Activation, Internet Surveys, etc.
Well, apparently the emulator failed to emulate GlobalMemoryStatus, which contrary to the documentation will not clamp its result to 2GB. That causes some… interesting behavior on x64 machines with 4 (or 8, or…) GB of RAM.
Apparently five years and a few billion dollars can’t afford MS a backcompat test machine with 4 GB of RAM. I mean, the amount of stuff this breaks is staggering. Any amount of testing would have caught it.
For really old stuff not all hope is lost though: Windows 98/95 compatibility mode does clamp it to 256 megs. Nice way of saying "we almost got it right! Almost…"
BTW, the real system32 is available as %windir%Sysnative under emulation. But shh, don’t tell anybody. Consider this a retaliation for not showing the Compatibility Files button under x64 Explorer.
SuperKoko:
This is the lamest answer. Ever.
The simple fact is someone was an idiot and I mean a world class idiot when they made the system32 being 64bit and the SysWow64 being 32bit call. Linux does it right
/lib – 64bit libs if thats your think
/usr/lib – ditto
/usr/local/lib – ditto
/lib32 – 32bit libs (ia32-libs installed)
/usr/lib32 – ditto
/usr/local/lib32 – ditto
etc, etc, etc
It works for me and every other Linux user on the planet and I have no trouble with applications getting confused because ld takes care of all of the magic for me. If your running a 32bit app, the LD_LIBRARY_PATH is set to the correct locations automagically. If your hardcoding your app, your asking for trouble. Its not like Microsoft does not even have a solution for older broken apps if someone really needs to run a hardcoded Win32 app on a 64bit host. Its called a VM. They had lots of shots for a clean break with Win64 and they blew it here.
The simple fact is someone at Microsoft really blew it with this insane design and they should be canned. In fact they should publish the name of whoever it was that did it so they can never work in software development again.
In case anyone wants to play this off as standard Microsoft bashing, I’ll give credit where credit is due, the Visual Studio guys did an outstanding job to support the 32 to 64bit transition. Its whoever on the OS team that made this call that is a total idiot.
Isn’t it cleaner to just make the installer a fat binary containing both the 32 and 64 bit installers instead of chaining them ?
@Steven Edwards
I’m not sure I follow you. You say Linux works better because it works magically. Unless "If your hardcoding your app, your asking for trouble." then you go on to imply Microsoft sucks because they don’t even have a solution for older broken apps if someone really needs to run a hardcoded Win32 app on a 64bit host.
So linux is ok, but you are in trouble if you hardcode, and Microsoft sucks because you hardcoded?
chrismcb: I think he’s complaining because the directory for 64-bit binaries has "32" in the name, and the directory for 32-bit binaries has "64" in the name… after all, the only difference between the Windows solution and the Linux one is the name of the directories.
I don’t see how this is anything to get so worked up over!
Koko,
“will probably overlook changing “system32” to “system64″”
1. grep or “Find in files” VS function makes WONDERS. You won’t believe that, but it’s possibly to find all “system32” strings in the code.
2. Most general purpose software doesn’t have a business to install or read files in Windows and system32. Folks, you have your program directory already, and any LoadLibrary call will look there. Stop tossing your crap into Windows.
@Chris Charabaruk
@Killer: In a perfect world, this would have been the solution. However, common sense isn’t, especially when it comes to application development.
Its just a words. I’m intresting what binary backward compatibility problem was solved by this redirection.
And I think that simple grep’ping for hardcoded "system32" in source and changing it to predefined macro would be simpliest problem from all stuff that usually neccessary to port x86 app written by ignorant developers to x64 platform. And this small text replacement doesn’t worth that FS redirection hack. Futhermore, I think that System32-SysWOW64 redirection brought more chaos into cross platform development than solved problems.
If you really, really, really have to use a 32-bit app on x64 for this, there’s an AppCompat patch that can help you out here that I wrote a few months ago:
http://support.microsoft.com/kb/950407
@Steven Edwards
As others have pointed put, Linux is hardly any better than Windows on this front. Perhaps OS X gets it right? There aren’t special locations for the 64bit and 32bit apps. Everything is in /Applications, /bin, /sbin, …, but the binaries themselves can contain 32 and 64 bit code. Indeed they can contain x86 or PPC code of varying bit-iness. The loader selects the correct segments on executing the binary.
And as for registry – it also can be done by another way, instead of silent redirection one key to another. MS could just claim that x64 software should use something like HKLMSoftware64.
And all ‘shared’ registry keys could be implemented via old good but almost not documented registry key links. Did MS forget about feature that it was invented many years ago? –
HKEY k;
RegCreateKeyExW(HKEY_LOCAL_MACHINE, L"Software\Test", 0, 0, REG_OPTION_CREATE_LINK, KEY_ALL_ACCESS, 0, &k, 0);
std::wstring target = L"\REGISTRY\MACHINE\Software";
RegSetValueExW(k, L"SymbolicLinkValue", 0, REG_LINK, (LPBYTE)target.c_str(), target.size()*2 );
RegCloseKey(k);
So it was possible to keep simple plain FS and registry, without any problems with binary compatibility with legacy 32 bit code, and without additional complexity level that introduced by redirection.
Nitpicking on the commenters: Why do so many people who sound like native English speakers have trouble spelling "definitely"? Someone spelled it wrong in comments to the Seattle weather post, and someone else (or maybe the same person) spelled it wrong in these comments.
Bring back the teching of English grammar and spelling! :-)
DWalker59,
Check Muphry’s Law
chrismcb, others, et al…
Yes I am because the solution is the same on both. If you have an older broken app, you can VM it. This is not 1995 and its not like the transition to 64bit is forced like the Win 3.1 to Win9x switch. 32bit support is going to be around for a long long long time. If you have a broken app, run it on a 32bit system or VM it. You don’t need to introduce OS level convoluted hacks like the System32 being 64bit an SysWow64 being 32bit mess.
I don’t see how this is so hard to understand. The loader can figure this out. Take a PE32 app, the loader should be smart enough to know, hey I can’t call a PE+ 64bit dll and try to lookup functions. Ditto the reverse case. The loader knows, oh I loaded a 64bit executable, it would not be a good idea for me to try and load a normal 32bit dll.
I just cannot wrap my mind around how this is for ‘compatibility’ as the libs that are in system32 are now 64bit! If you’ve got a hardcoded 32bit app that tries to do a LoadLibrary then its going to get redirected using the Microsoft system. If they had just left
system32 – 32bit crap
system64 – 64bit crap
The only problem you would have would be some idiot writing a 64bit program and then hardcoding it to system32.
Raymond: My point which I thought I made clear was that you do not want to VM everything. Of course it has limitations. The point was that if you have a really broken application then a VM is an acceptable solution.
My point in vocalizing the system32 being 64bit and SysWow64 being 32bit nonsense, is that none of the arguments for it make sense. Fine, you think a VM is never an acceptable solution. I can buy that. As I stated, the logic on why 64bit apps have to have a folder called System32 and how 32bit apps are not supposed to get even more confused by finding a system32 folder contain 64bit binaries is lost I me. Instead the system does all of this extra magic behind your back in the name of ‘compatibility’ which does nothing but adds more complexity to the system.
Back to the original point about Program Files and your filesystem question, I think its a question that needs more information. Why does he need to interact with the 64bit Program Files directory from the 32bit application? Without knowing that I can’t see why given the even oddities of WOW64, he needs to do it. But I don’t think its unreasonable for him to ask if he has a valid reason. After all you guys give 32bit applications the ability to detect if they are running under Wow via IsWow64Process().