| Date: | April 19, 2005 / year-entry #98 |
| Tags: | history;when-people-ask-for-security-holes-as-features |
| Orig Link: | https://blogs.msdn.microsoft.com/oldnewthing/20050419-50/?p=35853 |
| Comments: | 40 |
| Summary: | By default, Explorer does not show files that have the FILE_ATTRIBUTE_HIDDEN flag, since somebody went out of their way to hide those files from view. You can, of course, ask that such files be shown anyway by going to Folder Options and selecting "Show hidden files and folders". This shows files and folders even if... |
By default, Explorer does not show files that have the You can, of course, ask that such files be shown anyway by going to Folder Options and selecting "Show hidden files and folders". This shows files and folders even if they are marked as On the other hand, files that are marked as both If you want to see those files, too, then you can uncheck "Hide protected operating system files". Let's look at how far this game of hide/show ping-pong has gone:
You'd think this would be the end of the hide/show arms race, but apparently some people want to add a sixth level and make something invisible to Explorer, overriding the five existing levels. At some point this back-and-forth has to stop, and for now, it has stopped at level five. Adding just a sixth level would create a security hole, because it would allow a file to hide from the user. As a matter of security, a sufficiently-privileged user must always have a way of seeing what is there or at least know that there is something there that can't be seen. Nothing can be undetectably invisible. If you add a sixth level that lets a file hide from level five, then there must be a level seven that reveals it. |
Comments (40)
Comments are closed. |
If a no program can read a file, does it really exist?
On this Win2000 box, those options appear under the "Advanced Settings" list box, which is driven by information in HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindowsCurrentVersionExplorerAdvancedFolder .
That should make it possible to remove the options entirely, which for a determined sysadmin, could be viewed as the "sixth level".
(In any event, this reminds me of caller ID and the block-all-withheld-number-calls features offered by telephone companies, a few years ago)
Programs that want to hide information already do so by utilizing a Alternate Data Stream function within NTFS. If that is considered to be the 6th level, there are no documented 7th level options to display these ADS (we can use 3rd party tools like from sysinternals though). Any reason, why explorer does not show the ADSs in the list or properties?
Although I usually argue the opposite side, on this one I have to side with the Unix folks and blame this on poor user/admin separation in the NT base. "root" on unix would obviously see all those files and be able to do whatever they wanted; normal user would not; and that works well in most cases.
If a sixth level, and corresponding 7th level, were to appear, I’m sure many users, including myself, would configure Explorer all the way to the 7th level without a thought, by default, just as we do with level 5 today.
Now there’s progress :)
Just in case I’m missing something, aren’t the Temporary Internet Files hidden successfully from Explorer? They don’t all show up, even with "Show hidden files and folders" on and "Hide protected operating system files" off. (They’re sort-of hidden from DOS; "dir" doesn’t show them all, but the TAB filename completion thing finds them.) I was sure you’d written something on this in the past, but the search engine fails to turn up anything for "temporary internet files", so perhaps I imagined it.
The major problem of "Explorer" settings is that they are "global". You can’t have the views in which the files with hidden attributes are shown and the views in which all the files are visible. And that’s exactly could be convenient. For example, when I search for documents (or programs) I’d like to be able to search through the disk but to ignore all the folders which are hidden (for example, skipping "Temporary Intrnet Files"). On the other hand, when I manage files (look what’s in some folder) I really want to see what’s really there.
When I would be able to run something like
"Explorer /showHidden /root,." (which would mean "open new explorer window of the directory I’m currently in, but with all files shown) (BTW, "." is not working now, I always have to specify the full path, anybody knows why?)
then I guess I would leave hidden files not shown by default. Good UI alternative would be if I would be able to press some toolbar button to show/hide "hidden" files.
I guess right now most people discover that the Explorer doesn’t show everything they’re interested in, they turn it on (it’s not so pleasant to find where to flip the feature) and leave like this.
NTFS streams are a bit like that. Nothing, barring ACL:s I guess, stops me from adding a parallel data stream to C:autoexec.bat, containing my supersecret data.
Explorer won’t see it, but streams.exe may be considered a seventh level in this case.
The Temporary Internet Files folder has magic applied to it via a shell namespace extension.
http://www.microsoft.com/mind/0298/cutting0298.asp
Notice that if you add "content.ie5" to your TIF folder path you can actually see the files. You can also get to the actual files through the command line as long as you use /ah or /as to view hidden/system files.
this is right up there with that INCREDIBLY stupid option to hide file extensions. 99% of those viruses in email that are now stripped out would not have been a problem if the hide file extensions of known types check box didn’t exist or at least it was defaulted to OFF.
I know you were trying to compete with mac back in the day, but really…. You’re not anymore, put back the file extensions!
What I would like instead of the simple shown/hidden possibilities would be a flag based on user rights.
So you could log in as administrator and see all the files on the system; but when logged in as a standard user, some files would be hidden.
A kind of "permissions" on the "show file" flag if you will…
I’ve encountered folders that were not visible even with both options set correctly. I finally got them to show up in a cmd prompt by using dir telling it to show all attributes. It seems to have something to do with desktop.ini settings. But that was quite a while ago and I could be mistaken.
Really couldn’t this just lead to Explorer Replacements. I know I have already gotten sick with this Hiding crap.
For Example the other day I go to burn a DVD with about 4 gigs of MP3s on it. I can’t figure out what in the heck is taking up all the space, it doesn’t seem like 4 gigs of music where the heck is it going. I know somethings wierd but I didn’t really want to sit there and figure it out I just wanted to throw the DVD in my DVD player which also plays MP3s that are burnd on disk.
Well it also shows pictures. My DVD from the manuafacturer specifically states and this is a great feature. Burn a dvd of images and it will treat them all as a photo show. Burn a DVD of MP3s I can set up play lists and so on or it will just start playing the music I can set randomly and so on.
Now however if I put in a dvd with both images and mp3s it won’t do anything but display the files structure if I want to play a song I have to go choose each song if I want to see a pic I have to go select it.
Well thanks Microsoft for hiding the fact that you download the album cover images from Media Player and put them in that songs directory. I never Unhid files in that directory because I created it, I put things there, I organized it. I wasn’t expecting when I just dragged the entire music folder to the DVD Burning software that there were images in there. Now I sat there waiting for the entire thing to burn, Verify the burn. Then put the dang thing in there expecting it to work and it doesn’t because you hid files on me. Unhide them and look. Yep there are all these JPG files.
Oh and then it gets better. So I write a little script, fine I will just go delete all files in that directory *.jpg, reburn dvd again. ARRRRRRGGGGGGGGG System files thumbs.db that was hidden from being hidden and thubs.db only gets created if there are images in there….. And they want to add another level.
"Putting GDI in kernel mode in NT4, for speed reasons they claimed (reasons, that now, are completely irellevant!)."
Why are those reasons irrelevant? Context switches between user mode and kernel mode are always going to be an expensive operation. Even if you run User and GDI in Ring 3 as NT 3.x did, the Win32 subsystem is regarded as so critical to the operation of the OS, that NT is actually designed to halt if that subsystem goes down.
While we’re ranting about explorer. Does anybody know how to disable that message box that shows up when you rename a file but also change the extension?
Why are we fighting an arms race with "hide protected operating system files" when we can simply "protect operating system files"? Who cares who gets to see the goodies when a foot thick bulletproof glass separates them from the Village Idiot?
And if visibility itself is core to somebody’s security, why not safeguard against a file’s enumeration via the ACL?
Ah, but there is a sixth level. What about the hidden "desktop.ini" file that holds a GUID that alters how explorer shows files? That’s why fonts, IE history, etc all look different.
So it’s there. It’s just not a file attribute, it’s a directory thing.
But what if somebody wants to modify their system files? If they have the privileges, the system shouldn’t stop them from doing it.
This post is not using traditional wording, for me, but I think it’s an issue important enough, and something that has had (negative) impact on me, my working and even private life, why I can’t sit silent. Should someone want to censor this: screw them.
I think, basically, Microsoft screwed up so bad with Win98 (and therefore NT5), that the race is lost.
This could be changed with one change; fix IE.
Really. Remove IE and all its hated HTML parsers, controls and viral infected crap. Make that *** <i>per user</i>. Remove the HTML crap (for crap is what it is) from the Explorer used to view operating system filesystems. Just remove it! NT4 and Win95 Explorer.exe’s worked perfectly fine. Why destroy them? Just to display your power over customers, to open the doors to infect them with viruses, spyware and other crap that could be seen as the plague of Microsoft?
Break compatibility. Yes, I know that sucks, but would Microsoft rather forever support virus spreading and other *** with your software (which you do today, you freakin’ idiots!), or break compatibility to do something right again for a change?
"Right again"? Yes. With NT (I’m thinking of the original 1.0/3.1), Microsoft hired (not bought, that started later) personnel did something good. Very good, if you ask me.
With NT4, they started to fsck up. GDI and USER worked like a charm in 3.51, but nooo… speed. Look at it today. Would you even recognize the speed difference between the 3.51 and 4.0 (meaning 4.0, 5.0, 5.1 and 5.2, aka 2003) model, using machines of today?
I dubt you would, meaning Microsoft committed the cardinal sin of premature optimization in around 1996. What’s especially bad with this "violation", is that they opened the door for many, perhaps uncountable, security holes, and they NEVER displayed any interest in reverting that horribly bad decision. Much like anyone with tunnel sight doing something, Microsoft steamed ahead with this fucked up "new" design, making NT more into Win9x wrt. security. Way to go, morons!
Do you f.ex. know why Citrix, or TS, costs you so much, but the similar but often better functionality over X, using even free software if you like, costs less or is even free? It’s due to Microsoft:
1. Putting GDI in kernel mode in NT4, for speed reasons they claimed (reasons, that now, are completely irellevant!).
2. Making the interface for drivers "secret" (try getting [b]all[/b] data you need as a free software developer, and feel your anus stretch with an MS representative standing behind you and pressing hard).
Putting HTML junk into kmode, or "even just" the system itself, making HTML parsers and renderers system wide (and allowed to CreateProcess() through ShellExecute(), was likely the worst, ever, decision any human has ever made. Any sane person would never have ever "invented" it, but Microsoft not only created it, they have defended it for a decade now! It has estimated costed society something like 10-40 TIMES more than Microsoft itself is worth. What’s next, including a vial of Ebola with every license of XP sold?!
Sorry Raymond, for possibly abusing your blog like this. Feel free to edit, or even remove this post. I just had to get it off my chest.
Totally agreed. Feature that "disallow people from edit/deleting essential files unless another action is explicitly taken first" type of protection is enough, further requests of "protection" can indeed only create security holes.
P.S.: IMHO, *nix’s sticky-bit type of protection is quite good in terms of this – not "user friendly" enough for casual users to mess with it(accidentially delete the file under protection, or accidentialy set up the bit), but experienced users can easily play with it.
And what about Windows File Protection- you know, that dllcache thing that MS hacked together. Whats up with that? Really makes you wonder what goes on in Redmond???
The thing about windows is that it suffers from a fatal flaw- it has no self respect. It allows any printer driver that a user installs to muck with the system files, and, by the way, make the app that you spend hunderds of hours on stop working! Happened to me. Now MS has a word for it- DLL HELL! or maybe they just picked that up from NNTP somewhere.
But seriously, DLLCACHE? Gotta make you wonder, who dreams this SH*T up for MS?
There is already at least one more level. Files which are hidden from the Win32 API by the kernel, such as NTFS metadata, which is where the next breed of root kits, spyware and virii will hide. Once you’ve got something running in kernel mode there is no way to know for certain that what your computer shows you is real.
Wait a minute – there IS a sixth layer of "hiddenness" – I don’t see Explorer revealing the super-hidden files such as C:$Boot, C:LogFile, C:$MFT, and others.
Explorer doesn’t have a chance, mind you, since they are protected by the filesystem API itself.
Exactly what I was talking about, wound.
Try putting a file in the recycle bin directory. It’ll be effectively invisible from the ui.
"And what about Windows File Protection- you know, that dllcache thing that MS hacked together. Whats up with that? Really makes you wonder what goes on in Redmond???"
It’s quite simple really. Stopping applications from overwriting system files would break the apps, so they’ve allowed to overwrite and then the original system file is restored from the cache.
"It’s quite simple really. Stopping applications from overwriting system files would break the apps, so they’ve allowed to overwrite and then the original system file is restored from the cache."
True – though a better implementation of WFP would have used a filesystem write filter to let the application THINK it was writing the binary, but never overwrite it at all. Then there would be nothing to replace, and the bloat of DLLCache wouldn’t be required.
The only things that should have ever been able to overwrite a Microsoft signed binary in %windir% are Microsoft-sourced hotfixes, service packs, and Windows upgrades. Not installers for random applications.
But you can’t throw the writes away because some installers do a checksum of the file to make sure it got copied okay. So now you have to emulate all the file operations – and now you’re emulating a filesystem – why not just use the filesystem?
LOL… with desktop.ini and alternate data stream, we have two ways to hide files that Raymond didn’t even mention. Does that mean we are at level nine now? ;-)
"But you can’t throw the writes away because some installers do a checksum of the file to make sure it got copied okay. So now you have to emulate all the file operations – and now you’re emulating a filesystem – why not just use the filesystem?"
You don’t have to immediately discard the writes – keep them around until the installer has exited… whatever. Don’t get me wrong – the last thing I think Windows needs is duct-tape (as I just proposed) holding things like WFP together. But if the problem is installers stomping on protected binaries, then preventing stomping is step 1. Step 2 is going back and cleaning up the fundamental flaws of file redistribution where every installer in the world ships a different version of every MS binary – causing DLLHell, and leading to the over-engineered solution that is side-by-side (see my duct-tape reference above).
I’m just sayin’… :-)
It appears that people misunderstood the scope of the original request. It wasn’t "Hi, I’m making a rootkit and I want to hide stuff". It’s "Hi, I’m a program that has some important data files and I don’t want the user messing with them because my program will stop working if they get deleted or corrupted."
I think even non-hidden files are well enough hidden when located in places like
C:Documents and SettingsUserLocal SettingsApplication DataMicrosoftInfoPathFormCache15A6420C5.AF6$7bbe17b22d2a6bc8_XDInfo_props.dat
Funny you should mention that, memet. A while ago Nero blew up while creating an image on my computer. Now I’m stuck with this "LOM_SE." file that Explorer and the command prompt have no clue how to delete (they always say that the file doesn’t exist). My best guess is that there’s a null or some other invalid character in the extension, that neither of them even know how to display.
I’ve recently seen some spyware perform a REALLY vicious trick. The program in question creates a file with an invalid filename in the file table (I’m not sure exactly how this is turn, but it’s undoubtedly low-level voodoo). The trick is to give the file the same name as a real file (in this case, mshta.exe in the system32 directory) except for a wildcard ("?") placed just before the extension.
The effect of this seems to be to make the file invisible in Explorer, (perhaps because the wildcard is "filtered out", and duplicate file names are not shown either, so the original file thus hides the intruder).
The only way to see the file is to do a "dir" with the a switch and place a wildcard "?" in the correct position. But of course you have to suspect that the file exists, otherwise you have no chance of finding it.
Has anybody else come across this? Can anyone shed any light on exactly why the spyware file doesn’t show up in Explorer? I’ve had to clean this infection off a couple of computers and it makes me wonder how many more "ultra-hidden" files there are lurking around out there.
You know, there’s a way to actually hide files from the Win32 subsystem, and that’s by naming files with null embedded unicode names via the Native API.
Now, when I first read of this, I thought it was a very ‘evil’ trick that could be done out of malicious intent. But recently, I realized that it’s not at all that difficult. For some reason, the java runtime writes files with Unicode names via something other than the Win32 API. Why do I say this? because recently, I downloaded something via Azureus (a java based bit torrent client).
One of the files that came with the download is this obnoxiously long named file. You can view it in explorer, but you can’t select, edit or delete it. This sounded exactly like a Unicode filename issue to me, since explorer displays the NULL terminated string returned by the Find Files API. But when it tries to open that NULL terminated file name, it doesn’t exist.
I had to spend quite a few hours writing a native application to open the file and friggin rename it…
Anyways, as much of a headache as it was to me, I still don’t mind that there is this loophole in the system. But it *is* a loophole.
(ps. why don’t I mind? you might ask. Because "if a bad guy gets you to run his code on your computer, it’s not your computer anymore". The loophole is not the reason why…)
Aha. That worked. Can’t imagine why I never thought to try that in the couple of years that file’s been there. Thx.
Justin:
How did you try to delete the file on the command line? I’ve had this problem with some files before, and I’ve always been able to do it with wildcards. If it’s the only file in the directory, did you try "del *.*"?
If there are other files in the directory, you can try wildcards like:
"del LOM*" or "del LOM?SE?*" or whatever. If those don’t work, you have a more difficult problem than I’ve experienced before.
Looks like there will be a sixth level after all:
http://blogs.technet.com/windowsserver/archive/2005/04/20/404005.aspx
http://spaces.msn.com/members/wardralston/PersonalSpace.aspx?_c01_photoalbum=showdefault&_c02_owner=1&_c=photoalbum
[quote]
A while ago Nero blew up while creating an image on my computer. Now I’m stuck with this "LOM_SE." file that Explorer and the command prompt have no clue how to delete (they always say that the file doesn’t exist).
[/quote]
It was so much easier in DOS times with Norton Disk Edit…
While we on Explorer, I would love to read about those HT files, which allow to configure L&F for each directory.