| Date: | August 6, 2014 / year-entry #188 |
| Tags: | code |
| Orig Link: | https://blogs.msdn.microsoft.com/oldnewthing/20140806-00/?p=323 |
| Comments: | 34 |
| Summary: | When Explorer copies files around, it doesn't really know what the maximum file size supported by any file system happens to be. (That information is not reported by GetVolumeInformation.) So it guesses. If the file system name is "FAT" or "FAT32", then Explorer assumes that the maximum file size is 4GB − 1. Also, if... |
When Explorer copies files around, it doesn't really know what the maximum file size supported by any file system happens to be. (That information is not reported by If the file system name is Also, if a file operation fails with the error Why does Explorer map "invalid parameter" to "file size too large"? Because some file systems use Therefore, if you're implementing a file system, and you're getting these spurious "File too large" errors, one thing to check is whether you are reporting "invalid parameter" for a case where all the parameters are actually valid, but something else prevents you from doing what you want. (Maybe "access denied" would be a better error code.) |
Comments (34)
Comments are closed. |
I've been known to use invalid parameter for you bonehead (for an on-topic example move file inside of file).
Ooh. Ooh! Here's a thought!
How about, instead of making unfounded assumptions, Explorer stops reporting "file too large" except when the file system actually reports "ERROR_FILE_TOO_LARGE". If Explorer can't figure out why the parameter is invalid, then it should just report to the user an error message that says _that_ instead of translating the error into some other error message that may or may not be appropriate.
"Explorer assumes that the maximum file size is 4GB − 1."
– 1 GB? 1 Byte? 1 slice of left over pizza? 1 months salary?
If Explorer reported ERROR_INVALID_PARAMETER without fiddling, then people who are knowledgeable or not knowledgeable enough might say "Explorer is calling the file system with an invalid parameter, it's a bug!!". There is no way Microsoft can win, it's an artefact of a mass market. Sometimes you have to do what is intuitively "wrong" so that you can stop people complaining en masse and get on with the real work.
@pete.d Then users say "This new version of Windows is too confusing, it took me forever to figure out why I couldn't copy this file to my flash drive when it could have just told me it was too big! Parameter invalid? I don't know what that means! I'm going to get a Mac for my next computer".
At the end of the day, MS has to work with the third party file system drivers they have, not the third party file system drivers they want (at least I assume they are third party or legacy or something, or they would have just had them fixed).
@Jimmy: 1 byte. The maximum value of an unsigned 32-bit integer is 4294967295. 4gb is 4294967296 bytes (or only 4000000000 bytes, if you market hard drives).
@dave This entire article is about cases involving a "custom file system". The context of this debate is framed by what Explorer has to do to meet that case.
It would be nice if the shell said "That file was too big," and then logged an event that said "We think that file may have been too big, the FS returned 'Invalid Parameter'"
Hiding an error from the end-user is smart.
Hiding an error from the administrator just makes you an ass.
Clearly you can't tell the user "Invalid Parameter" because even a user like me wouldn't know what that means.
Perhaps a better message would be, "Explorer encountered an unknown error. This may mean the file is too large."
@Danny Moules: there can be a custom file system provided by Microsoft – like the exFAT driver for XP and Server 2003, available for download.
An instance where Explorer incorrectly reports a file size error is when the Explorer Shell's caching of file system metadata fails to properly cache reparse point data. As a result the shell queries the wrong volume for volume information. If the queried volume is readonly or does not have enough free space, the shell reports an error indicating that the file is too large. There has been an open bug report about this issue going back several years now.
In all my life, I've seen one (1!) computer with a custom file system, and it was a Microsoft file system.
Rather than making Explorer report spurious errors, maybe Microsoft should make sure that at least its own file systems do the right thing. This also does away with the entire mass-market argument.
Furthermore, if you encounter an invalid parameter on a custom file system, you may try to guess but Gabe's message is much better than a potentially wrong message. Also, if it's known that the cause might be the file size, why not probe with a one byte file?
I've had "insufficient space available" while copying a file to a USB stick before; 6GB to an empty 32GB stick. Took me a while to twig and go "oh hang, what file system is on that stick?". One reformat to NTFS later and problem solved. A slightly more helpful "file too large for the file system" would have been welcome.
@Dave There's an ext2/3/4 filesystem driver for windows, iirc. I'm sure there are others. It's entire possible some businesses may have an in-house proprietary driver of some variety…for whatever reason.
@Danny Moules : This entire article is about cases involving a "custom file system".
Duh, yeah. I was misled by "file system name is FAT" into thinking "the file system is FAT". Even though I have told that exact same lie in a remote file system (reached via SMB) of my own concoction.
The "file system name" is a terrible idea, since applications use it to make assumptions about capabilities rather than using the capabilities bits (and yes I know there is no max-file-size indication). And if you create a new file system and give it a new name, existing apps tend to have hissy fits (Robocopy: OMG! It can't have ACLs if it's not NTFS!)
@Chris Crowther.
Yes, but as I mentioned before, I was confusing 'name of file system on-disk format' and 'the identity of the file system driver'. We were talking about the former, but I assumed the latter.
<i>And then I have to write blog entries saying "This means that the file is too large." -Raymond</i>
I laughed quite hard at that. And it is so true. You can't win, you can't break even, and it's probably unwise for one's financial future to get out of the game.
@T. West: I think somebody just broke even. See impossible microwave drive.
The most common custom FS drivers I come across is the Apple provided HFS+ driver – which is awful and fails in all kinds of weird ways. I've seen it crash lots of apps just trying to enumerate files – so it wouldn't surprise me if it pulled stuff like this.
@Thomas Jepp
> The most common custom FS drivers I come across is the Apple provided HFS+ driver – which is awful and fails in all kinds of weird ways. I've seen it crash lots of apps just trying to enumerate files – so it wouldn't surprise me if it pulled stuff like this.
Except Apple's driver is read-only, so Explorer couldn't possibly be copying TO the HFS+ partition. And I've never had that much problem with Apple's driver. The last problem I had was back in Lion when FileVault 2 came out and it blue-screened your computer trying to read the encrypted volume.
I love the appropriate usage of U+2212: MINUS SIGN. I suppose most people would have used U+002D (acceptable) or U+2013 (nightmare).
Wouldn't the shell saying "Invalid ParameternnThis may be due to the file being to large." or something similar be better then saying something *completely* different to the return code's meaning?
>MS has to work with the third party file system drivers
Does a typical Windows system *have* third party file system drivers? (At least for the case where we're talking about file systems on persistent storage, not just weirdo drivers that are conveniently designated as 'file systems')
In particular, I would imagine that all *FAT* drivers are Microsoft-supplied.
@dave: Some comes with the OS, other are added when new devices come to exist.
I think I've used 3rd party driver for ISO-9660 (i.e.: CDROM driver) on Win3.1, and then an Ext2FS driver for Win98.
Take a look on FDISK's list partition types command and see how many filesystems were around at that time.
@Dave >Does a typical Windows system *have* third party file system drivers?
Perhaps it's uncommon, but definitely not unheard of. For example, any Mac user with Windows installed will have Apple's HFS+ file system driver installed.
This article reminds me of my dad, who would constantly be buying RAM for his computer as he had a corrupt EXE that, upon launching, caused Explorer to display an "Out of memory" error (even though the error had nothing to do with memory.. )
blogs.msdn.com/…/519388.aspx
@everyone: mapped network drives. You shouldn't assume that the filesystem is local to the PC reporting the error.
>How about, instead of making unfounded assumptions, Explorer stops
>reporting "file too large" except when the file system actually
>reports "ERROR_FILE_TOO_LARGE". If Explorer can't figure out why
>the parameter is invalid, then it should just report to the user an
>error message that says _that_ instead of translating the error into
>some other error message that may or may not be appropriate.
Sometimes you have to report a best-case guess because the actual error information that's being provided is insufficient. Consider a situation where you're making a non-blocking connect to a remote server. select() returns success with an exceptfd set indicating that there's a problem, WSAGetLastError() says "no error". What do you tell the user? You can't say "select() returned with an exceptfd set but WSAGetLastError() reports no error" because it's meaningless, you can't say "We tried to connect to the server but couldn't, absolutely no idea why" because that's useless, so you sort of make things up and report a WSAECONNREFUSED or something because it's the least awful thing you can do and it's probably close to the reason why there was a problem.
The ClearCase source control system mounts as a file system. (At least it used to.)
@Anon: Windows Explorer (like most Windows components) looks up all error messages with FormatMessage. You get whatever messages were built into Windows, for that particular error. There are still, I think, a few messages that have a replacement in them (%1 etc) which causes problems when the component trying to get the error message doesn't know that the error in question is expecting parameters.
The WinError.h file in the SDK is generated by the message compiler, using the .mc file that generates the resource that FormatMessage looks for. The comments above each #define give the message in the version of Windows that the SDK was built for. This means you can search WinError.h for the message you got, to get the #define and the actual error number. Usually, I find searching for one of those will give more useful information than searching for the error code.
@640k: it seems you missed my comment above. What if the file system is on a Novell Netware (or whatever) file server accessed by a mapped network drive?
Compensating for your own bugs again?
Why not fix the return values in *your* file system driver instead?
3rd party usb driver is actually ms own usb driver, no need to blame others.
@Mike Dimmick
err.exe will look them up, no need to dig around.
In this case:
C:err>err =ERROR_INVALID_PARAMETER
# matching string "ERROR_INVALID_PARAMETER" :
0x00000057 ERROR_INVALID_PARAMETER winerror.h
# The parameter is incorrect.
# 1 matches found for "=ERROR_INVALID_PARAMETER"
"The parameter is incorrect" is not helpful.
0x00000057 is also not helpful.
The actual incorrect parameter, and/or why it was incorrect, would be helpful.
@Dave (no, a different Dave)
If you're reporting another component's error, report the error that component gave you. You don't have to present it in a dialog, but you need to present it, at a minimum, to either the logging subsystem or a channel over which dbgview can intercept and display it.
>If you're reporting another component's error, report the error that component gave you.
See my original message, WSAGetLastError() is reporting ERROR_SUCCESS for the reason why it can't connect. The only thing that reporting that as an error status will give me is a screenshot on the daily WTF. This is why I have to invent an error, in my case WSAECONNREFUSED, in order to have something to report that at least may be the cause of the problem.
@Dave: Hmmm I think this needs WASEYOURDRIVERISBROKEN.