| Date: | October 26, 2005 / year-entry #321 |
| Tags: | history |
| Orig Link: | https://blogs.msdn.microsoft.com/oldnewthing/20051026-44/?p=33613 |
| Comments: | 100 |
| Summary: | An example, all too frequent, of ways programs assume that the user interface will never change is reaching into system binaries and sucking out undocumented resources. In the shell, we have fallen into the reluctant position of carrying "dead" icons around for the benefit of programs that assumed that they would always be available. However,... |
An example, all too frequent, of ways programs assume that the user interface will never change is reaching into system binaries and sucking out undocumented resources. In the shell, we have fallen into the reluctant position of carrying "dead" icons around for the benefit of programs that assumed that they would always be available. However, we often make these "orphaned" icons blank so that these programs don't crash, but they don't necessarily look all that wonderful either. Recently, I learned of a new type of resource stealing: Stealing animations. For Windows Vista, there have been many changes to the way the system internally organizes its resources in order to support the Multilingual User Interface feature. One of the things we found was a handful of programs that reach directly into Shell32.dll in order to obtain the file copy animation. Too bad (1) the animation isn't kept there any more, and even if we moved the animation back to its original location, (2) the animation uses a new compression scheme that older programs can't handle. Remember, resources in system DLLs should be treated as implementation details, unless explicitly documented otherwise. I'm not sure what we're going to do about the animation resource problem. Suggestions anyone? Should we declare these programs broken and make the people who bought them go back to their vendors for upgrades? (What if there is no vendor?) Should we develop a shim and hope that those four are the only programs that need it? Should we carry the original animation in shell32.dll as junk? If I told you that one of the programs that did this won "Game of the Year" recently, would that change your answer? What if another turned out to be a top-rated anti-spyware program? |
Comments (100)
Comments are closed. |
I would replace the animation with a picture of a finger flying across the screen pulling a banner that reads.
"You buy software made by morons."
Now forgive me while I go change some in house software that did this very thing.
Wow … the lengths for compatibility. Here is a question; what factors go into deciding whether an app will be flagged in the user’s face at Vista install time as incompatible? Or when the app installs later? For example, if you install SQL 2000 on Win2003, it warns you. Perhaps this is in order? Otherwise, sigh, keep the old animation. Or dumb it down like you did the icons.
The most backwards-compatible solution, of course, would be to just leave the old animation in there. I suspect that’s what you’d end up doing.
So, what’s the documented way to display the system’s file copy animation? Is it easy to find?
Ooh, I know, it’s the file copy animation for SpyBot when it checks for updates! I need a hobby.
Sometimes I wonder what is being sacrificed for the sake of compatibility with third party developers using completely undocumented methods of doing things. Especially since it creates code paths that can not be tested any longer.
Anywho, my suggestion is that you return an ugly, 2-bit international NO symbol for any resource that doesn’t exist. Although the program will not crash, it will be obvious that there is something wrong and it means you’d only have to encode the ugly NO once.
Bah, forgot something. I also strongly suggest that if so many people need these animations, make a public, supported, well-rounded API for doing this (if one doesn’t exist already).
The sad part is that these vendors probably thought they were doing the right thing. By reaching into the a DLL to get an icon or animation rather than making their own, they probably figured they would always appear up-to-date with the latest look and feel changes.
I say break them. If it’s a vendor that doesn’t exist anymore, c’est la vie. If it’s Game of the Year and a popular spyware program, they’ll issue updates, and learn that they committed a no-no.
Another idea would be to offer to ship the patches for these vendors with the Vista CDs. When a user upgrades a machine to Vista, it can check to see if it has patches for any of the installed programs and offer to apply them. That may sound complicated and expensive–it probably is–but so is having layers of backward compatibility shims adding complexity to the Windows code base over time.
The option of adding a "backwards-compatibility icon" which is blank seems to follow through quite neatly into other areas, like the shell32.dll animation. That seems to strike the balance well between wanting to keep apps working (and avoid the "MS broke my apps!" shriek) and not wanting to encourage people to rely on bad behaviour. It might be better to do something other than blankness, though, since blankness doesn’t immediately mean "wrong", it just means "something odd is going on". While I wouldn’t recommend having a "coded by morons" flag, and obviously you wouldn’t take that solution seriously, perhaps something akin to the "broken image" icon? So when the app’s manufacturers test it (!) on Vista they’ll note the problem, but it doesn’t kill their app and annoy them and their customers.
One other thing, and this is where you laugh hysterically because I don’t know what I’m talking about; can you make shell32.dll not include the animation but instead load another DLL (in AppPatch or similar) if something tries to use it? So shell32 isn’t bulked up with a backward compatibility patch? Like I say, this is probably totally impossible and ridiculous, but be gentle with me, I’m a web guy. :)
OK, the vendors have no excuse for this — everybody knows you should just copy the icon/animation if it’s not documented!
I say you teach them a lesson. Keep the animation there, but make it look really stupid. Like make the files dance instead of elegantly float.
If icon resources are supposed to be private implementation details, why does Windows Explorer (and previously Program Manager) allow users to select shortcut icons from arbitrary files?
"Should we declare these programs broken and make the people who bought them go back to their vendors for upgrades?"
Yes. Ok, maybe not. But it’s not a bad idea really!
"What if there is no vendor?"
Then the app is perminently dead. So sad. Programs do not have to survive forever.
"Should we carry the original animation in shell32.dll as junk?"
No. Make it blank.
"If I told you that one of the programs that did this won "Game of the Year" recently, would that change your answer?"
a) They will release a patch.
b) Someone at the game company is going to get a good talking to.
"What if another turned out to be a top-rated anti-spyware program?"
a) They will release a patch.
b) Someone at the game company is going to get a good talking to.
Give us a standard API to load system resources! give us standard save/open/copy/paste/print/etc. icons, standard animations, etc.!
Well, give us a standard API to load (and customize!) system resources, then. Like standard toolbar icons, standard animations, etc.
Mike Gunderloy once used an animation of a small piglet flying out of a pair of buttocks (to help symbolize the feeling that a particular program would only work when pigs flew out of our butts). That would teach people, putting in an animation like that!
Microsoft introducing bugs and anomalies into Windows is an excellent way for us software vendors to persuade people to carry on paying their support subscriptions.
I would cite (in recent months): the disabling of access to help files held on a network file server; the setting of the Kill bit for PSDispatch (that object is needed for Windows Scripting to work); and a change in file sharing that caused programs for older versions of Windows to complain that the SHARE command needed to be run.
The nice thing is that Windows Update installs these bugs automatically so we don’t have to worry about some users not needing our help.
I have to agree with some of those above comments… its time to start moving forward faster and stop supporting programs that don’t have vendors anymore. With any luck, there will already be a newer and better program to take its place.
I also agree with the replacing of the images with one catch-all "No" symbol – something along those lines… its time developers started following the rules rather than snooping around and getting into undocumented stuff.
To answer your question, though: if the programmer was doing something in an undocumented way when a documented way existed then do what you can to keep compatibility (or at least avoid crashes) but don’t bust a gut doing it.
But if the programmer had to do something in an undocumented way because no documented way existed, then work night and day to keep compatibility, and if you can’t make a future version of Windows compatible, cancel that version of Windows.
Okay, if I understand it, Vista isn’t a released product yet. In that case, I would go back to the vendor and indicate that they should change the behavior. That gives them fair warning.
In this situation, what is the right thing to do? Is it legal to copy the animation used by the shell and include it as a resource within the application? Doesn’t seem like it to me, even though everyone rips off the XP icons for their own websites and reuses them. At least in this case using undocumented resources in your application doesn’t seem as illegal even if the end result is the same: presenting someone else’s work as your own. :)
"I am a developer, not an artist. My file copy animation attempts are not fit for public consumption. "
I never said that YOU should make it personally. That’s why God made graphic artists. As for consistency, I would say to feel free to copy the dialog, but don’t access it in a way that is not documented, and therefore may be changed without notice.
As far as I’m concerned, it’s just as bad as hard-coding a path to a file that exists, and you just keep assuming it’s going to be there for every new version. You wouldn’t do that, so why would you hard-code to the resource in a DLL? (I am of course referring to undocumented resources, it should be safe to assume documented resources will continue to be available, and that microsoft should bust their ass to always make it available in future releases).
The problem with icons is that there’s no straight forward way of getting a shell icon for a file type given the size wanted… there’s always the shell size and icons look pixelated even though my dll provided all sizes including 256×256…
IExtractIcon just doesn’t work as it should, so that’s one reason why a lot of developers just loaded the icons directly from DLLs…
1) Come up with an API for accessing animations of this kind.
2) Inform the vendors of the new API.
3) Break the apps with a clear conscience.
The sad thing here is that Microsoft creates these nice user interface widgets but then doesn’t bother making them available to developers, forcing the developers to either waste time trying to recreate them (and match the Microsoft "standard") or else hack into the Microsoft implementation.
Microsoft that needs a "widget czar" that watches for icons, animations, menus, controls, etc. in operating systems and major products (especially Office) and makes sure that these are properly and "officially" exposed to developers.
"Should we carry the original animation in shell32.dll as junk?"
Yes. This is the philosophy that made Microsoft Windows the most commercially successful software product in history. It is way too late to start thinking about a clean architecture now.
-Wang-Lo.
} If it’s Game of the Year and a popular spyware program,
} they’ll issue updates, and learn that they committed
} a no-no.
If a Game of the Year is using a file copy animation, it’s probably doing so in its installer. Installers tend to come on write-only media, and are accordingly hard to patch.;)
The real problem is that it seems that any program is an abusing bunch of stuff in the eyes of the author, except for the programs he wrote himself of course.
Fact is, most applications used that animation because there was no documented way to do that without redesigning the animation yourself (no you cannot copy it without going into licensing issues). So the solution is to use some article you find on the net (probably even on MSDN) which tells you how to do that. You can cry out loud about abusive applications developers all the time you want, but it’s your fault for having some thing which does not have an API.
All this except of course Microsoft applications which have neither the undocumented nor the licensing problem.
These articles about how third party software is bad and forces us to adapt are getting boring.
It should be possible to create a dummy animation that a) is really small and b) shows that something is not quite right, and put that in shell32.dll as a replacement.
Well my vote on this would maybe create a dll with animations and stuff for developers to use. Kind of like the Icons that comes with Visual studio, While the icons are kind of lame and if you want something better then you need to create your own they are a nice handy way to get this kind of thing. So create a library, and not a huge library or common things like animations, icons and stuff for developers to use and then make it well known. Spyware guys and so on are always going to try to dig into the system32. Thier software changes weekly to try to avoid detection. In reality I would love to see a breaking change to spyware. Game of the year, those developers should have known better. But due to backwards compatability I think you have to leave that in there hence leaving more people to complain about code bloat but I would change it to a lower res or slightly different. But I would come up with a common resource library for developers to use and document it well.
One of the things us developer do to make our apps acceptable to the users is follow the Microsoft Design guidlines which even Raymond has posted back to. When those designs change over the years http://blogs.msdn.com/oldnewthing/archive/2004/07/28/199589.aspx We hope that the changes in the Microsoft culture reflect automatically in our own apps.
Also I noticed a comment I made back in another of Raymonds post where he was point out the windows design specs is http://blogs.msdn.com/oldnewthing/archive/2004/07/29/200653.aspx I think that is why game of the year might be game of the year. They didn’t try to reinvent the wheel, they realized, windows was doing something and they just wanted to reuse it. Apps that totally try to reinvent the wheel usually fall flat on thier face because users are confy in going app to app seeing a familiar UI. There are a few exceptions though like http://downloads.theregister.co.uk/Windows/Utilities/Launchers/jet-toolbar.html Jet tool bar I think is much better than windows start bar. Again though only my opinion on that.
::looks at feet::
Guilty as charged. My setup program uses the SysAnimate control, displaying the file copy animation for intallation, and the file delete animation for uninstallation. I guess this serves as fair warning that I may need to change my app for Vista.
I guess there’s no reason to not use private copies — in total, the two animations are just 28 KB. (I believe some editions of Visual Studio include the animation files.) It just means potentially less consistency with Windows, depending on their version.
Hopefully, if the resources aren’t there, the SysAnimate control will degrade gracefully, so the worst that would happen is no animation showing. If that’s all that happens, then "breaking" apps is surely the way to go.
If the resources are there and but using a new compression scheme… well, hopefully the SysAnimate control is being updated to support the new compression scheme.
(To jd regarding a "widget czar" — hear hear! That would be awesome.)
To all of the people saying that old applications should die, please bear in mind that there’s a lot of custom-developed software out there in the business world that was made by some overpaid consultant five or ten years ago but continues to do its job. The company using the software has no desire to pay again to have it altered, and so their only recourse is not to upgrade to Windows Vista.
With all that said, I think a good compromise would be to make a smaller version of the animation which perhaps skips some frames or whatever and include that in the place of the old video resource. Of course, this assumes that there are no applications out there relying on the video being a certain framerate or having a certain number of frames, and it wouldn’t surprise me to find that assumption invalid.
I like Jeff Parker’s idea, except instead of a DLL somewhere, I think it would make more sense as a new API of some kind.
Why?
Well, it might be nice for users to be able to set up ‘themes’ for these sort of things, so that (for example) the file copy animation could be customized (maybe even for an OEM).
If you do this via DLL, you can’t as easily or flexibly alter or extend the visual element in question.
Downside, of course, is that it might require a lot more work than a simple DLL.
I totally feed into the whole "we need backward compatibility" and the "no junk" mantras.
I say take the middle group and impliment an empty animation, even if it’s a static image that says "copying".
Considering 2 of the 4 programs are highly visible/popular apps even more reason to at least support it. Keep in mind, the average person is going to run the bits from the CD, they aren’t going to patch and reburn their game so hoping the vendor fixes their coding mistake (on an old game, good luck!) is pointless.
Breaking old apps–or declaring them broken–in Vista makes Vista a less valuable platform and increases consumer resistence to upgrading to Vista. Some would label that "bad."
Or have I simplified too much? :)
Change Vista’s DLL to Shell64 and leave Shell32 alone?
Oh, and make the animations in Vista available through a public API so people don’t steal them?
I’ve always used the resources, GUI and otherwise, included with the IDE. When I needed an aninated icon or other trendy GUI candy, I used those that shipped with the IDE, or bought a third-party resource.
Various people have grumbled that there’s no documented way of doing this. In fact, the animations can be found here:
C:Program FilesMicrosoft Visual Studio .NET 2003Common7Graphicsvideos
The "ooh but I wanted to get new animations automatically" argument is very weak. If MS updated the UI, the chances of the animation being even the same size were pretty slim.
But my vote is don’t break the apps. Make them ugly.
Various people have grumbled that there’s no documented way of doing this. In fact, the animations can be found here:
C:Program FilesMicrosoft Visual Studio .NET 2003Common7Graphicsvideos
Yes!! Of course everyone develops ONLY with VS.NET 2003. Just. Plain. Dumb.
I can see both sides of the compatibility debate, and I would lean towards impairing w/o breaking (i.e. a NO animation).
What I really think MS should spend some time on is trying to figure out ways to prevent this sort of thing from happening going forward. I realize there will always be a way to dig into system code, but if that path can be made sufficiently difficult (and therefore VERY VERY OBVIOUSLY unsupported), it may be easier to make future decisions about breaking compatibility.
"What if there is no vendor?"
Then the app is perminently dead. So sad. Programs do not have to survive forever.
===
Actually, some do. There are programs and devices out there that are too expensive or useful to replace. There’s been companies willing to pay big money to replace a 286 that died because it’s the system designed to run with a very expensive piece of hardware.
The solution, as I see it, is to figure out how to use virtual machines to easily run previous versions of the OS. I have to wonder if it would be worthwhile to do so.
If you’re going to break compatibility to remove the bloat from the code (and add some OS-level security?), phase in the changes slowly over time like NT did.
My uneducated two cents:
1) So you look in the dll, you see a resource… how do you tell if it’s supported?
If the only way to discover that something is undocumented is to read the entirety of the documentation and find it’s not in there, then you will get this kind of problem with every single resource you ever include.
So if it’s possible, I’d say to mark all unsupported can-change-at-any-time resources as such, /by name/. So, "file copy animation" is not a suitable name for an unsupported animation. But "file copy animation, internal use only" is.
2) When a resource becomes deprecated, visually mark it as such. Make the animation display the animation grayed out, with text over it saying "Deprecated animation, see MSDN article XYZ". Modify audio resources so that there is speech over the audio, saying something similar. Replace text resources with something similar. And so on.
It does not HAVE to be user friendly. It just needs a message they can type into google and find that the authors of their program screwed up. And it has to still work, just giving that warning without changing the program functionality in any way.
Text and speech need to be localized, and there’s no way MS will do that on animations and sound files.
Also realize that a) people whose programs don’t work when they upgrade Windows blame Windows, not the program. Thus MS gets a bad rep.
b) vendors don’t like to patch (especially games) because it costs them money to develop and distribute them
Inevitably I suspect this will have to end up like the shell folders registry key (kept forever because of bad programs!)
However I do agree that there should be documented ways to do everything that people end up doing through kludge. I know you can’t anticipate everything, but past experience with situations like this should give you clues.
Yes, what is the documented way? I’m not sure there is one…
While I "feel your pain", in this case (and others) I think it’s partly self-inflicted.
Everybody sees that copy annimation and thinks "I’ll use that myself!". But what if it changes? (folder colors, slightly different annimation, etc.) I don’t want to have to update that in *my* app…and what if it’s different on NT and XP? Then I’ve got to maintain two annimation resources too. It’s "much easier" just to grab whichever one the Shell is using.
You’ve got to assume that just about anything Microsoft wants to do in its applications (especially widely used ones like Office and Explorer) developers will also want to do in their apps. So give them a documented solution from day-one or be stuck supporting the undocumented hacks.
Yes, I realize it’s not that simple as that…the code has to ship at some point. That’s why it’s "Software Engineering", not just "Programming" :-)
You should
A) Create a new ShellVista.dll with all the new stuff.
B) Change the Vista SDK link libraries to point people at ShellVista.dll
C) Create a shim Shell32.dll which has the identified resources and shim exports needed by all PreVista applications.
In otherwords, you should play be the same rules we have to :)
i’d say break them :)
like win64 needs new drivers
and ship with a free virtual pc :p
I’d argue that the old programs shouldn’t break in any way. If it’s not technically infeasible, have the animation become something that fits with the Vista look and feel. Perhaps a version of the new animation in an XP format?
Stop thinking about business cases and proper corporate practices and proper programming practices and blame and who’s responsible for the mess… Instead, focus on the user experience. The user experience is using a program in Windows Vista with a Vista look and feel. End of story.
"answer: there isn’t any!
so ms has to share the blame for what’s going on with current programs "
Why should Microsoft be blamed for a third party using resources in an unintended, *undocumented* way? If you want to use an animation, make it yourself!
Although, I do think that it is very courteous of MS to put in fixes so the programs at least function, I don’t think it’s anything more than that: a courtesy.
Ytram: "Why should Microsoft be blamed for a third party using resources in an unintended, *undocumented* way? If you want to use an animation, make it yourself!"
Well,
1. I am a developer, not an artist. My file copy animation attempts are not fit for public consumption.
2. I belive a consistent end user UI experience is good: For exactly the same reason that I (or any other developer) would choose to use the GetOpenFileName common dialog, I’d choose to use the standard file copy animation.
Simple answer is to replace the icons and animations with something similar to the IE "broken image" icon.
That way, app developers will see that there is something wrong with their app.
And users will see it too and understand what it means.
Jeff Parker — Raymond mentioned an *anti* spyware app, not spyware itself…
Regarding a DLL for animations & stuff — Vista has a new dll called "imageres.dll" that contains new icons. And there are some new functions in the Vista SDK shell header files that appear to be ways to load these new icons. So maybe they can expand this to animations as well…
Well there’s always a chance. :-/
No-one is going to use Windows Vista – any more than anyone is currently using Windows XP. People use Windows, full stop. Anything more specific is technical mumbo-jumbo as far as they’re concerned.
The downside with all of this "let the apps break" and "they should have known better" is that there’s a lot of people out there that doesn’t really actually *know* better; and finding all this information about what to do or what not to do in the tidal wave of API’s, fixes, articles, help files and blogs from Microsoft isn’t the easiest thing in the world — provided that you actually take time to sift through it all in the first place. For most of these "secret Linux admirers but works in the Win32 world because everyone else does"-programmers (like many of my colleagues), that’s mostly wishful thinking.
What will probably happen is that a lot of developers notice their programs just broke (again), and their response will typically not be "oops, I must have done something wrong", but rather "those darn *** at Microsoft changed it all again!" Either way, MS gets the blame.
Not everyone reads Raymond Chen’s blog. It’s been a wakeup call for me.
> (I believe some editions of Visual Studio include the animation files.)
PatriotB has it right. Most of these were included in Visual Studio.
Even if you can’t afford Visual Studio (say you are using gcc and one of the free tool chains) and you just *have* to use the animation or icon, just extract the resource and use your own local copy.
*Sigh* laziness kills you every time.
IMHO, Vista will need a shim Shell32.dll.
My call will be: If the program won’t hang by the lack of icon, just let it be. Otherwise, mark the application in the incompatability list and call the software house to tell them release a new version that fix this.
Actually, what’s the "Compatable with /Design for Windows whatever version" label for if you can’t fail them because they’re using "hacks" that is never supported? I’ll agree that all "supported"/"documented" feature must not be broken, for anything else it’s the software house’s responsibility.
Here’s my take
1. Break it. Let vendors know about it, post a TID.
2. Create a new API. This has shown you there is need for one. rather than being annoyed by this example of vendors hacking, realize that it SHOWS you a speciic need.
Dewi Morgan:
But why the resources are to be renamed? Aren’t "Everything is undocumented unless it’s documented" logical? If you don’t see it documented, make you own version or you risk it may not work in the future versions. I think this is fair.
People are not supposed to "steal" features out of Windows unless it’s stated "you can use it if you like". If your program enjoyed undocumented NT kernel functions, blame noone when you program broke when Microsoft later change it.
Ok, as I understand it:
[1] The animations are not in the DLL.
[2] Vista is not yet shipping.
[3] Hence the original animations (in the original format) could be put back in the DLL for compatibility reasons, using the original IDs (because Vista hasn’t shipped, so no hard-coded usage of these DLL/resource IDs exists in Vista).
[4] These animations are pretty small (yes?)
[5] Vista has monster system requirements.
[6] In conclusion, no-one will notice if you just put the animations back where they were.
[7] In summary, JFDI.
Am I missing something? Or is it just the usual Microsoft "implement some standard part of Shell/UI, and then don’t provide devs with a way to reproduce it in their apps if they want to be consistent?" thing?
Job #1 would be to *learn* from this. At the very least, all such resources should be available for devs to use just by downloading them from the MS site so they can use them if they want, without resorting to system hacks. No need to buy VS.net/VC etc – just give them away.
In other words, don’t repeat this in Vista by implementing some fancy UI for a core feature (like *copying data*! gee, no program other than Explorer would *ever* want to do that, surely?) without providing a way for devs to use it too.
Or else just accept that people will hack around your new stuff too :-)
Wow, just checking my favourite programming language, Delphi 7, Borland have been kind enough to provide a TAnimate control on the tool palate. I have checked the source code for this control (which Borland also supply with their wonderful tools) and lo and behold, this control grabs the resource from shell32.dll when you use the one of the standard animations. I am guessing that most programs with animations built using the Borland VCL will encounter this problem.
This is not the programmer’s fault, rather the fault of Microsoft for not providing a standard API for these animations so that Borland could leverage this work and build it into its tools.
It goes to show that Microsoft should be very careful when they introduce new features to include API’s and documentation.
PatriotB is not right, not everyone uses Microsoft tools to develop windows applications, and hence do not have access to these resources. These animations are the types of this that have become standard to represent common operations, and they should be provided by the OS.
All gripes aside, if the system was reorganized to have the animations removed, let that decision stay. If the vendor no longer exists, well, do you really want software not designed for Vista to run on Vista? I realize how vital compatibility is to Windows’ adoption in many places, but this is Windows 6.0, not 5.3. Take a stand and show that Vista is the *new* Windows, not an upgraded Server 03 by any means. Of course, if you don’t already have an API for an app to use system animations, make one available. It will only help give Windows a visual coherency that will only be appreciated.
I understand your usual stance on compatibility, but in this case I would look forward arrogantly and leave these bugs to the vendor, extant or not. With Windows being the world-class software it is, it needs to look good too. Know why people like OS X so much? Well, look at it! Besides its technical features/accomplishments, it’s visually pleasing to use, and I believe MS is moving in the right direction to make Vista look like a 21st century OS.
On the flipside, how is Microsoft addressing this issue so it won’t happen again?
asdf: How do you propose preventing people from doing undocumented things? There was already a documented and supported mechanism (using the animations that came with Visual Studio), so "provide a documented alternative" already failed.
There is no documented alternative! There would be if using Visual Studio was a requirement for developing a Windows application, but it’s not. And while I haven’t looked at the animations that came with VS 2003, I recall that its icons were ancient and ugly, compared to what Windows provided. If the animations were of a similarly low quality they were no real alternative anyway.
Those animations are a part of Windows, and it’s a very reasonable desire to use consistent animations for common tasks in your own applications. Consistency is one of the fundamentals of Microsoft’s own GUI guidelines, after all.
Usually I agree with blaming developers for using undocumented features, but not here.
In this case, the fault lies clearly with the OS team for failing to provide a public API to access those resources. You might as well expect developers to write their own Open File dialogs.
If Microsoft make changes to their OS, that break mis-behaving applications, users will see this as Microsoft’s fault. Infact it is the problem of the applications’ developers, but users will normally unfairly blame MS first.
"My_favourite_app worked in Window 2000, but now it doesn’t work in Vista. Therefore MS broke it". This is how people think.
"There was already a documented and supported mechanism (using the animations that came with Visual Studio), so "provide a documented alternative" already failed. "
Eh – well, you know there are other development tools than Microsoft’s own, like Borland’s…
I agree with that the original problem here is the lack of an API to use the animations.
Hallvard: Although this might be nitpicking, the animations are actually included in most (if not all) of Borlands tools, atleast since Delphi 3. It’s actually even a VCL component.
marcus: That was the gist of Hallvard’s comment.
For frontend applications (or probably any app that won’t benefit from Win64, seeing as Win64 sadly won’t be supported by blundering Borland until 2007), Borland Delphi is the best dev tool there is. VS doesn’t come close. Shame really that MS don’t use Borland tools (but rather hire their people instead).
Raymond’s suggestion that filecopy animations are included with VS (presumably a copy rather than loading the OS resource like Delphi does), misses the mark, because the goal is to use the OS’ animation, not a copy of a legacy OS’ animation! Vista users expect apps to use the identical animations as the shell uses.
I enjoy reading Raymond’s blog, but a general trend seems to be Raymond saying "don’t do this", but then fail to come up with good suggestions on how to implement it properly. E.g. his entries concerning system dialog — so which font are we supposed to use? Searching MSDN for "MS Shell Dlg 2" returns some rather confusing documentation. (don’t worry, I’ll find the answer)
Sadly, this topic is certainly no exception. :(
—
Rune
"My abstracting default look/feel graphics into a public API…"
Heh… I meant "By abstracting default look/feel graphics into a public API…" Sorry. :)
One aspect of this that nobody else has touched on yet: presumably the "get video clip" function returns an HRESULT (or similar), so a well-written program should test for failure rather than assuming success?
So, if the app does that test (and handles failure sensibly), removing the video clip shouldn’t cause any major problems. Conversely, if the app doesn’t do that test, then the programmers have done more wrong than just "stealing" the clip (opinions seem to vary on whether that’s good/bad), and should get less sympathy.
As for the specific question, I’d echo other people’s views – tell the companies about the problem, and replace the video with a less flattering one in Vista.
Michael Teper said:
"I realize there will always be a way to dig into system code"
I am not sure about that. I may be overlooking something, but I think one can do run-time enforcement of the ‘MS Windows’-other stuff layer by introducing two different kinds of entry point: public ones and ‘for internal use only’ ones. A way to achieve that is as follows:
– digitally sign all Windows DLLs with a ‘made by Microsoft’ key
– Introduce a ‘for MS internal use only’ bit, either on a per-DLL basis or on a per-DLL entry point (function, resource, whatever) basis.
– Make the linker/loader obey that bit, by only allowing DLLs signed by the ‘made by Microsoft’ key to link with or load resources with the ‘for MS internal use only’ bit.
– Allow only Windows Update to install DLLs signed by the ‘made by Microsoft’ key
– Have the OS refuse to launch executables that are signed by the ‘made by Microsoft’ key (side effect would be that the Windows team can guarantee that the Office team does not use any ‘side entry’ into the OS)
Hacking around this would involve one of the following:
– have your installer toggle the ‘for MS internal use’ bit in a DLL
– crack the ‘made by Microsoft’ key, and sign your own DLLs with the key
I think it should not be hard too prevent either of those from happening accidentally. If someone does this on purpose, (s)he deserves his app to break on an update.
For extra safeguards, split the System32 directory into two: one with external entry points (for use by MS applications and third parties), and one with DLLs containing all ‘for MS internal use only’ stuff, seriously locking the second one. You might want to introduce a third one for 3rd party DLLs.
Reinder: Hacking around your proposal is much easier.
– parse the PE header of the DLL and extract the procedure address manually, then call it.
Windows 95’s kernel32.dll had a similar trick to prevent non-OS components from linking to the internal-only exports. It didn’t stop people for very long.
John C. Kirk said: "One aspect of this that nobody else has touched on yet: presumably the "get video clip" function returns an HRESULT (or similar), so a well-written program should test for failure rather than assuming success?"
Documentation for the relevant window message is at: http://msdn.microsoft.com/library/en-us/shellcc/platform/commctls/animation/messages/acm_open.asp
The function "returns nonzero if successful, or zero otherwise". So, there is a way for a developer to test for failure.
Bryan: Even if you isolate the internal functions, that internal DLL needs to talk to the "external" DLL through… an internal function! So the external DLL needs to expose an internal function anyway -> You’re back where you started.
Even if you manage to keep internal functions out of the external DLL, an app doesn’t need to load the internal DLL directly. It can load an external DLL that in turn loads the internal DLL. Now the internal DLL is in memory as code and the app can call it.
Well, looks like I get to re-iterate what others have said.
1) Don’t break old apps — backwards compatibility is one of the major reasons why Windows maintains its dominance of the OS market.
2) Fix the *reason* behind the problem. Folks want their UI to be consistent with the newest/latest/greatest version of the OS. *Users*, I mean, not programmers. My abstracting default look/feel graphics into a public API that could be used by programmers, users can retain known visual cues like, say, the file copy animation, regardless of the program or the version of the OS being used. The overall effect is a better experience for the user, which can only help Microsoft in general.
The phrase "It just works" actually represents people going to great lengths to accomplish this task seemingly simple task.
Unfortunately, if your company uses the phrase and doesn’t go the distance, "It JUST works" is what you end up with.
My opinion is that [you] go the distance: Honor their original desire to keep their app looking up to date by replacing the resource with one that looks good and fits in with Vista’s Theme.
Then, I would draft up a Theme-like API for System Animations and Icons. Then I would draft up an internal API / File structure to load these resources from disk. It’s not anti-trust if you let people use the resource, but control how and when it’s used.
This solves the problem of today and tomorrow. And I would thank [you].
But you have to load the DLL first, right? (LoadLibrary or something similar.) Otherwise the code won’t be there to run. LoadLibrary could check the signed-status of the calling code.
Doing it in LoadLibrary would mean you’d have to separate internal-use functions and resources from external-use ones, and put the internal ones in a separate DLL. At that point, the bit makes more sense if it’s put in the file header instead of per function.
Well, I suppose there’s still one way around it: if the program opens the file using the normal OpenFile/CreateFile/whatever APIs, and reads the data into memory, then tries to call the in-memory address. But that *should* be preventable if the OS marks data pages non-executable (DEP on), and doesn’t let the program override that. (If a program could override that, then so could malware if the user ran it directly, so it’s pretty pointless.)
This won’t prevent the EXE from using a resource — like an animation — though, because that’s just data. Not sure how to get around that one; there may not be any technical way.
(This may raise other issues too — some people may take it as a monopolistic type of action. They might claim that the office team, for instance, might be able to get its executables signed with this key, so they could use internal stuff, but outside developers (obviously) wouldn’t be able to do that. Even though we’re proposing that only a small number of MS teams would be able to get their code signed, that may not be the perception in the outside world. That’s a question for the MS legal people to look at, I think, not Raymond.)
And as for what Reinder said about not launching EXEs that have been signed at all — that may not be possible. A stock Windows install stuffs in a lot of services that are contained in EXEs, and these are quite internal. I am guessing that at least some of them would need access to the internal APIs.
cmd.exe *might* be another exception. Explorer too, *possibly*.
Oooh, wait a minute. If MS did this, it would force them to document any APIs or resources or whatever that Explorer, cmd, these services, and whatever else use. This would remove the cause of this particular issue (because either the animation resource would be non-internal, or there’d be a non-internal way of getting at it, which Explorer would use). So maybe it is possible. (Whether that documentation would be possible is another issue, though.)
> What I really think MS should spend some time
> on is trying to figure out ways to prevent
> this sort of thing from happening going
> forward. I realize there will always be a way
> to dig into system code, but if that path
> can be made sufficiently difficult (and
> therefore VERY VERY
> OBVIOUSLY unsupported), it may be easier to
> make future decisions about breaking
> compatibility.
You’re assuming that the cost will be identical for all developers, and I think you’re wrong in all but the initial case.
The cost for the first developer *will* be high. He’ll likely be a hacker (let’s call him Hacker Fred) of the E.S.R. school, who enjoys fiddling with OS internals. He’ll spend hours or days tinkering and testing, in order to make a particularly arcane undocumented API work the way he wants it to.
Once he’s done, Hacker Fred will post a blog article explaining in simple steps how he did what he did, because it’s ‘kewl’ … without necessarily explaining why it’s bad, or what the limitations are. That’s because he assumes that anyone using it would take the care to understand how it all works – or, perhaps, he just doesn’t care what it’d be used for – it was fun to do, and that’s what counts for him.
Then along comes Drone Joe, without the requisite skill or experience to understand exactly how Hacker Fred did what he did. He will see the article, think "hey, that’s a cool trick", and implement it in his production code without giving it a moment’s more thought.
What Microsoft needs to do isn’t try to obfuscate the way DLLs work or APIs are named – but provide a clear set of documented APIs, and explain that using APIs outside the documented set is likely to cause problems with future releases of Windows.
Oh wait, it does just that already.
Really, this isn’t a problem caused by Microsoft, it’s a problem caused by unprofessional developers, but which Microsoft gets the blame for & has to fix in order to maintain the fantastic reputation for backwards compatibility that Windows has achieved.
I’m curious about something. I’m not a GUI programmer so animations are waaay out of my league. I have seen several posts about how various animations are included with VS.NET 2003 in C:Program FilesMicrosoft Visual Studio .NET 2003Common7Graphicsvideos.
My question is – how would I, starting from pretty much zero knowlege, find out about those files and that location? I can’t find anything in the MSDN included with VS.NET about it. So unless I’m missing something (quite possible since I’ve only spent a few minutes looking), it seems to me that the animations included with VS.NET are also undocumented. You just have to happen to stumble on them or have someone in-the-know tell you.
If I search for "animations" I get way to many hits.
A search on "common animation" gets me a close hit (out of 216) for "Using an Animation Control (MFC)" – but that doesn’t have a pointer to the actual location of the AVI files or even a hint that such a thing exists.
And finally, a direct search for the answer – "common7 graphics videos" – nets zero hits.
How exactly is this ‘documented’ (leaving aside the question of trying this using non VS tools based only on the SDK)?
mikeb — I googled for "visual studio image library", and got this link:
http://msdn2.microsoft.com/en-us/library/ms247035.aspx
It’s short, but directly to the point.
Hmm. Raymond, you’re right, that would make it hard. If it were possible, I think it’d be a pretty nice way around the issue. But oh well, I suppose.
mikeb — I make it a habit to look around in the VS commonX directory whenever I install a new version of VS. But that’s only because a few years ago, I happened to see some icons and bitmaps in there when I installed an older version of VS; if I hadn’t seen that, I wouldn’t know it existed.
Randolpho — is that only for VS.Net 2005 (beta only, I believe)? I don’t see it in my VS.Net 2003. It looks like a good thing to have, though.
Raymond, I understand that Microsoft has a large interest in keeping existing and prior customers happy, but it seems that Microsoft is also constantly getting beaten up by a second audience for the crufty hacks it has to implement to ostensibly appease the former. Look at what Apple does. It sells you a box that "just works". The price is that when supreme commander decides it should all work differently, out goes your box and you have to buy, essentially, new physical hardware and new software (yes I realize this is changing recently but go with me). Now that has a lot of value to /new/ customers.
At some point I think Microsoft really needs to abandon backwards, arguably bug, compatibility. Otherwise it risks stagnation. It’s hard to innovate when you are chained to a mountain of history. I for one would gladly accept intentionally, purposefully, and obnoxiously breaking applications that do Stupid Things – I may have to upgrade some apps (or use that as a reason to switch apps), but if this is not done, the responsibility of each app is just dispersed into the operating system. I realize there are probably a lot of users of Microsoft software that will be unable to make a distinction, but I think at some point you just have to let it go.
Regarding some responses to how would I know about the VS animation library’s existence…
Randolpho: >> I googled for "visual studio image library", and got this link <<
This seems like cheating a few ways:
1) it seems like Googling after knowing the answer; I, for one, would probably have never thought to use those keywords
2) I don’t find any hits for VS products before VS 2005, so you can hardly fault someone who developed something a couple of years ago from missing that document.
3) If I search using those keywords in VS 2003 (using the July 2005 MSDN), I get 500 hits, which – at first glance anyway – are not helpful. Do we have to rely on Google for documentation on programming for MS platforms? Proabably not a bad idea in practice, but surely it shouldn’t be the definitive source.
On the other hand, this does now seem to be actually documented in VS 2005, as opposed to just some files that you might happen to stumble on.
Bryan: >> I make it a habit to look around in the VS commonX directory… <<
This is surely a good idea, but is it ‘documentation’?
I think those that are going to be hit hardest are Borland Delphi users. Included in the VCL is a component called TAnimate for displaying animated AVI’s. One "feature" is a property called "CommonAVI" described in the documentation as:
"Indicates whether the animation control displays a one of the common Windows AVI clips provided in Shell32.dll"
I.e. by setting a single property you can display the Windows copy, deletion or other animation.
No indication is given in their documentation that they have accessed an undocumented Windows resource.
I’m sure many developers, like myself, assumed that Delphi was simply wrapping a standard windows control and such usage was quite valid.
I guess I will wait for Vista to see whether i need to implement a custom animation ;-)
For the record, I think it is obviously in everyone’s best interest that Windows provides a standard set of animations and graphics for reuse by developers, because:
– Apps will look more consistent with Windows
– Newer windows versions can improve the styling of the library and instantly give all applications a face lift
– Developers can spend more time improving their software rather than creating their own crappy looking animations
– Smaller downloads because extra resources are not included in the installer (less internet traffic)
My vote is keep compatibility by keeping the old animations. For new animations/UI gimmickry, please be sure to provide an API so this situation can be avoided better in the future. =)
Why are you removing codecs from windows?
The problem isn’t due to codec removal; quite the opposite. The Windows Vista animation uses a *new* codec that the old programs don’t support.
I discussed the limitations of the SysAnimate control earlier this year: http://blogs.msdn.com/oldnewthing/archive/2005/02/16/374397.aspx
Remember also that there are multiple versions of the common controls library, so even if the version 7 common controls library supports the new codec, the old version 5’s still won’t.
<quote>But if the programmer had to do something in an undocumented way because no documented way existed, then work night and day to keep compatibility, and if you can’t make a future version of Windows compatible, cancel that version of Windows.</quote>
But there IS a documented way! Make your own animation. Oh, you want a documented way to get the MS animation? Well, there is no documented way for that since (drumroll) it is undocumented!
There are lots of ways to show animations in Windows. You can’t blame this on MS.
<quote>
There is no documented alternative! There would be if using Visual Studio was a requirement for developing a Windows application, but it’s not. And while I haven’t looked at the animations that came with VS 2003, I recall that its icons were ancient and ugly, compared to what Windows provided. If the animations were of a similarly low quality they were no real alternative anyway.
</quote>
You may not LIKE the alternative. But it is there. And no-one said you couldn’t solve it another way (by getting animations from elsewhere).
<quote>
Those animations are a part of Windows, and it’s a very reasonable desire to use consistent animations for common tasks in your own applications. Consistency is one of the fundamentals of Microsoft’s own GUI guidelines, after all.
</quote>
Think it through. If you apply this logic, MS could never change anything.
<quote>
Usually I agree with blaming developers for using undocumented features, but not here.
In this case, the fault lies clearly with the OS team for failing to provide a public API to access those resources.
</quote>
They don’t provide one since THEY DON’T WANT YOU TO DO IT. How complex is this folks?
<quote>
You might as well expect developers to write their own Open File dialogs.
</quote>
The fact remains, whether it was a good idea or not, they never said ‘YOU ARE ALLOWED TO DO THIS’. In fact, they said ‘DO THIS INSTEAD’.
"Should we develop a shim and hope that those four are the only programs that need it?"
If you develop a shim administrators and MS themselves can easily add applications to the "need shimming" list. Shimming is the best solution by far. The AppCompat database could be stuck on Windows Update and updated monthly or quarterly to add new fixes and expand the database of known fix requirements (in fact, I suspect that something like this is already done, as XPSP2, for example, has different AppCompat fixes than XP RTM).
"The Windows Vista animation uses a *new* codec that the old programs don’t support."
But, surely the SysAnimate control will support the new codec?
(I can understand that people who read the resource and try to interpret its bytes manually will run into problems, but if you merely pass the HINSTANCE and Resource ID to ACM_OPEN, I would hope that it would still work…)
"I discussed the limitations of the SysAnimate control earlier this year: http://blogs.msdn.com/oldnewthing/archive/2005/02/16/374397.aspx"
There’s no reason those limitations can’t be lifted for Vista. If Vista’s shell animations are now more complex, then the shell’s animation control should be upgraded to handle it. (At least, that’s the way I would have done it. Of course I’m not on the team so my opinion is just based on what I know/surmise :-)
"Remember also that there are multiple versions of the common controls library, so even if the version 7 common controls library supports the new codec, the old version 5’s still won’t."
Dang. Side-by-side is one of those things that seems good at the time, but here’s a classic example of where forward compatibility would be the way to go. (Nick Kramer was just talking about this earlier today: http://blogs.msdn.com/nickkramer/archive/2005/10/31/487151.aspx)
PatriotB: Yikes! Whoever wrote that MSDN article should be given a really stern talking-to… It’s already broken in Windows Vista (no shdoclc.dll).
"Aren’t "Everything is undocumented unless it’s documented" logical?".
Not really for a number of reasons. This line of reasoning is one of the things that gets Microsoft into trouble time and time again.
In fact, the way the world works is that developers will assume everything is documented unless you tell them it isn’t. Gigabytes of documentation will do that to you.
As counter-intuitive as this may seem, I think Microsoft needs to do a vastly better job of documenting which things are "undocumented".
A prominant list of things not to count on staying there would go a long way. Then it would be *documented* that someone is breaking the rules and there would be significantly less angst over whether to just break them when those things do change.
Of course, at the same time, it would be polite to provide a hyperlink to documentation of an *alternate* way to achieve similar functionality, and/or documentation promising to break things only in particular ways in the future.
Actually, the word "undocumented" is itself a cowards way out. What they should call these features is "unsupported" or "subject to change".
One MSDN article which has always intrigued me is "WebBrowser Customization" (http://msdn.microsoft.com/library/default.asp?url=/workshop/browser/hosting/wbcustomization.asp). In it, it talks about loading strings and menus directly from SHDOCLC.DLL–doing stuff that surely isn’t kosher and surely won’t be the same with Vista.
"The WebBrowser Control gets its shortcut menu resources from Shdoclc.dll. That knowledge and a few #defines gives you a chance to manipulate the browser’s menu…" Hmm… those #defines aren’t documented *any*where, except for that article. If nothing else, this article encourages the kind of mucking around that this blog entry discourages.
That article is one that raises more questions than it answers. How much other stuff is there that should be documented but isn’t?
Anyways, looks like this article is headed for the scrap heap with Vista…
PingBack from http://blogs.msdn.com/oldnewthing/archive/2006/08/17/704232.aspx