| Date: | September 6, 2005 / year-entry #252 |
| Tags: | tipssupport |
| Orig Link: | https://blogs.msdn.microsoft.com/oldnewthing/20050906-11/?p=34313 |
| Comments: | 34 |
| Summary: | If you are running Windows Server 2003, you owe it to yourself to enable the Volume Shadow Copy service. What this service does is periodically (according to a schedule you set) capture a snapshot of the files you specify so they can be recovered later. The copies are lazy: If a file doesn't change between... |
If you are running Windows Server 2003, you owe it to yourself to enable the Volume Shadow Copy service. What this service does is periodically (according to a schedule you set) capture a snapshot of the files you specify so they can be recovered later. The copies are lazy: If a file doesn't change between snapshots, a new copy isn't made. Up to 64 versions of a file can be recorded in the snapshot database. Bear this in mind when setting your snapshot schedule. If you take a snapshot twice a day, you're good for a month, but if you take a snapshot every minute, you get only an hour's worth of snapshots. You are trading off snapshot quality against quantity. Although I can count on my hand the number of times the Volume Shadow Copy service has saved my bacon, each time I needed it, it saved me at least a day's work. Typically, it's because I wasn't paying attention and deleted the wrong file. Once it was because I make some changes to a file and ended up making a bigger mess of things and would have been better off just returning to the version I had the previous day. I just click on "View previous versions of this folder" in the Tasks Pane, pick the snapshot from yesterday, and drag yesterday's version of the file to my desktop. Then I can take that file and compare it to the version I have now and reconcile the changes. In the case of a deleted file, I just click the "Restore" button and back to life it comes. (Be careful about using "Restore" for a file that still exists, however, because that will overwrite the current version with the snapshot version.) One tricky bit about viewing snapshots is that it works only on network drives. If you want to restore a file from a local hard drive, you'll need to either connect to the drive from another computer or (what I do) create a loopback connection and restore it via the loopback. Note that the Volume Shadow Copy service is not a replacement for backups. The shadow copies are kept on the drive itself, so if you lose the drive, you lose the shadow copies too. Given the ability of the Volume Shadow Copy service to go back in time and recover previous versions of a file, you're probably not surprised that the code name for the feature was "Timewarp". John, a colleague in security, points out that shadow copies provide a curious backdoor to the quota system. Although you have access to shadow copies of your file, they do not count against your quota. Counting them against your quota would be unfair since it is the system that created these files, not you. (Of course, this isn't a very useful way to circumvent quota, because the system will also delete shadow copies whenever it feels the urge.) |
Comments (34)
Comments are closed. |
Does this do anything that normal version control doesn’t? For instance, I put all my project files in <a href="http://subversion.tigris.org/">Subversion</a>, and commit whenever I’ve made a set of changes. If I make a mistake, I can roll back to any previous version.
Sorry for the munged-up link. That should have been:
http://subversion.tigris.org/
Windows XP also has a Volume Shadow Copy service, but I can’t find any UI to configure it. Does anyone know how to configure it, or if it supports the file recovery functionality Raymond is talking about?
Shadow Copies are transparent to the end user. They are more like mounting a Subversion repository via WebDAV.
With a version control system, you have check out and commit files.
What’s the motivation for supporting only network drives?
I always found *NIX solutions to versioning to be extremely over complex and lack good GUI intergration… It is almost like trying to use FTP with an extension to do your versioning.
Shadow copies (in this form) seems a lot more natural and human friendly, nobody needs to worry about hitting commit, thier files are just backed up for them.
Jonas Grumby: actually it works wonderfully on local drives too, except the GUI doesn’t support it
But if you 1) know the exact device name for the snapshot (each snapshot acts like its own read-only volume; device names are in the form "DeviceHarddiskVolumeShadowCopyX", X being a progressive number), 2) have a tool to create arbitrary device names (shadow copies aren’t registered with the mount manager, so you can’t use mountvol) and (optionally, only required to create globally-visible drive letters) 3) have write access to the GLOBAL?? directory in the object namespace, you can assign each of them a drive letter
Play with it with the vssadmin command line tool, have fun. Works great for backups too, since it lets you back up files opened in exclusive read mode (e.g. registry hives), and indeed ntbackup does support it (altough it’s been kind of broken in SP1)
It’s not the same as source control. Suppose you are working on something that will take two days to complete. After day 1, do you commit all your changes? No, because it doesn’t work yet. If you committed it, then everybody else on your project would have your half-finished job and the project would be broken.
And I suspect you don’t put every single file on your computer under version control. Imagine having to create a new project for every grocery list you write and every podcast you download…
I really liked how VMS handled file versioning. No UI was required since it was built into the file system.
bob.txt;1
bob.txt;2
bob.txt;3
etc.
With version control, I commit everything after 1 day, yes. That’s what branches are for.
This way, I can go back without negatively affecting other users. It’s simple.
Regarding the quota issue: I hope the system makes sure that cannot be used to perform a DoS attack by using more memory than my quota allows?
Carlos: I think the VSC on XP is just the "client" half of the Server 2003 feature, usually refered to as Previous Versions.
Jonus: I think the motivation behind only having a GUI for network shares is that the target audience were people storing profiles/document folders on a WS2003 server.
strik: You can’t DoS a server with VSC, because the server admin designates what percentage of drive space can be used for shadow copies. I believe Windows will also happily trash previous file versions if space becomes a premium.
Timewarp sounds a lot like the "Snapshot" functionality provided by NetApp filers (http://www.netapp.com/products/snapshot.html).
At Tellme we put home directories and CVS repositories on a NetApp filer and from any directory you can access a .snapshot "directory" (it’s not a real directory, so it won’t show up if you do something like "ls" or "readdir()" but you can get to it regardless). In there are hourly and nightly copies, up to a week back, of the files in the current directory.
As Raymond says I don’t think typically people use this functionality a lot, but each time you do, it saves you a LOT of time. It’s fantastic.
I use source control fairly religiously (I just use a personal repository in my home directory when I want only versioning, and not sharing), but I still find Snapshots indispensible for recovering from screwups.
Does anyone know if we will see this in Vista? I would love to be able to shadow a folder (instead of a drive), it would also be great if you could specify what location the shadow gets placed in.. That way you could have it shadowing your Documents folder constantly, so if you make any stupid changes or it corrupts on you it will be easy to go back to a previous version.
Note: Yes I am a lazy guy who would use it to backup documents ^Shrug^
strik: Well you have much better discipline than most people. Try convincing the Marketing department that they need to "branch" a file before they can edit it.
Carlos:
The version of Volume Shadow Copy Service in XP doesn’t support "persistent" shadow copies which survive across reboots. As a result, it doesn’t support the "view previous versions" or "timewarp" feature that Raymond is describing.
What it is used for (and Server 2003 does this as well) is for backups. When you start a backup using a backup app which supports VSS, a temporary shadow copy is created, and the backup is read from that instead of the live volume.
This serves two purposes: Allowing backup of files which would otherwise be difficult to back up because they were exclusively open or being written to during the backup process; and synchronizing the files so that all the files in a given backup are consistent with each other, because they all reflect their state at a precise point in time.
You can see VSS in action on XP by starting a backup using Ntbackup. The first thing you will see before the backup starts is a message telling you that it’s creating a shadow copy of the volume.
Incidentally, the offical abbreviation for Volume Shadow Copy Service is VSS (not VSC or VSCS). This may seem odd, but it has to do with a name change made during the development cycle.
strik, chris mear:
Version control and Volume Shadow Copy serve completely different usage scenarios – which explains why SourceSafe 2005 is still coming out.
Volume Shadow Copy is simplified source/version control "without thinking about or knowing about it" but at the same time, you lose the functionality (branching, merging, etc.) provided by subversion/cvs/vss.
The fact that it requires no client software, is automated, etc. makes it more useful for document shares on the network.
Wow. This removes yet another reason for using Novell Netware around here, knowing that Windows 2003 has the "Salvage" feature also… just by a different name.
A couple of comments:
The lazy copying is actually done at a block level. This distinction is transparent to a user, but it means that if one block of a file changes no other blocks need to be copied.
The 64 shadow copy limit is per volume. Each volume in your systems can have a maxiumum of 64 shadow copies.
Storage for shadow copies is on the same volume by default, but the system can be configured to store the shadow copies on a different volume. This doesn’t help you with reliability (if you lose either volume, you’ve lost your shadow copies), but it does improve performance.
In response to KJK::Hyperion
The internal device name of the shadow copy of the form DeviceHarddiskShadowCopyN is subject to change every time you reboot, making your mechanism very fragile. The correct way to expose a shadow copy is through the ExposeSnapshot API, which will expose the shadow copy in a more reliable way. Unfortunately, this API is currently blocked for this type of shadow copy.
With Volumen Shadow Copy you also lose the ability to commit files more frequently than what the administrator setup, to see the diffs between file versions or even to see who’s copy it is you’re looking at (if I see a file version 14 days old I have no idea if it’s different from the version 13 days old or even who saved that file). As Andy said, they serve completely different purposes.
On an off-topic note, why do people branch files just to work on them in source control, only to merge them back into the main tree? It looks to me like most developers (at least most I know) are still not used to work in teams, where changes happen at the same time, they all seem to want to play in their little sandbox, so they "don’t break anybody else’s changes". As far as I’ve seen trying to merge all these branches (I’ve seen teams of 10 or developers where everyone had their own branch) just creates many more problems than simply working in the main branch, which works quite well for thousands of open source projects and probably three or four closed source ones (but who can really tell).
Of course I could be an arse and point out Novell has had the equivilant of Shadow Copy since before 1995 (give or take).
It’s nice to see it, but jesh, it took almost 10 yrs.
Shadow Copy is great and I hope it’s available and enabled by default on Vista Home edition.
Or better yet, the home user is asked to optimize Vista for "speed" or "reliablitiy" and it will enable many settings, such as Shadowing.
This sounds much like the snapshots offered by Veritas, Linux’s LVM2, and other storage management systems. Nice to see it natively available in Windows, as it’s something I’ve often missed when working on Windows servers.
Copy-on-write snapshots are just wonderful when you’re backing up databases, mail servers, etc. Ask the service to finish its disk tasks and work from RAM, snapshot, tell the service it can continue writing. Nobody even notices the interruption.
So you have to specify the files to be shadow copied in advance?
Surely that’s one level of activity alone will mean most users will never use it until its too late (and I’m talking about Marketing et al here).
When I used to be a Netware admin back in the mid-90s it did this kind of thing, but automatically with all the files on the server (until it ran out of disk space, then it just discarded the oldest files first).
It was a life-saver for when we were on a nightly backup routine and users messed up files in the middle of the afternoon. It was command line originally, but easy enough that the biggest trouble makers picked up how to do it themselves pretty quickly, and once we started using the Win 95 clients it was only a right-click away.
You have to turn it on manually because on servers, everything defaults off for security reasons. (System Restore does something similar and everybody is furious that it’s on by default. Clearly this needs to be off by default too!)
I’ve used Executive Softwares Undelete for the same type of thing. It doesn’t do scheduled copies but instead catches each file modification and saves a copy to a different hidden folder. I set mine to keep the last 2Gb of changes, which is enough to undo most slipups.
As someone earlier pointed out, it sounds like VSC is closest to VMS’s file backups. Comparisons to cvs/sub are somewhat humorous. The quota question is interesting. Maybe VMS changed later, but when I was in school circa 1990, file versions definitely counted against my quota on the student VAX cluster. If my FORTRAN compile ran out of space I had to delete some backups (or some uuencoded usenet pr0n) to make room. I think our student accounts had 5000 block quotas. What are the pros and cons of putting VSC backup management into a grey zone? It seems that if they are accounted against your quota, you would have better control–the space is preallocated, so to speak, so any scavanging to reclaim freespace has to be self-directed. From a user’s point of view you know that backup copies won’t disappear just because some other user does something unrelated that causes the system to randomly reap backups. Is this how Novell’s "scavange" worked? Could VMS’s backup strategy be tuned to behave differently?
The server administrator specifies which files to build shadow copies for. I discussed deletion in the original article: "In the case of a deleted file". If a file moves it looks like a deletion and a creation. (Imagine the confusion if moving a folder moved its history.)
(Imagine the confusion if moving a folder moved its history.)
Imagine the confusion when it doesn’t. Microsoft is not the only company that’s shipped revision control software that doesn’t get this stuff right, but it’s no surprise that in 2005 they’re the ones still getting it wrong. A move is a move, you should be able to undo the /move/, but it’s still the same file, not "a deletion and a creation". How’d this come to be wrong? Microsoft doesn’t understand the difference between a /file/ and a filename, see also the inability to rename files that are "in use", and the consequent need to reboot Windows PCs all the time so that the OS can move a file out of the way…
As to the earlier comments, firstly yes, for most purposes this is equivalent to the existing snapshot functionality in your volume management system which everyone (again, except Microsoft) now seems to ship as the default way of handling fixed disks.
Secondly no, this isn’t version control, because it doesn’t know about versions. VMS and any revision control system keep every version, not just whatever you had 12, 24 and 36 hours ago (per the administrator’s arbitrary choices). If you do a lot of minor edits and once in a while you screw up big-time, you probably want (a decent version control system) and transparent (e.g. WebDAV) access, not this.
Okay, let’s walk through it.
File "dir1foo" exists on Jan1.
On Jan2 it is moved to "dir2foo".
On Jan3, you go to "dir1" and say "View previous versions of this directory" and select Jan1.
You expect to see foo there because foo existed in dir1 on Jan1.
Therefore, the Jan1 history for dir1foo needs to stay in dir1.
On Jan3, you go to "dir2" and say "View previous versions of this directory’ and select Jan1.
You do not expect to see foo there because foo did not exist in dir2 on Jan1.
Therefore the Jan1 history for dir2foo should not exist in dir2.
Volume shadowing is not version control and is not intended to replace version control. Volume shadowing is just a collection of recent volume snapshots.
(Enlighten me since I don’t know: Does "everyone" else really already implement automatic volume snapshots by default? Linux has "lvcreate" and Solaris "fssnap", but these are manual processes, not an automatic one, and these snapshots do not persist across reboots. I don’t see anything for the Mac or FreeBSD but then again I didn’t look very hard.)
The Plan9 file server had exactly the same feature in 1995.
See for instance
http://plan9.bell-labs.com/sys/doc/9.html
and search "The File Server"
Not all folders in the user profile are stored in the user profile.
I’m a big fan of being productive by not losing work. I don’t care how optimized your system and development