| Date: | September 9, 2005 / year-entry #257 |
| Tags: | code |
| Orig Link: | https://blogs.msdn.microsoft.com/oldnewthing/20050909-24/?p=34263 |
| Comments: | 48 |
| Summary: | The FOR command has become the batch language's looping construct. If you ask for help via FOR /? you can see all the ways it has become overloaded. For example, you can read the output of a command by using the for command. FOR /F "tokens=*" %i IN ('ver') DO echo %i The /F switch... |
The
FOR /F "tokens=*" %i IN ('ver') DO echo %i
The /F switch in conjunction with the single quotation
marks indicates that the quoted string is a command to run, whose output
is then to be parsed and returned in the specified variable (or variables).
The option The kludgy batch language gets even kludgier. Why is the batch language such a grammatical mess? Backwards compatibility. Any change to the batch language cannot break compatibility with the millions of batch programs already in existence. Such batch files are burned onto millions of CDs (you'd be surprised how many commercial programs use batch files, particularly as part of their installation process). They're also run by corporations around the world to get their day-to-day work done. Plus of course the batch files written by people like you and me to do a wide variety of things. Any change to the batch language must keep these batch files running. Of course, one could invent a brand new batch language, let's call it Batch² for the sake of discussion, and thereby be rid of the backwards compatibility constraints. But with that decision come different obstacles. Suppose you have a 500-line batch file and you want to add one little feature to it, but that new feature is available only in Batch². Does this mean that you have to do a complete rewrite of your batch program into Batch²? Your company spent years tweaking this batch file over the years. (And by "tweaking" I might mean "turning into a plate of spaghetti".) Do you want to take the risk of introducing who-knows-how-many bugs and breaking various obscure features as part of the rewrite into Batch²? Suppose you decide to bite the bullet and rewrite. Oh, but Batch² is available only in more recent versions of Windows. Do you tell your customers, "We don't support the older versions of Windows any more"? Or do you bite another bullet and say, "We support only versions of Windows that have Batch²"? I'm not saying that it won't happen. (In fact, I'm under the impression that there are already efforts to design a new command console language with an entirely new grammar. Said effort might even be presenting at the PDC in a few days.) I'm just explaining why the classic batch language is such a mess. Welcome to evolution. [Raymond is currently away; this message was pre-recorded.] |
Comments (48)
Comments are closed. |
"Of course, one could invent a brand new batch language, let’s call it Batch² for the sake of discussion, and thereby be rid of the backwards compatibility constraints. But with that decision come different obstacles. "
I don’t think these obstacles need to be explained to anybody with any dependance on VB6 code.
I know it’s hard to conceive of, but Microsoft doesn’t actually have to solve every problem under the sun. Perl is a great scripting language, and is portable to Unix systems. There’s also Python, Ruby, tcsh; all are free, I believe.
I’ll grant the big advantage of knowing your language is already installed (.BAT).
I think that calls for command namespaces.
Yeah, but aren’t these new batch features added on a pretty ad-hoc basis already? I don’t recall having access to FOR/F under Win98, for instance.
What if you were to provide a converter along with the new batch language? You run your old batch program through the converter, and voila, you’re now using the new langauge…
but why the hell was the choice command removed from the NT line?
I hate to admit it, but the scripting languages in Unix blow away what Microsoft offers. It would be awesome if a future version of Windows included the scripting capabilities of tcsh.
New batch language? I thought it was already here in WSH and cscript.exe.
It’s probably the legacy and excess backward compability, but every time I try to do something with BATs, it almost get the feeling that the syntax has delibarately been designed such that you can’t do anything useful with it. For example:
FOR /F "tokens=*" %i IN (‘ver|find version’) DO echo %i
| was unexpected at this time.
Why on Earth can’t you use | there?
This is just one example out of million. Fortunately we have cygwin these days…
I’m with AndyB: these days, if I have the urge to write a batch file, I write javascript and execute with cscript.exe.
However quirky (a lot!), I love the batch language! my only gripe is that there’s no clean way to break out of a for loop. Sure, it looks slightly more like line noise than Perl (can you tell what does the below code do?), but you can do very interesting stuff with it:
@echo off
setlocal
setlocal enableextensions
setlocal enabledelayedexpansion
set startdir=%1
if .!startdir!. == .. ( set startdir=!CD! )
call :visit "!startdir!"
goto :EOF
:visit
setlocal
set dir="%~dpnx1"
pushd !dir! && ( ( ^
for /f "tokens=*" %%D in (‘dir /ad/b’) do ( ^
call :visit "%%D" "%%~aD" ) ) & popd ) || ^
goto :EOF
( echo %~a1| findstr /R /X ……..l 2>&1 > NUL ) || ( ^
rd !dir! 2> NUL && ( echo !dir:~1,-1!>&2 || exit /b ) )
KJK::Hyperion: Thanks for the tip.
"despite not being really documented anywhere, it’s not hard to understand by long and painful trial-and-error" – Exactly what I mean :) Trial-and-error is what I’ve been doing, Trial-and-eventual-success muvh much less. BAT must have the lowest productivity rate among the languages I’ve ever used (and that includes AutoLISP…)
Regarding find vs. findstr – my mistake, didn’t verify that.
"I hate to admit it, but the scripting languages in Unix blow away what Microsoft offers. It would be awesome if a future version of Windows included the scripting capabilities of tcsh."
What do you need that WSH doesn’t provide?
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/script56/html/wsoriWindowsScriptHost.asp
"How did one get the output of a program (or input from the user) into a variable in the DOS era?"
For simple things, my approach was to redirect output into another batch file that set the variable, and then call the second batch file. It’d look something like this (making allowances for long file names):
copy __set_cd_template.bat __set_cd.bat
cd >> __set_cd.bat
call __set_cd.bat
del __set_cd.bat
__set_cd_template.bat would contain something like the text "set CURRENT_DIRECTORY=^Z". The lack of a space between the equal sign and the EOF (^Z) was important. Otherwise, the output from the file got redirected into a second line.
You could also do interesting things with edlin. I remember pretty clearly implementing the equivalent of pushd/popd with edlin and batch files. pushd would append a line that read "CD<current_directory>" to a global batch file in a well known location. popd would call the global batch file and use edlin to truncate the last line. It would (quickly, behind the scenes) take you through every directory in the stack, but you’d end up where you were when you said ‘pushd’.
Around the time time, chkdsk would report percentage fragmentation of the files in a given directory. I built a set of batch files atop pushd/popd that traversed the entire directory hierarchy of a hard disk and dumped a report of the fragmentation percentages in a well known location.
(Or you could use Norton… :-)
I used to work at a company whose complex build system ran entirely on 4DOS/4NT batch files. It was probably the first system that was designed in-house back in the DOS days, and they just stuck with it. It scaled horribly (ie, not at all) and was hard for newbies to learn, but since every product used it, no one wanted to change it. So breaking batch files would literally make the company unable to function.
Perl is not good as a replacement. You can’t easy use it on CD or etc. Lot’s of files needed to run it. (ok actually I have single-file perl4.exe for DOS ;)
And it is not batch language – It’s not easy to generate programmatically perl script which will be run next. It’s easy to do with BAT.
Thanks all for hints on JS/BAT/find* and explanations of BAT parsing.
Nekto2: "Perl is not good as a replacement. You can’t easy use it on CD or etc."
I like Perl, use it daily, but this is 100% true.
Then I have discovered KiXtart (http://www.kixtart.org/). Not perfect, or as powerfull as Perl, but worth more than a look.
mschaef: the
copy sethack.bat tmp.bat
foo >> tmp
call tmp.bat
del tmp.bat
hack is what I used, too :). But if I recall correctly ancient DOS versions did not even have CALL… :)
"Suppose you decide to bite the bullet and rewrite. Oh, but Batch² is available only in more recent versions of Windows. Do you tell your customers, "We don’t support the older versions of Windows any more"? Or do you bite another bullet and say, "We support only versions of Windows that have Batch²"?"
So all features available in batch files under WinXP are also backported to Windows 95? I find that hard to believe…
"Welcome to evolution."
So we can agree it’s not Intelligent Design? [wag]
" but why the hell was the choice command removed from the NT line?"
Because you can accomplish the same thing with set /p:
set /p answer=Select from [a, b, c]:
if %answer% == "a"…
" …"Those who don’t know Unix are doomed to reinvent it, poorly.""
The general concensus from what I’ve read is that MSH (aka Monad) is as (or more) powerful a shell than anything offered by *nix.
"So all features available in batch files under WinXP are also backported to Windows 95? I find that hard to believe…"
If you read all of what Raymond wrote, you see that the argument is that any batch file will run in Windows XP and 2003, not the other way around. If Windows XP and 2003 used another shell, with another batch language syntax, no batch file from prior to XP would run in them. In other words, if any version of Windows to ship with Monah (MSH) expected .bat files to contain Monah (MSH) syntax, all existing .bat files would break.
From the link provided…
""Once you have access to the beta site, download the beta version of the .NET Framework 2.0 (a 24mb download), and the MSH Preview version (4.1 mb) from the downloads page. Once downloaded, run Dotnetfx.exe to install the .NET Framework, then run Windows command shell preview.exe to install MSH.""
I consider 28Mb of downloads (or inclusions on CD/DVD) and an installation of the .NET framework at a minimum, just to get a new shell scripting language is a little too heavy. One of the light-weight standalone scripting languages around seems a better solution. e.g. 4DOS (although it is not free) or one of the programs listed above.
Did anybody say "Cygwin"?
Raymond just explained a cool trick possible with batch files and noted how popular they are inspite of being so ‘Kludgy’. *Nobody* mentioned anything about competing with other scripting languages. LOL Perl ! Ruby !
Btw, given this is The *Old* New Thing, let me ask an outdated question. How did one get the output of a program (or input from the user) into a variable in the DOS era? That is, before the WinNT era FOR /F and other modern folly.
bat man: you need to escape the "|". Try to get familiar with the extremely idiosyncratic and extremely quirky multi-stage parsing of batch files (for example: don’t *ever* try to nest FOR loops if you use the !variable! syntax – string the loops in a pipeline instead); despite not being really documented anywhere, it’s not hard to understand by long and painful trial-and-error
It helps to know, for example, that the batch language has *no* constructs whatsoever (with the possible exception of labels) – rem, echo, for, if, etc. are all built-in commands; being built-in means, among other things, that they have a custom parser (note how echo outputs the string argument verbatim, preserving whitespace), and that they are handled by a slightly different tokenizer (which is why "cd.." is equivalent to "cd .." but "findstr.exe" isn’t equivalent to "findstr .exe"); even the parenthesis isn’t a construct, it’s a built-in command with a custom parser that handles line breaks itself, which is why you can write this:
if .%var%. == .. (
command
command
) else (
command
)
but not this:
if .%var%. == ..
(
command
command
)
else
(
command
)
since the "if" built-in doesn’t handle line breaks in any special way
In your case what you *really* mean is:
FOR /F "tokens=*" %I IN (‘ver^|findstr /i version’) DO @echo %I
Note the use of uppercase letters for variables (it’s unambiguous if you’ll ever use the %~xxxX syntax, and chanches are you will), the escaped "|", the use of findstr instead of find (find can only search disk files), and the @ to suppress the echoing of the echo command
…"Those who don’t know Unix are doomed to reinvent it, poorly."
;-)
About find vs. findstr — find can search more than just disk files. For example, just today I’ve done a:
netstat -n | find ":25"
to see the states of all the open, closed, etc. SMTP connections on a 2K Server box. Unless you have to use findstr in XP? If so, then *that* reeks of backwards-incompatibility…
One of the things not mentioned is that batch files can’t be self modifying. (Or rather they can if you are extremely careful).
The batch file is closed before each command is executed, and then reopened and the next command read from the absolute location that the previous one ended. You would get wierd errors if you moved the contents of the file around at all.
Cooney wrote on Friday, September 09, 2005 6:29 PM:
> Why is that? We have established
> methods for dealing with this already.
> Leave standard batch files alone and
> make something new and clean.
There is something new and clean already. As many others pointed, it’s called WSH (Windows Scripting Host). WSH exists since Win98 and has two excellent scripting languages out of the box: JScript and VBScript. Implementation of other popular languages can be found easily, including Tcl, Python and Perl, so beloved by Unix admirers (search web for ActivePerl).
Personally, I almost always prefer WSH over batch files. I agree with others here that batch script has done its job well for many years, but now it’s dead-end path of scripting evolution.
Moreover, I find batch script as counterproductive quite often, because it gives you illusion that no programming efforts are required; "it’s just few CMD commands away" thinks typical developer and starts typing. Then he/she needs to peek in documentation before typing almost every command, because nobody remembers cumbersome syntax. Then developer is stunned by results of first run and starts to debug it (with echo, of course, there is nothing else). Then poor guy cursing his way through opaque rules of parsing and execution context of batch scripts. After couple of hours of exhausting coding it works finally. However, nobody can understand it from first glance including the author. What about maintenance now? That’s why I don’t like batch scripts.
There is mental barrier that developers often need to overcome before start writing JScript or Perl: it looks like "programming". It has functions and variables, debuggers, even editors with syntax highlight. It makes impression as full blown programming in contrast with "simple" batch files. Many times people think "what? I won’t start coding now for such simple task; there is batch script for that!", but they often forget that tasks tend to expand quickly (usually as early as you’re writing very first solution) and batch script reaches its limits even quicker. As a result, coding efforts invested in batch scripts are much higher than those for JScript/Perl for comparable tasks. And I even don’t want to think about batch script maintenance vs JScript maintenance.
"You pipe text around."
Actually, often binary data is piped around, for instance, on some *nix systems, where ‘tar’ dosent support gzip directly, the following command will extract a .tar.gz archive:
gzcat file.tar.gz | tar xvf –
> The general concensus from what I’ve read is that MSH (aka Monad) is as (or more) powerful a shell than anything offered by *nix.
I’d love to see evidence of that. However, by stating "shell," scripting languages that don’t have their own shell are excluded. Perl, for example, is ubiquitous on *nix, but there isn’t a perl shell. (Incidently, TCL/TK does have a shell, tclsh, so a comparison with it would be warranted.)
I also feel the need to pull up a quote from Jeffrey Snover, Architect for Administration Experience Platform (and one of Monad’s lead developers).
"So, Monad is a way to automate the system. It has four components. First is it’s as interactive and composable as kshell or bash. So if you’re familiar with those types of shells, we have those capabilities. It’s also as programmatic as say Perl or Ruby. Third is we’ve focused in on some production-oriented capabilities of it, like VMS’s DCL or the AS400. We’re really focused in on trying to solve admins’ problems. And then fourth, we go and we take all the management information in your system and make it as easy to find as files in the file system." — http://msdn.microsoft.com/theshow/transcripts/Episode043Transcript.aspx
One and two are already covered by *nix, as is the fourth. In fact, the fourth is one of UNIX’s strong points, because process and device information already appear as files in the file system (see /dev and /proc).
The third point seems to deal with DCL’s ties to VMS’s security system… but in *nix this isn’t all that necessary, because security settings are already managed through shell utilities. Even ACL lists are managed that way by the *nix systems that support them (Linux and some of the BSDs).
Anyway, I think I’ve said enough on the subject, because this is starting to get rambly.
>> The general concensus from what I’ve read is that MSH (aka Monad) is as (or more) powerful a shell than anything offered by *nix.
>I’d love to see evidence of that. However, by stating "shell," scripting languages that don’t have their own shell are excluded. Perl, for example, is ubiquitous on *nix, but there isn’t a perl shell. (Incidently, TCL/TK does have a shell, tclsh, so a comparison with it would be warranted.)
Why is it more powerful? Because the script languages in traditional UNIX is based on pure text streams. You pipe text around, which you then have to parse. MSH pipes *objects* around instead, which make it far more powerful. In fact, these are .NET objects. And if you know .NET then you can do a lot of magic in there :)
Some videos of "Monad" are available at Channel9: http://channel9.msdn.com/tags/Monad
And they also have a blog where you can read more about it: http://blogs.msdn.com/monad/.
Anyways, I really like it :]
Raymond: you can’t use that "backward compatability" excuse for all idiotic "features" that windows have. And with batch files windows misses the point big time.
It is hard to believe, but *nixes have virtually unchanged shell languages for about 30 years. And it is all backward compatible.
So please, have a respect for your readers: call things by their name. Bug is a bug and mistake is a mistake and mess is a mess. And batch files are design mistake – plain and simple.
YEAH!!!
I wish the tar and gzip people used XML so I wouldn’t have to create my own parsers. I hate it when those unix guys can’t follow a standard.
note: sarcasm at many levels.
O…K… I’m thoroughly confused. ;-)
> You pipe text around, which you then have to parse
No, *you* (as the programmer) don’t usually have to parse it, unless you dream up your own file format(s). That’s what expat, libxml2, Lisp’s s-expression interpreter, etc. are for.
As for the utility of passing objects around, yes, sometimes it would be helpful (but then, in most scripting languages, you’d just use XMLRPC, and in Lisp, you’d use s-expressions), but not always.
Especially not when the format of the objects you’re passing around is undocumented, and binary, and you’re having problems with whatever’s going on on one side or the other. It’s really easy in Unix to dump the output of a command to a file and look at it in a text editor; it’s not too much more difficult to redirect an XMLRPC "call" to a file and look at it. It’s also not too difficult to dump a bunch of s-expressions to a file. And if you know a bit about XML, or the structure of Lisp, you can generally figure out what the problem is.
Whereas, what do you do if you’re having problems with the way MSH is passing objects?
The difference is transparency.
I have yet to find anything that beats the power of the Bash shell. Combine that with say, Perl/Python/Ruby/whatever, and you have the most complete shell environment you could wish for.
Real men use c
http://root.cern.ch/root/Cint.html
Backwards compatibility means being compatible with your mistakes, too.
#! means never having to say you’re sorry.
/couldn’t resist
PingBack from http://blogs.msdn.com/oldnewthing/archive/2006/03/22/558007.aspx
PingBack from http://blogs.msdn.com/oldnewthing/archive/2006/03/22/558012.aspx
PingBack from http://blogs.msdn.com/oldnewthing/archive/2006/04/27/585047.aspx