| Date: | May 16, 2007 / year-entry #172 |
| Tags: | other |
| Orig Link: | https://blogs.msdn.microsoft.com/oldnewthing/20070516-00/?p=26843 |
| Comments: | 59 |
| Summary: | Because most fonts don't work in command prompts. |
In Windows 95, we experimented with other fonts for the console window, and it was a disaster. In order to be a usable font for the console window, the font needs to be more than merely monospace. It also needs to support all the characters in the OEM code page. Testing this is easy for SBCS code pages, since they have only 256 characters. But for DBCS code pages, testing all the characters means testing tens of thousands of code points. The OEM code page test already rules out a lot of fonts, because the 437 code page (default in the United States) contains oddball characters like the box-drawing characters and a few astronomical symbols which most Windows fonts don't bother to include.
But checking whether the font supports all the necessary characters
is a red herring.
The most common reason why a font ends up unsuitable for use in
a console window is that the font contains characters with negative
A- or C-widths.
These A- and C-width values come from the
Consider, for example, the capital letter W. In many fonts, this character contains both under- and overhang:
Notice how the left and right stems "stick out" beyond the putative cell boundaries. I wrote code in Windows 95 to allow any monospace font to be used in console windows, and the ink was hardly even dry on the CD before the bugs started pouring in. "When I choose Courier New as my font, my console window looks like a Jackson Pollock painting with splotches of pixels everywhere, and parts of other characters get cut off." (Except that they didn't use words as nice as "splotches of pixels".) The reason is those overhang pixels. The console rendering model assumes each character fits neatly inside its fixed-sized cell. When a new character is written to a cell, the old cell is overprinted with the new character, but if the old character has overhang or underhang, those extra pixels are left behind since they "spilled over" the required cell and infected neighbor cells. Similarly, if a neighboring character "spilled over", those "spillover pixels" would get erased. The set of fonts that could be used in the console window was trimmed to the fonts that were tested and known to work acceptably in console windows. For English systems, this brought us down to Lucida Console and Terminal. "Why isn't there an interface for choosing a replacement font, with a big annoying message box warning you that 'Choosing a font not on the list above may result in really ugly results. Don't blame me!'?" First of all, because we know that nobody reads those warnings anyway. Second, because a poor choice of font results in the console window looking so ugly that everybody would rightly claim that it was a bug. "No, it's not a bug. You brought this upon yourself by choosing a font that results in painting artifacts when used in a console window." "Well, that's stupid. You should've stopped me from choosing a font that so clearly results in nonsense." And that's what we did. Of course, if you're a super-geek and are willing to shoulder the blame if the font you pick happens not to be suitable for use in a console window, you can follow the instructions in this Knowledge Base article to add your font to the list. But if you end up creating a work of modern art, well, you asked for it. Nitpicker's corner In the title of this entry, s/console windows/Windows console windows/† †s/Windows console windows/Windows console windows when displayed inside a GUI window, as opposed to consoles that have gone to hardware fullscreen, which is another matter entirely/. |
Comments (59)
Comments are closed. |
I really do enjoy the nipickers corner. Makes the articles that much more interesting.
Speaking of hardware fullscreen, why’d it disappear in Vista? Not that I use it very frequently or that it’s important to me, but last night I just felt like doing it with cmd and I got a message box saying "This system does not support fullscreen mode."
I’m using Consolas on consoles since some time and I quite like it. It doesn’t have the overhang problem, I don’t know if there are some chars missing in the extended set but for my use of cmd is good enough and I find it more readable.
Interesting article anyway, I always found strange not having a font selection but now I know the answer!
Jerry: I tried Consolas but it didn’t work, I supposed that it doesn’t meets the requirements depicted in the KB article. How did you get it working?
A question: is there any method to change the font of the console programmatically? even just from raster to lucida
[Semi off-topic warning]
Why in XP does the console window not use the themed non-client area drawing when Windows XP style is selected? The buttons are at least rescaled to the differing title bar height, but it still looks odd.
Linux terminal emulators and PuTTY have no problem using fonts like Courier and even non-fixed width fonts (if you feel like doing something like that). If they can do it, it seems to me that the Windows console can do it too.
Lorenzo: read the KB article. The font must have all those characteristics. I personally hacked the BitStream Vera Mono font to use in the Console. While the font is free, I doubt I can redistribute the hacked version. Find a font editor and edit the font of your choice.
Programmers by their very nature are nit pickers. Finding bugs forces you to that state.
Didn’t Windows 95 RTM only allow Courier New? I thought that it was the Plus! pack or Windows 98 that switched to LC — I seem to remember having to hack RTM to switch the font.
Unfortunately, I don’t think this line of reasoning is valid anymore, as with Windows XP ClearType enabled even Lucida Console will leave trails due to the filter applied to the text. Re-rendering the surrounding characters around a changed character with a cliprect would seem to be the next solution short of just re-rendering the entire window on each update.
Thanks for the explanation! Still I’m wondering if you couldn’t have changed the console renderer to work nicely with such overhanging pixels? If it wouldn’t just overwrite characters, but put new characters over old characters with some kind of alpha-blending, the characters should like nicely in most cases (except for cases like “WW” where the Ws would run into each other). I mean, Notepad works with monospace fonts as well and it seems to work well… what’s the difference to console?
Joel wrote "Linux terminal emulators and PuTTY have no problem using fonts like Courier and even non-fixed width fonts (if you feel like doing something like that)."
While that’s true now, I did have a version of "konsole" once that suffered from exactly the overhang/underhang bug that Raymond describes. I think it was the version that shipped with SuSE 7.1, whatever that was.
These programs you’re talking about are also substantially more recent than Windows 95, which is what Raymond is talking about working on. The performance concerns of having to overwrite neighbouring character cells on change would have been much more of an issue back in ’95 than they are for a modern terminal emulator. Back then, a large proportion of graphics hardware was completely unaccelerated and may even have been accessed across the 8MHz 16-bit ISA bus. rxvt may be a more suitable comparison than a modern program, as it was written to target similar hardware, and I believe it only supports raster fonts (it’s been a while, so I’m not sure…?).
I use Consolas also.
Once it was installed on my machine, I just selected it as the font. I think it meets all the criteria, since I didn’t have to modifying anything.
+1 using Consolas as the Console font. And you don’t have to edit the font file; it just works (TM) once you’ve changed the registry according to the KB article. One downside I’ve found is that the font config isn’t `global’; if you start a command-line application directly, and it opens in a new console, this console will have the default font, instead of the font you chose.
Thanks for this post Raymond. Ever since I changed the console font by editing the registry, I’ve wondered why such an arcane method where it would have taken one simple dialog :).
One question that remains is that why are the raster fonts set to display by default even in Windows XP considering that they no longer work with Unicode? Was that an oversight?
Additional follow up: is chcp supposed to by used on modern text-mode applications? Changing it to UTF8 mode results in some weird screen artifacts for the F7 history pop-up.
Is it not possible to programmatically probe the font itself to see if it has those types of characters? Granted, too much for this instance, but it would be in the universe of possible, albeit impractical, solutions, if true.
I tried using Consolas as the console font, but the version I had was missing all the box drawing characters. In my book, this is a showstopper; two-panel console-based file managers look ugly with borders composed of the “default glyph”.
Andale Mono makes for a good console font, although it suffers the overhang problem when rendered with ClearType.
Sorry if this is off topic, but is there a reason why the scrollbar in the console window is still unthemed, even in Vista? Vista finally gave us console windows with themed (e.g. Aero) window borders, but the scrollbar’s still looking very long in the tooth.
I have always assumed, perhaps incorrectly, that there is so much arcane, legacy code depending on particular (and probably undocumented) behaviour of the console window that MS dare not touch it in case they break it.
(Another Vista console question: What happened to being able to drop files on the console? That was really useful but it’s gone now, even for non-admin consoles.)
There’s lots I wish would improve about the console in addition to the cosmetics but that’s going way off topic. People have tried writing replacements but they don’t seem to work very well, I assume because of the arcane, legacy stuff that’s very hard to recreate, although how often do we run Edit.exe or anything that doesn’t simply printf text these days? Maybe I should try the alternative consoles again.
You brought it up, so now I’m curious.
Why is hardware fullscreen another matter entirely?
I understand a different method is used to get the pixels to show up on the screen. But aren’t you using the same mechanism to figure out where the pixels go?
<blockquote>Speaking of hardware fullscreen, why’d it disappear in Vista? Not that I use it very frequently or that it’s important to me, but last night I just felt like doing it with cmd and I got a message box saying "This system does not support fullscreen mode."</blockquote>
I imagine it has to do with the lack of unicode support in hardware fonts.
Hardware full-screen is an actual hardware text-only mode implemented by the video card, it is not a bitmapped display.
What does it say about me that I look forward to reading "Nitpicker’s corner" as soon as my feed reader informs me of new hotness on "The Old New Thing?"
Thanks, Raymond!
Thanks for clearing that up. I had wondered why things like Bitstream Mono didn’t appear in cmd.
Still, I’d do murder for a decent terminal emulator in Windows (something similar to urxvt would be wonderful…). I don’t suppose there’s any chance of Microsoft ever deciding to ditch the current one and write something new – possibly even one that doesn’t inexplicably ignore themes?
I feel bad about pointing this out, Raymond, but I don’t think you want a period at the end of your s/// expression.
why not just disable comments?
sounds like you should, if it’s causing you that much stress. but i don’t see why you let it affect you so personally. it’s just a sign of respect and admiration from them, really.
"That’s a period that ends a sentence. You people are driving me nuts. I’m tempted to just stop blogging for a year to get away from all this. -Raymond"
I personally like to draw as much attention as possible to incredibly stupid things people say (and, just as important, who said them) to me or others. Maybe you could create a comment hall of shame, featuring comments like that one :P
“sounds like you should, if it’s causing you that much stress. but i don’t see why you let it affect you so personally. it’s just a sign of respect and admiration from them, really.”
Seems like some kind of psychological disorder where he can’t ignore stupid, little stuff. One can only hope he isn’t like this in person, but it’s probably what makes him good at what he does.
Raymond,
You should stop this nitpicker’s corner. It is creating more noise. You are making it easier to nitpick.
You are doing a valuable service in this blog by sharing the knowledge that you have gained over years with thousands of people.
You don’t have any contract or commitment to provide this service.
No matter what you do, with free style commenting some one will always be there to nitpick or hijack the topic.
So if you need to take a break, that’s fine. Meanwhile there is already a wealth of information from existing entries that people are going to refer to for some time.
Now THAT’S an idea – Raymond sell commenter care support contracts. If people want to guarantee that he’ll respond to their comments, they have to pay for it. Hopefully that will either shut up the nitpickers or make some money on the side (or, if he’s really lucky, do both).
Meh, I personally have no issue with switching off comments most of the time. Sure, a blog is nice when it’s a dialogue, but if that dialogue is with infuriating nitpickers, perhaps it’s better to encourage them to bugger off.
I would have suggested having a write-in form instead so you could post the comments and respond to them, in effect doing a letter to the editor style post, but it occurs that it’s the wankers who nitpick who are going to be more likely to send you a stupid letter than the people whose comments add something to the discussion.
…maybe it’d be possible to mark some comments as ‘pedantry’, so they load with a background image of, say, a big stamped ‘PEDANT’? You probably don’t have the latitude (or the time) to edit the blog software so you can mark comments, though.
In any case, I’d love to see a solution to worthless posters because it’s something that worries me a lot. I’ve been very selective with the places I reply to on the net and haven’t run into this problem yet, so I’d love to know how to solve it.
Please do not feed the nitpickers.
Every time I see that nitpicker’s corner ("ZOMW LUL shudnt it be nitpickers’ cuz thers mor than 1 of us????"), I cringe, because I realize how difficult it must be when people deliberately misunderstand, misinterpret and generally just twist what you’re trying to say.
Granted, there are times when the nitpick is justified – I don’t recall the exact article, but on one occasion, because I wasn’t familiar with the subject matter, the nitpick helped clarify things for me – but usually, they’re just a waste of everyone’s time.
I’d like to suggest that you let your regular commenters police this kind of thing. If someone feels that a comment is a nitpick, let them add a new comment, pointing that out. This does away with the nitpicker being able to make accusations of being ignored. Likewise, if none of your commenters feel the comment is a nitpick, then perhaps there is some valid confusion regarding something you’ve said.
It really is a pity you don’t have control over the blogging software. But it’s a greater pity that you’d have a need to change it in the first place. :/
Thanks for all the effort you’ve put in thus far. I’m more informed and better educated for it, and I appreciate that. Thank you.
Raymond, ignore the pickers of nits if you can. If not here are a couple of ideas, I have no idea if they’ll work for you or not — each of these have some fairly obvious downsides I won’t point out, but maybe there’s something in there that strikes the right balance (and thanks for the blog, it’s always a good read).
Comment moderation.
Comment moderation by someone else (there’s got to be someone who’ll do this for a couple of bucks an hour, or the odd something from the Microsoft Shop or simply as a favour).
Declare June as "I’m not reading any comments month.".
Declare June as The Old New Thing Magazine Month (eg disable comments).
Delete comments that annoy you. It’s Your Blog(tm).
Does community server support a ranking system? If so enable it and experiment with different thresholds.
Can you add anything in the text near the comments section: "No nitpicking, please!"
As above but: "I will delete/ban you or your IP/subnet/country".
Declare June "The Old New Thing Clip Show", schedule some repeats, and don’t even look/think about it.
Change blog providers. No rules against it.
Run two blogs. First blog is RO or member-only, second blog is public free-for-all. Entries mirror from first to second.
It is a bitmapped display. Notice how the instructions in the boot menu for different language versions of Windows get corrupted to varying degrees when the system partition’s BOOTFONT.BIN file gets added or overwritten.
Win 2000 used the normal video card font. I don’t know why XP changed this, but I bet they just load a new font.
Notice that Vista instead really switches to graphics mode and that the bootloader there contains and supports TTF-fonts? How’s that?
Guess it’s time to shift from cmd.exe and ntvm.exe to Virtual PC or DosBox :-(
If Vista does not support hardware full text mode, will it support things like Duke Nukem 3D or Doom? Or is that disabled as well? Maybe they don’t care because with 64Bit it’s gone too.
Sorry for going off-topic and please just disable comments. A Raymond Chen magazine column is also a very good thing. Too bad that all the comments already affected very much what you write and leave out.
meh, methinks Raymond should chill out a bit over the comments. People are idiots on the internet, what’s new. Just ignore them. No big deal.
I just tried using Consolas and it does work. However, I’m getting the Italic font instead of the regular one.
Any ideas on why this happens?
> But for DBCS code pages, testing all the
> characters means testing tens of thousands of
> code points.
Thank you for being aware. A related observation is that the default list of console window fonts really isn’t Lucida Console and raster fonts, it contains fonts which *can* handle the default character set.
Wednesday, May 16, 2007 2:04 PM by Mozilla
> is chcp supposed to by used on modern text-
> mode applications? Changing it to UTF8 mode
> results in some weird screen artifacts for
> the F7 history pop-up.
Oh you’re right, XP still has chcp. It also changes the default list of selectable fonts. In MSDN versions of XP and Vista I had tried "mode con cp select=932" but got error messages, so I wonder if chcp will work there.
Wednesday, May 16, 2007 4:06 PM by jon
> Hardware full-screen is an actual hardware
> text-only mode implemented by the video card,
> it is not a bitmapped display.
It is a bitmapped display. Notice how the instructions in the boot menu for different language versions of Windows get corrupted to varying degrees when the system partition’s BOOTFONT.BIN file gets added or overwritten.
Although still having some glitches, this project: http://sourceforge.net/projects/console/ might make the handling of console windows a bit saner.
14pt Bitstream Vera Sans Mono works perfectly as a console font, though some font sizes behave weirdly. I highly recommend it
Thursday, May 17, 2007 4:09 AM by Christian
>
8×8, 8×14, and 8×16 would display only a tiny fraction of the characters that are displayed by the bootloaders of the NT and Vista series, by full-screen consoles in the NT and 9x series, and BSODs in the 9x series.
I’ll take your word for it that BOOTFONT.BIN can be loaded into the video card. I can partly understand if this means that the screen is divided into 16×16 bitmapped cells without the entire screen being bitmapped. But then the next question is how the BIOS displays bitmapped logos ^_^
Did not. It uses BOOTFONT.BIN, the same as XP and NT4. Notice how the instructions in the boot menu for different language versions of Windows 2000 get corrupted to varying degrees when the system partition’s BOOTFONT.BIN file gets added or overwritten.
Vista stores its boot font somewhere else.
Actually, this problem never existed in the X Window System (in 85, 10 years before 95 :-), because the font description was more complete, including bearing, ascent, descent, baseline, bearings… (see http://freetype.sourceforge.net/freetype2/docs/glyphs/glyphs-3.html ). Once you have this information, these kind of problems are impossible.
Of course, the other problems of having ugly font display (due to non-homogeneous bearings of individual glyphs for instance), and the few fonts offering all the needed characters (an nearly impossible task now in the days of Unicode) will still plague all the console app developers…
Nice article anyways, it is always enlightening to see these informative "behind the curtain" insights.
Pierre B.: IIRC, the Bitstream Vera font licenses allow you to redistribute the fonts, as long as you don’t name the changed font Bitstream Vera.
Norman Diamond: 8×8 character cell size is used for 80×50 text, 8×14 for 80×30 (rarely seen on VGA displays, though the same size was used for 80×25 on EGA), and 8×16 is used for 80×25 text mode. If you remember the EGA.CPI file from DOS/Win9x, it was used to change the look of these glyphs. The textmode display isn’t bitmapped, and eg. on Hercules cards (anybodey remembers them?), we used to replace the ROM to get national characters. The resolution of text mode on VGA is actually 720×480 (and the character cells are 9 pixels wide), but the last column on most characters is always empty, and when it isn’t, it just duplicates the 8th column.
The EPA logo in older BIOSes is made by changing the look of high-ASCII characters (if you have a parallel printer connected to the computer, try hitting PrintScreen while booting up, and you should get the printout of the POST screen with a block of characters where the EPA logo is displayed on-screen). Newer BIOSes usually use graphic mode, so this might not work anymore.
"I’ll take your word for it that BOOTFONT.BIN can be loaded into the video card. I can partly understand if this means that the screen is divided into 16×16 bitmapped cells without the entire screen being bitmapped. But then the next question is how the BIOS displays bitmapped logos"
I’d suggest you try Impulse Tracker ( http://www.lim.com.au/ImpulseTracker/ ). It’s a DOS music composer that features a GUI with mouse, waveform display, etc. Now here’s the big secret: IT’S ALL IN TEXT MODE. Run it; you won’t believe it until you see it.
I find that the console gets better and better. One nice new thing is with .Net 2.0, you can colorize your output. Nice if we had fonts also, but color is better in a way. I find I write small C# programs to extend the console’s function.
EGA.CPI is still around in Windowses up to XP, for fullscreen console sessions. You may find that various file protection mechanisms stop you from changing it, though.
Steve: The fullscreen rendering is not a new problem it has been around for a few years it has to do with the graphics hardware, some graphics cards drivers can’t display the full screen terminal. I ran into this problem with XP while working as an administrator for a company that used accounting software that was an old DOS program and on certain machines the program would hang if the user switched from the full screen console to windows and back again. When I changed the users settings to use the console window the problem did not appear anymore.
So your machine not being able to display fullscreen console in Vista suggests that NVidia have taken the final step of taking the support completely out of the driver
My machine does not have an NVidia card and I can enter the fullscreen console
PingBack from http://www.gooli.org/blog/blogs-i-frequent/
>
This can’t be the issue, since my video card is an onboard Intel, and fullscreen worked in XP on the same machine.
I can understand that at some point in the distant past, a release had to go out the door and some corners were cut, but it’s been a while since Windows 95 shipped.
There are open source products for Windows (for example, PuTTY and gVim) that can use fonts without leaving overhang droppings or performing poorly.
Van Dyke Software’s SecureCRT has has a fontpoop problem when using ClearType, but that seems to be the exception rather than the rule. It SEEMS like this problem has been solved.
For me, JP Software’s TCI (Tabbed Console Interface) has completely replaced the neglected Windows console, so I have a dropping-free console experience with a broad choice of fonts.
Steve: This works fine on my machine running Vista, no problem entering fullscreen mode for cmd or powershell
Thursday, May 17, 2007 8:21 AM by ender
OK, I believe you. Nonetheless the screen displays 16×16 in text mode, and the font comes from file BOOTFONT.BIN in the root of the system partition (in NT4, 2000, XP, and 2003). You can observe this by having NTLDR and/or NTDETECT.COM come from a different language version than BOOTFONT.BIN.
I didn’t notice EGA.CPI in Windows 9x but I’ll take your word for it that it was involved. So it yielded 16×16 glyphs that weren’t restricted to a subset of foreign characters. Fine, it did the same as BOOTFONT.BIN.
Colas: that’s not the point. The Windows console was designed to emulate VGA text mode in a window, so it expects to print characters as fixed-size cells of pixels, not as strings of typographic glyphs
A couple of people have mentioned above that PuTTY permits the use of any monospaced font, and yet doesn’t suffer from character droppings. I thought I’d post the reason why not, in case anyone’s interested.
The reason why PuTTY doesn’t suffer from droppings is because I devised a redraw policy which is proof against it. When drawing text in the console window, we always clip it to its theoretical containing rectangle, and then we remember that we drew that rectangle as a single unit and hence that the character cells within it may overlap each other. So the next time we draw anything that overlaps that rectangle, we make sure we redraw the entire rectangle, never only half of it.
So you’ll still occasionally see edges being chopped off characters which extend beyond their character cells, but not too much: within any string of text that was displayed in one go, the characters should look normal. And you should never see a dropping being left behind after its associated character has gone.
PuTTY is a truly awesome product, no two ways about it. I’ve heard of various hacks to turn it into a generic terminal emulator connected to a local process instead of a connection, but it seems to be a pretty involved. Have you considered building support for such a mode into the main PuTTY distribution?
Right now I use cygwin rxvt, which is okay, but it has no unicode support :(
My first thought was, although it would have been too slow in 95, redraw the entire window every time a character is changed.
After reading through the comments, I think a better solution would be, recommend bitmapped or Lucida Console fonts, but if another font is chosen, autodetect whether it contains characters with overhangs (which, as was pointed out, should be possible).
If it does have overhangs, then switch on a mode where when any character is changed, redraw not only its cell but the cells of all the characters around it.
The console window would then not perform as well with some fonts, but it would look good with all of them, and anyone could get the full performance by sticking to the recommended fonts. A big project, I’ll admit.
Thanks, Raymond, for the link to that KB article, and thanks to the commenters who pointed out Consolas, which I was unaware of.
I guess people who bother with the console font would bother with the console window in the first place. I asked myself for a long time why it’s so ugly and doesn’t follow the themes in Windows XP.
I got a partial answer from Mark’s book, Windows Internals. The drawing of console windows is one of the few pieces of the Windows subsystem that runs in user-mode. Non-console windows are drawn entirely in kernel mode.
I’d love to see my console windows look like all others, however. Looks like that has been solved in Vista.
PingBack from http://plexuk.co.uk/blogs/microsoft_products/archive/2008/01/08/why-aren-t-console-windows-themed-on-windows-xp.aspx
PingBack from http://rodonist.wordpress.com/2008/02/06/microsoft-way-or-why-windows-must-die/
PingBack from http://www.jeffhung.net/blog/articles/jeffhung/1114/