The formats for scalable fonts used by XFree86, X.Org and Keith Packard's FreeDesktop X server, TrueType, OpenType and Type 1, as well as the FreeType library used for accessing them, have a number of desirable features that both users and system administrators have become used to. The bitmap formats (notably PCF) and the core bitmap backends, on the other hand, show their age by failing to have a number of these characteristics.
Very roughly speaking, the PCF format is a memory dump of the data structures used by older versions of the X.Org SI (and by XFree86 prior to version 3.9.15) for representing bitmap fonts. These data structures represent a font as a linear vector of 12-byte glyph structures.
Modern fonts are usually encoded according to Unicode, a 21-bit encoding that uniquely identifies characters. A Unicode-encoded font will typically have glyphs with indices in the range of 0 to 65000 even if it only contains a couple hundred glyphs. Representing such a font in the PCF format imposes an overhead of over 750 KB.
XFree86 works around this issue by compressing PCF fonts using gzip.
The traditional font model in X11 assumes a one-to-one correspondence between font files and client-visible font instances. In particular, if a client sees multiple encodings of a single font (for example, ISO 8859-1 and ISO 8859-2 versions of 12pt Courier), they correspond to multiple font files.
In the model implied by Type 1 and TrueType/OpenType fonts, a font file is merely a collection of glyphs which does not presume on the order in which these glyphs are presented to the client. A single font file can be presented to the client under multiple encodings, and this is what XFree86 3.9.15 and later does with scalable fonts.
Presenting a font under a given encoding usually implies that the font is subsetted, i.e. that only some of the glyphs it contains are actually presented to the client. Both the Type 1 and TrueType/OpenType formats allow a rasteriser to only load into memory those glyphs of a font that it needs. Additionally, the TrueType/OpenType format is a random access format, allowing the rasteriser to only read those parts of a font file that it is interested in. (The Type 1 format requires the rasteriser to read even those parts that it will discard.)
While the PCF format would in principle allow random access, for reasons outlined above PCF fonts are usually compressed, and thus need to be read sequentially. Due to a further mis-design of the PCF format, it is not possible to efficiently subset a PCF font when reading it sequentially — memory needs to be temporarily allocated for the whole font, the whole font needs to be read in, and only later can the unused parts be discarded.
(The problem is quite subtle — information on the type of encoding used, linear or matrix, only appears after the glyph data. Thus, at the time when the glyph data is being read in, it is impossible to determine what codepoint a given glyph corresponds to.)
Modern font formats, notably OpenType, include a large amount of metadata together with the glyph data. The PCF format is limited to unstructured metadata in the form of non-standard “font properties”.
The PCF format is no longer suitable as the standard format for bitmap fonts, and it is being replaced. This change will not involve a flag day: PCF will still be supported for the foreseeable future.
We will use “bitmap-only TrueType fonts”, more properly known as “bitmap fonts in a SFNT wrapper” as the preferred format for bitmap fonts. This format has all the features that we require:
A fonttosfnt utility for converting legacy bitmap fonts to the bitmap-only SFNT format is available from my web site. A tweaked version is available in Keith Packard's CVS repository.
Current versions of the FreeType library are able to use bitmap-only SFNT fonts out of the box. Thus, most Xft-based applications should be able to deal with such fonts without change.
(Note, however, that currently FreeType requires glyf and loca tables to be present in such fonts. By default, fonttosfnt generates fonts with an empty glyf table.)
Switching to a new font format for bitmap fonts makes the current bitmap backends obsolete. A version of the FreeType backend that can make use of bitmap fonts (both in the SFNT format and in the legacy PCF and BDF formats) is included in XFree86 and X.Org CVS.
Contrary to what is implied above, this version of the FreeType backend does perform on-the-fly reencoding for fonts in the legacy PCF and BDF formats, but will do so very slowly, especially if they are compressed.
(The new backend does not support the SNF format, which has been obsolete for as long as anyone can remember. If you still use this format, it is high time to move into the 1980s.)
The fontfile interfaces — the part of the X server that deals with local font backends — contain the assumption that there is a one-to-one correspondence between bitmap font files and font instances (this is the assumption on which the design of the mkfontdir utility is based). The simplest way of lifting this limitation is to deprecate the traditional bitmap interface to font backends, and make the scalable interface the only one used. This can fortunately be done without breaking compatibility with legacy backends.
Such a change breaks the mkfontdir utility, which has been rewritten from scratch.
(The new interfaces consider two types of backends. A “traditional” backend has an “Open” method and optionally an “OpenScalable” method; the former is used for non-scalable entries, the latter, if present, for scalable ones. A “new-style” backend only provides an “OpenScalable” method, which is used for both scalable and non-scalable entries.
What about non-XLFD entries? Currently, we fail them, on the assumption that the only non-XLFD entries are aliases.)
The introcution of Fontconfig/Xft in XFree86 4.2 provided a standard means of locating fonts. The core fonts system does not use Fontconfig; instead, core fonts must still be enumerated using fonts.dir files. Having two distinct configuration mechanisms is likely to confuse users.
In the core fonts system, and object that enumerates fonts is known as a Font Path Element (FPE). I suggest that a new class of FPE, the Fontconfig FPE, should be implemented. A user who wants to use Fontconfig for locating fonts will simply set his font path to “fontconfig”.
Such an FPE will need a few extensions to be made to fontconfig, and Keith Packard, the Fontconfig maintainer, appears willing to make such an addition to Fontconfig.
A preliminary version of the Fontconfig FPE has been conjured, and I think it sometimes works. Feel free to grab a copy if you want your share of the crashes.
(Keith is the body who originally suggested that a Fontconfig-based FPE should be implemented.)
After some events at XFree86 that are outside the scope of this document, I have moved on to greener pastures, and no longer hack at X fonts. Here is the status of the transition at the time I left: