Formatting Font Formats

Page 1

Luc Devroye

McGill University, Montréal, Canada H3A 2K6

luc@cs.mgill.ca

Abstract

Font formats are a tug of war between artists (designers and drawers), programmers (computer

scientists), the business world, and users. Each of these four groups has had an influence on the

path that font formats have followed. We review the successes and failures, and present a wish list

of properties that a good font format should have.

Résumé

Les formats de fontes ont depuis toujours été un tir à la corde entre les artistes (graphistes et des-

sinateurs de fontes), les programmeurs (informaticiens), le monde des affaires et les utilisateurs.

Chacun parmi ces groupes a influencé l’itinéraire historique que les formats de fonte ont suivi ces

vingt dernières années.

Nous allons, dans cette présentation, revoir les succès et les échecs des formats de fonte, et nous

allons présenter une liste de vœux des propriétés que nous considérons qu’un bon format de fonte

devrait avoir.

Introduction

Let us try to imagine what format fonts will be living in

several decades from today. That question is very rel-

evant in 2003, as the type world is ready for yet an-

other overhaul. In this paper, we briefly comment on

the present situation, in which the TrueType and Post-

Script font formats are dominant, and the OpenType for-

mat, which was proposed about eight years ago, is being

promoted. We then take a broader and more long-term

view and touch upon various issues related to the design

of electronic font formats.

Before we embark on the more technical aspects of

electronic font formats, ithelpsto identify the forces that

are helping to shape these formats.

First and foremost, the users would like to see sim-

ple, useful formats, that are easy to manipulate and edit.

They want to have access to the art created by great type

artists and the technical refinement provided by digital

font experts. In addition, professional users may demand

a certain degree of flexibility in a font, in order to incor-

porate personal choices.

The artists and typographers had a lot of influence

in pre-electronic font formats. The early typographers

were nearly all craftsmen. In the twentieth century, var-

ious technological advances were made at companies like

Linotype and Monotype, that were driven by the de-

mands of the type designers, and we witnessed a shorten-

ing of the time between design on paper and actual glyph

production. In the electronic era, the artists and typog-

raphers have been largely left out of the decisions on font

formats, and this has led to an unfortunate split in the

family of typographers: on the one hand, there are those

who never adapted to the mouse and the screen, and con-

tinued designing typefaces using pen and ink. Perhaps

the medium or perhaps the all too mathematical font for-

mats and font editors acted as deterrents for them. On

the other hand, we have seen the emergence of digi-

tal artists who design glyphs directly on the screen, and

do so with extreme efficiency. In this category, we can

place prolific artists such as Lucas De Groot, Jean-Franois

Porchez and David Berlow. A few evenmastered the bit-

map format, and became the ultimate digital technicians.

Matthew Carter’s Verdana, an outline font designed and

tweaked for optimal screen output, is a prime example

of the output of a master digital technician. For more on

the designer’s perspective, read Hermann Zapf’s 1991

book [53]. For both groups of designers, however, the

font format came first, and they had to adapt to the tech-

nology. Perhaps, in the future, we should ask them for

some input, and create a medium in which their freedom

is undiminished.

The engineers have a say in the matter as they report

about the limitations of certain media. Screen render-

ers, printer specifications and other physical facts limit

the format in which fonts are presented in those media.

There is a movable boundary defined by the partition of

the responsibilitiesbetweencomputer and peripheralde-

vice. For example, a “lazy” computer may send a raw

font to a printer, and the printer must do all the process-

ing internally to putink on a page [thisisthe strategy used

in native PostScript printers, for example]. Other media

expect a device-specific font format, often a bitmap or

102

TUGboat, Volume 24 (2003), No. 3— Proceedings of EuroT

X 2003

Page 2

Formatting Font Formats

pixel font, adapted to the resolution and device specifi-

cations. The onus here is on the computer, not the de-

vice. In these cases, font formats are sometimes designed

by engineers, who have very little typographic training.

Computerscientistsandprogrammers(softwareartists)

are increasingly important players. The creators of Post-

Script, Geschke and Warnock, who developed PostScript

based on the page description language

PDL

by John

Gaffneyin 1976 [1], and the Type 1 and Type 3 font

formats [2, 3], were computer scientists with a graphical

vision. PostScript succeeded thanks to its simplicity and

flexibility. The influence of Adobe today is infact largely

due to the invention of PostScript. In font software and

format design, the computer scientists are largely preoc-

cupied with logical organizations of files and with issues

like standardization. This endeavour often carries them

away, so, just as with the engineers, this group of people

should remain dedicated to the users and the font design-

ers, not the other way around.

Going up the ladder, we find the vendors, foundries

and companies, whose interests are often commercial, and

who by definition are concerned with company reputa-

tion, sales volume, market share, proprietary formats,

and software strategies. Fonts are often developedas part

of larger software packages or in conjunction with cer-

tain operating systems. This world also revolves around

patents, trademarks and copyrights, the various ways in

which software and typefaces may be protected. The ac-

tual font format itself that is supported by this group is

often the result of various market decisions, the prime

example being the story of PostScript, TrueType and

OpenType that will be recounted a bit further on.

The final force at work in the creation of a font for-

mat is inertia, driven by tradition and historical models.

An electronic font format is often the result of a mod-

ification of a previous format or technology. Backward

compatibility is often cited as a requirement for a new

format, but this been contradicted by the historic record,

with dramatic incompatible quantum jumps in the tech-

nology.

The typeface repertoire is rich, with many type-

faces existing in only one of several possible formats.

Many historic faces only exist in print (in specimen books

or old manuscripts), while hundreds if not thousands are

only available in metal or wood. In the phototypeset-

ting era—the 1950s to 1980s—, typefaces were stored

in photographic format. And finally, in the 1980s, elec-

tronic font formats were introduced. Among these, the

earliest are the bitmap formats such as “fon”, “bdf”, and

“fnt”. In 1982, Jim Warnock and Charles Geschke in-

troduced PostScript [1], and suggested storing glyphs by

describing their outlines as Bezier curves. This led to

the Type 3 and Type 1 font formats. Knuth also used

Bezier curves for outlines, but had the idea of describ-

ing glyphs by programs in his

METAFONT

[32], which

was introduced and perfected in the period from 1977–

1985. In 1987–1989, Apple’sSampo Kaasila developed

the TrueType format, which was an economic decision

to counteract the stranglehold Adobe had on the type

technology market at the time with its proprietary Post-

Script. Finally, Microsoft and Adobe joined forces in

the 1990s to create OpenType in the hope of reconcil-

ing TrueType and PostScript. The discussion below will

show that this is only a minor technological step. When

we look into the future, we must take this varied histor-

ical record into account. The electronic era is the first

one in which font formats were proprietary—they were

designed and “belonged” to one or more companies. In

taking the next step in formats, we should stear clear of

this trap, and agree on a route that is open to everyone.

It is very likely that the present computer data

model, in which the bits are the atoms, and in which bit

storage is somehow achieved at the microscopic physical

level, will survivefor at leasta few decades, so we will use

words like files and bits in this paper, with the caveat that

a future reader may find this vocabulary old-fashioned.

Taking a long-term view, we will describe the hub model

for font storage and manipulation. The detailswill be de-

scribed in subsequent sections.

The hub model

A font is an implementation of a typeface: ideally, it con-

tains the full description of that typeface. It is like a com-

plete book—anyone can read it, nothing is missing, the

author is clearly identified, and so on. Similarly, a font

should thus be implemented in a human-readable “open

book” format. None of the previous formats had this. In

the metal days, valuable information about the creative

process was missing, and only foundries actually owned

metal type. TrueType, Type 1 and OpenType fonts are

only computer-readable.

METAFONT

and Type 3 can

only be interpreted by programmers and computer sci-

entists. In fact, because of the proliferation of formats,

we have TrueType, Type 1 and OpenType versions for

hundreds of typefaces, and each version is slightly differ-

ent from the other one because of technical incompati-

bilities. In other words, at present, one typeface “lives

on” in many fonts, and this is a chaotic situation.

The human-readable mother font for a typeface

should exist once, and ideally be frozen forever, just as

with a “version” of a piece of software. Additions and

modifications of it then yield new fonts. One can swim

downstream from the mother font to popular implemen-

tations (TrueType, Type 1, etcetera) by filters, but hor-

izontal swimming between OpenType and Type 1, for

example, is not recommended, and upstream conversions

are to be avoided at all costs.

FonteditorsatpresentincludeFontographer(owned

TUGboat, Volume 24 (2003), No. 3—Proceedings of EuroT

X2003

103

Page 3

Luc Devroye

by Macromedia, described by Moye [41], FontLab (by

Yuri Yarmola), Font Studio (by Letraset), Ikarus (by Pe-

ter Karow at

URW

), FontForge (by George Williams),

FontCreator (by Erwin Denissen), Softy (by Dave Em-

mett), Manutius (by A. Gebert) and Noah (by Yeah

Noah). Each operates on one or more formats on one or

more computer platforms. New editors should be de-

signed to create or manipulate that mother font, thus

leading to a more logical situation. Artists too should

be able to directly access that mother font. Printers,

screens, applications, and handheld devices can oper-

ate on compact electronic formats obtained downstream

from the mother font. It should be noted that most seri-

ous editors store fonts in an internal human-readable for-

mat, and have in fact created models for mother fonts.

Most of these do not go beyond a one-to-one translation

of the corresponding binary format, however. For sur-

veys on font technology, we refer to the books by André

[6], Karow [28, 29] and Knuth [35] and the articles by

Gonczarowski [19, 20] and André and Hersch [7].

Each of the sections below treats one of the aspects

of the mother font in more detail.

Outline and pre-outline

One of the main contributions to computational geom-

etry and computer-aided geometric design was the de-

velopment of the Bézier curve by James Ferguson, an

airplane designer, Pierre Bézier, an engineer with Re-

nault, and de Casteljau, an engineer at the competing

French automobile company, Citroën. Two and three-

dimensional objects could be described and approxi-

mated rather simply by concatenating sections of curves.

This is, in fact, a way of transforming a physical object

into a number of bits, and thus, a way of compaction.

One can take a 1

high-detail photograph or scan of a

letter, which after compaction by standard methods such

as “zip” (which uses a mix of Huffman and Lempel-Ziv

coding) may be reduced to 200 kilobytes or so. Yet, by

just storing the collection of Bézier curves, the same let-

ter can be locked in memory using under a kilobyte, as

the formula for a

-th order Bézier curve just requires

the knowledge of

n + 1

control points

, x

, . . . , x

the plane:

x(t) =

i=0

(1 − t)

n−i

· x

, 0 ≤ t ≤ 1.

Here

x(t)

is a parametric curve, a continuous convex

combination of the control points (hence, the Bézier

curve stays within the polygon formed by the control

points), starting at

and ending at

. The mathemat-

ical properties of Bézier curves and splines in general are

described by Farin [17], Su and Liu [47] and Bu-Qing

and Ding-Yuan [12].

It was only natural that PostScript and

METAFONT

adopted the Bézier curve: their creators settled on the

cubic Bézier curve (

n = 3

). TrueType uses quadratic

Bézier curves (

n = 2

), which was an unfortunate de-

cision, as a quadratic Bézier curve can without loss be

transformed into a cubic one (given

, x

, set

, y

= (2x

+ x

)/3, y

= (2x

+ x

)/3, y

= x

to obtain the cubic control points

), but not vice versa.

So, Type 1 is downstream from TrueType, yet, cubic ap-

proximations are usually heralded as being more compact

than quadratic approximations. Artists report that cubic

curves have a richer palette than quadratic curves.

Bézier curves cannot represent circles without er-

ror, no matter how large

is [for the mathematically in-

clined, this is an excellent exercise]. For example, a 90-

degree circle arc is best approximated by a cubic Bézier

if we take the control points

(0, 1), (a, 1), (1, a), (1, 0)

and

a = (4/3)(√2 − 1) = 0.5522847498 . . .

. This

omission could have been rectified if Bézier had allowed

parametric descriptions involving either a square root or

a trigonometric function.

Type designers who work with type on screen are in

fact Bézier point placement artists. Their instrument is

the mouse. This is very hard, as many control points are

not on the curves, and continuity of derivatives between

adjacent Bézier sections is difficult to achive by the naked

eye. Some designers still use pen and paper, and rely on

scanners for computer input. Yetothers, used to software

for artists, are good at placing points that are related to

Bézier curves indirectly, such as demonstrated by Böhm

splines [10], where smooth continuous derivative Bézier

sections are implied.

Hobby [25] and Knuth [32] developed an algo-

rithm for constructing a sequence of Bézier curves that

is forced to visit the designer’s set of points. This algo-

rithm is built in

METAFONT

[32, p. 131], and can be a

great on-line tool for some. We call such ways of describ-

ing outlines “pre-outlines”.

To summarize, the mother font should be flexible

and permit choices between any of a number of outline

and pre-outline formats, as long as each format defines

a mathematical curve in a unique manner. Concatena-

tions of Bézier curves of any degree (with

being a pa-

rameter) should be allowed, as well as several pre-outline

formats to accommodate the typographers at large. At

least one spline model should be included that stores cir-

cle arcs without any error, so that we can finally have ex-

act approximations of those fantastic geometric ruler and

compass creationsofmasters likePhilippeGrandjean, the

designer of the Romain du roi (1693–1745).

Ink models

The two main formats, TrueType and Type 1, and their

104

TUGboat, Volume 24 (2003), No. 3— Proceedings of EuroT

X 2003

Page 4

Formatting Font Formats

derivative, OpenType, are all based on a primitive ink

model, based on the principle that a character is defined

by a number of closed outlines, which are then filled with

ink according to the non-zero windingnumber rule. This

means that if a point or pixel is in a given region, then its

color, black or white, can be determined by drawing a ray

from that point to infinity (in any direction!) and keep-

ing a weighted count of the outlines crossed. A weight

of one is given to a clockwise turning contour and mi-

nus one to a counter-clockwise contour at each crossing

point. But this is clearly not how we place ink on pa-

per at home, where overwriting and erasing are two pri-

mary operations. Also, one should be able to work with

many black/white images, perhaps levels of images, and

define a final image as a logical operation on component

images, using operators like “or”, “and”, “exclusive or”,

and “not”. One should be able to mark regions black or

white by pointing to it—in other words, the region con-

taining “x” should always be black.

Stroke fonts are distinguished from outline fonts by

their ink model: a stroke is defined, perhaps by a collec-

tion of splines of Bézier curves, and ink is placed by fol-

lowing the stroke with a brush or nibbed pen, perhaps

tilted at an angle or suitably shaped. Japanese and Chi-

nese seem like prime territory for such fonts. But closer

to home, we should not forget about the characters that

are created by the interaction between a pen and a tablet,

as on palm-held devices, or signatures made with a mag-

netic pen, or input from a computer tablet. A person’s

handwriting is often better captured by letting the per-

son write on a tablet (so that we obtain the stroke points

inchronological order, withdynamicinformation), asop-

posed to scanning the individual’s handwriting. Tablet

input is naturally translated into strokes.

The recommendation to allow many ink models

sounds like an extension of the PostScript graphical

model, but it can be organized by rasterizers and printers

without too much trouble as all can be internally reduced

to outlines (out of sight of the font designer!) and to the

classical non-zero winding number rule. The extension

is suggested, once again, to make type design easier, more

universal, more current and more accessible.

Path complexities

Outlines and curve data are not unrestricted in our

present electronic formats. For example, paths in Post-

Script and thus Type 1 are limited to about 750 control

points. Such limitations make it impossible to store cer-

tain complex characters as are found in ornaments, dec-

orative initial caps, and outlines based on high resolution

scans. TrueType has higher limits, but the mother font

should in principle have no limit. Limits could be in-

troduced by various formats downstream, and by various

viewing media even further downstream, but it should

not be introduced at the mother font level.

Accuracy

Outlines in any form require mathematical input. As

points need to be represented in a unique manner across

all platforms, it is imperative that all mathematical de-

scriptions be in terms of integers. For example, a point

can simply be

(x, y)

, where

and

are integers, but it

can also be

(x/x , y/y )

where

x, x , y, y

are integer, so

that we can attain all rational numbers. At present, as-

suming that a character occupies the square

[0, 1]

, points

in that square can be addressed as

(x/1000, y/1000)

with

x, y

integer, as is common in Type 1. Type 1 per-

mits higher values than 1000, but not all interpreters of

Type 1 fonts are happy with such. In TrueType, the

1000

1000 box is replaced by 2048

2048. The dif-

ferent box sizes shows that there is no lossless horizontal

conversion between TrueType and Type 1, as

x/1000 =

y/2048

implies that

must be a multiple of 125 and

multiple of 256, and any other values imply a loss in ac-

curacy. OpenType inherits the Type 1 restriction for its

CFF

style implementation, and the TrueType restriction

otherwise.

It is incomprehensible that no one has even at-

tempted to increase these limits of accuracy. Picture a

complex character consisting of 50 rows and 50 columns

of circles that touch other. In a 1000

1000 integer box,

this would force the radius of each circle to be 5. In a cu-

bic Bézier implementation of a quarter circle, we need to

place the control points at

(0, 5), (a, 5), (5, a), (5, 0)

,and

must select the values

1, 2, 3

for

, recalling that the

ideal value is about

2.75

(see above). By picking

a = 3

the circles will be far from perfect!

There is an even more compelling reason why the

accuracy must be increased: the historical record. As we

scan historical designs, in our quest to store everything

in some electronic format for the future, we must en-

sure that as little as possible is lost in the process. Just as

the noise in old

s was due to mechanical limitations, so

is the noise introduced by storing valuable designs using

less-than-ideal accuracy. Reconstruction and de-noising

will be difficult once the damage is done.

In a 1000

1000 box, storing a point

(x, y)

requires

about 20 bits. In a 1 million by 1 million box, the stor-

age increases to about 40 bits, and for an unimaginable 1

billion by 1 billion box, the storage increases to about 60

bits. Thus, by doubling the storage requirements, we can

in fact increase the number of point positions by a factor

of one million! By tripling, that multiplication factor be-

comes one trillion. In other words, this is a change that

comes relatively cheaply. Furthermore, since the mother

font is upstream of everything else, one can always drop

down to lower accuracies when moving downstream. For

storing points, perhaps the best method is to work with

TUGboat, Volume 24 (2003), No. 3—Proceedings of EuroT

X2003

105

Page 5

Luc Devroye

(x, y, n)

, where

is the accuracy, and

and

are inte-

gers in or near the range

[0 . . . n]

. The triple then rep-

resents

(x/n, y/n)

. The value of

should not a priori

be restricted. Accuracy should be a variable parameter,

perhaps different from font to font.

It must be mentioned that accuracy is not an issue in

a pure PostScript type format such as Type 3, and that

theoretically, in a Type 1 font, it can be controlled by

the Font Matrix, although, in practice, many applications

expect a 1000

1000 matrix.

Programming and fonts

The current crop of electronic font formats are just ta-

bles. Just as with their metal counterparts, they are dead

objects that require manipulation by an external mas-

ter or computer program. Even though some companies

claim that their fonts are programs, this is false, with

the exception of

METAFONT

and Type 3, which were

both major steps forward in font technology. In addi-

tion, some TrueType fontshavesome bitsof codeintheir

hinting sections, but it is debatable whether this should

be considered as a program or a table.

The Type 3 format allows the use of the full Post-

Script language: there are parameters, variables, condi-

tional instructions and loops. It is possible to make ran-

domized fonts, e.g., for the simulation of handwriting,

and to create connected context-sensitive glyphs. Char-

acters can be programmed in terms of tunable parame-

ters. Perhaps the simplest tunable fonts are the multiple

master fonts that Adobe proposed in the 1990s, in which

one can vary one or more parameters to interpolate be-

tween extremal fonts. Of course, this can be emulated in

Type 3 fonts.

METAFONT

has similar capabilities, and,

in fact, Knuth demonstrated with his Computer Modern

family [34] that one program per glyph suffices to cre-

ate a family of 72 component fonts, ranging from type-

writer type to serif and sans serif (see also [21]). Other

attempts at parametrization, such as Infinifont (McQueen

and Beausoleil, [40]) and LiveType (Shamir and Rap-

poport, [44, 45]) were short-lived.

The disadvantage of such programmable fonts is the

necessity to have at one’s fingertips, in printers, and in

applications, powerful interpreters or on-the-fly convert-

ers to other formats. Furthermore, the danger of a virus

lurks in every piece of code—indeed, executing a Type 3

“font” can have as side effect the creation or deletion

of one or more files. Finally, interpreters for powerful

languages are often legally protected and can only be li-

censed at enormous fees. With language features wisely

restricted to purely mathematical and graphical opera-

tions, one should be able to flag mother fonts that contain

active code, analogous to the present flagging of multiple

master fonts.

Reviving the idea of programmable fonts will have

enormous benefits for mathematical typesetting. Knuth’s

model (

METAFONT +

X, [33]) is now over 20 years

old, and has a few shortcomings that require an update.

There should be a continuum of optically adjusted sym-

bols like brackets and parentheses, with line thickness and

size adapted to the surrounding text. At present, the

symbols are selected from a finite set, which often leads

to aesthetic mismatches. Improvements should be made

in optical size matching of subscripts and superscripts.

Of course, optically and continuously adjusted sym-

bols are only part of my mathematical typesetting wish-

list. There should ideally be a symbiosis of figures, for-

mulas and text, all playing and interacting on the page, a

bit as with blackboard mathematics in the hands of a mas-

ter mathematician. This requires a paradigm that tran-

scends T

In the area of randomized fonts for the simulation of

handwriting, we refer to Devroye and McDougall [15]

for a theoretical development and some crude exam-

ples, to Desruisseaux [14] for a thoroughly researched

font called MetamorFont, and to André and Borghi [5],

Dooijes [16] and van Blokland and van Rossum [51] for

earlier attempts in this direction. All these develop-

ments used the programming power of Type 3 to cre-

ate random-looking characters that are either based on

a sample of one’s handwriting (as in the first reference

above) or that are constructed artificially by program-

ming the randomness in the outlines (as in Metamor-

Font). It would be a shame not to include a random num-

ber generator in the specification of the mother font. Of

course, one should make sure that the random sequence

generated can be “replayed” for debugging purposes.

Ligatures and context sensitivity

Ligatures are combinations of two or more characters.

Context sensitive characters are single characters that

change shape as a function of their context or neighbor-

hood. The activation of a context sensitive change should

always be the responsibility of the application—the font

should only contain the various shapes without getting in-

volved in questions related to context.

This separation of form and application should also

apply to ligatures. Fonts provide the shapes only. This

division has been rigorously supported in the