533 lines
23 KiB
Plaintext
533 lines
23 KiB
Plaintext
gxvalid: TrueType GX validator
|
|
==============================
|
|
|
|
|
|
1. What is this
|
|
---------------
|
|
|
|
`gxvalid' is a module to validate TrueType GX tables: a collection of
|
|
additional tables in TrueType font which are used by `QuickDraw GX
|
|
Text', Apple Advanced Typography (AAT). In addition, gxvalid can
|
|
validates `kern' tables which have been extended for AAT. Like the
|
|
otvalid module, gxvalid uses Freetype 2's validator framework
|
|
(ftvalid).
|
|
|
|
You can link gxvalid with your program; before running your own layout
|
|
engine, gxvalid validates a font file. As the result, you can remove
|
|
error-checking code from the layout engine. It is also possible to
|
|
use gxvalid as a stand-alone font validator; the `ftvalid' test
|
|
program included in the ft2demo bundle calls gxvalid internally.
|
|
A stand-alone font validator may be useful for font developers.
|
|
|
|
This documents documents the following issues.
|
|
|
|
- supported TrueType GX tables
|
|
- fundamental validation limitations
|
|
- permissive error handling of broken GX tables
|
|
- `kern' table issue.
|
|
|
|
|
|
2. Supported tables
|
|
-------------------
|
|
|
|
The following GX tables are currently supported.
|
|
|
|
bsln
|
|
feat
|
|
just
|
|
kern(*)
|
|
lcar
|
|
mort
|
|
morx
|
|
opbd
|
|
prop
|
|
trak
|
|
|
|
The following GX tables are currently unsupported.
|
|
|
|
cvar
|
|
fdsc
|
|
fmtx
|
|
fvar
|
|
gvar
|
|
Zapf
|
|
|
|
The following GX tables won't be supported.
|
|
|
|
acnt(**)
|
|
hsty(***)
|
|
|
|
The following undocumented tables in TrueType fonts designed for Apple
|
|
platform aren't handled either.
|
|
|
|
addg
|
|
CVTM
|
|
TPNM
|
|
umif
|
|
|
|
|
|
*) The `kern' validator handles both the classic and the new kern
|
|
formats; the former is supported on both Microsoft and Apple
|
|
platforms, while the latter is supported on Apple platforms.
|
|
|
|
**) `acnt' tables are not supported by currently available Apple font
|
|
tools.
|
|
|
|
***) There is one more Apple extension, `hsty', but it is for
|
|
Newton-OS, not GX (Newton-OS is a platform by Apple, but it can
|
|
use sfnt- housed bitmap fonts only). Therefore, it should be
|
|
excluded from `Apple platform' in the context of TrueType.
|
|
gxvalid ignores it as Apple font tools do so.
|
|
|
|
|
|
We have checked 183 fonts bundled with MacOS 9.1, MacOS 9.2, MacOS
|
|
10.0, MacOS X 10.1, MSIE for MacOS, and AppleWorks 6.0. In addition,
|
|
we have checked 67 Dynalab fonts (designed for MacOS) and 189 Ricoh
|
|
fonts (designed for Windows and MacOS dual platforms). The number of
|
|
fonts including TrueType GX tables are as follows.
|
|
|
|
bsln: 76
|
|
feat: 191
|
|
just: 84
|
|
kern: 59
|
|
lcar: 4
|
|
mort: 326
|
|
morx: 19
|
|
opbd: 4
|
|
prop: 114
|
|
trak: 16
|
|
|
|
Dynalab and Ricoh fonts don't have GX tables except of `feat' and
|
|
`mort'.
|
|
|
|
|
|
3. Fundamental validation limitations
|
|
-------------------------------------
|
|
|
|
TrueType GX provides layout information to libraries for font
|
|
rasterizers and text layout. gxvalid can check whether the layout
|
|
data in a font is conformant to the TrueType GX format specified by
|
|
Apple. But gxvalid cannot check a how QuickDraw GX/AAT renderer uses
|
|
the stored information.
|
|
|
|
3-1. Validation of State Machine activity
|
|
-----------------------------------------
|
|
|
|
QuickDraw GX/AAT uses a `State Machine' to provide `stateful' layout
|
|
features, and TrueType GX stores the state transition diagram of
|
|
this `State Machine' in a `StateTable' data structure. While the
|
|
State Machine receives a series of glyph IDs, the State Machine
|
|
starts with `start of text' state, walks around various states and
|
|
generates various layout information to the renderer, and finally
|
|
reaches the `end of text' state.
|
|
|
|
gxvalid can check essential errors like:
|
|
|
|
- possibility of state transitions to undefined states
|
|
- existence of glyph IDs that the State Machine doesn't know how
|
|
to handle
|
|
- the State Machine cannot compute the layout information from
|
|
given diagram
|
|
|
|
These errors can be checked within finite steps, and without the
|
|
State Machine itself, because these are `expression' errors of state
|
|
transition diagram.
|
|
|
|
There is no limitation about how long the State Machine walks
|
|
around, so validation of the algorithm in the state transition
|
|
diagram requires infinite steps, even if we had a State Machine in
|
|
gxvalid. Therefore, the following errors and problems cannot be
|
|
checked.
|
|
|
|
- existence of states which the State Machine never transits to
|
|
- the possibility that the State Machine never reaches `end of
|
|
text'
|
|
- the possibility of stack underflow/overflow in the State Machine
|
|
(in ligature and contextual glyph substitutions, the State
|
|
Machine can store 16 glyphs onto its stack)
|
|
|
|
In addition, gxvalid doesn't check `temporary glyph IDs' used in the
|
|
chained State Machines (in `mort' and `morx' tables). If a layout
|
|
feature is implemented by a single State Machine, a glyph ID
|
|
converted by the State Machine is passed to the glyph renderer, thus
|
|
it should not point to an undefined glyph ID. But if a layout
|
|
feature is implemented by chained State Machines, a component State
|
|
Machine (if it is not the final one) is permitted to generate
|
|
undefined glyph IDs for temporary use, because it is handled by next
|
|
component State Machine and not by the glyph renderer. To validate
|
|
such temporary glyph IDs, gxvalid must stack all undefined glyph IDs
|
|
which can occur in the output of the previous State Machine and
|
|
search them in the `ClassTable' structure of the current State
|
|
Machine. It is too complex to list all possible glyph IDs from the
|
|
StateTable, especially from a ligature substitution table.
|
|
|
|
3-2. Validation of relationship between multiple layout features
|
|
----------------------------------------------------------------
|
|
|
|
gxvalid does not validate the relationship between multiple layout
|
|
features at all.
|
|
|
|
If multiple layout features are defined in TrueType GX tables,
|
|
possible interactions, overrides, and conflicts between layout
|
|
features are implicitly given in the font too. For example, there
|
|
are several predefined spacing control features:
|
|
|
|
- Text Spacing (Proportional/Monospace/Half-width/Normal)
|
|
- Number Spacing (Monospaced-numbers/Proportional-numbers)
|
|
- Kana Spacing (Full-width/Proportional)
|
|
- Ideographic Spacing (Full-width/Proportional)
|
|
- CJK Roman Spacing (Half-width/Proportional/Default-roman
|
|
/Full-width-roman/Proportional)
|
|
|
|
If all layout features are independently managed, we can activate
|
|
inconsistent typographic rules like `Text Spacing=Monospace' and
|
|
`Ideographic Spacing=Proportional' at the same time.
|
|
|
|
The combinations of layout features is managed by a 32bit integer
|
|
(one bit each for selector setting), so we can define relationships
|
|
between up to 32 features, theoretically. But if one feature
|
|
setting affects another feature setting, we need typographic
|
|
priority rules to validate the relationship. Unfortunately, the
|
|
TrueType GX format specification does not give such information even
|
|
for predefined features.
|
|
|
|
|
|
4. Permissive error handling of broken GX tables
|
|
------------------------------------------------
|
|
|
|
When Apple's font rendering system finds an inconsistency, like a
|
|
specification violation or an unspecified value in a TrueType GX
|
|
table, it does not always return error. In most cases, the rendering
|
|
engine silently ignores such wrong values or even whole tables. In
|
|
fact, MacOS is shipped with fonts including broken GX/AAT tables, but
|
|
no harmful effects due to `officially broken' fonts are observed by
|
|
end-users.
|
|
|
|
gxvalid is designed to continue the validation process as long as
|
|
possible. When gxvalid find wrong values, gxvalid warns it at least,
|
|
and takes a fallback procedure if possible. The fallback procedure
|
|
depends on the debug level.
|
|
|
|
We used the following three tools to investigate Apple's error handling.
|
|
|
|
- FontValidator (for MacOS 8.5 - 9.2) resource fork font
|
|
- ftxvalidator (for MacOS X 10.1 -) dfont or naked-sfnt
|
|
- ftxdumperfuser (for MacOS X 10.1 -) dfont or naked-sfnt
|
|
|
|
However, all tests were done on a PowerPC based Macintosh; at present,
|
|
we have not checked those tools on a m68k-based Macintosh.
|
|
|
|
In total, we checked 183 fonts bundled to MacOS 9.1, MacOS 9.2, MacOS
|
|
10.0, MacOS X 10.1, MSIE for MacOS, and AppleWorks 6.0. These fonts
|
|
are distributed officially, but many broken GX/AAT tables were found
|
|
by Apple's font tools. In the following, we list typical violation of
|
|
the GX specification, in fonts officially distributed with those Apple
|
|
systems.
|
|
|
|
4-1. broken BinSrchHeader (19/183)
|
|
----------------------------------
|
|
|
|
`BinSrchHeader' is a header of a data array for m68k platforms to
|
|
access memory efficiently. Although there are only two independent
|
|
parameters for real (`unitSize' and `nUnits'), BinSrchHeader has
|
|
three additional parameters which can be calculated from `unitSize'
|
|
and `nUnits', for fast setup. Apple font tools ignore them
|
|
silently, so gxvalid warns if it finds and inconsistency, and always
|
|
continues validation. The additional parameters are ignored
|
|
regardless of the consistency.
|
|
|
|
19 fonts include such inconsistencies; all breaks are in the
|
|
BinSrchHeader structure of the `kern' table.
|
|
|
|
4-2. too-short LookupTable (5/183)
|
|
----------------------------------
|
|
|
|
LookupTable format 0 is a simple array to get a value from a given
|
|
GID (glyph ID); the index of this array is a GID too. Therefore,
|
|
the length of the array is expected to be same as the maximum GID
|
|
value defined in the `maxp' table, but there are some fonts whose
|
|
LookupTable format 0 is too short to cover all GIDs. FontValidator
|
|
ignores this error silently, ftxvalidator and ftxdumperfuser both
|
|
warn and continue. Similar problems are found in format 3 subtables
|
|
of `kern'. gxvalid warns always and abort if the validation level
|
|
is set to FT_VALIDATE_PARANOID.
|
|
|
|
5 fonts include too-short kern format 0 subtables.
|
|
1 font includes too-short kern format 3 subtable.
|
|
|
|
4-3. broken LookupTable format 2 (1/183)
|
|
----------------------------------------
|
|
|
|
LookupTable format 2, subformat 4 covers the GID space by a
|
|
collection of segments which are specified by `firstGlyph' and
|
|
`lastGlyph'. Some fonts store `firstGlyph' and `lastGlyph' in
|
|
reverse order, so the segment specification is broken. Apple font
|
|
tools ignore this error silently; a broken segment is ignored as if
|
|
it did not exist. gxvalid warns and normalize the segment at
|
|
FT_VALIDATE_DEFAULT, or ignore the segment at FT_VALIDATE_TIGHT, or
|
|
abort at FT_VALIDATE_PARANOID.
|
|
|
|
1 font includes broken LookupTable format 2, in the `just' table.
|
|
|
|
*) It seems that all fonts manufactured by ITC for AppleWorks have
|
|
this error.
|
|
|
|
4-4. bad bracketing in glyph property (14/183)
|
|
----------------------------------------------
|
|
|
|
GX/AAT defines a `bracketing' property of the glyphs in the `prop'
|
|
table, to control layout features of strings enclosed inside and
|
|
outside of brackets. Some fonts give inappropriate bracket
|
|
properties to glyphs. Apple font tools warn about this error;
|
|
gxvalid warns too and aborts at FT_VALIDATE_PARANOID.
|
|
|
|
14 fonts include wrong bracket properties.
|
|
|
|
|
|
4-5. invalid feature number (117/183)
|
|
-------------------------------------
|
|
|
|
The GX/AAT extension can include 255 different layout features, but
|
|
popular layout features are predefined (see
|
|
http://developer.apple.com/fonts/Registry/index.html). Some fonts
|
|
include feature numbers which are incompatible with the predefined
|
|
feature registry.
|
|
|
|
In our survey, there are 140 fonts including `feat' table.
|
|
|
|
a) 67 fonts use a feature number which should not be used.
|
|
b) 117 fonts set the wrong feature range (nSetting). This is mostly
|
|
found in the `mort' and `morx' tables.
|
|
|
|
Apple font tools give no warning, although they cannot recognize
|
|
what the feature is. At FT_VALIDATE_DEFAULT, gxvalid warns but
|
|
continues in both cases (a, b). At FT_VALIDATE_TIGHT, gxvalid warns
|
|
and aborts for (a), but continues for (b). At FT_VALIDATE_PARANOID,
|
|
gxvalid warns and aborts in both cases (a, b).
|
|
|
|
4-6. invalid prop version (10/183)
|
|
----------------------------------
|
|
|
|
As most TrueType GX tables, the `prop' table must start with a 32bit
|
|
version identifier: 0x00010000, 0x00020000 or 0x00030000. But some
|
|
fonts store nonsense binary data instead. When Apple font tools
|
|
find them, they abort the processing immediately, and the data which
|
|
follows is unhandled. gxvalid does the same.
|
|
|
|
10 fonts include broken `prop' version.
|
|
|
|
All of these fonts are classic TrueType fonts for the Japanese
|
|
script, manufactured by Apple.
|
|
|
|
4-7. unknown resource name (2/183)
|
|
------------------------------------
|
|
|
|
NOTE: THIS IS NOT A TRUETYPE GX ERROR.
|
|
|
|
If a TrueType font is stored in the resource fork or in dfont
|
|
format, the data must be tagged as `sfnt' in the resource fork index
|
|
to invoke TrueType font handler for the data. But the TrueType font
|
|
data in `Keyboard.dfont' is tagged as `kbd', and that in
|
|
`LastResort.dfont' is tagged as `lst'. Apple font tools can detect
|
|
that the data is in TrueType format and successfully validate them.
|
|
Maybe this is possible because they are known to be dfont. The
|
|
current implementation of the resource fork driver of FreeType
|
|
cannot do that, thus gxvalid cannot validate them.
|
|
|
|
2 fonts use an unknown tag for the TrueType font resource.
|
|
|
|
5. `kern' table issues
|
|
----------------------
|
|
|
|
In common terminology of TrueType, `kern' is classified as a basic and
|
|
platform-independent table. But there are Apple extensions of `kern',
|
|
and there is an extension which requires a GX state machine for
|
|
contextual kerning. Therefore, gxvalid includes a special validator
|
|
for `kern' tables. Unfortunately, there is no exact algorithm to
|
|
check Apple's extension, so gxvalid includes a heuristic algorithm to
|
|
find the proper validation routines for all possible data formats,
|
|
including the data format for Microsoft. By calling
|
|
classic_kern_validate() instead of gxv_validate(), you can specify the
|
|
`kern' format explicitly. However, current FreeType2 uses Microsoft
|
|
`kern' format only, others are ignored (and should be handled in a
|
|
library one level higher than FreeType).
|
|
|
|
5-1. History
|
|
------------
|
|
|
|
The original 16bit version of `kern' was designed by Apple in the
|
|
pre-GX era, and it was also approved by Microsoft. Afterwards,
|
|
Apple designed a new 32bit version of the `kern' table. According
|
|
to the documentation, the difference between the 16bit and 32bit
|
|
version is only the size of variables in the `kern' header. In the
|
|
following, we call the original 16bit version as `classic', and
|
|
32bit version as `new'.
|
|
|
|
5-2. Versions and dialects which should be differentiated
|
|
---------------------------------------------------------
|
|
|
|
The `kern' table consists of a table header and several subtables.
|
|
The version number which identifies a `classic' or a `new' version
|
|
is explicitly written in the table header, but there are
|
|
undocumented differences between Microsoft's and Apple's formats.
|
|
It is called a `dialect' in the following. There are three cases
|
|
which should be handled: the new Apple-dialect, the classic
|
|
Apple-dialect, and the classic Microsoft-dialect. An analysis of
|
|
the formats and the auto detection algorithm of gxvalid is described
|
|
in the following.
|
|
|
|
5-2-1. Version detection: classic and new kern
|
|
----------------------------------------------
|
|
|
|
According to Apple TrueType specification, there are only two
|
|
differences between the classic and the new:
|
|
|
|
- The `kern' table header starts with the version number.
|
|
The classic version starts with 0x0000 (16bit),
|
|
the new version starts with 0x00010000 (32bit).
|
|
|
|
- In the `kern' table header, the number of subtables follows
|
|
the version number.
|
|
In the classic version, it is stored as a 16bit value.
|
|
In the new version, it is stored as a 32bit value.
|
|
|
|
From Apple font tool's output (DumpKERN is also tested in addition
|
|
to the three Apple font tools in above), there is another
|
|
undocumented difference. In the new version, the subtable header
|
|
includes a 16bit variable named `tupleIndex' which does not exist
|
|
in the classic version.
|
|
|
|
The new version can store all subtable formats (0, 1, 2, and 3),
|
|
but the Apple TrueType specification does not mention the subtable
|
|
formats available in the classic version.
|
|
|
|
5-2-2. Available subtable formats in classic version
|
|
----------------------------------------------------
|
|
|
|
Although the Apple TrueType specification recommends to use the
|
|
classic version in the case if the font is designed for both the
|
|
Apple and Microsoft platforms, it does not document the available
|
|
subtable formats in the classic version.
|
|
|
|
According to the Microsoft TrueType specification, the subtable
|
|
format assured for Windows and OS/2 support is only subtable
|
|
format 0. The Microsoft TrueType specification also describes
|
|
subtable format 2, but does not mention which platforms support
|
|
it. Subtable formats 1, 3, and higher are documented as reserved
|
|
for future use. Therefore, the classic version can store subtable
|
|
formats 0 and 2, at least. `ttfdump.exe', a font tool provided by
|
|
Microsoft, ignores the subtable format written in the subtable
|
|
header, and parses the table as if all subtables are in format 0.
|
|
|
|
`kern' subtable format 1 uses a StateTable, so it cannot be
|
|
utilized without a GX State Machine. Therefore, it is reasonable
|
|
to assume that format 1 (and 3) were introduced after Apple had
|
|
introduced GX and moved to the new 32bit version.
|
|
|
|
5-2-3. Apple and Microsoft dialects
|
|
-----------------------------------
|
|
|
|
The `kern' subtable has a 16bit `coverage' field to describe
|
|
kerning attributes, but bit interpretations by Apple and Microsoft
|
|
are different: For example, Apple uses bits 0-7 to identify the
|
|
subtable, while Microsoft uses bits 8-15.
|
|
|
|
In addition, due to the output of DumpKERN and FontValidator,
|
|
Apple's bit interpretations of coverage in classic and new version
|
|
are incompatible also. In summary, there are three dialects:
|
|
classic Apple dialect, classic Microsoft dialect, and new Apple
|
|
dialect. The classic Microsoft dialect and the new Apple dialect
|
|
are documented by each vendors' TrueType font specification, but
|
|
the documentation for classic Apple dialect is not available.
|
|
|
|
For example, in the new Apple dialect, bit 15 is documented as
|
|
`set to 1 if the kerning is vertical'. On the other hand, in
|
|
classic Microsoft dialect, bit 1 is documented as `set to 1 if the
|
|
kerning is horizontal'. From the outputs of DumpKERN and
|
|
FontValidator, classic Apple dialect recognizes 15 as `set to 1
|
|
when the kerning is horizontal'. From the results of similar
|
|
experiments, classic Apple dialect seems to be the Endian reverse
|
|
of the classic Microsoft dialect.
|
|
|
|
As a conclusion it must be noted that no font tool can identify
|
|
classic Apple dialect or classic Microsoft dialect automatically.
|
|
|
|
5-2-4. gxvalid auto dialect detection algorithm
|
|
-----------------------------------------------
|
|
|
|
The first 16 bits of the `kern' table are enough to identify the
|
|
version:
|
|
|
|
- if the first 16 bits are 0x0000, the `kern' table is in
|
|
classic Apple dialect or classic Microsoft dialect
|
|
- if the first 16 bits are 0x0001, and next 16 bits are 0x0000,
|
|
the kern table is in new Apple dialect.
|
|
|
|
If the `kern' table is a classic one, the 16bit `coverage' field
|
|
is checked next. Firstly, the coverage bits are decoded for the
|
|
classic Apple dialect using the following bit masks (this is based
|
|
on DumpKERN output):
|
|
|
|
0x8000: 1=horizontal, 0=vertical
|
|
0x4000: not used
|
|
0x2000: 1=cross-stream, 0=normal
|
|
0x1FF0: reserved
|
|
0x000F: subtable format
|
|
|
|
If any of reserved bits are set or the subtable bits is
|
|
interpreted as format 1 or 3, we take it as `impossible in classic
|
|
Apple dialect' and retry, using the classic Microsoft dialect.
|
|
|
|
The most popular coverage in new Apple-dialect: 0x8000,
|
|
The most popular coverage in classic Apple-dialect: 0x0000,
|
|
The most popular coverage in classic Microsoft dialect: 0x0001.
|
|
|
|
5-3. Tested fonts
|
|
-----------------
|
|
|
|
We checked 59 fonts bundled with MacOS and 38 fonts bundled with
|
|
Windows, where all font include a `kern' table.
|
|
|
|
- fonts bundled with MacOS
|
|
* new Apple dialect
|
|
format 0: 18
|
|
format 2: 1
|
|
format 3: 1
|
|
* classic Apple dialect
|
|
format 0: 14
|
|
* classic Microsoft dialect
|
|
format 0: 15
|
|
|
|
- fonts bundled with Windows
|
|
* classic Microsoft dialect
|
|
format 0: 38
|
|
|
|
It looks strange that classic Microsoft-dialect fonts are bundled to
|
|
MacOS: they come from MSIE for MacOS, except of MarkerFelt.dfont.
|
|
|
|
|
|
ACKNOWLEDGEMENT
|
|
---------------
|
|
|
|
Some parts of gxvalid are derived from both the `gxlayout' module and
|
|
the `otvalid' module. Development of gxlayout was supported by the
|
|
Information-technology Promotion Agency(IPA), Japan.
|
|
|
|
The detailed analysis of undefined glyph ID utilization in `mort' and
|
|
`morx' tables is provided by George Williams.
|
|
|
|
------------------------------------------------------------------------
|
|
|
|
Copyright 2004-2017 by
|
|
suzuki toshiya, Masatake YAMATO, Red hat K.K.,
|
|
David Turner, Robert Wilhelm, and Werner Lemberg.
|
|
|
|
This file is part of the FreeType project, and may only be used,
|
|
modified, and distributed under the terms of the FreeType project
|
|
license, LICENSE.TXT. By continuing to use, modify, or distribute this
|
|
file you indicate that you have read the license and understand and
|
|
accept it fully.
|
|
|
|
|
|
--- end of README ---
|