PLS Lexicons

The first of the new synthetic speech enhancement layers we’ll look at is PLS files, which are xml lexicon files that conform to the W3C Pronunciation Lexicon Specification. The entries in these files identify the word(s) to apply each pronunciation rule to. The entries also include the correct phonetic spelling, which provides the text-to-speech engine with the proper pronunciation to render.

Perhaps a simpler way of thinking about PLS files, though, is as containing globally-applicable pronunciation rules: the entries you define in these files will be used for all matching cases in your content. Instead of having to add the pronunciation over and over every time the word is encountered in your markup, as SSML requires, these lexicons are used as global lookups.

PLS files are consequently the ideal place to define all the proper names and technical terms and other complex words that do not change based on the context in which they are used. Even in the case of heteronyms, it’s good to define the pronunciation you deem the most commonly used in your PLS file, as it may be the only case in your ebook(s). It also ensures that you know how the heteronym will always be pronounced by default, to remove the element of chance.

But let’s take a look at a minimal example of a complete PLS file to see how they work in practice. Here we’ll define a single entry for “acetaminophen” to cure our pronunciation headaches:

<lexicon
    version="1.0"
    alphabet="x-sampa"
    xml:lang="en"
    xmlns="http://www.w3.org/2005/01/pronunciation-lexicon">
    <lexeme>
        <grapheme>acetaminophen</grapheme>
        <phoneme>@"sit@'mIn@f@n</phoneme>
    </lexeme>
</lexicon>

To start breaking this markup down, the alphabet attribute on the root lexicon element defines the phonetic alphabet we’re going to use to write our pronunciations. In this case, I’m indicating that I’m going to write them using X-SAMPA.

The version and xmlns namespace declaration attributes are static values, so nothing exciting to see there, as usual. The xml:lang attribute, however, is required, and must reflect the language of the entries contained in the lexicon. Here we’re declaring that all the entries are in English.

The root would normally contain many more lexeme elements than in this example, as each defines the word(s) the rule applies to in the child grapheme element(s). (Graphemes, of course, don’t have to take the form of words, but for simplicity of explanation I’ll stick to the general concept.) When the string is matched, the pronunciation in the phoneme element gets rendered in place of the default rendering the engine would have performed.

Or, if it helps conceptualize, when the word “acetaminophen” is encountered in the prose, before passing the word to the rendering engine to voice, an internal lookup of the defined graphemes occurs. Because we’ve defined a match, the phoneme and the alphabet it adheres to are swapped in instead for voicing.

That you can include multiple graphemes may not seem immediately useful, but it enables you to create a single entry for regional variations in spelling, for example. British and American variants of “defense” could be defined in a single rule as:

<lexeme>
    <grapheme>defense</grapheme>
    <grapheme>defence</grapheme>
    <phoneme>dI'fEns</phoneme>
</lexeme>

It is similarly possible to define more than one pronunciation by adding multiple phoneme elements. We could add the IPA spelling to the last example as follows, in case reading systems end up only supporting one or the other alphabet:

<lexeme>
    <grapheme>defense</grapheme>
    <grapheme>defence</grapheme>
    <phoneme>dI'fEns</phoneme>
    <phoneme alphabet="ipa">dɪˈfɛns</phoneme>
</lexeme>

The alphabet attribute on the new phoneme element is required because its spelling doesn’t conform to the default defined on the root. If the rendering engine doesn’t support X-SAMPA, it could now possibly make use of this embedded IPA version instead.

The phoneme doesn’t have to be in another alphabet, however; you could add a regional dialect as a secondary pronunciation, for example. The specification unfortunately doesn’t provide any mechanisms to indicate why you’ve included such additional pronunciations or when they should be used, so there’s not much value in doing so at this time.

There’s much more to creating PLS files than can be covered here, of course, but you’re now versed in the basics and ready to start compiling your own lexicons. You only need to attach your PLS file to your publication to complete the process of enhancing your ebook.

The first step is to include an entry for the PLS file in the EPUB manifest:

<item href="lexicon.pls" id="pls" media-type="application/pls+xml"/>

The href attribute defines the location of the file relative to the package document and the media-type attribute value “application/pls+xml” identifies to a reading system that we’ve attached a PLS file.

Including one or more PLS files does not mean they apply by default to all your content, however; in fact, they apply to none of it by default. You next have to explicitly tie each PLS lexicon to each XHTML content document it is to be used with by adding a link element to the document’s header:

<html …>
    <head>
        …
        <link
            rel="pronunciation"
            href="lexicon.pls"
            type="application/pls+xml"
            hreflang="en" />
        …
    </head>
    …
</html>

There are a number of differences between the declaration for the PLS file in the publication manifest above and in the content file here. The first is the use of the rel attribute to include an explicit relationship (that the referenced file represents pronunciation information). This attribute represents somewhat redundant information, however, since the media type is once again specified (here in the type attribute). But as it is a required attribute in HTML5, it can’t be omitted.

The HTML link element also includes an additional piece of information to allow selective targeting of lexicons: the hreflang attribute. This attribute specifies the language to which the included pronunciations apply. For example, if you have an English document (as defined in the xml:lang attribute on the html root element) that embeds French prose, you could include two lexicon files:

<link
    rel="pronunciation"
    href="lexicon/en.pls"
    type="application/pls+xml"
    hreflang="en" />

<link
    rel="pronunciation"
    href="lexicon/fr.pls"
    type="application/pls+xml"
    hreflang="fr" />

Assuming all your French passages have xml:lang attributes on them, the reading system can selectively apply the lexicons to prevent any possible pronunciation confusion:

<p>It's the Hunchback of <i xml:lang="fr">Notre Dame</i> not of Notre Dame.</p>

A unilingual person reading this prose probably would not understand the distinction being made here: that the French pronunciation is not the same as the Americanization. Including separate lexicons by language, however, would ensure that readers would hear the Indiana university name differently than the French cathedral if they turn on TTS:

<lexicon
    version="1.0"
    alphabet="x-sampa"
    xml:lang="en"
    xmlns="http://www.w3.org/2005/01/pronunciation-lexicon">
    <lexeme>
        <grapheme>Notre Dame</grapheme>
        <phoneme>noUt@r 'deIm</phoneme>
    </lexeme>
</lexicon>

<lexicon
    version="1.0"
    alphabet="x-sampa"
    xml:lang="fr"
    xmlns="http://www.w3.org/2005/01/pronunciation-lexicon">
    <lexeme>
        <grapheme>Notre Dame</grapheme>
        <phoneme>n%oUtr@ d"Am</phoneme>
    </lexeme>
</lexicon>

When the contents of the i tag are encountered, and identified as French, the pronunciation from the corresponding lexicon gets applied instead of the one from the default English lexicon.

Now that we know how to globally define pronunciation rules, let’s turn to how we can override and/or define behavior at the markup level.

SSML

Although PLS files are a great way to globally set the pronunciation of words, their primary failing is that they aren’t a lot of help where context matters in determining the correct pronunciation. Leave the pronunciation of heteronyms to chance, for example, and you’re invariably going to be disappointed by the result; the cases where context might not significantly influence comprehension (e.g., an English heteronym like “mobile”), are going to be dwarfed by the ones where it does.

By way of example, when talking about PLS files I mentioned bass the instrument and bass the fish as an example of how context influences pronunciation. Let’s take a look at this problem in practice now:

<p>The guitarist was playing a bass that was shaped like a bass.</p>

Human readers won’t have much of a struggle with this sentence, despite the contrived oddity of it. A guitarist is not going to be playing a fish shaped like a guitar, and it would be strange to note that the bass guitar is shaped like a bass guitar. From context you’re able to determine without much pause that we’re talking about someone playing a guitar shaped like a fish.

All good and simple. Now consider your reaction if, when listening to a synthetic speech engine pronounce the sentence, you heard both words pronounced the same way, which is the typical result. The process to correct the mistake takes you out of the flow of the narrative. You’re going to wonder why the guitar is shaped like a guitar, admit it.

Synthetic narration doesn’t afford you the same ease to move forward and back through the prose that visual reading does, as words are only announced as they’re voiced. The engine may be applying heuristic tests to attempt to better interpret the text for you behind the scenes, but you’re at its mercy. You can back up and listen to the word again to verify whether the engine said what you thought it did, but it’s an intrusive process that requires you to interact with the reading system. If you still can’t make sense of the word, you can have the reading system spell it out as a last resort, but now you’re train of thought is completely on understanding the word.

And this is an easy example. A blind reader used to synthetic speech engines would probably just keep listening past this sentence having made a quick assumption that the engine should have said something else, for example, but that’s not a justification for neglect. The problems only get more complex and less avoidable, no matter your familiarity. And that you’re asking your readers to compensate is a major red flag you’re not being accessible, as mispronunciations are not always easily overcome depending on the reader’s disability. It also doesn’t reflect well on your ebooks if readers turn to synthetic speech engines to help with pronunciation and find gibberish, as I touched on in the last section.

And the problems are rarely one-time occurrences. When the reader figures out what the engine was trying to say they will, in all likelihood, have to make a mental note on how to translate the synthetic gunk each time it is re-encountered to avoid repeatedly going through the same process. If you don’t think that makes reading comprehension a headache, try it sometime.

But this is where the Synthetic Speech Markup Language (SSML) comes in, allowing you to define individual pronunciations at the markup level. EPUB 3 adds the ssml:alphabet and ssml:ph attributes, which allow you to specify the alphabet you’re using and phonemic pronunciation of the containing element’s content, respectively. These attributes work in very much the same way as the PLS entries we just reviewed, as you might already suspect.

For example, we could revise our earlier example as follows to ensure the proper pronunciation for each use of bass:

<p>
    The guitarist was playing a
    <span ssml:alphabet="x-sampa" ssml:ph="beIs">bass</span> that was shaped
    like a <span ssml:alphabet="x-sampa" ssml:ph="b&amp;s">bass</span>.
</p>

The ssml:alphabet attribute on each span element identifies that the pronunciation carried in the ssml:ph attribute is written in X-SAMPA, identically to the PLS alphabet attribute. We don’t need a grapheme to match against, because we’re telling the synthetic speech engine to replace the content of the span element. The engine will now voice the provided pronunciations instead of applying its own rules. In other words, no more ambiguity and no more rendering problem; it really is that simple.

It bears a quick note that the pronunciation in the ssml:ph attribute has to match the prose contained in the element it is attached to. By wrapping span elements around each individual word in this example, I’ve limited the translation of text to phonetic code to just the problematic words I want to fix. If I put the attribute on the parent p element, I’d have to transcode the entire sentence.

The upside of the granularity SSML markup provides should be clear now, though: you can overcome any problem no matter how small (or big) with greater precision than PLS files offer. The downside, of course, is having to work at the markup level to correct each instance that has to be overridden.

To hark back to the discussion of PLS files for a moment, though, we could further simplify the correction process by moving the more common pronunciation to our PLS lexicon and only fix the differing heteronym:

<lexicon
    version="1.0"
    alphabet="x-sampa"
    xml:lang="en"
    xmlns="http://www.w3.org/2005/01/pronunciation-lexicon">
    <lexeme>
        <grapheme>bass</grapheme>
        <phoneme>beIs</phoneme>
    </lexeme>
</lexicon>

<p>
    The guitarist was playing a bass that was shaped like a
    <span ssml:alphabet="x-sampa" ssml:ph="b&amp;s">bass</span>.
</p>

It’s also not necessary to define the ssml:alphabet attribute every time. If we were only using a single alphabet throughout the document, which would be typical of most ebooks, we could instead define the alphabet once on the root html element:

<html … ssml:alphabet="x-sampa">

So long as the alphabet is defined on an ancestor of the element carrying the ssml:ph attribute, a rendering engine will interpret it correctly (and your document will be valid). (The root element is the ancestor of all the elements in the document, which is why these kinds of declarations are invariably found on it, in case you’ve ever wondered but were afraid to ask.)

Our markup can now be reduced to the much more legible and easily maintained:

<p>
    The guitarist was playing a bass that was shaped like a
    <span ssml:ph="b&amp;s">bass</span>.
</p>

But heteronyms are far from the only case for SSML. Any language construct that can be voiced differently depending on the context in which it is used is a candidate for SSML. Numbers are always problematic, as are weights and measures:

<p>
    There are <span
    ssml:ph="w&quot;Vn T&quot;aUz@n t_hw&quot;En4i f&quot;O:r">1024</span> bits
    in a byte, not <span ssml:ph="t_h&quot;En t_hw&quot;En4i f&quot;O:r">1024</span>,
    as the year is pronounced.
</p>

<p>
    It reached a high of <span
    ssml:ph="'T3rti s&quot;Ev@n 'sEntI&quot;greId">37C</span> in the sun as I stood
    outside <span ssml:ph="'T3rti s&quot;Ev@n si">37C</span> waiting for someone
    to answer my knocks and let me in.
</p>

<p>
    You'll be an <span ssml:ph="Ekstr@ lArdZ">XL</span> by the end of Super Bowl
    <span ssml:ph="'fOrti">XL</span> at the rate you're eating.
</p>

But there’s unfortunately no simple guideline to give in terms of finding issues. It takes an eye for detail and an ear for possible different aural renderings. Editors and indexers are good starting resources for the process, as they should be able to quickly flag problem words during production so they don’t have to be rooted out after the fact. Programs that can analyze books and report on potentially problematic words, although not generally available, are not just a fantasy. Their prevalence will hopefully grow now that EPUB 3 incorporates more facilities to enhance default renderings, as they can greatly reduce the human burden.

The only other requirement when using the SSML attributes that I haven’t touched on is that you always have to declare the SSML namespace. I’ve omitted the declaration from the previous examples for clarity, and because the namespace is typically only specified once on the root html element as follows:

<html … xmlns:ssml="http://www.w3.org/2001/10/synthesis">

Similar to the alphabet attribute, we could have equally well attached the namespace declaration to each instance where we used the attributes:

<span
    xmlns:ssml="http://www.w3.org/2001/10/synthesis"
    ssml:ph="x-sampa"
    …>

But that’s a verbose approach to markup, and generally only makes sense when content is encapsulated and shared across documents, as I just noted, or expected to be extracted into foreign playback environments where the full document context is unavailable.

The question you may still be wondering at this point is what happens if a PLS file contains a pronunciation rule that matches a word that is also defined by an SSML pronunciation, how can you be sure which one wins? You don’t have to worry, however, as the EPUB 3 specification defines a precedence rule that states that the SSML pronunciation must be honored. There’d be no way to override the global PLS definitions, otherwise, which would make SSML largely useless in resolving conflicts.

But to wrap up, a final note is that there is no reason why you couldn’t make all your improvements in SSML. It’s not the ideal way to tackle the problem, because of the text-level recognition and tagging it requires, at least in this author’s opinion, but it may make more sense to internal production to only use a single technology and/or support for PLS may not prove universal (it’s too early to know yet).

CSS3 Speech

You might be thinking the global definition power of PLS lexicons combined with the granular override abilities of SSML might be sufficient to cover all cases, so why a third technology? But you’d be only partly right.

The CSS3 Speech module is not about word pronunciation, however. It includes no phonetic capabilities, but defines how you can use CSS style sheet technology to control such aspects of synthetic speech rendering as the gender of voice to use, the amount of time to pause before and after elements, when to insert aural cues, etc.

The CSS3 Speech module also provides a simpler entry point for some basic voicing enhancements. The ability to write X-SAMPA or IPA pronunciations requires specialized knowledge, but the speak-as property masks the complexity for some common use cases.

You could use this property to mark all acronyms that are to be spelled out letter-by-letter, for example. If we added a class called ‘spell’ to the abbr elements we want spelled, as in the following example:

<abbr class="spell">IBM</abbr>

we could then define a CSS class to indicate that each letter should be voiced individually using the spell-out value:

.spell {
    -epub-speak-as: spell-out
}

It’s no longer left to the rendering engine to determine whether the acronym is “wordy” enough to attempt to voice as a word now.

The speak-as property provides the same functionality for numbers, ensuring they get spoken one digit at a time instead of as a single number, something engines will not typically do by default.

.digits {
    -epub-speak-as: digits
}

Adding this class to the following number would ensure that readers understand you’re referring to the North American emergency line when listening to TTS playback:

<span class="digits">911</span>

The property also allows you to control whether or not to read out punctuation. Only some punctuation ever gets announced in normal playback, as it’s generally used for pause effects, but you could require all punctuation to be voiced using the literal-punctuation value:

.punctuate {
    -epub-speak-as: literal-punctuation
}

This setting would be vital for grammar books, for example, where you would want the entire punctuation for each example to be read out to the student. Conversely, to turn punctuation off you’d use the no-punctuation value.

The speak-as property isn’t a complex control mechanism, but definitely serves a useful role. Even if you are fluent with phonetic alphabets, there’s a point where it doesn’t make sense to have to define or write out every letter or number to ensure the engine doesn’t do the wrong thing, and this is where the Speech module helps.

Where the module excels, however, is in providing playback control. But this is also an area where you may want to think twice before adding your own custom style sheet rules. Most reading systems typically have their own internal rules for playback so that the synthetic speech rendering doesn’t come out as one long uninterrupted stream of monotone narration. When you add your own rules, you have the potential to interfere with the reader’s default settings. But in the interests of thoroughness, we’ll take a quick tour.

The first stop is the ability to insert pauses. Pauses are an integral part of the synthetic speech reading process, as they provide a non-verbal indication of change. Without them, it wouldn’t always be clear if a new sentence were beginning or a new paragraph, or when one section ends and another begins.

The CSS3 Speech module includes a pause property that allows you to control the duration to pause before and after any element. For example, we could define a half-second pause before headings followed by a quarter-second pause after by including the following rule:

h1 {
    -epub-pause: 50ms 25ms
}

Aural cues are equally helpful when it comes to identifying new headings, as the pause alone may not be interpreted by the listener as you expect. The Speech module includes a cue property for exactly this purpose:

h1 {
    -epub-pause: 50ms 25ms;
    -epub-cue: url('audio/ping.mp3') none
}

(Note that the addition of the none value after the audio file location. If omitted, the cue would also sound after the heading was finished.)

And finally, the rest property provides fine-grained control when using cues. Cues occur after any initial pause before the element (as defined by the pause property), and before any pause after. But you may still want to control the pause that occurs between the cue sounding and the text being read and between the end of the text being read and the trailing cue sounding (i.e., so that the sound and the text aren’t run together). The rest property is how you control the duration of these pauses.

We could update our previous example to add a 10 millisecond rest after the cue is sounded to prevent run-ins as follows:

h1 {
    -epub-pause: 50ms 25ms;
    -epub-cue: url('audio/ping.mp3') none;
    -epub-rest: 10ms 0ms
}

But again, if I didn’t say it forcefully enough earlier, it’s best not to tweak these properties unless you’re targeting a specific user group, know their needs, and know that their players will not provide sufficient quality “out of the box.” Tread lightly, in other words.

A final property, that is slightly more of an aside, is voice-family. Although not specifically accessibility related, it can provide a more flavorful synthesis experience for your readers.

If your ebook contains dialogue, or the gender of the narrator is important, you can use this property to specify the appropriate gender voice. We could set a female narrator as follows:

body {
    -epub-voice-family: female
}

and a male class to use as needed for male characters:

.male {
    -epub-voice-family: male
}

If we added these rules to a copy of Alice’s Adventures in Wonderland, we could now differentiate the Cheshire Cat using the male voice as follows:

<p>
    Alice: But I don't want to go among mad people.
</p>

<p class="male">
    The Cat: Oh, you can't help that.
    We're all mad here. I'm mad. You're mad.
</p>

You can also specify different voices within the specified gender. For example, if a reading system had two male voices available, you could add some variety to the characters as follows by indicating the number of the voice to use:

.first-male {
    -epub-voice-family: male 1
}

.second-male {
    -epub-voice-family: male 2
}

At worst, the reading system will ignore your instruction and only present whatever voice it has available, but this property gives you the ability to be more creative with your text-to-speech renderings for those systems that do provide better support.

Whatever properties you decide to add, it is always good practice to separate them into their own style sheet. You should also define them as applicable only for synthetic speech playback using a media at-rule as follows:

@media speech {

    .spell {
        -epub-speak-as: spell-out
    }

}

As I noted earlier, reading systems will typically have their own defaults, and separating your aural instructions will allow them to be ignored and/or turned off on systems where they’re unwanted.

For completeness, you should also indicate the correct media type for the style sheet when linking from your content document:

<link rel="stylesheet" media="speech" href="synth.css" />

And that covers the full range of synthetic speech enhancements. You now have a whole arsenal at your disposal to create high-quality synthetic speech.