Alzheimer’s Disease and Speech Background

Abstract

Developing a possible diagnostic test for Alzheimer’s disease (AD) based on speech is used throughout this book to illustrate the application of the machine learning methods we describe. This application has many of the characteristics typical of such tasks: one has some idea that a set of features characterizing samples that one possesses might be able to classify the objects into two or more classes. Lacking sufficient depth of understanding of how this might be caused, one goes searching for useful patterns in the data—a fishing expedition.

This chapter provides background material on AD for the reader interested in how it is defined, what is known about its underlying pathology, how it is usually diagnosed in the clinic, and some of its known effects on speech. We then quickly summarize previous attempts at this task, older ones doing linguistic operations largely by hand, and more current attempts using computers. We then provide details of the speech samples we have collected, and how an array of speech features was extracted from them fully automatically, including speech to text with punctuation. Brief comments are included on issues related to experimental design.

Appendix: Features Extracted from Each Voice Sample

Here, we provide some details of the processing of each speech sample. Figure 4.3 gives an overview of the processing chain.

Table 4.2 lists the features extracted from the speech signal: wav format file with 16 bits sampled at 16 kHz. We implemented three approaches to breaking up the signal into periods of speaking and intervening pauses. One approach was based on detecting the pitch of the speaker’s voice using an approach called YAAPT (Yet Another Algorithm for Pitch Tracking) (Zahorian and Hu 2008). This produces results as illustrated in Fig. 4.3. A second approach, using energy in the signal and setting a threshold to separate speech from pause, was based on work by Sakhnov et al. (2009). A third approach used the Voice Activity Detector (VAD) of Sohn et al. (1999). Within each approach, a pause detector decided that any un-voiced period longer than 200 ms was a pause, and voiced segments had to be longer than 50 ms. In the literature on pauses, one may find thresholds ranging from 100 ms to 1 s. Our thresholds were set after some experimentation comparing manually determined pause segmented samples with the algorithmic outputs (Bologna et al. 2013).

In addition, we also sought a method for assessing emotion or affect that may be diminished in AD. Using the data from the pitch-based pause detector, and following the approach of Hewlett (2007), we calculated the mean, median, variance, min, and max of the pitch in each sample. We hypothesized that a low-pitch variance might capture a blunted affect. We also captured a speech rate measure from the pitch-based signal with the pauses removed (i.e. a shorter utterance length). These are also listed in Table 4.2.

Next, we describe the syntactic complexity features. These are computed using software generously shared by Roark et al. (2011). Two methods are used: Frazier (1985) and Yngve (1960). Both methods begin with a parse tree as produced by the Charniak parser (Charniak 2000). Parse trees indicate the syntactic structure of a sentence or phrase. They are made up of nodes and branches, with each node representing a grammatical category that is present in the sentence. Branches connect parent nodes and child nodes; child nodes are embedded within the grammatical structure that the parent node indicates. In Frazier scoring, each non-terminal node (any node that is not the end of a word’s grammatical structure) is given a score of 1. In some cases, such as sentence nodes, a score of 1.5 is given (Roark et al. 2011). Each word’s score is obtained by summing up the scores that are covered while tracing upward from the word to the top of the tree (Fig. 4.6). In Yngve scoring, each right-most branching node of the parse tree is given a score of 0, and each consecutive left branch has a 1 added to its score (Fig. 4.6). The score for a word is obtained by summing up scores as you move up the tree from a given word. Note that this example has no punctuation. The parser adds more nodes for punctuation, so the Frazier and Yngve scores will change. Our speech-to-text software inserts punctuation.

Fig. 4.6

Example of a parse tree showing Frazier (dashed lines) and Yngve scoring (numbers on each branch)

Another complexity measure was proposed by Rosenberg and Abbeduto (1987) called Developmental Level. The score on Developmental Level, a scale of speech complexity, ranges from 0 to 7 points. A score of 0 is given to a simple sentence with one clause. A score of 7 is given to a “complex” sentence that contains at least two of the pre-defined grammatical structures that are involved in D-Level scoring. We used code graciously provided by Roark et al. (2011) and summed the D-Level scores for all sentences in the sample (Table 4.3).

Table 4.3 Thirteen features computed by syntactic complexity code from parse tree

Another set of concepts derive from the findings from the Nun’s study and relate to idea density (Snowdon et al. 1996). Brown et al. (2008) implemented a set of rules outlined in the Computerized Propositional Idea Density Rater (CPIDR) program. Roark et al. (2011) developed their own extension of this rule set, called p_density. Starting with these rules, we conducted extensive experiments using the examples provided by Turner and Greene (1977), making additional changes to improve performance (idea_density). Another metric is content density, the ratio of open-class words to closed-class words. Open-class words are nouns, verbs, adjectives, adverbs, and symbols. Any other parts of speech are considered closed-class words. These metrics shown in Table 4.4, including the word count used for computing speech_rate metrics in Table 4.2.

Table 4.4 Five features computed by idea density and related code from parse tree

Various measures have been proposed to assess the richness of a vocabulary. They apply different formulae to some simple word counts: number of unique words used in the utterance, the total number of words (N), and the numbers of words used i times (V(i,N)) and V the frequency of the most prevalent word (Table 4.5).

Table 4.5 Nine features assessing vocabulary richness

The Unix Operating system long ago provided a suite of software tools to help users preparing documents to assess various elements of writing style. These are listed in Table 4.6. The metrics from the Lexical Inquiry Word Count system from Pennebaker et al. (2015) are also computed, but not listed here. The interested reader may consult the documentation from this system (LIWC2015) Pennebaker et al. (2015).

Table 4.6 Fourteen features computed by the Unix Style Code from transcripts

Table 4.7 Fraction of words in transcript by word type from the Charniak parser (Charniak 2000) and the PennTreeBank (Marcus et al. 1993) 40 features

Finally, we developed a simple word counting program that counts words and their synonyms in a transcript as listed in a control input file. We used this program to count words appropriate to a picture content and also words deemed to convey positive, neutral, and negative emotions. Such an approach has been taken previously by Hux et al. (2008) for the cookie theft picture. These control files we assembled by hand, and included words found in our transcripts, so we may anticipate additional effort may be needed to augment them as new voice samples are acquired. One may hope that once sufficient samples are in hand, this need to augment may diminish eventually to zero.

For each category of word counts (picture content, positive, neutral, negative emotion words), the program has a number of key concepts (n_keys), it counts the number of key concept words found in the transcript (n_keys_matched: each key counted only as present or not), and it reports n_keys_matched/n_keys. The keys are listed in Tables 4.8, 4.9, 4.10, 4.11, and 4.12.

Table 4.8 Key concepts for cookie theft picture

Table 4.9 Key concepts for lake picture

Table 4.10 Key concepts for positive emotion words

Table 4.11 Key concepts for neutral emotion words

Table 4.12 Key concepts for negative emotion words