Citations

Purpose: This study examined whether phoneme-level posterior probabilities generated by an articulatory-informed deep learning model (Phonet) can serve as interpretable and sensitive markers of articulatory impairment in motor speech disorders (MSDs). The goal was to determine whether these features distinguish individuals with and without MSDs and reflect varying severity levels of articulatory impairment. Method: A total of 676 participants (199 with MSDs, 477 without MSDs) completed a remotely captured word repetition task. Speech recordings were phonetically aligned, and posterior probabilities were extracted using a retrained version of the Phonet model, which estimates frame-level probabilities for 23 phonological classes. Segment-level posterior values were computed and compared across groups and severity levels using linear mixed-effects regression models. Results: Posterior probabilities were significantly reduced in the MSD group across all phoneme types. Within the MSD group, posterior probabilities decreased progressively with greater clinical severity, as measured by both perceptual sound distortion ratings and a cumulative articulation error score. Segment-specific analyses revealed systematic differences across phoneme classes, indicating that posterior probabilities capture distinct patterns of articulatory degradation associated with motor speech impairment. Conclusions: Phoneme-level posterior probabilities serve as interpretable, segment-specific indicators of articulatory impairment in MSDs. These features differentiate clinical status and reflect gradations of severity, making them promising tools for augmenting traditional perceptual speech assessments. Their linguistic transparency and scalability support future clinical applications in diagnosis, monitoring, and targeted intervention.

Abstract Purpose To evaluate whether two automated measures of nasality, Nasal Cognate Distinctiveness (NCD) and nasal posterior probability (NPP), provide clinically meaningful markers of hypernasality in speakers with motor speech disorders (MSDs) and to determine whether hypernasality-related information extends beyond stop consonants, which are typically targeted in cognate-based approaches. Method Sentence-level speech recordings were obtained from 561 patients, including 374 neurologically typical controls and 187 individuals with MSDs. NCD was computed for voiceless stops (/p/, /t/, /k/) using likelihood ratios between stops and their nasal cognates. NPP was extracted using a phonological feature-based neural network model and analyzed for stops, consonants, and vowels. Hypernasality severity was rated by speech-language pathologists. Group differences and associations with hypernasality severity were evaluated using linear mixed-effects models and Spearman correlations, and receiver operating characteristic (ROC) analyses were used to evaluate discrimination of absent versus present hypernasality. An additional 20 controls with repeat recordings were used to evaluate test-retest reliability and minimal detectable change (MDC). Results NCD generally decreased and NPP increased with increasing hypernasality severity, with phoneme-specific severity effects observed for NCD. Across stops, NCD and NPP were strongly inversely correlated and showed comparable discrimination of absent versus present hypernasality (AUCs = 0.796 and 0.730, respectively). Consonant and vowel NPP measures were both positively associated with hypernasality severity and demonstrated identical classification performance (AUC = 0.853). No significant differences in ROC performance were observed between NCD and NPP or between consonant and vowel NPP measures. Test-retest change exceeded the MDC threshold in 2 of 20 controls for NCD and 1 of 20 for NPP. Conclusions Both NCD and NPP provide objective markers of hypernasality in speakers with MSDs. The measures capture related aspects of nasalization and demonstrate comparable ability to identify clinically meaningful hypernasality. NPP captures hypernasality-related acoustic information across both consonants and vowels, with similar relationships to severity and classification performance across sound classes. Test-retest findings provide preliminary MDC benchmarks for interpreting longitudinal change.

Purpose: Depression is prevalent yet frequently underdiagnosed. Although speech-based detection methods show promise, most prior studies have emphasized binary classification in nontonal languages. This study examined whether acoustic features of spontaneous Mandarin speech reflect depression as a categorical condition or a continuous, dimensional construct. Method: A validated emotional Mandarin speech corpus paired with self-reported depression severity scores was analyzed. Acoustic features were extracted using the extended Geneva Minimalistic Acoustic Parameter Set. A multistage analytic framework was applied, including random forest classification to distinguish depressed from nondepressed clips, linear mixed-effects modeling to examine group and severity effects, and unsupervised clustering to examine latent structure in the acoustic feature space. Results: The random forest classifier achieved an accuracy of 72.3% and an area under the receiver operating characteristic curve of 0.826 on the held-out test set. Model performance favored sensitivity over precision in identifying depressed clips. SHapley Additive exPlanations analysis identified features across multiple acoustic domains, including spectral, pitch-related, voice quality, cepstral, and intensity measures, as important contributors to model predictions. Among the Top 10 features, no acoustic features showed statistically significant group differences. In contrast, median and mean fundamental frequencies were significantly positively associated with depression severity. Clustering analysis revealed overlapping group structures and a continuous distribution of severity scores across clusters. Conclusions: These findings provide preliminary support for a dimensional conceptualization of depression in spontaneous Mandarin speech. While classification models can distinguish depressed from nondepressed speech with moderate accuracy, acoustic features appear to vary more consistently with symptom severity than with diagnostic group. Speech-based measures may therefore have potential for continuous monitoring of depression, although this possibility requires further longitudinal verification.

Abstract Motor speech disorders (MSDs) are early markers of neurological disease, but expert perceptual analysis is rarely available outside specialized centers. Automated speech analysis offers a scalable alternative, yet prior studies have not systematically compared modeling approaches or assessed clinically relevant metrics in independent datasets. This study compared static acoustic features, articulatory informed Phonet features, and self-supervised pretrained models for binary and multi label MSD classification. We trained and evaluated models on 583 speech samples using speaker level splits. Baseline models included logistic regression and Gated Recurrent Units (GRUs) trained on eGeMAPS and MFCCs. We extracted three types of Phonet derived features and evaluated pretrained HuBERT and SSAST models in frozen, partially fine-tuned, and fully fine-tuned configurations. Binary classification distinguished MSDs from controls, while multi label classification identified six MSD subtypes. Models were assessed using validation AUC, and cut points were tested on two independent datasets. Pretrained and Phonet based models substantially outperformed static acoustic features. In binary classification, HuBERT achieved the highest AUC (0.95), while compact Phonet derived GRUs achieved comparable performance (up to 0.94). These models generalized well to independent datasets, maintaining high sensitivity (0.94) and specificity (0.97). In multi label classification, Phonet models achieved the highest macro average AUC (0.86), but threshold-based subtype performance declined on unseen data. Automated MSD detection is feasible and clinically promising. Binary classification generalized well, whereas multi label classification showed limited threshold stability across datasets.

Abstract This study examined the extent to which goodness of pronunciation (GoP) scores and phonological posterior probabilities capture perceptual ratings of speech severity in individuals with motor speech disorders (MSD). Speech recordings of the word catastrophe were obtained from 489 participants, including 333 neurologically typical controls and 156 individuals with MSD. GoP scores were derived using traditional acoustic features and self-supervised speech representations, including WavLM and XLS-R, across multiple modeling approaches, while phonological posterior probabilities were extracted using Phonet. Model performance was evaluated using Kendall’s rank correlations, regression, and receiver operating characteristic analyses against speech-language pathologists’ perceptual ratings of sound distortion and intelligibility. Both GoP and phonological posterior probabilities were significantly associated with perceptual ratings. Self-supervised speech representations substantially outperformed traditional acoustic features, with WavLM-based GoP using k-nearest neighbors achieving the strongest performance. Across correlation, regression, and classification analyses, GoP consistently outperformed phonological posterior probabilities for both sound distortion and intelligibility. Age and gender had minimal influence on model-derived measures or their relationships with perceptual ratings. These findings demonstrate the value of self-supervised GoP as an objective measure of speech impairment while highlighting the complementary role of phonological posterior probabilities in characterizing articulatory aspects of motor speech disorders.

This study investigates how word-initial /l/ and /n/ are phonetically realized in Mandarin Chinese as produced by speakers from three regions: Beijing (close to Standard Mandarin), Changsha (where Xiang, which contains /l/ but not /n/, is also spoken locally), and Meizhou (where Hakka, which contains both /l/ and /n/, is also spoken locally), focusing on acoustic cue use and phonological context effects. Using acoustic analysis and random forest classification, we analyzed five acoustic measures (F2–F1 spacing, F3 frequency, BW1, ΔA1, and A1–P0) and evaluated their contribution to distinguishing the two sounds. Results showed that speakers from different regions used distinct cue weighting strategies: Beijing Mandarin speakers relied more on spectral cues, Meizhou Mandarin speakers showed greater reliance on nasality-related cues, and Changsha Mandarin speakers combined both, with greater variability. Phonological context, including preceding nasals and following vowels, further modulated cue realization in region-specific ways. These findings demonstrate how native phonological background and phonological environment shape the phonetic implementation of a shared Mandarin contrast and illustrate the value of combining acoustic and classification-based approaches for modeling gradient phonetic variation.

Abstract This study examined the acoustic profiles of five basic emotions in American English and Mandarin Chinese using a big data approach. A total of 6,373 features were extracted using the openSMILE toolkit, and key discriminative features were identified through random forest classification. In American English, vocal emotions were primarily conveyed through pitch-related features, while Mandarin Chinese, shaped by its tonal constraints, relied more on spectral and voice quality cues, including MFCCs, HNR, and shimmer. Linear mixed-effects models confirmed significant effects of emotion on the top-ranked features, and Cohen’s d further supported distinct acoustic profiles for each emotion. K-means clustering revealed both categorical groupings and dimensional overlaps, such as the clustering of high-arousal emotions like happy and surprised, and low-arousal emotions like sad and neutral. These results suggest that vocal emotion expression is shaped by language-specific prosodic systems, as well as by both discrete emotion categories and continuous affective dimensions, supporting an integrated model of emotional prosody.

Dysarthric speech poses significant challenges for clinical assessment and diagnosis due to wide variability in the nature and severity of speech impairments associated with it. To address this, this study investigates the phonetic variations in different types of dysarthric speech using the deep learning model Phonet. Recordings of the word “Catastrophe” from 54 patients diagnosed with dysarthria were analyzed to extract posterior probabilities of consonants and vowels. The analysis revealed distinct posterior probability patterns for consonants and vowels across three types of dysarthria, indicating disorder-specific articulatory-based speech characteristics. Moreover, within each dysarthric group, variations in posterior probabilities aligned with the severity of rated articulation difficulty, reflecting nuanced intra-disease phonetic differences. These findings demonstrate Phonet’s ability to capture subtle speech variations and link them to both dysarthria type and severity. By quantifying these phonetic alterations, this study offers a novel and robust framework for objectively evaluating articulation difficulties in dysarthric speech. The results emphasize the clinical potential of Phonet as an advanced tool for improving the diagnosis and monitoring of dysarthria, paving the way for innovative applications in speech-based clinical assessments.