Assessing Performance on Speech Tasks Via Crowdsourced Participants (AMYWEB)

AMYWEB Validation Studies: Assessing Performance on Speech Tasks Via Crowdsourced Participants

The purpose of the study is to examine the properties of the Automatic Story Recall Test (ASRT) and its parallel variants, as well as letter fluency and category fluency cognitive tests. Tests will be completed in crowdsourced populations, to derive normative data, and examine test properties in demographically diverse and cognitively impaired participants recruited and tested online.

Study Overview

• Status: Terminated
• Conditions: Cognitive Impairment; Mild Cognitive Impairment
• Intervention / Treatment: Other: No intervention/exposure

Detailed Description

The AMYWEB study will validate speech tasks developed by NOVOIC in different population samples, including normative samples, samples recruited for enhanced demographic diversity, and participants with reported cognitive impairment.

Participants will be research volunteers recruited online and via crowdsourcing platforms. Six rounds of recruitment will be completed, evaluating different parallel variants of the Automatic Story Recall Task (ASRT), and fluency tasks. During each recruitment round participants will complete a speech test battery and questionnaires during one assessment (Baseline). Speech tasks will be recorded and later analyzed on Novoic's technology platform. All assessments will be fully online and remote.

• Study Type: Observational
• Enrollment (Actual): 1034

Contacts and Locations

• Study Contact: Name: Marton Meszaros, MD; Phone Number: 07849522891; Email: amyweb@novoic.com

Study Locations

• United Kingdom
  • London, United Kingdom
    • Novoic Ltd

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: 18 years and older (Adult, Older Adult)
• Accepts Healthy Volunteers: N/A

• Sampling Method: Non-Probability Sample

Study Population

Participants will be recruited online, primarily through crowdsourcing websites.

Description

Inclusion Criteria:

• First language must be English
• Residing in the USA
• Willing to participate in a study investigating speech and dementia
• Able to provide informed consent
• Access to personal computing device that is capable of audio and video recording, and able to connect to the internet

Since key research questions are focused primarily on older-age speech and cognition, initial recruitment will be focused on older adult age brackets, which may be extended to younger adulthood if the desired sample sizes are not attained.

Exclusion Criteria:

• No formal exclusion criteria will apply at the point of recruitment and testing. however, participants with a reported history of neurological conditions or head injury, and current depression will be excluded from normative data development.

Study Plan

How is the study designed?

Number of groups / cohorts

3

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
AMYWEB-Normative; This cohort assessed over six recruitment and test phases, will be sampled to be as closely representative of the demographic population as possible.	Other: No intervention/exposure; No interventions/exposures will be administered or evaluated
AMYWEB-Diversity; This cohort assessed over two recruitment and test phases will oversample participants from minority and mixed ethnic and racial backgrounds as well as participants from lower educational backgrounds.	Other: No intervention/exposure; No interventions/exposures will be administered or evaluated
AMYWEB-MCI; This cohort assessed and recruited over one test phase, will include participants with a reported diagnosis of mild cognitive impairment or dementia.	Other: No intervention/exposure; No interventions/exposures will be administered or evaluated

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Age, sex, and education adjusted normative scores for the Automatic Story Recall Task (ASRT) test variants derived via linear regression analysis.	Participants will complete a story recall task (ASRT), which will be administered in triplets (3x stories) with immediate and delayed recall assessed. Six different test phases, recruiting different participant samples, will be completed in which six parallel ASRT triplets will be separately evaluated. Normative data will be generated for each test triplet.	Baseline

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Normative data will be derived for shorter variants of the ASRT test battery derived via linear regression analysis.	Age, sex, and education adjusted normative scores for shorter test batteries established in the same way as the primary outcome for each distinct ASRT story triplet, using data from: 1 immediate recall, 2 immediate recalls, 3 immediate recalls, 3 immediate + 1 delayed recall, 3 immediate + 2 delayed recalls.	Baseline
Age, sex, and education adjusted normative scores for the category fluency task performance derived via linear regression analysis.	During two of the study phases, participants will complete category fluency tests (participants will be asked to name as many Animals or Vegetables as they can within one minute).	Baseline
Age, sex, and education adjusted normative scores for the letter fluency task performance derived via linear regression analysis.	During two of the study phases, participants will complete letter fluency tests (participants will be asked to say as many words as they can think of beginning with the letters F or S).	Baseline
The effect of demographic differences on task performance will be evaluated via linear regression analysis.	The effects of age, sex, education and ethnicity on cognitive outcome measures will be evaluated for all cognitive tasks separately.	Baseline
Demographic and cultural biases will be examined via group or pairwise comparison of transcription error rate and scoring errors on a simple reading task.	In data from a simple reading task evaluated in phases 1 and 2 of the study, transcription errors and automated scoring metrics will be evaluated alongside key demographic variables to test for cultural and demographic biases in automated transcription and scoring.	Baseline
Performance differences in parallel ASRT variant tasks will be examined via task-wise comparison.	The distribution of test scores across parallel ASRT variants will be evaluated.	Baseline
Area under the curve (AUC) of the receiver operating characteristic (ROC) curve of the binary classifier distinguishing between the MCI sample and the normative sample using as input the speech elicited during cognitive tasks.	Case control comparison of task performance between participants with AMYWEB-MCI and the AMYWEB-Normative population	Baseline
Area under the curve (AUC) of the receiver operating characteristic (ROC) curve of the binary classifier distinguishing between depressed and non-depressed participants using as input the speech elicited during cognitive tasks.	All participants will complete the Participant Health Questionnaire (PHQ-8) self-report measure, alongside cognitive assessment. Participants will be separated into low depressive symptomatology (PHQ-8 scores<10), and high depressive symptomatology (PHQ-8 score>=10) groups.	Baseline
The agreement between the PHQ-8 and the corresponding regression model predicting depressive symptomatology from speech data elicited during tasks will be evaluated.	All participants will complete the Participant Health Questionnaire (PHQ-8) self-report measure, alongside cognitive assessment. A continuous predictor of PHQ-8 score from speech data will be derived and covariation with continuous PHQ-8 scores will be evaluated.	Baseline

Collaborators and Investigators

• Sponsor: Novoic Limited

Study record dates

Study Major Dates

• Study Start (Actual): April 5, 2022
• Primary Completion (Actual): July 8, 2022
• Study Completion (Actual): July 8, 2022

Study Registration Dates

• First Submitted: March 17, 2022
• First Submitted That Met QC Criteria: March 25, 2022
• First Posted (Actual): March 28, 2022

Study Record Updates

• Last Update Posted (Actual): January 31, 2024
• Last Update Submitted That Met QC Criteria: January 29, 2024
• Last Verified: March 1, 2022

More Information

Terms related to this study

• Additional Relevant MeSH Terms: Mental Disorders; Neurocognitive Disorders; Cognition Disorders; Cognitive Dysfunction
• Other Study ID Numbers: NOV-0300

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: UNDECIDED

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No