Developing a Multimodal Cancer Pain Database to Support AI-Based Automatic Pain Assessment (SENSAI-DBD)

SENSAI: Seeing, hEaring, seNsing: Smart, Effortless and Objective Pain Assessment With Mobile AI Technology - DataBase Development

The goal of this observational study is to collect short video and sound recordings of people with cancer to create a secure database that can be used in future research to develop an artificial intelligence (AI) tool for pain assessment. The main aim is to build a large, high-quality collection of audiovisual data showing how people with cancer express themselves when they do and do not have pain.

Participants will include adults with cancer who are admitted to the oncology ward for pain treatment and a control group admitted for chemotherapy who have no pain. After giving consent, participants will:

• Be recorded on video (from the shoulders up) for up to 60 seconds while reading a short sentence and describing their pain or daily experience.
• Complete a short questionnaire about their mood and pain expression.
• Allow researchers to collect some information from their medical record, such as their pain score, medications, and cancer type.

These recordings will be securely stored and used to create a database for future AI research. No medical tests, new treatments, or extra hospital visits are involved. This study will provide the foundation for developing future AI-based tools that could support doctors and patients in monitoring and managing pain more accurately and easily.

Study Overview

• Status: Recruiting
• Conditions: Cancer; Oncology; Facial Expression; Voice; Pain Assessment; Database; Cancer-related Pain; Oncology Pain; Artificial Intelligence (AI)
• Intervention / Treatment: Other: Audiovisual and questionnaire-based data collection (consecutive); Other: Audiovisual and questionnaire-based data collection (once)

Detailed Description

This is a prospective, single-centre database development study conducted at the Erasmus MC Cancer Institute. The purpose of the study is to create a high-quality audiovisual database for future research on automated pain assessment in people with cancer. The database will include short (up to 60 seconds) video and sound recordings of participants' facial and vocal expressions, alongside relevant clinical and demographic information.

The study will include two groups: (1) adults admitted to the oncology ward for cancer-related pain (pain group) and (2) adults admitted for chemotherapy who report no pain (control group). Participants will provide written informed consent before any recording takes place.

Each participant will be recorded from the shoulders up while reading a neutral prompt and, if applicable, describing their pain experience. The recordings will be securely stored in compliance with the General Data Protection Regulation (GDPR). In addition, researchers will collect secondary parameters from the participants' electronic medical record, such as pain scores, analgesic use, tumour type, and clinical status. Participants in the pain group may be recorded on multiple days during admission, while controls will be recorded once.

The audiovisual recordings will be used to extract facial and vocal features (e.g., Facial Action Units, voice parameters) with open-source software such as OpenFace and OpenSmile. These features will form the foundation for future artificial intelligence (AI) model development aimed at automatic pain assessment.

No experimental interventions, additional clinical procedures, or diagnostic tests are part of this study. The study carries minimal risk to participants, primarily related to the handling of identifiable audiovisual data. All data will be stored on Erasmus MC's secure Research Storage & Compute infrastructure, accessible only to authorised researchers.

The resulting dataset will be used to develop and validate AI models that can objectively estimate pain intensity from audiovisual data, supporting more accurate and continuous pain monitoring in clinical care.

• Study Type: Observational
• Enrollment (Estimated): 200

Contacts and Locations

• Study Contact: Name: Marsha Kamsteeg, MSc; Phone Number: +31 107040704; Email: m.kamsteeg@erasmusmc.nl

Study Locations

• Netherlands
  • South Holland
    • Rotterdam, South Holland, Netherlands, 3015 GD
      • Recruiting
      • Erasmus University Medical Centre
      • Contact: Marsha Kamsteeg, MSc; Phone Number: +31 107040704; Email: m.kamsteeg@erasmusmc.nl
      • Contact: Email: m.kamsteeg@erasmusmc.nl
      • Sub-Investigator: Marsha Kamsteeg, MSc.
      • Principal Investigator: Mark Mulder, MD, PhD

Participation Criteria

Eligibility Criteria

• Ages Eligible for Study: Adult; Older Adult
• Accepts Healthy Volunteers: No

• Sampling Method: Probability Sample

Study Population

Hospitalised adult cancer patients recruited from the oncology and palliative care departments of Erasmus University Medical Center (Erasmus MC) in Rotterdam, the Netherlands. The study includes both patients admitted for cancer-related pain and those admitted for treatment without pain. Participants represent an inpatient clinical population from a tertiary referral centre, providing a controlled setting for audiovisual and clinical data collection.

Description

Inclusion Criteria:

• Adult patients (≥18 years)
• Diagnosed with cancer, active
• Able to communicate verbally in Dutch or English
• Able to provide written informed consent.
• Pain group specific:
• Experiencing pain related to cancer
• Admitted to the hospital due to pain
• Control group specific:
• Not experiencing pain (NRS = 0)
• Admitted to the hospital for chemotherapy

Exclusion Criteria:

• Cognitive, physical, or medical limitations that prevent participation in the audiovisual recording sessions or affect facial expressions or voice (e.g. facial paralysis, tracheostomy, severe speech impairment).
• Critical illness or end-of-life care where participation would impose an additional burden.
• Experiencing pain not associated with cancer
• Infectious isolation precautions that prevent safe data collection

Study Plan

How is the study designed?

Number of groups / cohorts

2

Cohorts and Interventions

Group / Cohort	Intervention / Treatment
Pain Group; Adults (18 years and older) with active cancer who are admitted to the oncology ward for cancer-related pain. Participants will be recorded on video (from the shoulders up) for up to 60 seconds while reading a short sentence and describing their pain experience. They will also complete a short questionnaire about their mood and pain expression. Audiovisual data and selected clinical information (such as pain scores and medications) will be collected from their medical record. Participants may be recorded on several consecutive days during admission.	Other: Audiovisual and questionnaire-based data collection (consecutive); Participants will be video recorded for up to 60 seconds from the shoulders up while reading a short sentence and describing their pain or daily experience. They will also complete a brief questionnaire about their mood and how they express pain. No drugs, medical devices, or clinical treatments are used. This non-invasive data collection is performed on several consecutive days during admission for participants with pain.; Other Names: Video recording procedure; Mood questionnaire; Pain questionnaire
Control Group; Adults (18 years and older) with active cancer who are admitted to the oncology ward for chemotherapy and report no pain (Numerical Rating Scale score = 0). Participants will be recorded on video (from the shoulders up) for up to 60 seconds while reading a short sentence and describing their daily experience. They will also complete a short questionnaire about their mood and pain expression. Only one recording will be made for each participant.	Other: Audiovisual and questionnaire-based data collection (once); Participants will be video recorded for up to 60 seconds from the shoulders up while reading a short sentence and describing their pain or daily experience. They will also complete a brief questionnaire about their mood and how they express pain. No drugs, medical devices, or clinical treatments are used. This non-invasive data collection is performed once for participants without pain.; Other Names: Video recording procedure; Mood questionnaire; Pain questionnaire

What is the study measuring?

Primary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Audiovisual recordings of facial and vocal pain expression	The main endpoint consists of audiovisual recordings of each patient's face (from the shoulders up) and voice while the patient reads a short standardised sentence and describes their pain experience. Recordings (maximum 60 seconds after filtering for relevant parts) will be stored digitally and used as input data for the training and validation of an AI-based pain assessment algorithm.	Up to 2 weeks

Secondary Outcome Measures

Outcome Measure	Measure Description	Time Frame
Clinical and patient-reported parameters from electronic health records	Secondary data used to label the audiovisual recordings and provide context for model development, including patient-reported pain intensity (NRS 0-10), patient-reported change in pain (increasing, decreasing, no change), patient description of pain (short annotations), quality of life (if reported), observer-reported pain intensity (NRS 0-10), analgesics in use (drug name, dose, frequency), cancer tumour type, stage, Karnofsky score, and current treatment, demographics: age, sex, height, weight	At each recording session
Patient-reported mood and pain questionnaire outcomes	Assessment of patients' self-reported mood and perceived pain expression using a short questionnaire administered alongside audiovisual recordings. The questionnaire evaluates: current mood state, perceived intensity and expressiveness of pain, ease or difficulty in expressing pain verbally and non-verbally. Responses will be used to explore associations between subjective reports and multimodal AI-derived pain indicators.	During each recording session

Collaborators and Investigators

• Sponsor: Dr. Mark Mulder
• Collaborators: Delft University of Technology
• Investigators: Principal Investigator: Mark Mulder, dr.ir., Erasmus University Medical Centre

Study record dates

Study Major Dates

• Study Start (Actual): December 1, 2025
• Primary Completion (Estimated): December 31, 2027
• Study Completion (Estimated): December 31, 2027

Study Registration Dates

• First Submitted: November 20, 2025
• First Submitted That Met QC Criteria: December 1, 2025
• First Posted (Actual): December 3, 2025

Study Record Updates

• Last Update Posted (Actual): March 11, 2026
• Last Update Submitted That Met QC Criteria: March 10, 2026
• Last Verified: March 1, 2026

More Information

Terms related to this study

• Keywords: Oncology; Database; Machine learning; Pain assessment; Acoustic; Artificial Intelligence (AI); Cancer-related Pain; Vocalizations; Oncology Pain; Facial Action Unit; Automatic assessment; Audiovisual data; Paralinguistics
• Additional Relevant MeSH Terms: Pain; Neurologic Manifestations; Pathological Conditions, Signs and Symptoms; Behavior; Signs and Symptoms; Behavior, Animal; Communication; Nonverbal Communication; Animal Communication; Cancer Pain; Neoplasms; Facial Expression; Vocalization, Animal
• Other Study ID Numbers: 11388; MEC-2024-0601 (Other Identifier: Medical Ethics Review Committee Erasmus University Medical Centre)

Plan for Individual participant data (IPD)

• Plan to Share Individual Participant Data (IPD)?: YES

IPD Plan Description

Individual participant data (IPD) in the form of non-identifiable, derived features will be shared for research purposes. The shared data will include automatically extracted facial and vocal features (e.g., facial action units, voice parameters) obtained from audiovisual recordings of participants, along with relevant clinical and demographic variables. The original audiovisual recordings will not be shared, as they contain identifiable information (faces and voices).

Access to the derived dataset will be provided to qualified researchers upon reasonable request and after approval of a data sharing agreement that ensures compliance with privacy regulations, including the General Data Protection Regulation (GDPR). Data will be shared only for scientific and non-commercial research.

• IPD Sharing Time Frame: The de-identified dataset containing extracted facial and vocal features, along with relevant clinical and demographic variables, will be made available after publication of the main study results and within 12 months of study completion. Data will remain accessible for at least 5 years after the initial release, or longer if there is continued scientific interest and compliance with privacy regulations.

IPD Sharing Access Criteria

Access to de-identified participant data and supporting materials will be granted to qualified academic researchers upon reasonable request. Interested researchers must submit a written proposal outlining their research objectives and analysis plan. Access will be provided only after approval of a data sharing agreement that ensures ethical use of the data and compliance with General Data Protection Regulation (GDPR) requirements. The original audiovisual recordings containing identifiable information will not be shared.

Data will be made available through controlled access managed by the research team at Erasmus MC.

• IPD Sharing Supporting Information Type: STUDY_PROTOCOL; SAP; ICF

Drug and device information, study documents

• Studies a U.S. FDA-regulated drug product: No
• Studies a U.S. FDA-regulated device product: No