Jakub Sikora

dblp:288/1180 · DBLP profile ↗
← Back
1ranked-venue papers
0as first author
1since 2021 · last 2026
—ORCID · unresolved

Domains — the database's venue-derived domains; a paper can count in several

Graphics, computer vision, multimedia, augmented reality and games · 1 · 1 since 2021Human-computer interaction and ubiquitous computing · 1 · 1 since 2021

Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.

Interdisciplinary, comprehensive, and emerging computing
1 paper
Medical and health informatics · 100%
Artificial intelligence
1 paper
Speech recognition and synthesis · 50% Information extraction and text analysis · 50%

Topics — the 3 heaviest of 3, each with the papers that count most for it

TopicWeightPapersLastEvidence papers
Medical and health informatics › digital health
digital biomarkers
1.012026
Multimodal Analysis of Speech-Gaze Fusion in Mixed Reality for the Detection of Neurodegenerative Disorders · VR 2026
Natural language and speech › Information extraction and text analysis › data annotation
linguistic annotation
0.312026
Multimodal Analysis of Speech-Gaze Fusion in Mixed Reality for the Detection of Neurodegenerative Disorders · VR 2026
Natural language and speech › Speech recognition and synthesis
speech analysis
0.312026
Multimodal Analysis of Speech-Gaze Fusion in Mixed Reality for the Detection of Neurodegenerative Disorders · VR 2026

Methods — techniques the papers use, named apart from their topics

nearest-centroid classification · 2.0mann-whitney u · 2.0force alignment · 2.0cliff's delta · 2.0FDR control · 2.0
YearPublicationVenuePosition
2026 Multimodal Analysis of Speech-Gaze Fusion in Mixed Reality for the Detection of Neurodegenerative Disorders
abstract
Mixed reality (MR) headsets can synchronously capture eye movements and speech during ecological tasks, enabling interpretable, multimodal behavioural assessment. This study introduces an MR-native pipeline that fuses gaze and speech to characterize Parkinson’s disease (PD) in 10 PD patients and 18 healthy controls (HC) during a 40 s picture description on Microsoft HoloLens 2.Audio is transcribed and force-aligned with word-level timestamps, linguistically annotated and converted into lexical (tokens, unique tokens, MTLD (measure of textual lexical diversity)), syntactic (proper nouns per 100 tokens), fluency (words-per-minute), and pause-based temporal features. Gaze is filtered and summarized into kinematic measures (mean gaze speed, mean gaze acceleration, and acceleration variability) and fixation rate. Aligned gaze speech segments were independently rated for correspondence, yielding a per-participant alignment accuracy used in downstream analysis.Group contrasts use Mann–Whitney U, Cliff’s δ, and FDR control (global and family-wise) and show a distributional shift toward lower alignment in PD. Speech-derived markers (total/voiced words per minute, tokens, unique tokens, MTLD) are reduced in PD, gaze fixation rate also trends lower; proper nouns per 100 tokens is higher in PD, indicating a higher rate of proper-noun usage relative to transcript length in this task. A compact Top-K set (K=7) yields meaningful multivariate separability (centroid distance 2.826, 95% CI [1.900,4.113]) and nearest-centroid balanced accuracy 0.733, which further improves when adding alignment as an 8th feature (distance 2.854, CI [1.981,4.149]; accuracy 0.783).MR offers clear advantages over conventional setups: the headset co-registers gaze and speech in situ without external rigs, preserves ecological validity, and supports repeatable, low-burden, time-synchronized capture in clinics and at home. These findings indicate that MR gaze–speech fusion can capture complementary PD deficits and suggests a scalable path toward interpretable digital biomarkers. However, the conclusions are constrained by the limited sample size, and future validation in larger, independent cohorts is required to confirm generalizability and clinical utility.
Milosz Dudek, Jakub Sikora, Daria Hemmerling, Mateusz Daniol, Marek Wodzinski, Magdalena Wójcik-Pedziwiatr
VR2