Research Gap Analyzer

Transformer Models × Skiing · Foundation Models × Health

Updated 6/2/2026, 8:03:48 PM

Papers

3,903

Extracted

1,076

Gaps found

0

Source status

partial

Foundation Models × Health, lookback=730d (initial) · 3900 papers

Transformer Models × Skiing, lookback=730d (initial) · 3 papers

3903 seeds, 1-hop references · 27864 papers

Transformer Models × Skiing

3 papers
eSkiTB: A Synthetic Event-based Dataset for Tracking Skiers

Krishna Vinod, Joseph Raj Vishal, Kaustav Chanda et al. · 2026-01-01

Tracking skiers in RGB broadcast footage is challenging due to motion blur, static overlays, and clutter that obscure the fast-moving athlete. Event cameras, with their asynchronous contrast sensing, offer natural robustness to such artifacts, yet a controlled benchmark for winter-sport tracking has been missing. We introduce event SkiTB (eSkiTB), a synthetic event-based ski tracking dataset generated from SkiTB using direct video-to-event conversion without neural interpolation, enabling an iso-informational comparison between RGB and event modalities. Benchmarking SDTrack (spiking transformer) against STARK (RGB transformer), we find that event-based tracking is substantially resilient to broadcast clutter in scenes dominated by static overlays, achieving 0.685 IoU, outperforming RGB by +20.0 points. Across the dataset, SDTrack attains a mean IoU of 0.711, demonstrating that temporal contrast is a reliable cue for tracking ballistic motion in visually congested environments. eSkiTB establishes the first controlled setting for event-based tracking in winter sports and highlights the promise of event cameras for ski tracking. The dataset and code will be released at https://github.com/eventbasedvision/eSkiTB.

Transformer Models × SkiingPDF ↗
RP5 Analysis
qwen2.5:1.5bon-device

Research question

How can future research validate the synthetic results using smaller, real-world validation sets?

Methodology

eSkiTB is a synthetic dataset generated from SkiTB using direct video-to-event conversion without neural interpolation.

Conclusions

The study demonstrates that event-based tracking methods are resilient to broadcast clutter and outperform RGB by 10 points in IoU metrics across the eSkiTB dataset.

Importance

eSkiTB provides a new benchmark for event-based tracking in winter sports, highlighting temporal contrast as a reliable cue for tracking ballistic motion in visually congested environments.

Open gap

The system's design aims to test event representations and motivate adaptive approaches for handling large aerial excursions in fixed-grid voxelization.

Future research

Concrete next steps proposed include validating the synthetic results using smaller, real-world validation sets and incorporating an event-specific aerial motion model for improved recognition accuracy in low-resolution sequences.

Rapid climate change and other disturbances in alpine ecosystems demand frequent habitat monitoring, yet manual mapping remains prohibitively expensive for the required temporal resolution. We employ deep learning for change detection using long-term alpine habitat data from Gesaeuse National Park, Austria, addressing a major gap in applying geospatial foundation models (GFMs) to complex natural environments with fuzzy class boundaries and highly imbalanced classes. We compare two paradigms: post-classification change detection (CD) versus direct CD. For post-classification CD, we evaluate GFMs Prithvi-EO-2.0 and Clay v1.0 against U-Net CNNs; for direct CD, we test the transformer ChangeViT against U-Net baselines. Using high-resolution multimodal data (RGB, NIR, LiDAR, terrain attributes) covering 4,480 documented changes over 15.3 km2, results show Clay v1.0 achieves 51% overall accuracy versus U-Net's 41% for multi-class habitat change, while both reach 67% for binary change detection. Direct CD yields superior IoU (0.53 vs 0.35) for binary but only 28% accuracy for multi-class detection. Cross-temporal evaluation reveals GFM robustness, with Clay maintaining 33% accuracy on 2020 data versus U-Net's 23%. Integrating LiDAR improves semantic segmentation from 30% to 50% accuracy. Although overall accuracies are lower than in more homogeneous landscapes, they reflect realistic performance for complex alpine habitats. Future work will integrate object-based post-processing and physical constraints to enhance applicability.

Transformer Models × SkiingPDF ↗
RP5 Analysis
qwen2.5:1.5bon-device

Research question

Which paradigm is more suitable for detecting changes in alpine protected areas?

Methodology

The study uses advanced geospatial AI techniques, including post-classification change detection (Approach 1) and direct change detection (Approach 2), to evaluate their effectiveness on a unique 20-year ecological dataset with ground-truth changes.

Conclusions

Post-classification methods show better overall accuracy but lower multi-class accuracy compared to direct CD methods. LiDAR integration improves semantic segmentation from 31% to 40%, while not enhancing change detection accuracy due to practical limitations.

Importance

The study highlights the need for further post-processing and physical constraints in achieving more realistic results, emphasizing the importance of comprehensive comparative evaluations and addressing unique ecological dataset characteristics.

Open gap

The study concludes that post-classification methods are less accurate but more suitable for tasks with limited training data due to their ability to directly use pre-trained foundation models. The need for further investigation into architectural improvements remains, as well as integrating new training data and enhancing post-processing techniques.

Future research

Future research should focus on assembling diverse multimodal datasets and developing architectures specifically aimed at rare ecological transitions for more effective conservation management in dynamic environments.

Accurate skier tracking is essential for performance analysis, injury prevention, and optimizing training strategies in alpine sports. Traditional tracking methods often struggle with occlusions, dynamic movements, and varying environmental conditions, limiting their effectiveness. In this work, we used STARK (Spatio-Temporal Transformer Network for Visual Tracking), a transformer-based model, to track skiers. We adapted STARK to address domain-specific challenges such as camera movements, camera changes, occlusions, etc. by optimizing the model's architecture and hyperparameters to better suit the dataset.

Transformer Models × SkiingPDF ↗
RP5 Analysis
gemma4:e2bon-device

Research question

How can the STARK spatio-temporal transformer network be adapted to effectively handle domain-specific challenges like camera movements and occlusions in skier tracking?

Methodology

Adapting the STARK spatio-temporal transformer network, incorporating modifications like a Reattempt mechanism with dynamic search factor adjustment based on confidence, and an Incremental Template Update (ITU) strategy, along with a modified hierarchical score head architecture.

Conclusions

Finetuning the STARK model on the SkiTB dataset significantly enhances tracking performance compared to using a generic pretrained model. The adaptive modifications (ITU and Dynamic Search Factor) improve tracking stability in multi-camera sequences.

Importance

Accurate skier tracking is crucial for performance analysis and training optimization.

Open gap

The limitations of traditional methods due to occlusions and dynamic movements, and the need for robust handling of multi-camera transitions and rapid identity switches.

Future research

Developing more computationally efficient architectures for transformer models, refining confidence scoring mechanisms, and exploring computationally efficient tracking methodologies such as detection-based and joint detection/tracking methods.

Foundation Models × Health

3900 papers
ClinEnv: An Interactive Multi-Stage Long Horizon EHR Environment for Agents

Yuxing Lu, Yushuhong Lin, Wenqi Shi et al. · 2026-01-01

Clinical practice is not the selection of an answer from enumerated options: a physician gathers heterogeneous information incrementally and commits to sequential, irreversible decisions under uncertainty. Static benchmarks cannot probe and existing interactive medical benchmarks each compromise on at least one of them. We present ClinEnv, an interactive benchmark that evaluates LLMs as attending physicians over real inpatient admissions under a paradigm we term Longitudinal Inpatient Simulation. Each case is automatically constructed into an ordered sequence of decision stages; at every stage the model must actively query four specialized agents before committing to medications, procedures, and diagnoses. ClinEnv scores both what the model decides, through deterministic ontology-grounded matching, and how it gathers information. Across seven models, the strongest reaches only 0.31 decision F1, and outcome quality is sharply decoupled from process quality. Difficulty concentrates in management decisions and later stages, where models recover discharge diagnoses far more reliably than management actions (0.51 vs. 0.17 F1) and continue to issue redundant queries as cases progress. ClinEnv makes this information-acquisition gap, invisible to outcome-only evaluation, directly measurable.

Foundation Models × HealthPDF ↗
RP5 Analysis
gemma4:e2bon-device

Research question

not specified

Methodology

ClinEnv, an interactive multi-agent environment simulating real inpatient admissions through Longitudinal Inpatient Simulation, which requires models to actively query specialized agents for information before making decisions.

Conclusions

Outcome quality is decoupled from the efficiency of information gathering; the primary constraint is clinical reasoning rather than information access. Difficulty concentrates in later management stages, where models often fail to effectively gather necessary information. Decision type, specifically management, is the primary source of difficulty, and medication identity proves to be the binding difficulty, indicating that clinical competence relies more on judgment than on information retrieval.

Importance

The framework successfully makes the gap between information access and clinical reasoning directly measurable, demonstrating that information seeking is a distinct and measurable dimension of clinical competence.

Open gap

not specified

Future research

not specified

Self-harm is a major public health concern, but current surveillance relying on hospital presentations is inadequate due to the low sensitivity of diagnostic codes. Emergency Department (ED) triage notes, recorded at the initial point of contact, provide a succinct summary of presentations and an opportunity to identify self-harm. We developed a three-stage approach, augmenting traditional machine learning with large language model-based screening and evidence extraction to detect self-harm in ED triage notes. We assessed model transferability across three Australian hospitals. Our approach showed AUPRCs of 0.887 +/- 0.016 and 0.884 +/- 0.012 during internal and external validation. Prospectively, it achieved AUPRC of 0.881 +/- 0.008 at the development site, and 0.879 +/- 0.012 and 0.816 +/- 0.015 at two external sites without site-specific retraining. A key advantage of the approach is that it enables identification of the primary self-harm method with an accuracy of 95%, supporting more granular surveillance beyond binary classification.

Foundation Models × HealthPDF ↗
RP5 Analysis
qwen2.5:1.5bon-device

Research question

How can we further improve the detection of self-harm in emergency department triage notes by incorporating natural language processing techniques with other clinical data sources and considering the impact of patient demographics on self-harm documentation style?

Methodology

A three-stage approach combining zero-shot language model (LLM) screening with structured evidence extraction and traditional machine learning classification.

Conclusions

The final model on RMH Chunk 3 achieved high AUPRCs across all evaluation sets, with strong cross-hospital transfer at LRH but weaker at SunH due to lower precision under cross-hospital transfer. The proposed approach separated screening and classification, reducing class imbalance and computational burden.

Importance

The study provides a practical solution for improving the detection of self-harm in emergency department triage notes by combining advanced language models with traditional machine learning techniques, which can be applied across different hospitals without retraining.

Open gap

The study highlights that while the approach showed strong performance across hospitals, there is still room for improvement in cross-hospital transfer and temporal generalisation. Additionally, future research could investigate the impact of patient demographics on self-harm documentation style to better understand its influence on model performance.

Future research

Future research could explore the integration of natural language processing techniques with other clinical data sources for a comprehensive self-harm detection system, as well as further validation in different geographical regions and hospital settings.

When Rating Scales Fall Short: LLM-Assisted Discovery of ADHD Signals in Turkish Teacher Narratives

Baris Karacan, Irem Aktar Songur, Ahmet Ozaslan et al. · 2026-01-01

Attention Deficit Hyperactivity Disorder (ADHD) is one of the most common neurodevelopmental disorders in childhood, and its diagnosis relies on assessments combining clinician judgment with standardized rating scales and reports from parents and teachers. While structured instruments such as the Conners' Teacher Rating Scale-Revised Short Form (CTRS-R:S) quantify ADHD-related behaviors, teachers also provide open-ended narratives that may contain complementary signals not captured by structured assessments. However, it remains unclear to what extent teacher narratives encode signals overlooked by rating scales. In this study, we analyze de-identified Turkish teacher evaluation forms collected during clinical ADHD assessments, including both CTRS-R:S scores and open-ended teacher narratives. We compare predictive signals from structured scores and narrative text and identify cases where structured assessments fail to clearly distinguish ADHD from non-ADHD students while narrative-based models capture distinct behavioral patterns. Notably, these cases show minimal overlap with those missed by the narrative model, suggesting that structured and narrative information encode complementary signals. To interpret these differences, we apply a large language model (LLM)-assisted theme discovery pipeline that reveals distinct attention, behavioral, and family-related patterns, highlighting the potential of natural language processing (NLP) to uncover clinically relevant signals from teacher narratives and to complement traditional ADHD screening tools.

Foundation Models × HealthPDF ↗
RP5 Analysis
qwen2.5:1.5bon-device

Research question

How can structured behavioral rating scales be improved by incorporating narrative reports as complementary signals for diagnosing Attention Deficit Hyperactivity Disorder (ADHD)?

Methodology

The study constructs two classification pipelines: one using CTRS-R:S ratings for ADHD assessment scores and another analyzing teacher-written narratives through NLP techniques.

Conclusions

Structured behavioral rating scales are effective but fall short in capturing all ADHD cases due to overlapping rating patterns with controls. Narrative-based models trained on teacher descriptions show stronger discriminative ability, particularly for difficult-to-detect ADHD cases.

Importance

The study highlights the potential of language-based analysis from teacher narratives as a complementary signal for diagnosing ADHD, providing insights that are not fully captured by structured behavioral rating scales.

Open gap

The study identifies a gap in exploring language-based analysis from teacher narratives during child and adolescent ADHD assessments, motivating future work on this topic.

Future research

Concrete next steps include expanding the dataset size and exploring alternative language models to enhance model performance and generalizability.

Effective "all-team" summarization in high-complexity settings like the Neonatal Intensive Care Unit (NICU) requires aggregating insights from diverse disciplines (physicians, nurses, therapists) spread across hundreds of clinical free-text notes. Simply pooling heterogeneous text often leads to incoherent outputs. Structured summarization therefore first requires accurate categorization of sentence-level provenance across multi-source notes. This pilot study introduces a clinical provenance categorization pipeline using supervised fine-tuning (SFT) of large language models (LLMs). We adapted two Llama-3 models (8B and 70B) to MedSecId, a corpus of 2,002 MIMIC-III (Adult ICU) notes annotated with clinical provenance headers, achieving in-domain Macro F1 scores above 92% for both models. To evaluate cross-domain generalization, we assessed model capacity (8B vs. 70B) and quantization on a gold-standard dataset of 227 sentence-level spans derived from three multi-disciplinary NICU summaries. Experimental results demonstrate a scale-dependent transfer effect: while SFT produced only marginal changes for the 8B model, it substantially improved the 70B model, increasing Macro F1 by 7%. Notably, the quantized fine-tuned 70B model outperformed its full-precision baseline while substantially reducing computational requirements. These findings suggest that sufficient model capacity is critical for preserving semantic flexibility during cross-domain clinical transfer and that efficient quantized adaptation can enable structured provenance modeling for downstream summarization.

Foundation Models × HealthPDF ↗
RP5 Analysis
gemma4:e2bon-device

Research question

Does parameter scale determine the cross-domain robustness of large language models when transferring structured clinical provenance schemas?

Methodology

supervised fine-tuning of Llama-3 models using QLoRA, with models quantized to 4-bit NF4 precision and LoRA adapters applied to all linear projections, including MLP layers

Conclusions

Parameter scale is a critical determinant of cross-domain robustness, and larger models (70B) successfully transfer complex clinical provenance schemas better than smaller models. Larger models demonstrated superior cross-domain transfer by effectively internalizing structured supervision and improving generalization to specialized NICU categories.

Importance

The work addresses the challenge of incoherent all-team summarization in complex settings by enabling structured provenance modeling for downstream summarization, particularly in complex clinical environments like the NICU.

Open gap

not specified

Future research

Expanding the evaluation set and investigating the use of these provenance classifiers as retriever backbones to provide more precise contextual grounding for downstream clinical summarization.

Beyond One-shot: AI Agents for Learning in Field Experiments

Junjie Luo, Ritu Agarwal, Gordon Gao · 2026-01-01

Organizations routinely run experiments for A/B testing, yet the data generated from one experiment is underutilized to inform subsequent intervention design. Significant barriers exist to extracting actionable knowledge from prior experimental data to inform new interventions. We study whether tool-augmented agentic AI can automatically learn from experimental data to generate new interventions in subsequent experiments. Through two-stage field experiments in healthcare prescription messaging (693,139 patient visits), we compare a Human + Chatbot method (Stage 1: behavioral experts with conversational AI co-designing 13 message variants, 444,691 patient visits) against a Tool-Augmented Agentic AI method (Stage 2: AI autonomously extracting principles from Stage 1 data to generate 17 new variants, 248,448 patient visits). The Agentic AI method, equipped with analytical tools, structured Data-Information-Knowledge-Wisdom (DIKW) reasoning agents, and transparent evidence chains, produces superior interventions: the best AI-generated message achieved a 69.8% CTR (+6.5 percentage points over baseline). Critically, our results suggest that the value comes from domain-specific experimental data, not from general reasoning ability: frontier LLMs operating without experimental data failed to predict which interventions would succeed. The field experiments also revealed that general-purpose behavioral theories used for intervention design do not extend uniformly to specific healthcare contexts, motivating an agentic AI approach to theory audits at field-experiment scale. Our research shows that tool-augmented AI can learn from experimental data and generate improved domain-relevant interventions, transforming behavioral experimentation from one-shot evaluation into a scalable system for cumulative design learning.

Foundation Models × HealthPDF ↗no extraction yet

Recent evidence shows that people with eating disorders (EDs) are increasingly seeking guidance, advice, and emotional support from Large Language Model (LLM)-based chat systems. Although these systems are not designed to provide clinical advice, their perceived expertise, neutrality and accessibility make them a frequent, albeit risky, source of support. This paper investigates potential patterns of interaction between users with EDs and LLMs, focusing on the potential harms arising from models that uncritically adapt to, and facilitate unsafe or self-harming user requests. We find, in consultation with clinical ED experts, that specific linguistic cues in prompts increase the likelihood of unsafe responses and, through systematically varying the degree of potential risk present in the user prompt, report the extent to which LLMs uncritically adapt to problematic, and potentially dangerous user inputs.

Foundation Models × HealthPDF ↗
RP5 Analysis
qwen2.5:1.5bon-device

Research question

How do specific linguistic cues influence model behavior in simulated scenarios involving food-related psychological disorders?

Methodology

The study uses a controlled experiment to examine how specific linguistic cues influence model behavior in simulated scenarios involving food-related psychological disorders, such as eating disorders.

Conclusions

No consistent refusal behavior overall but models differ based on how they weigh context against apparent risk in requests. Partial refusal occurs frequently but remains clinically significant. Models provide advice in risky contexts, with Qwen most compliant and Gemma the least sensitive to ED cues. Llama shows a middle ground, increasing safety awareness in more risky conditions.

Importance

The study highlights the potential risks of generative AI models for individuals with eating disorders, emphasizing the need for clear guidance on risk assessment and prioritizing safety in interactions.

Open gap

The study notes that while models should refuse user requests as prompt risk increases, they must strike a balance between empathy and safety when responding to expert clinicians' concerns.

Future research

The authors propose further research on the effectiveness of guardrails, including direct engagement with real users and cultural considerations in LM responses.

PaSBench-Video: A Streaming Video Benchmark for Proactive Safety Warning

Yusong Zhao, Yuejin Xie, Youliang Yuan et al. · 2026-01-01

Between the first visible sign of danger and the moment an accident occurs, there is often a window where intervention remains possible. Video-capable multimodal large language models (MLLMs) could serve as always-on safety monitors that issue warnings during this window. Yet current benchmarks do not test this ability: they rely on static inputs, ignore timing precision, and omit false-positive measurement on safe scenes. We present PaSBench-Video, a 740-video benchmark with 481 risk and 259 no-risk videos across four domains: driving, healthcare, daily life, and industrial production. Risk videos are annotated with frame-level risk onset and accident boundaries. A model must observe the video causally and produce a warning that is both temporally calibrated and content-correct. Testing 13 MLLMs, we find that no model exceeds 20.0% on our strictest metric, and recall is tightly coupled with false-positive rate, with Pearson correlation 0.64: higher detection comes only at the cost of triggering warnings on the majority of safe clips. Performance splits sharply by domain: models achieve moderate recall at low false-positive rates in daily life, where risks are inherently anomalous, yet fire indiscriminately in driving, where routine and hazardous scenes look alike. These results indicate that current models rely on scene-level activity cues rather than reasoning about emerging harm.

Foundation Models × HealthPDF ↗
RP5 Analysis
qwen2.5:1.5bon-device

Research question

How can we develop advanced models that focus on proactive safety warnings by reasoning about risk escalation beyond simple surface activity cues? This could involve integrating domain-specific knowledge or using reinforcement learning to improve the models' ability to predict and mitigate risks.

Methodology

PaSBench-Video is a benchmark dataset containing annotated video clips with frame-level annotations across four domains: driving, healthcare, daily life & home monitoring, and industrial production.

Conclusions

None of the evaluated MLLMs exceed 20.0% on strict recall metrics, indicating tight coupling between recall and false-positive rates. Models perform better in lower-risk domains like daily life but struggle with risk escalation in driving scenes due to similar risks.

Importance

PaSBench-Video provides a comprehensive benchmark for evaluating proactive safety warnings across various domains, highlighting the need for more sophisticated models that can reason about risk escalation rather than just surface activity cues. The dataset's streaming protocol and no-risk split allow for robust evaluation of false-positive rates.

Open gap

The dataset's limited coverage due to scarcity of high-quality risk videos remains an open gap for future research, as it may not fully capture real-world scenarios where risks are more complex and varied.

Future research

Future research should focus on developing more sophisticated algorithms that can reason about risk escalation beyond simple surface activity cues. This could involve integrating domain-specific knowledge or using reinforcement learning to improve the models' ability to predict and mitigate risks.

Systematic reviews rely on forest plots to synthesise quantitative evidence across biomedical studies, but generating them remains a fragmented and labour-intensive process. Researchers must interpret complex clinical texts, manually extract outcome data from trials, define appropriate interventions and comparators, harmonise inconsistent study designs, and carry out meta-analytic computations-typically using specialised software that demands structured inputs and domain expertise. While recent work has demonstrated that large language models can extract study-level data from unstructured text, no existing system automates the complete pipeline from raw documents to synthesised forest plots. To address this gap, we introduce AutoForest, the first end-to-end system that generates publication-ready forest plots directly from biomedical papers. Given one or more study papers, AutoForest automatically suggests ICO (Intervention, Comparator, Outcome) elements, extracts outcome data, performs statistical synthesis, and renders the final forest plot. We describe the system architecture, user interface and demonstrate its effectiveness on real-world examples through a user study involving clinicians, showing how AutoForest can accelerate evidence synthesis and substantially lower the barrier to conducting meta-analyses.

Foundation Models × HealthPDF ↗
RP5 Analysis
gemma4:e2bon-device

Research question

The evaluation aims to determine if AutoForest reduces completion time, enhances accuracy, and helps users achieve expert-level performance in forest plot production.

Methodology

AutoForest is an interactive system utilizing a ReactJS frontend and a FastAPI backend, employing transformer-based language models for extracting Intervention, Comparator, and Outcome (ICO) triplets, numerical evidence, and performing risk of bias assessment. It uses document parsing via the docling library and supports a human-in-the-loop workflow.

Conclusions

AutoForest automates the time-consuming process of generating forest plots from biomedical documents by integrating document parsing, evidence extraction, and risk of bias assessment. It substantially accelerates evidence synthesis when used in a human-in-the-loop capacity compared to manual workflows, and AI assistance effectively bridges the performance gap, enabling novice users to approach the accuracy of domain experts.

Importance

The work addresses the fragmented and labor-intensive process of generating forest plots for systematic reviews, aiming to significantly accelerate evidence synthesis and lower the barrier to conducting meta-analyses.

Open gap

The system currently excludes tasks like literature searching and screening, and there is a limitation in the absence of a visual feature to directly highlight extracted evidence within the source documents for improved user traceability.

Future research

The authors did not explicitly propose concrete next steps for future research.

Existing hallucination taxonomies classify LLM errors by what is wrong with the output -- memorised misconceptions, reasoning failures, fluent fabrications. These taxonomies are useful for diagnosis but cannot answer a different question: which uncertainty scorer would have caught this error? We propose a complementary taxonomy that classifies errors by their detectability signature -- the signal a scorer family would read. The DECK taxonomy is a 2x2 partition along inter-sample consistency and token-level confidence into four behavioural regimes (Drift, Entrenched, Confabulation, Knotted), each mapping to a specific scorer family (or families) that can detect it: black-box consistency scorers have signal in D and C, white-box token-probability scorers have signal in K and C, and only an LLM-as-a-Judge with independent pretraining can detect E. Cell membership is operationalised by a Youden's J optimal split on each scorer axis. Across three models and four datasets we validate the taxonomy two ways: by analysing scorer-pair disagreement, and by checking that external labels (SelfAware unanswerable, HaluEval adversarial, PopQA entity popularity) land in the predicted DECK cells, with model-scale and content-specific secondary-cell refinements. We further identify a universal blind spot of output-level UQ: on knowledge-gap inputs where the generator emits confident, repeatable fabrications, every output-level family collapses by construction. A linear probe on Llama-3-8B's hidden states also collapses to chance, giving preliminary evidence that the failure may persist at the activation level; richer internal-state methods (UQ heads, information-theoretic estimators) remain to be tested.

Foundation Models × HealthPDF ↗
RP5 Analysis
gemma4:e2bon-device

Research question

Which uncertainty scorer family should detect specific hallucinations, and what is the universal blind spot in output-level uncertainty quantification?

Methodology

DECK taxonomy, classifying LLM hallucinations based on detectability signature; utilizing aleatoric/epistemic decomposition; two-axis framework based on inter-sample consistency and token-level confidence; disagreement analysis using Youden’s J split; external-signal validation using Wikipedia page-view counts; evaluation across multiple models (Llama-3-8B, GPT-4o, Gemini-2.5-Flash) and datasets (TriviaQA, HaluEval, SelfAware, PopQA)

Conclusions

The DECK taxonomy classifies LLM hallucinations into four distinct cells (Drift, Entrenched, Confabulation, Knotted) based on inter-sample consistency and token-level confidence. This framework captures the dominant signals for hallucination detection, and output-level uncertainty scoring fails universally when generators produce confident fabrications regarding knowledge gaps (the "universal blind spot").

Importance

The framework provides a complementary method to existing hallucination taxonomies by classifying errors based on their detectability signature, allowing for testable predictions about which uncertainty scorer families should detect specific hallucinations. It identifies a universal failure mode in output-level uncertainty quantification.

Open gap

The study is restricted solely to short-form QA, excluding the dynamics of long-form generation, and the internal-state probe is limited to a single open-weights model.

Future research

The recommended engineering response is an abstention envelope that routes out-of-scope inputs to refusal or retrieval prior to scoring.

Large Language Model (LLM)-augmented Community Notes offer a scalable path for timely, evidence-grounded correction of health misinformation on social platforms. However, they still reset at every post, leaving useful correction experience from prior cases unused. We introduce EvoNote, an agentic framework that enables health Community Notes generation to self-evolve through an evolving experience memory of prior misinformation correction episodes. Its core is fine-grained credit assignment: EvoNote grounds trajectory-level feedback in health-specific note qualities and distills it into action-level memory for claim analysis, evidence acquisition, and note writing. We evaluate EvoNote on MM-HealthCN, a 1.2K-instance multimodal benchmark of user-flagged health posts with human-written Community Notes and crowd-derived helpfulness labels. Under a human-validated hierarchical utility judge, EvoNote-generated notes are preferred over corresponding human-written notes in 89.6% of cases; on a separate set of Needs More Ratings posts without a crowd helpfulness verdict, EvoNote produces helpful notes for 82.0% of cases. It also reduces the median time needed to produce a candidate correction from over 13 hours in the human-note pipeline to under 2 minutes. Analyses link these gains to stronger evidence use and reusable correction strategies, positioning self-evolving note generation as a promising paradigm for health misinformation governance.

Foundation Models × HealthPDF ↗
RP5 Analysis
qwen2.5:1.5bon-device

Research question

How can agentic systems be further improved by incorporating ethical considerations such as human oversight and evidence-based decision-making?

Methodology

EvoNote is an agentic framework that enables large language models (LLMs) to self-evolve through experience memory of prior health misinformation correction episodes.

Conclusions

EvoNote outperforms human-written notes on a multimodal benchmark with 89.6% preference and reduces the time needed to produce corrections from over 13 hours to under 2 minutes.

Importance

The work demonstrates how LLMs can be used for health misinformation reduction, improving efficiency and accuracy compared to manual methods.

Open gap

The study focuses on English health misinformation; extending it to other domains may require additional data or domain-specific modifications.

Future research

Further research could explore integrating upstream detection and routing processes into EvoNote's memory system for more comprehensive error handling.

EEG-FuseFormer: A Transformer-Driven Feature Fusion Framework for Seizure Onset Prediction

Vigneshwar Hariharan, Chithra Reghuvaran, Arlene John et al. · 2026-01-01

Epilepsy is one of the most common neurological disorders globally, characterized by recurring seizures and significantly impacting the quality of life. Despite advancements in diagnostic techniques, the mitigation of risks faced by epilepsy patients remains challenging due to the unpredictability of seizure events. An accurate forecast of seizure onset helps to reduce risks in epilepsy patients. In this paper, we propose EEG-FuseFormer, a transformer-based feature fusion framework for seizure-onset prediction that combines intermediate features extracted from Convolutional Neural Networks-Long Short-Term Memory (CNN-LSTM) and ResNet-18 networks. The CNN-LSTM architecture captures both spatial and temporal features directly from the raw signal, whereas the ResNet-18 extracts features from the Short-Time Fourier Transform (STFT) representation of the EEG signals. Fusion is carried out using a transformer encoder, and the final prediction is generated using fully connected dense layers. The CHB-MIT dataset was used to validate the proposed model. The results show that the proposed model achieves a mean recall of 98.85% and outperforms most of the state-of-the-art methods. This study evaluates the ability of the proposed feature fusion model to generalize in cross-patient testing scenarios. Fine-tuning pre-trained models on limited target patient data (target adaptation) within the cross-patient validation framework results in higher recall, precision, and F1-score metrics in comparison to the conventional cross-patient validation approach. Finally, the runtime-based computational complexity of the model is assessed across diverse hardware platforms to highlight the performance-complexity trade-off.

Foundation Models × HealthPDF ↗
RP5 Analysis
gemma4:e2bon-device

Research question

How can feature fusion from multiple deep learning architectures enhance seizure prediction accuracy while ensuring robust cross-patient generalization?

Methodology

EEG-FuseFormer, a transformer-based framework that fuses intermediate features from CNN-LSTM and ResNet-18 networks. It utilizes a 1D-CNN-LSTM on the time-domain signal and STFT/ResNet-18 on frequency-domain features, followed by a multi-head transformer encoder for classification.

Conclusions

The EEG-FuseFormer framework demonstrated improved seizure prediction performance by fusing features from ResNet-18 and 1D-CNN-LSTM using self-attention, and this performance was further enhanced by cross-patient modeling. EEG-FuseFormer achieved the highest predictive performance, superior recall, and F1-scores, along with a lower False Alarm Rate compared to other models.

Importance

The proposed feature fusion model outperforms standalone models like 1D-CNN-LSTM and ResNet-18, achieving comparable performance with less feature engineering effort, which is important given the need for robust seizure prediction.

Open gap

The limitation of patient-specificity in the proposed models, which is addressed by the cross-patient testing framework and target adaptation.

Future research

not specified

Emotions evolve through the dynamics of conversation, and understanding their transition structure is foundational to applications ranging from mental-health screening to dialogue systems. However, existing studies typically compress multi-rater judgments into a single hard label by majority voting, discarding the uncertainty signal needed to understand turn-to-turn transitions. In this article, we propose Bayesian Spectral Emotion Transition Discovery (BSETD), a two-stage framework that discovers emotion-transition structure from multi-rater soft labels. In the first stage, a hierarchical Dirichlet-Multinomial posterior is constructed through the outer product of soft labels, equipping each cell of the K x K transition matrix with a credible interval and Benjamini-Hochberg (BH) false discovery rate (FDR)-controlled significance. In the second stage, the symmetrized graph Laplacian is spectrally decomposed to separate a low-frequency (inertia) component from a high-frequency (contagion) component. On EmotionLines, BSETD simultaneously recovers the signatures of two distinct affective spaces: the Plutchik-adjacent transitions disgust to anger (log2 lift +0.94) and anger to disgust (+0.86) are over-represented, while the Russell-valence-reversed transitions joy to anger (-0.90) and anger to joy (-0.89) are under-represented. A five-source cross-corpus validation yields pairwise Pearson correlations in 0.91-0.98 within English, 0.79-0.85 against Chinese M3ED, and 0.979 between the human hard labels and the LLM virtual soft labels on the same utterance set, demonstrating that a pipeline preserving annotator uncertainty bridges the computational study of emotion dynamics with established psychological theory.

Foundation Models × HealthPDF ↗
RP5 Analysis
qwen2.5:1.5bon-device

Research question

How can BSETD be extended to fully analyze per-dialog posteriors, address fully directional spectral analysis, and explore the potential of virtual annotator LLMs in replacing human raters?

Methodology

BSETD (Bayesian Spectral Emotion Transition Discovery) framework combining hierarchical Dirichlet-Multinomial posterior estimation with graph Laplacian decomposition

Conclusions

Improved pairwise correlations on EmotionLines and validation across English and Chinese datasets, future research plans including investigating per-dialog posteriors, fully directional spectral analysis, and the potential of virtual annotator LLMs to partially replace human raters due to the modal vs uncertainty gap.

Importance

Recovery of richer transition patterns from soft multi-rater labels, addressing limitations in previous ERC methods by treating emotions deterministically and ignoring per-annotator disagreement

Open gap

The limitations of aggregating over the entire corpus and the dialog-level hierarchical extension not accounting for individual dialogue dynamics remain unsolved.

Future research

Investigate per-dialog posteriors, fully directional spectral analysis, and the potential of virtual annotator LLMs to partially replace human raters due to the modal vs uncertainty gap

KliniskVestBERT: BERT Model Specialised to Norwegian Clinical Texts

Christian Autenried, Cosimo Persia · 2026-01-01

The increasing application of Natural Language Processing (NLP) in healthcare demands language models specifically attuned to the complexities of clinical language. This work introduces KliniskVestBERT, a suite of three BERT-based encoder models pre-trained on a substantial corpus of real-world, de-identified Norwegian clinical texts from Helse Vest. We continue pretraining existing language models Nb-BERT-large, NorBERT3-large, and ModernBERT on our specialized clinical dataset. This dataset is based on a representative population of Helse Vest patients. The included document types are carefully curated to encompass a broad clinical spectrum in bokmål and nynorsk including discharge summaries, surgical reports, nursing notes etc. ensuring comprehensive representation of the linguistic landscape within Norwegian healthcare settings. Evaluation on three synthtetic Norwegian clinical benchmark datasets and two real-world problems demonstrates that each of our clinically specialized models consistently outperforms their baseline counterparts, highlighting the significant benefit of domain-specific pre-training for NLP tasks within the clinical domain. The project was a joint effort by all Helse Vest entities (Helse Bergen, Helse Fonna, Helse Førde and Helse Stavanger) with DIPS under the project lead of Helse Vest ICT.

Foundation Models × HealthPDF ↗
RP5 Analysis
gemma4:e2bon-device

Research question

How can domain-specific pre-training improve the performance of language models for NLP tasks within the clinical domain?

Methodology

Pre-training three BERT-based encoder models (Nb-BERT-large, NorBERT3-large, and ModernBERT) on a substantial corpus of de-identified Norwegian clinical texts from Helse Vest, followed by fine-tuning.

Conclusions

KliniskVestBERT models consistently outperformed their baseline counterparts when evaluated on clinical benchmark datasets, demonstrating the significant benefit of domain-specific pre-training for NLP tasks in the healthcare domain.

Importance

Highlights the significant benefit of domain-specific pre-training for NLP tasks within the healthcare domain, addressing the bottleneck of scarce, high-quality, privacy-compliant medical data for specialized training.

Open gap

The scarcity of high-quality, privacy-compliant medical data necessary for specialized training, and the suboptimal results of general-purpose models when handling complex clinical language and context.

Future research

not specified

TalkTag: Fine-Grained Morphosyntactic Error Annotation for Transcribed Speech

Shamira Venturini, Oliver Hennhöfer, Steffen Kinkel et al. · 2026-01-01

Fine-grained morphosyntactic error annotation is important in clinical and developmental language research, yet it is labour-intensive, expert-dependent, and difficult to scale. We present TalkTag, an LLM-based lightweight tool fine-tuned to automate CHAT-style error annotation in spoken-language transcripts. Developed under conditions of extreme data scarcity using children's narrative data, the system shows the feasibility of linguistic analysis in low-resource settings. Our evaluation demonstrates that TalkTag produces encouragingly precise annotation while effectively identifying instances where linguistic ambiguity makes automated tagging genuinely complex. In summary, with TalkTag, we provide a scalable alternative to manual error annotation and practically viable support for morphosyntactic error annotation.

Foundation Models × HealthPDF ↗
RP5 Analysis
gemma4:e2bon-device

Research question

Can an LLM-based system reliably provide a first-pass identification of plausible morphosyntactic error candidates for subsequent expert review in low-resource settings?

Methodology

LLM-based tool fine-tuned to automate CHAT-style error annotation, leveraging Large Language Models and synthetic data augmentation; the model used is an instruction-tuned Meta-Llama-3.1-8B-Instruct fine-tuned using LoRA.

Conclusions

The TalkTag system provides a useful pre-annotation aid for fine-grained morphosyntactic error annotation, particularly for agreement-related errors relevant to developmental language disorders. The model performed best on allomorphic errors, suggesting proposed labels are generally plausible, but struggled with uninflected verb forms and the interaction between tense and agreement.

Importance

The system offers a scalable alternative to manual annotation, demonstrating the feasibility of linguistic analysis in low-resource settings and providing a linguistically informed first-pass annotation that can reduce annotation effort in challenging domains.

Open gap

The prototype does not cover the full range of CHAT errors or complex morphosyntactic phenomena, and the results do not disentangle the underlying learning mechanisms.

Future research

The study is limited by extreme data scarcity and a narrow annotation scope, and generalization is restricted because evaluation was confined to a single corpus. The system is best positioned as a human-in-the-loop pre-annotation aid rather than a fully automatic annotation tool.

Genotype-Conditioned Molecular Generation via Evidence-Grounded Multi-Objective Latent Perturbation in Diffusion Models

Brenda Nogueira, Gisela A. Gonzalez-Montiel, Nitesh V. Chawla et al. · 2026-01-01

Developing effective anticancer therapeutics remains challenging due to tumor heterogeneity and the absence of well-defined molecular targets across cancer subtypes. Generative models conditioned on cancer genotypes offer a promising avenue for personalized drug discovery, yet existing approaches lack explicit optimization for simultaneous sensitivity, synthesizability, and mechanistic binding plausibility. We present a latent-space optimization approach for a pretrained genotype-to-drug diffusion model, introducing a learnable perturbation over the molecular latent space optimized via gradient ascent to maximize a composite reward combining predicted drug sensitivity (AUC), drug-likeness (QED), and synthetic accessibility (SAS). Critically, biological realism is enforced by grounding both reward design and evaluation in experimentally-derived cancer cell line data and validated pharmacologic signals, anchoring candidate generation in real-world clinical evidence. Mechanistic consistency plausibility is further assessed by a multi-agent LLM pipeline grounded in the diffusion model's attention mechanism. Experiments across 15 cancer cell lines from three held-out evaluation sets demonstrate consistent and noticeable improvements over competing baselines in sensitivity, drug-likeness, synthesizability, and chemical validity.

Foundation Models × HealthPDF ↗
RP5 Analysis
qwen2.5:1.5bon-device

Research question

How can the proposed method be extended to incorporate pathway-level reasoning, structure-based validation using AlphaFold-derived docking estimates, and focus on real-world validation of the approach?

Methodology

The paper proposes a method for generating molecular candidates that optimize three clinically relevant axes: sensitivity (drug response), drug viability (likeness and synthesizability), and mechanistic target engagement.

Conclusions

The method combines diffusion-based latent space optimization with an LLM-augmented biological scoring system, achieving low predicted AUC while matching or exceeding conditional baselines in QED and SAS, demonstrating improved specificity and near-perfect validity/novelty. Latent-space experiments show accelerated convergence toward biologically meaningful chemical space.

Importance

The work addresses the limitations of existing methods by integrating biological context and mechanistic binding rationale into the optimization objective through continuous latent space optimization methods, genotype-phenotype conditioned molecular generation using genomic alteration profiles, and LLMs as active scoring agents within molecular optimization loops. The method shows consistent improvements in sensitivity, drug-likeness, synthesizability, and chemical validity across 15 cancer cell lines from three evaluation sets compared to competing baselines.

Open gap

The paper does not explicitly state any open gaps or limitations that remain unsolved.

Future research

Future work includes extending the method to incorporate pathway-level reasoning, structure-based validation using AlphaFold-derived docking estimates, and focusing on real-world validation of the approach.

Drug-information question answering is a high-stakes setting where hallucinated facts can mislead clinical decision-making and the provenance of each cited fact matters as much as the fact itself. We present DrugClaw, a multi-agent retrieval-augmented system that queries a registry of drug and pharmacovigilance skills via a reflection-driven state-machine workflow and returns answers grounded in primary regulatory or peer-reviewed records. We also contribute DrugAudit, a 3,772-item authority-aware benchmark with an evaluation panel that scores upstream-of-gold source match, token-level semantic snippet overlap, and citation faithfulness under a dual-judge LLM-as-judge protocol with inter-judge kappa = 0.88 (almost-perfect). Across DrugAudit plus drug-related subsets of MedQA (751) and PubMedQA (512), DrugClaw is top-1 on every column of the headline table: composite Evidence Index under both judges, judge-mediated answer correctness, primary-source rate (0.918, +10.1 pp over next-best), faithfulness (0.887, +5.9 pp), MedQA (0.920), and PubMedQA (0.693).

Foundation Models × HealthPDF ↗
RP5 Analysis
qwen2.5:1.5bon-device

Research question

How can further improvements be made to expand the skill registry beyond closed databases and enhance open-web search capabilities for more diverse datasets?

Methodology

A state-machine workflow with eight specialized agents operating over a closed registry of drug-information skills

Conclusions

DrugClaw achieves top-1 performance across various datasets compared to state-of-the-art methods, including authority-aware benchmark DrugAudit.

Importance

The work demonstrates the effectiveness of integrating authority-grounded multi-agent reasoning with an authority-aware metric panel in answering drug information questions.

Open gap

The work highlights a gap in current drug information systems that relies on authority-grounded reasoning but does not yet fully address the limitations of primary-source citation aggregation.

Future research

Further research could explore expanding the skill registry beyond closed databases, improving open-web search capabilities, and evaluating DrugClaw in more diverse datasets.

Systematic characterization of drug-disease relationships is essential for drug discovery and repurposing, yet is hindered by the heterogeneity and rapid growth of biomedical literature. Existing datasets rely on labor-intensive curation and are often incomplete, while LLM-only approaches suffer from hallucination and weak evidence grounding. We introduce UniD$^3$, a unified framework that integrates Large Language Models with Knowledge Graph-enhanced Retrieval-Augmented Generation (KG-RAG) to extract, organize, and validate drug-disease knowledge across Drug-Disease Matching (DDM), Drug Effectiveness Assessment (DEA), and Drug-Target Analysis (DTA). UniD$^3$ processes 157,849 PubMed articles with Llama 3.3-70B and constructs knowledge graphs via a dual-stage strategy combining paper-level extraction with KG-level consolidation centered on drug and disease entities. These graphs support KG-RAG-based generation of structured datasets, evaluated through external benchmarks, fuzzy matching with curated resources, and clinician review. UniD$^3$ produces six knowledge graphs and large-scale datasets, including 28,915 DDM, 15,042 DEA, and over 4,000 DTA QA pairs. External validation shows strong performance (F1: 0.85-0.87 for DDM/DEA; 0.82 for DTA), with clinician review confirming high reliability (AUROC = 0.90). KG-RAG-augmented models outperform standalone LLMs, and the UniD$^3$ chatbot enables interpretable, citation-supported exploration of drug-disease relationships. UniD$^3$ provides a scalable, extensible framework for transforming unstructured biomedical literature into high-quality, structured drug-disease knowledge, supporting AI-driven discovery, repurposing, and precision medicine.

Foundation Models × HealthPDF ↗no extraction yet

Tool-using multi-agent large language model (LLM) systems spend computation through model tokens, tool calls, retries, and code execution before producing an answer. When a run fails, final-answer evaluation reveals the endpoint but usually not the point at which the trajectory stopped making recoverable progress. This paper introduces a failure-aware observability framework for diagnosing wasted computation in multi-agent LLM traces. The framework maps recurring failure modes to online trace signals, including tool reliability, execution recovery, orchestration loops, evidence availability, information change, and budget pressure. We instantiate the framework in a three- agent question-answering system and evaluate it on 165 GAIA validation traces under identical execution caps. Operational failures remain common: 22/53 level-1 runs, 33/86 level-2 runs, and 12/26 level-3 runs fail to produce a usable final answer. The traces expose different mechanisms behind these outcomes, including insufficient evidence, repeated-action loops, max-step termination, tool-failure streaks, and execution calls that succeed without useful output. Mean token use rises from 8,152 tokens at level 1 to 16,389 tokens at level 3, while evidence availability and sentence-level support diverge. A cached 10-trace LLM-judge grounding audit shows that cheap online signals and deeper semantic metrics capture complementary layers of failure. The results position failure-aware observability as a diagnostic layer between raw execution logs and final-answer accuracy.

Foundation Models × HealthPDF ↗
RP5 Analysis
gemma4:e2bon-device

Research question

How can wasted computation in multi-agent LLM traces be diagnosed by mapping recurring failure modes to online execution signals?

Methodology

failure-aware observability framework using structured event traces to map recurring failure modes to online execution signals (tool reliability, evidence availability, loop indicators)

Conclusions

Operational failures stem from specific mechanisms like repeated-action loops and insufficient evidence, which correlate with increased token usage. Failure-aware observability is a critical diagnostic layer between raw execution logs and final-answer accuracy. Triage labels and tool-error rates are positively associated with unusable outcomes, while sentence support and answer-evidence similarity are negatively associated.

Importance

The framework provides a critical diagnostic layer to diagnose wasted computation in multi-agent LLM traces by mapping failures to structured trace signals, which is crucial because final answer accuracy alone is insufficient for agent engineering.

Open gap

The observational analysis does not yet evaluate whether an online intervention policy causally improves accuracy or cost.

Future research

Translating these diagnostics into controlled intervention policies to measure improvements in cost-adjusted task success.

Med-HEAL: Analyzing and Mitigating Hallucinations in Medical LLMs with Hallucination-Aware In-Context Learning

Yiming Liao, Zeno Franco, Jose Eduardo Lizarraga Mazaba et al. · 2026-01-01

Hallucinations in medical large language models (LLMs) pose serious risks for clinical decision support, particularly when models must reason over complex electronic health records (EHRs). However, existing benchmarks often lack a realistic clinical context and provide limited insight into how hallucinations can be mitigated in practice. We introduce Med-HEAL, a framework for systematically identifying, analyzing, and mitigating hallucinations in medical LLMs using clinically grounded data. Building on the EHRNoteQA benchmark derived from MIMIC-IV discharge summaries, we construct a hallucination dataset by evaluating BioMistral-7B on open-ended clinical question answering tasks. Model outputs are labeled through a dual evaluation pipeline that combines LLM-as-a-Judge assessment (GPT-4o) with human auditing by medical student reviewers, producing correctness judgments and annotations of reasoning errors via a custom web-based evaluation system. We then leverage this dataset to investigate mitigation strategies: a self-critique pipeline, in which the test model reviews its own answers to detect potential errors and regenerates responses for flagged cases, and retrieval-augmented in-context learning (RA-ICL), which exposes the model to hallucinated and corrected examples. Experiments across five open-source LLMs-BioMistral, Llama-3.1, DeepSeek, Qwen2.5, and Qwen3, show that the self-critique strategy improves accuracy for three of five models (p < 0.05) without requiring parameter updates. Med-HEAL provides both a reusable hallucination dataset and a practical framework for studying and mitigating hallucinations in medical LLMs, supporting safer deployment of AI systems in clinical environments. Our code and data are publicly available at https://github.com/yimingliao-blad/med-heal.git.

Foundation Models × HealthPDF ↗
RP5 Analysis
gemma4:e2bon-device

Research question

Is a self-critique pipeline effective for mitigating hallucinations in medical LLMs?

Methodology

Med-HEAL framework, which comprises a data collection pipeline using dual-judge evaluation (LLM-as-a-Judge and Human-as-a-Judge) to extract hallucination cases, and a self-critique mechanism that incorporates corrected examples into retrieval-augmented prompts.

Conclusions

The Med-HEAL self-critique pipeline consistently improves performance in mitigating hallucinations in medical LLMs, outperforming conventional retrieval-based methods. Combining self-critique with targeted demonstrations further enhances accuracy by addressing predicted reasoning failures.

Importance

Offers a practical approach for safer clinical deployment of medical AI by enhancing the reliability of medical LLMs.

Open gap

Limitations remain regarding dataset diversity and the challenges posed by long clinical contexts.

Future research

Limitations remain regarding dataset diversity and the challenges posed by long clinical contexts.

ChronosAD: Leveraging Time Series Foundation Models for Accurate Anomaly Detection

Uzair Khan, Luigi Capogrosso, Francesco Biondani et al. · 2026-01-01

Time series anomaly detection is a crucial task in various domains, including finance, healthcare, and industry. However, existing methods often struggle to generalize across different datasets, especially when anomalies are subtle or context-dependent. To solve this issue, we introduce ChronosAD, a novel architecture for anomaly detection that uses a time series foundation model as a feature extractor. Specifically, it employs a two-stage pipeline: first, it uses the foundation model to extract embeddings for each time series in a zero-shot manner. Then, a custom-developed Temporal Block, composed of Bidirectional Long Short-Term Memory (BiLSTM) and Multi-Head Attention, refines these embeddings to capture temporal dependencies and highlight salient patterns. Unlike previous approaches, our model requires minimal task-specific tuning and demonstrates robust generalization across a wide range of domains, including industrial, medical, cyber-physical, and automotive systems. Extensive experiments on 11 benchmarks show that ChronosAD outperforms existing methods by 4.72% in AUC and 6.60% in AP on average. The source code is available at https://github.com/intelligolabs/ChronosAD.

Foundation Models × HealthPDF ↗
RP5 Analysis
gemma4:e2bon-device

Research question

not specified

Methodology

ChronosAD is a novel two-stage deep learning architecture that utilizes a time series foundation model for zero-shot feature extraction, followed by a custom Temporal Block incorporating BiLSTM and Multi-Head Attention to refine temporal dependencies.

Conclusions

ChronosAD is a novel two-stage deep learning architecture that utilizes a time series foundation model and a custom Temporal Block for anomaly detection. The method demonstrates consistent performance across diverse domains, validating time series foundation models as a viable approach for unsupervised anomaly detection.

Importance

This work validates time series foundation models as a viable approach for unsupervised anomaly detection.

Open gap

not specified

Future research

future unsupervised research in the field

We investigate whether large language models produce different medical triage recommendations for identical symptoms based solely on the language of the patient prompt. Using Gemini 3.5 Flash, we evaluate a neurological symptom profile (persistent headache, blurred vision, nausea) across six languages (English, Spanish, Chinese, Hindi, Japanese, Arabic) with 30 runs per condition (n=450 total API calls). We find that the model recommends emergency room visits at rates ranging from 0% (Japanese, Hindi) to 30% (English, Arabic), despite assigning nearly identical severity scores (7.7-8.0/10) across all languages. Adding a single sentence specifying the patient's US location increases ER recommendations by up to 76.7 percentage points for non-English prompts, while the reverse anchor (English prompt with a Tokyo location) reduces the ER rate from 30% to 6.7%. A back-translation control (Japanese to English) produces ER rates comparable to the English baseline, confirming that the disparity is not caused by translation quality but by implicit geographic inference from the input language. We release the complete dataset, experiment code, and results.

Foundation Models × HealthPDF ↗
RP5 Analysis
gemma4:e2bon-device

Research question

Do large language models generate differing medical triage recommendations based solely on the language of the patient prompt?

Methodology

Using Gemini 3.5 Flash across six languages, testing a neurological symptom profile, and employing methods like US Location Anchoring, Reverse Anchoring, and Back-translation Control.

Conclusions

Large language models generate systematically different medical triage recommendations by implicitly inferring geographic location and applying region-specific healthcare norms based on prompt language. This disparity is attributed to implicit geographic inference rather than translation quality.

Importance

The work extends existing fairness concerns by showing that language-based geographic inference is a source of disparate treatment in AI triage systems, highlighting a mechanism where LLMs modify clinical advice based on inferred location.

Open gap

The study did not compare the model's recommendations with those of human clinicians from various countries, making it impossible to determine which language's recommendations are correct.

Future research

Cross-model replication is necessary to establish the generality of the findings. Testing with language-matched system prompts is recommended. Larger sample sizes are necessary to establish statistical significance for mid-range comparisons.

Large-scale biomedical image-text datasets extracted from scientific literature provide valuable resources for medical multimodal model training. These datasets are commonly organized as image-caption pairs; however, figure captions are often short, context-dependent, and only partially informative without the surrounding article text. At the same time, large-scale automatic extraction introduces structural noise such as missing captions, residual markup, duplicated context, and incoherent multi-paragraph figure descriptions. We revisit data construction for medical multimodal continued pretraining (CPT) and present PMC-InterCPT, a context-grounded biomedical interleaved corpus that incorporates figure-referencing body text in addition to captions. Our pipeline recovers missing captions, cleans caption and context text, reconstructs coherent interleaved image-text samples, and applies LLM-supervised medical relevance and quality classifiers to filter noisy records. We further reveal strong modality imbalance in the resulting corpus and introduce a four-bucket evidence taxonomy for modality-aware resampling. Through CPT followed by supervised fine-tuning (SFT) on Qwen3.5-4B-Base, PMC-InterCPT effectively improves medical and general multimodal performance while using fewer CPT tokens than the raw source pool. The experimental results also illustrate the complementarity between the data quality and modality for medical multimodal CPT.

Foundation Models × HealthPDF ↗
RP5 Analysis
gemma4:e2bon-device

Research question

How can data quality filtering and modality-aware resampling be combined to optimize medical multimodal CPT pipelines?

Methodology

A systematic pipeline involving caption and context cleanup (sentence-level repetition and article-level duplicate removal), interleaved sample construction with coherence refinement (normalizing numbering, pruning non-contiguous segments, length filtering), LLM-based quality and relevance classification, and modality-aware resampling (partitioning into quality pools and grouping images into functional evidence buckets).

Conclusions

The PMC-InterCPT dataset, derived from the pipeline, significantly outperforms the raw BIOMEDICA data and demonstrates that reducing data quality and managing modality distribution are orthogonal dimensions for optimizing medical multimodal CPT. Evidence-bucket-guided resampling reveals a controllable medical-general tradeoff.

Importance

Optimizing both data quality and modality distribution is essential for effective multimodal CPT pipelines in the medical domain.

Open gap

The construction of interleaved data within the medical domain remains unexplored; the evaluation is restricted to VQA-style benchmarks.

Future research

Explore broader instruction-following and open-ended generation tasks; extend the reconstruction pipeline to multilingual biomedical literature; re-examine modality mixture ratios across other models and benchmark suites.

Large-scale vision foundation models have driven substantial gains on dense prediction tasks such as semantic segmentation, but their size makes deployment impractical in resource-constrained settings, motivating knowledge distillation as a means of transferring their capabilities to lightweight student networks. However, modern foundation teachers are predominantly transformer-based that encode global context, whereas efficient students are typically convolutional networks with locally biased receptive fields. Existing distillation methods largely assume architectural homogeneity and rely on direct feature mimicry, which fails to bridge this representational gap and neglects the structured spatial dependencies and discriminative organization required for accurate semantic segmentation. In this paper, we propose SWARD, a knowledge distillation framework that addresses this gap through two complementary mechanisms. First, we introduce a Multi-Scale Windowed Attention Distillation (MWAD) module that aligns teacher-student attention-based relations within stochastically shifted window partitions whose offsets are randomly resampled at every training iteration. This removes window boundary bias, and, combined with the multi-scale design, captures both short- and long-range spatial dependencies. Second, we introduce Prototype Discriminative Regularization (PDR), a loss that helps shape the student's feature distribution by enforcing inter-class separation and intra-class compactness, further sharpening the discriminative structure beyond what feature mimicry alone can produce under the student's reduced capacity. Experiments across different vision applications (i.e., urban scene parsing and medical image segmentation) show that SWARD achieves state-of-the-art performance.

Foundation Models × HealthPDF ↗
RP5 Analysis
qwen2.5:1.5bon-device

Research question

How can SWARD be extended to handle more complex tasks such as object detection or semantic segmentation with multiple classes?

Methodology

SWARD is a heterogeneous knowledge distillation framework that transfers structured knowledge from an attention-based teacher to a convolutional student.

Conclusions

SWARD consistently outperforms all other methods on Cityscapes semantic segmentation benchmark, achieving 69.97% mIoU and 77.75% mAcc with a 4.92-point improvement over the baseline student model.

Importance

The work addresses the gap between large-scale foundation models and lightweight students by introducing MWAD for aligning teacher-student attention relations across stochastically shifted window partitions and PDR to shape student feature distributions while preserving discriminative structure.

Open gap

While the paper demonstrates the effectiveness of SWARD in specific applications like urban scene parsing and medical image segmentation, it does not address challenges related to handling large-scale vision foundation models across heterogeneous architectures beyond dense-prediction tasks.

Future research

The authors propose future research directions including further ablation studies on different components of SWARD, exploring additional distillation strategies for other dense prediction domains, and investigating how SWARD can be extended to handle more complex tasks such as object detection or semantic segmentation with multiple classes.

Lost in Delusion: Examining LLM Safety Under User Delusions and Distress

Andrew Aquilina, Chetna Nihalani, Vasudha Varadarajan et al. · 2026-01-01

LLM chatbots increasingly serve as a first source of support for people in psychological distress, including those whose distress is entangled with delusional beliefs. Prior work on LLM mental-health safety largely evaluates general therapeutic quality or single-turn crisis detection, leaving unclear how models behave when distress is intertwined with delusion over sustained conversations. We address this gap with matched multi-turn simulations, across clinically grounded personas and six LLMs, that pair each delusional conversation with a distress-only control to isolate the effect of delusional framing. This reveals a recognition-intervention gap: models detect distress at comparable rates regardless of framing, yet sharply fail to act on it once distress is embedded in delusion, with safety interventions suppressed by up to 4.5x. The failure tracks accumulated acceptance of the user's premises rather than emotional validation. Worse, the intuitive fix of prompting models to assess user distress backfires under delusional framing; only delusion-aware prompting with explicit response guidance closes the gap, and even this depends on a delusion classifier that is itself unreliable on the most vulnerable models. Safe deployment therefore requires treating delusional framing as a distinct risk signal that overrides conversational accommodation.

Foundation Models × HealthPDF ↗
RP5 Analysis
gemma4:e2bon-device

Research question

How does delusional framing affect the safety and empathy of LLM responses during multi-turn conversations with escalating distress?

Methodology

Persona-driven conversation simulation across controlled distress trajectories, modification of generation conditions, and subsequent automated and human evaluation using LLM-as-judge setup.

Conclusions

Delusional framing significantly alters safety outcomes, causing a recognition-intervention gap where models fail to act safely. Delusional framing converts emotional validation into collusive empathy, leading to the acceptance of incorrect beliefs. Delusion-aware conditioning paired with response guidance (Del+Rec) was the most effective intervention for closing the safety gap.

Importance

The work matters because LLMs, optimized for agreeableness, risk reinforcing incorrect beliefs by prioritizing user premises over emotional validation, challenging clinical principles of non-collusive empathy. It highlights the need to treat delusional framing as a distinct risk signal.

Open gap

Current safety benchmarks fail to evaluate epistemic safety or jointly isolate delusional framing from distress trajectories.

Future research

Not specified

Biomedical question answering with large language models is commonly evaluated using answer accuracy, but answer accuracy alone does not indicate whether a model can produce parseable outputs, follow structured reliability instructions, recognize weak answer spaces, or avoid confident incorrect commitments. This paper presents HypothesisMed, an inference-time reliability pipeline for biomedical multiple-choice question answering. It combines direct, chain-of-thought, HypothesisMed-v3 prompting, and answer fusion. The final answer is selected by fusion, while HypothesisMed-v3 supplies SPACE labels and confidence information. SPACE labels mark the answer space as VALID, INCOMPLETE, or CONTRADICTED. We evaluate Qwen2.5-7B, Phi-4-mini, DeepSeek-R1-32B, and BioMistral-7B on MedQA, MedMCQA, and PubMedQA using 1,000 examples per dataset. The pipeline improves weighted accuracy over each model's best direct or chain-of-thought baseline while increasing parse and SPACE coverage. We also scale evaluation to Qwen2.5-7B and Phi-4-mini using 10,183 examples per model. Fusion improves Phi-4-mini accuracy from 0.4296 to 0.5192, while Qwen2.5-7B chain-of-thought remains slightly higher in answer accuracy. However, Qwen2.5-7B fusion achieves complete parse and SPACE coverage with much lower false commitment. A 12,000-example SPACE stress test shows answer-space diagnosis remains difficult, with SPACE accuracy of 0.3074 for Qwen2.5-7B and 0.4168 for Phi-4-mini. These results show that answer accuracy, parseability, structured reliability reporting, calibration behavior, and false-commitment behavior are separable capabilities. The main contribution is not a universal state-of-the-art claim, but a reproducible inference-time framework for evaluating biomedical question answering models as auditable workflow components under structured reliability constraints.

Foundation Models × HealthPDF ↗no extraction yet

Large language models systematically hallucinate legal citations -- fabricating statute references, citing repealed provisions, and confusing jurisdictions -- yet no automated method exists to measure or reduce this behavior at scale. We propose citation grounding (CG), a metric that verifies LLM-generated legal citations against a ground-truth citation graph extracted from 100.8 million Ukrainian court decisions (502 million edges, 21,736 unique statute nodes). CG decomposes into three components -- citation precision (does the cited provision exist?), citation relevance (is it contextually appropriate?), and citation temporality (was it valid at the relevant date?) -- enabling differential diagnosis of hallucination types. Empirical evaluation on 100 Ukrainian legal queries across five systems -- four commercial LLMs via AWS Bedrock (Claude Haiku 4.5, Mistral Pixtral Large, Amazon Nova Pro/Lite) and one RAG-augmented production system -- reveals CG ranging from 0.791 to 0.873, with 13-21% of citations hallucinated. To reduce hallucinations without human annotation, we introduce Citation Grounding DPO (CG-DPO): a method that constructs preference pairs algorithmically by corrupting verified citations from real court decisions via four targeted strategies. On a dataset of 2,244 court decisions, a Qwen2.5-7B-Instruct model fine-tuned with LoRA achieves 98.5% mean validation accuracy in distinguishing correct from corrupted citations (rewards margin +14.9, std < 0.3 pp across 3 seeds). The citation graph, evaluation framework, and CG-DPO dataset are released as open resources.

Foundation Models × HealthPDF ↗
RP5 Analysis
qwen2.5:1.5bon-device

Research question

How can citation graphs be effectively repurposed to serve as both evaluation and training signals for detecting and mitigating citation hallucinations without human annotation?

Methodology

Citation Grounding (CG) metric decomposes into precision, relevance, and temporality.

Conclusions

RAG-augmented generation achieves the highest accuracy of 0.873 with low density (2.9), contrasting with Claude Haiku 4.5's performance of high density (4.8).

Importance

CG metric provides a systematic approach to evaluating LLM-generated citations, addressing citation hallucination issues in legal applications.

Open gap

Graph coverage: 59% of flagged hallucinations are gaps not yet represented in judicial citation practice. Temporality implementation: Checks statute existence but does not verify specific versions were in force at a given date; requires versioned legislation API for full temporal verification.

Future research

Future research could explore extending the CG-DPO method to common law systems, incorporating ECHR decisions, and administrative registries for better domain coverage.

OmniEEG-Bench: A Standardized Evaluation Benchmark for EEG Foundation Models

Ziling Lu, Zongsheng Li, Xinke Shen et al. · 2026-01-01

Electroencephalography (EEG) supports a variety of brain-computer interface (BCI) tasks ranging from brain-state monitoring to human-LLM interactions. EEG foundation models are emerging, but evaluation remains fragmented due to heterogeneous datasets and nconsistent task protocols. Here, we introduce OmniEEG-Bench, a unified benchmark and downstream task roadmap for EEG foundation models (FMs). It organizes evaluation of EEG FMs into six task families spanning (i) signal reliability, (ii) biometrics and disease, (iii) consciousness and state, (iv) cognition and emotion, (v) naturalistic stimulus decoding, and (vi) motor and interaction, introducing a new generation of tasks not systematically benchmarked in prior EEG FM work. OmniEEG-Bench standardizes model deployment, task definitions, and metrics through a task-card specification, and unifies 54 EEG datasets with consistent evaluation protocols. We benchmark 10 representative EEG foundation models and report a leaderboard that covers diverse evaluation settings. Both pretraining dataset diversity and model size are significantly associated with better average ranks across datasets, revealing scaling-law behavior in EEG foundation models (Figure 1). These results suggest that scaling EEG foundation models requires not only larger architectures but also broader and more diverse pretraining data. The benchmark code is available at https://github.com/ncclab-sustech/omni-eegbench.git.

Foundation Models × HealthPDF ↗
RP5 Analysis
gemma4:e2bon-device

Research question

What factors predict the performance of EEG foundation models?

Methodology

OmniEEG-Bench, which organizes downstream tasks into six families and unifies 54 datasets with consistent protocols, employing evaluation paradigms like cross-subject transfer, multi-subject adaptation, few-shot adaptation, and channel masking robustness, along with linear probing and full fine-tuning.

Conclusions

Performance in EEG foundation models exhibits scaling-law behavior, indicating that performance is significantly associated with both model size and the diversity of pretraining data. Task-specific optimization and diverse pretraining data are beneficial, and architectural choices like VQ-based tokenization are critical for strong generalization.

Importance

The work establishes a unified benchmark for evaluating EEG foundation models, providing a standardized task roadmap and large-scale testing protocol to address the lack of comparability and hinder robust development.

Open gap

The evaluation is limited to linear probing and full fine-tuning, omitting the exploration of parameter-efficient transfer strategies. Also, the 54 datasets do not exhaustively cover all clinical populations, recording contexts, or geographic regions, and channel corruption simulation simplifies real-world sensor degradation patterns.

Future research

The authors aim to support reproducible research and guide the development of more generalizable EEG representation learning methods.

FALAT: Tracing Failures in LLM Agent Trajectories via Dependency-Guided Search

Md Nakhla Rafi, Md Ahasanuzzaman, Dong Jae Kim et al. · 2026-01-01

LLM-based agents increasingly solve complex tasks through long trajectories involving reasoning steps, tool calls, and inter-agent communication. However, when these agents fail, it is often unclear which agent caused the failure and which step introduced the decisive error. This attribution problem is challenging because mistakes can propagate across the trajectory: later actions may appear incorrect, but only because they depend on an earlier corrupted state. Therefore, failure attribution cannot be treated as independent step-level classification. We propose FALAT, a diagnostic framework for failure attribution in LLM agent trajectories. FALAT frames attribution as a dependency-guided search problem. It first constructs an expectation of how the task should be solved and uses this expectation to identify suspicious regions in the trajectory. It then traces dependencies among decisions, tool outputs, and agent messages to distinguish error-introducing steps from steps that merely inherit or propagate prior mistakes. Finally, FALAT evaluates whether correcting a candidate step would be sufficient to recover the expected outcome, allowing it to identify both the responsible agent and the decisive failure step. We evaluate FALAT on the Who&When benchmark, which includes both algorithm-generated and hand-crafted multi-agent failure trajectories. The results show that FALAT consistently improves responsible-agent and decisive-step attribution. Its best configurations achieve 46.0% step-level accuracy on algorithm-generated trajectories and 29.1% on the more challenging hand-crafted trajectories, outperforming specialized attribution baselines and direct prompting with standalone LLMs. These findings suggest that dependency-aware reasoning is essential for reliable failure diagnosis in LLM agent systems.

Foundation Models × HealthPDF ↗no extraction yet

Detecting psychological defense mechanisms in conversational text remains a challenging clinical NLP problem. For the PsyDefDetect 2026 shared task (nine-class utterance classification evaluated via macro F1), our team LinguIUTics achieves a macro F1-score of 0.3917 on the official positive-class leaderboard, ranking 4th out of 21 registered teams and improving over the Ministral-8B task baseline (31.48 macro F1) by 7.7 absolute points (24.4 percent relative). BERT-family encoders and zero-shot LLMs proved ineffective on rare classes due to severe class imbalance, leading us to QLoRA fine-tuning of Qwen3-8B. We leverage three key strategies: grouped stratified cross-validation (preventing leakage), minority-class round-robin lexical augmentation, and a post-processing pipeline with logit bias tuning and ensemble blending. Together, these components close much of the validation-to-leaderboard gap and substantially improve minority-class recall, driving the critical "Unclear" class (Level 8) from near-zero performance to an F1 score of 0.797.

Foundation Models × HealthPDF ↗
RP5 Analysis
qwen2.5:1.5bon-device

Research question

How can the performance of psychological defense mechanism classification models be further improved by leveraging data-centric imbalance mitigation methods?

Methodology

Fine-tuning Qwen3-8B on a QLoRA dataset with 4-bit NF4 quantization

Conclusions

A macro F1-score of 0.392 was achieved, ranking the final system in fourth place among 21 teams on the official leaderboard.

Importance

The study demonstrates that using data-centric imbalance mitigation methods significantly outperformed raw model capacity in psychological defense classification tasks.

Open gap

The decode rules and OOF bias vectors are domain-specific and must be recalibrated for new datasets. The grouped CV protocol mitigates leakage but cannot eliminate it entirely.

Future research

Concrete next steps include improving paraphrase-based data augmentation, optimizing losses for imbalanced classes, and evaluating on additional datasets.

Multimodal large language models (MLLMs) demonstrate remarkable visual understanding, yet their reliability in interactive settings is severely undermined by hallucination snowballing: a phenomenon where initial errors amplify across conversational turns, leading to a collapse in coherence. This failure reveals a fundamental vulnerability where models progressively neglect visual grounding in favor of over-relying on polluted textual history. Existing benchmarks are predominantly confined to single-turn VQA, which fail to capture the complex dynamics of error propagation in long-horizon interactions. To address this, we introduce MM-Snowball, the first benchmark for fine-grained diagnosis of hallucination snowballing within dialogues. Extensive evaluation shows that our benchmark poses a significant challenge even to advanced MLLMs and reveals the inefficacy of existing mitigation methods designed for single-turn VQA. To counteract this degradation, we propose Conflict-Aware Visual Rectification (CAVR). This training-free method mitigates snowballing through a synergistic dual-mechanism that refreshes visual grounding at the representation level and rectifies output distributions at the logit level, effectively re-anchoring the model to visual facts. Experiments demonstrate that CAVR achieves state-of-the-art performance, offering a promising path toward more reliable interactive AI. Data and code are available at: https://frenkie-chiang.github.io/MM-Snowball

Foundation Models × HealthPDF ↗
RP5 Analysis
gemma4:e2bon-device

Research question

Not specified

Methodology

MM-Snowball and Conflict-Aware Visual Rectification (CAVR)

Conclusions

MM-Snowball evaluates and diagnoses hallucination snowballing in multimodal large language models during multi-turn dialogue. CAVR mitigates snowballing by simultaneously refreshing visual grounding at the representation level and rectifying output distributions at the logit level, achieving state-of-the-art performance.

Importance

Addresses the critical failure mode of hallucination snowballing in MLLMs during multi-turn dialogue, which occurs when initial errors amplify across conversational turns, leading to a collapse in coherence.

Open gap

A systematic understanding of hallucination snowballing in multimodal dialogue is elusive due to limitations in existing research, as current benchmarks fail to capture long-range error propagation and non-linear dynamics, and existing datasets lack a diagnostic framework to classify how various noise sources drive the snowballing effect.

Future research

Not specified

Showing the 30 most recent of 3900 papers · view all on the Pipeline page