Publikation
Diagnostic Reasoning in Natural Language: Computational Model and Application
Nils Dycke; Matej Zecevic; Ilia Kuznetsov; Beatrix Suess; Kristian Kersting; Iryna Gurevych
In: Computing Research Repository eprint Journal (CoRR), Vol. abs/2409.05367, Pages 1-27, arXiv, 2024.
Zusammenfassung
Diagnostic reasoning is a key component of
expert work in many domains. It is a hard,
time-consuming activity that requires expertise,
and AI research has investigated the ways au-
tomated systems can support this process. Yet,
due to the complexity of natural language, the
applications of AI for diagnostic reasoning to
language-related tasks are lacking. To close
this gap, we investigate diagnostic abductive
reasoning (DAR) in the context of language-
grounded tasks (NL-DAR). We propose a novel
modeling framework for NL-DAR based on
Pearl’s structural causal models and instanti-
ate it in a comprehensive study of scientific
paper assessment in the biomedical domain.
We use the resulting dataset to investigate the
human decision-making process in NL-DAR
and determine the potential of LLMs to sup-
port structured decision-making over text. Our
framework, open resources and tools lay the
groundwork for the empirical study of collabo-
rative diagnostic reasoning in the age of LLMs,
in the scholarly domain and beyond.