Publication
Learning differentiable logic programs for abstract visual reasoning
Hikaru Shindo; Viktor Pfanschilling; Devendra Singh Dhami; Kristian Kersting
In: Journal of Machine Learning Research (JMLR), Vol. 113, No. 11, Pages 8533-8584, arXiv, 2024.
Abstract
Visual reasoning is essential for building intelligent agents that un-
derstand the world and perform problem-solving beyond perception. Differ-
entiable forward reasoning has been developed to integrate reasoning with
gradient-based machine learning paradigms. However, due to the memory in-
tensity, most existing approaches do not bring the best of the expressivity
of first-order logic, excluding a crucial ability to solve abstract visual reason-
ing, where agents need to perform reasoning by using analogies on abstract
concepts in different scenarios. To overcome this problem, we propose NEUro-
symbolic Message-pAssiNg reasoNer (NEUMANN), which is a graph-based
differentiable forward reasoner, passing messages in a memory-efficient man-
ner and handling structured programs with functors. Moreover, we propose
a computationally-efficient structure learning algorithm to perform explana-
tory program induction on complex visual scenes. To evaluate, in addition
to conventional visual reasoning tasks, we propose a new task, visual rea-
soning behind-the-scenes, where agents need to learn abstract programs and
then answer queries by imagining scenes that are not observed. We empiri-
cally demonstrate that NEUMANN solves visual reasoning tasks efficiently,
outperforming neural, symbolic, and neuro-symbolic baselines