Publikation
Neural Concept Binder
Wolfgang Stammer; Antonia Wüst; David Steinmann; Kristian Kersting
In: Computing Research Repository eprint Journal (CoRR), Vol. abs/2406.09949, Pages 1-39, arXiv, 2024.
Zusammenfassung
The challenge in object-based visual reasoning lies in generating concept repre-
sentations that are both descriptive and distinct. Achieving this in an unsupervised
manner requires human users to understand the model’s learned concepts and,
if necessary, revise incorrect ones. To address this challenge, we introduce the
Neural Concept Binder (NCB), a novel framework for deriving both discrete and
continuous concept representations, which we refer to as “concept-slot encodings”.
NCB employs two types of binding: “soft binding”, which leverages the recent
SysBinder mechanism to obtain object-factor encodings, and subsequent “hard
binding”, achieved through hierarchical clustering and retrieval-based inference.
This enables obtaining expressive, discrete representations from unlabeled images.
Moreover, the structured nature of NCB’s concept representations allows for intu-
itive inspection and the straightforward integration of external knowledge, such as
human input or insights from other AI models like GPT-4. Additionally, we demon-
strate that incorporating the hard binding mechanism preserves model performance
while enabling seamless integration into both neural and symbolic modules for
complex reasoning tasks. We validate the effectiveness of NCB through evaluations
on our newly introduced CLEVR-Sudoku dataset.