What's Wrong with Gradient-based Complex Query Answering?

Ouns El Harzli

Samy Badreddine

Tarek Besold

NeSy 2023

2023

Abstract

Multi-hop query answering on knowledge graphs is known to be a challenging computational task. Neurosymbolic approaches using neural link predictors have shown promising results but are still outperformed by combinatorial optimization methods on several benchmarks, including the FB15k dataset. We analyze the task on the FB15k dataset and propose two new gradient-based methods, one learning simultaneously the representations of several candidate answers and the other learning a skolem function projecting to candidate answers instead of learning direct candidate representations. We implement both using Logic Tensor Networks. As part of this investigation we identified two important factors that limit the ability of differentiable methods to learn correct answers. The first factor is the (un)reliability of the pre-trained neural link predictors which biases the guesses of the query solver. To account for this, we suggest new evaluation metrics using satisfiability scores, that better reflect the true performance of neurosymbolic approaches on multi-hop query answering. The second factor is the regularization technique proposed in previous works, which limits the exploration of the gradient-based solver. Our results provide the foundation for future work mitigating these bottlenecks.