On polynomial neural networks and experiments with relational data

Abstract

Artificial neural networks are directed finite acyclic graphswhere a value of a node is determined as a function of its predecessor nodes. Commonly, the function relationship between a node and its predecessors is a composition of an affine transformation and a continuous function, where coefficients of the affine transformation are determined node-wise in a training process for the artificial neural network. However, exceptions to this generic definition are more than common.

In this thesis a few polynomial neural network architectures are discussed. The formal meaning of a ``polynomial'' with respect to neural networks has not been fully established, so several similar architectures are presented. The main focus will be on factorization machines and neural networks with quadratic polynomial kernel. The fundamental definitions and key results are presented based on the existing literature to make the presentation self-contained to some extent.

Finally, the performance of the neural network architectures will be compared by running computational experiments. Based on these experiments, polynomial networks and factorization machines are not necessarily superior to traditional polynomial regression or kernel methods. The superiority of one method over another appears to be dependent on the given data.