Revisiting the XOR problem using spiking neural networks in a robotic context

The XOR problem still represents a challenge in the study of cognition and computational simulations, since its precise neural substrate remains to be discovered in natural organisms that succeeded in the task. This paper focuses on this particular problem, in a context of a spiking neural network acting as a robot's brain controller. Hence, the ability to solve ambiguous nonlinear problems from a neuro-robotic model may contribute to understand the natural function as well as proposing an artificial embedded solution. By using the same core components in both virtual and real environments and from operant conditioning procedures, the robot was able to correctly learn the XOR rule. Beyond resolving a XOR-like problem within real world constraints, this bio-inspired neural circuit also integrates related associative learning scenarios. Particularly, this study explores the consequences of passing from a 2-bit to a 3-bit task, analyzing the effects on the overall neural network, its neurons and synapses. The experiment consists in a robot choosing between a left or right action, depending on an image shown to it and seen by its camera. Within few trials, the robot was able to successfully learn all possible rewarding rules. Moreover, behavioral adaptation was shown by switching the rewarding rules once the robot had learned them.

Note : The following is all supplementary materials available for the article.

Architectures

2-bit XOR

Legend : P = predictor, L = left, R = right, BU = black up, BD = black down, WU = white up, WD = white down

3-bit XOR

Legend : P = predictor, L = left, R = right, BU = black up, BD = black down, WU = white up, WD = white down

Videos

Results

2-bit XOR

This Figure shows the XOR simulation when the input are randomized. Legend : P = predictor, L = left, R = right, BU = black-up, BD = black-down, WU = white-up, WD = white-down. Following a ashed image (graph A), a random decision is made (graphs B and E), but eventually learns (graphs I to X) from rewards (graph H) the correct motor action (graphs D and G). Graphs Y to AF represent simple associations (ex : black-up), not always rewarded by the XOR rule. Graphs I to AF shows the STDP coefficients, reaching its maximum value (100% of synaptic weight increase) in a single correct action.

3-bit XOR

In this figure, we can see that primary and secondary weights are decreased over time. The proper tertiary synapses increase after being rewarded. Legend : S = Synapse, N = Neuron, P = predictor, L = left, R = right, BU = black up, BD = black down, WU = white up, WD = white down

Real robot simulation

Other materials

We also did a simulation to evaluate the robustness of the SNN using the same type of architecture, but this time adapted for the 3-bit task. As expected, the SNN responds well and finds rapidly the solution, except that it takes longer to learn all the rules since many more input patterns were presented. As shown in the figure below, going from a 2-bit to a 3-bit problem requires a third layer of neurons. Here, only one action is shown. Thus, for the XOR problem where two actions are required, the number of neurons needs to be doubled. Also, only few excitatory synapses are illustrated on the figure, for readability. In this SNN implementation, only specific sensory neurons are connected to their predictors. The signal is redundantly propagated from the sensory layer to all predictors layers. Note that the only way to make a predictor neuron spike once learned is by receiving from all its incoming synapses.

Neural complexity comparison between the 2-bit and 3-bit.
  2-bit 3-bit
Primary 8 12
Secondary 8 24
Tertiary -- 16
Number of necessary predictor neurons to solve the XOR.

Data (as CSV)

XOR 2-bit

XOR 3-bit