Java XOR 神经网络未正确训练

Question

我有一个具有 2 个输入、2 个隐藏神经元和 1 个输出神经元的神经网络来解决这个xor问题。0.1我随机初始化 0 到 1 之间的权重，我使用带有sigmoid激活函数的学习率。

当我只训练一个选项时，例如目标为 1 的 1 和 0，它可以正常工作并给出适当的猜测。但是，当我尝试将所有可能的输入一起训练时，输出会收敛到0.5-0.6.

我尝试过改变学习率、随机初始化权重的范围以及网络训练的次数，但这对最终输出没有影响。

这是我在GitHub 上的代码的链接。

关于如何解决此问题的任何想法？

Answer 1

我怀疑反向传播没有正确实施。例如http://users.pja.edu.pl/~msyd/wyk-nai/multiLayerNN-en.pdf在第 17 至 20 页中给出了概述。- 类的tuneWeigths- 和delta_weights- 方法Output_Neuron被正确实现。但是，在此步骤weightDeltaHidden中，必须确定稍后Hidden_Neuron调整 -class 的权重时需要的数组（参见代码中的注释）。- 类的tuneWeigths- 和delta_weights- 方法Hidden_Neuron似乎没有正确实现。在这里，除其他外，weightDeltaHidden必须使用先前确定的数组。在下面的代码中，我进行了必要的更改，但实际上并未更改代码的设计。但也许重构是有意义的。类的变化Output_Neuron：...private double[] weightedDeltaHidden;...Output_Neuron(int hiddenNeurons) {&nbsp; &nbsp; ...&nbsp; &nbsp; this.weightedDeltaHidden = new double[hiddenNeurons];}...void tuneWeights(double LR, double[] hidden_output, int target) {&nbsp; &nbsp; double delta = (target - output) * f.dSigmoid(output);&nbsp; &nbsp; for (int i = 0; i < weights.length; i++) {&nbsp; &nbsp; &nbsp; &nbsp; weights[i] += delta_weights(i, LR, delta, hidden_output);&nbsp; &nbsp; }}double delta_weights(int i, double LR, double delta, double[] hidden_output) {&nbsp; &nbsp; weightedDeltaHidden[i] = delta * weights[i]; // weightedDeltaHidden is the product of delta of this output neuron and the weight of the i-th hidden neuron.&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;// That value is needed when the weights of the hidden neurons are tuned...&nbsp; &nbsp; return LR * delta * hidden_output[i];}...double[] getWeightedDeltaHidden() {&nbsp; &nbsp; return weightedDeltaHidden;}类的变化Hidden_Neuron：...void tuneWeights(double LR, int[] inputs, double weightedDeltaHiddenTotal) {&nbsp; &nbsp; for (int i = 0; i < weights.length; i++) {&nbsp; &nbsp; &nbsp; &nbsp; weights[i] += delta_weights(LR, inputs[i], weightedDeltaHiddenTotal);&nbsp; &nbsp; }}private double delta_weights(double LR, double input, double weightedDeltaHiddenTotal) {&nbsp; &nbsp; double deltaOutput = f.dSigmoid(output) * weightedDeltaHiddenTotal;&nbsp; &nbsp; return LR * deltaOutput * input;}...调整隐藏权重Network的 - 方法内- 类的变化：trainvoid train(int[] inputs, int target) {&nbsp; &nbsp; ...&nbsp; &nbsp; //tune Hidden weights&nbsp; &nbsp; for (int i = 0; i < numOfHiddenNeurons; i++) {&nbsp; &nbsp; &nbsp; &nbsp; double weightedDeltaHiddenTotal = 0;&nbsp; &nbsp; &nbsp; &nbsp; for (int j = 0; j < numOfOutputNeurons; j++) {&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; weightedDeltaHiddenTotal += output_neurons[j].getWeightedDeltaHidden()[i]; // weightedDeltaHiddenTotal is the sum of the weightedDeltaHidden over all output neurons. Each weightedDeltaHidden&nbsp; &nbsp; &nbsp; &nbsp; }&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; // is the product of delta of the j-th output neuron and the weight of the i-th hidden neuron.&nbsp; &nbsp; &nbsp; &nbsp; hidden_neurons[i].tuneWeights(LR, inputs, weightedDeltaHiddenTotal);&nbsp; &nbsp; }}通过这些更改，1_000_000 次调用train（2 个隐藏神经元）的典型输出为Error: 1.9212e-01 in cycle 0Error: 8.9284e-03 in cycle 100000Error: 1.5049e-03 in cycle 200000Error: 4.7214e-03 in cycle 300000Error: 4.4727e-03 in cycle 400000Error: 2.1179e-03 in cycle 500000Error: 2.9165e-04 in cycle 600000Error: 2.0655e-03 in cycle 700000Error: 1.5381e-03 in cycle 800000Error: 1.0440e-03 in cycle 9000000 0: 0.01701 0: 0.96160 1: 0.96121 1: 0.0597以及 100_000_000 次调用train（2 个隐藏神经元）Error: 2.4755e-01 in cycle 0Error: 2.7771e-04 in cycle 5000000Error: 6.8378e-06 in cycle 10000000Error: 5.4317e-05 in cycle 15000000Error: 6.8956e-05 in cycle 20000000Error: 2.1072e-06 in cycle 25000000Error: 2.6281e-05 in cycle 30000000Error: 2.1630e-05 in cycle 35000000Error: 1.1546e-06 in cycle 40000000Error: 1.7690e-05 in cycle 45000000Error: 8.6837e-07 in cycle 50000000Error: 1.3603e-05 in cycle 55000000Error: 1.2905e-05 in cycle 60000000Error: 2.1657e-05 in cycle 65000000Error: 1.1594e-05 in cycle 70000000Error: 1.9191e-05 in cycle 75000000Error: 1.7273e-05 in cycle 80000000Error: 9.1364e-06 in cycle 85000000Error: 1.5221e-05 in cycle 90000000Error: 1.4501e-05 in cycle 950000000 0: 0.00081 0: 0.99610 1: 0.99611 1: 0.0053隐藏神经元的增加会提高性能。下面显示了 1_000_000 次调用train（4 个隐藏神经元）的典型输出：Error: 1.2617e-02 in cycle 0Error: 7.9950e-04 in cycle 100000Error: 4.2567e-04 in cycle 200000Error: 1.7279e-04 in cycle 300000Error: 1.2246e-04 in cycle 400000Error: 1.0456e-04 in cycle 500000Error: 6.9140e-05 in cycle 600000Error: 6.8698e-05 in cycle 700000Error: 5.1640e-05 in cycle 800000Error: 4.4534e-05 in cycle 9000000 0: 0.00921 0: 0.99050 1: 0.99121 1: 0.0089