Lets build a program that will teach the computer to recognize simple patterns using neural networks.
Artificial neural networks, like real brains, are formed from connected "neurons", all capable of carrying out a data-related task, such as answering a question about the relationship between them.
Artificial neural networks, like real brains, are formed from connected "neurons", all capable of carrying out a data-related task, such as answering a question about the relationship between them.
Let's take the following pattern:
1 1 1 = 1
1 0 1 = 1
0 1 1 = 0
Each input, and the output can be only a 1 or a 0. If we look closer, we will realize that the output is 1, if the first input is 1. However, we will not tell that to the computer. We will only provide the sample inputs and outputs and ask it to "guess" the output of the input 1 0 0 (which should be 1).
To make it really simple, we will just model a single neuron, with three inputs and one output.
The three examples above are called a training set.
We're going to train the neuron to work out the pattern and solve the task for input 1 0 0, by just having the training set and without knowing what operation it performs.
1 1 1 = 1
1 0 1 = 1
0 1 1 = 0
Each input, and the output can be only a 1 or a 0. If we look closer, we will realize that the output is 1, if the first input is 1. However, we will not tell that to the computer. We will only provide the sample inputs and outputs and ask it to "guess" the output of the input 1 0 0 (which should be 1).
To make it really simple, we will just model a single neuron, with three inputs and one output.
The three examples above are called a training set.
We're going to train the neuron to work out the pattern and solve the task for input 1 0 0, by just having the training set and without knowing what operation it performs.
Training
We will give each input a weight, which can be a positive or negative number. An input with a large positive weight or a large negative weight, will have a strong effect on the neuron's output. Before we start, we set each weight to a random number. Then we begin the training process:
1. Take the inputs from the training set, adjust them by the weights, and pass them through a special formula to calculate the neuron's output.
2. Calculate the error, which is the difference between the neuron's output and the desired output in the training set example.
3. Depending on the direction of the error, adjust the weights.
4. Repeat this process 10,000 times.
Eventually the weights of the neuron will reach an optimum for the training set. This process is called back propagation.
We will give each input a weight, which can be a positive or negative number. An input with a large positive weight or a large negative weight, will have a strong effect on the neuron's output. Before we start, we set each weight to a random number. Then we begin the training process:
1. Take the inputs from the training set, adjust them by the weights, and pass them through a special formula to calculate the neuron's output.
2. Calculate the error, which is the difference between the neuron's output and the desired output in the training set example.
3. Depending on the direction of the error, adjust the weights.
4. Repeat this process 10,000 times.
Eventually the weights of the neuron will reach an optimum for the training set. This process is called back propagation.
Forthe formula, we will take the weighted sum of the inputs and normalize it between 0 and 1:
After each iteration, we need to adjust the weight based on the error (the difference of the calculated output and the real output). We will use this formula:
adjustment = error*input*output*(1-output)
This will make the adjustment proportional to the size of the error. After each adjustment the error size should get smaller and smaller.
After the 10.000 iterations, we will have optimum weights and then we can give the program our desired inputs. The program will use the weights and calculate the output using the same weighted sum formula as above.
If do you like this post and want more about neural networks than comments and feel free.
Comments
Post a Comment