These slides explain what a perceptron is and how a neural network is developed based on perceptrons.
1. Hello and welcome to this lecture on Perceptrons. In this lecture we will find out what a perceptron is and how you come from a perceptron to a neural network.
2. The perceptron is the base building block of a neural network. Stacking several perceptrons in line will result in a multi-layer perceptron, which is effectively a neural network.
a. A perceptron has four main parts. The first part is the input values. This is our input layer. In our example we have four inputs.
b. The input values are multiplied with their weights.
c. Now all these multiplication results are summed up and we get a weighted sum.
d. This value is passed to an activation function. As the name suggests, it might activate or not. That depends on the weighted sum and the activation function itself. The result is one output value.
e. Actually we missed one ingredient. There is a value of one that is multiplied with a weight w0. What is this? This is a bias term. The bias term adds a constant value to the weighted sum and this way, it allows you to shift the activation function up and down.
3. Let’s study an example.
a. We have some input values and corresponding weights. Initially the weights are set randomly, so let’s assume the are the weights we got. Now we calculate the product of one and 0.5, 2 and -1.25, and so on.
b. Of all these products, we take the sum, which is -0.5. This value is passed to the activation function.
c. There are different activation functions, that we will learn later in this lecture. For now, assume we take this activation function. If you pass a weighted sum that is smaller or equal to zero, it returns a zero, otherwise it return the value itself. Here, a negative value was passed, so we end up with an output of zero.
4. You can now stack that together to a more complex network.Here you see a simple neural network. Input values are shown in green. Hidden nodes in blue, and output nodes in orange. This is a two layer neural network. Why two? Because typically the input layer is not counted. There are three input values, four hidden nodes, and three output values. Everything is possible. You can use more or less hidden nodes than you have input values. The output nodes count is typically defined by the problem you want to solve. If you have a binary classification, the output node should only predict zero or one. That can be done with one node. But if you want to solve a multi-label classification problem, you need as many output nodes as you have classes.
5. Now we see a bit more complex example. We added one more hidden layer, so we now have a three layer neural network. Now we have two hidden layers, one input layer, and one output layer. All nodes are connected to all nodes of the subsequent layer. This is one specific type of layer. We will touch layer types in the following slides.
That’s it for this lecture. Thank you very much for watching and see you in the next one.
2. The perceptron is the base building block of a neural network. Stacking several perceptrons in line will result in a multi-layer perceptron, which is effectively a neural network.
a. A perceptron has four main parts. The first part is the input values. This is our input layer. In our example we have four inputs.
b. The input values are multiplied with their weights.
c. Now all these multiplication results are summed up and we get a weighted sum.
d. This value is passed to an activation function. As the name suggests, it might activate or not. That depends on the weighted sum and the activation function itself. The result is one output value.
e. Actually we missed one ingredient. There is a value of one that is multiplied with a weight w0. What is this? This is a bias term. The bias term adds a constant value to the weighted sum and this way, it allows you to shift the activation function up and down.
3. Let’s study an example.
a. We have some input values and corresponding weights. Initially the weights are set randomly, so let’s assume the are the weights we got. Now we calculate the product of one and 0.5, 2 and -1.25, and so on.
b. Of all these products, we take the sum, which is -0.5. This value is passed to the activation function.
c. There are different activation functions, that we will learn later in this lecture. For now, assume we take this activation function. If you pass a weighted sum that is smaller or equal to zero, it returns a zero, otherwise it return the value itself. Here, a negative value was passed, so we end up with an output of zero.
4. You can now stack that together to a more complex network.Here you see a simple neural network. Input values are shown in green. Hidden nodes in blue, and output nodes in orange. This is a two layer neural network. Why two? Because typically the input layer is not counted. There are three input values, four hidden nodes, and three output values. Everything is possible. You can use more or less hidden nodes than you have input values. The output nodes count is typically defined by the problem you want to solve. If you have a binary classification, the output node should only predict zero or one. That can be done with one node. But if you want to solve a multi-label classification problem, you need as many output nodes as you have classes.
5. Now we see a bit more complex example. We added one more hidden layer, so we now have a three layer neural network. Now we have two hidden layers, one input layer, and one output layer. All nodes are connected to all nodes of the subsequent layer. This is one specific type of layer. We will touch layer types in the following slides.
That’s it for this lecture. Thank you very much for watching and see you in the next one.