Neural Networks are a powerful tool for solving complex problems in a variety of domains, from computer vision to natural language processing. In recent years, the Go programming language has gained popularity for its simplicity, speed, and concurrency features. In this tutorial, we will explore how to implement neural networks with Go programming.
Overview
In this tutorial, we will implement a simple neural network using Go programming. The network will have one input layer, one hidden layer, and one output layer. We will use the backpropagation algorithm to train the network on a simple classification task.
The following steps will be covered:
- Preparing the data
- Creating the neural network structure
- Forward propagation
- Backpropagation
- Updating weights
- Training the network
- Evaluating the performance
- Improving the network
Preparing the Data
We will use a simple binary classification task to train our network. The task is to predict whether a point lies above or below a line. We will generate random points and label them based on their position relative to the line.
Let's generate some data:
import (
"math/rand"
)
type point struct {
x float64
y float64
}
func generateData(n int) ([]point, []float64) {
var data []point
var labels []float64
for i := 0; i < n; i++ {
x := rand.Float64()*10 - 5 // range [-5, 5]
y := rand.Float64()*10 - 5 // range [-5, 5]
label := 0.0
if y > 2*x+1 {
label = 1.0
}
data = append(data, point{x, y})
labels = append(labels, label)
}
return data, labels
}
This function generates n random points and labels them based on their position relative to the line y = 2x + 1. The generateData function returns two slices: one with the points and one with the labels.
Creating the Neural Network Structure
We will create a struct to represent our neural network. The struct will have fields for the number of input neurons, the number of neurons in the hidden layer, the number of neurons in the output layer, and the weights and biases for each layer.
type neuralNetwork struct {
inputNeurons int
hiddenNeurons int
outputNeurons int
weights1 [][]float64
weights2 [][]float64
biases1 []float64
biases2 []float64
}The weights1 and weights2 fields represent the weights for the connections between the input and hidden layers, and the hidden and output layers, respectively. The biases1 and biases2 fields represent the biases for the hidden and output layers, respectively.
We will use the sigmoid activation function for the hidden layer and the softmax activation function for the output layer.
func sigmoid(x float64) float64 {
return 1 / (1 + math.Exp(-x))
}
func softmax(xs []float64) []float64 {
ys := make([]float64, len(xs))
sum := 0.0
for _, x := range xs {
sum += math.Exp(x)
}
for i, x := range xs {
ys[i] = math.Exp(x) / sum
}
return ys
}
Forward Propagation
The forward propagation step computes the output of the neural network given an input. The input is propagated through the layers of the network using the weights and biases, and the activation functions.
func (nn *neuralNetwork) forward(input []float64) []float64 {
hidden := make([]float64, nn.hiddenNeurons)
for j := 0; j < nn.hiddenNeurons; j++ {
sum := 0.0
for i := 0; i < nn.inputNeurons; i++ {
sum += input[i] * nn.weights1[i][j]
}
sum += nn.biases1[j]
hidden[j] = sigmoid(sum)
}
output := make([]float64, nn.outputNeurons)
for j := 0; j < nn.outputNeurons; j++ {
sum := 0.0
for i := 0; i < nn.hiddenNeurons; i++ {
sum += hidden[i] * nn.weights2[i][j]
}
sum += nn.biases2[j]
output[j] = sum
}
return softmax(output)
}The forward method takes an input vector and returns the output of the neural network. The input is first passed through the hidden layer using the sigmoid activation function, and then through the output layer using the softmax activation function. The output is a probability distribution over the classes.
Backpropagation
The backpropagation step computes the gradient of the loss function with respect to the weights and biases, and updates them using the gradient descent algorithm.
func (nn *neuralNetwork) backprop(input []float64, label float64, learningRate float64) {
hidden := make([]float64, nn.hiddenNeurons)
for j := 0; j < nn.hiddenNeurons; j++ {
sum := 0.0
for i := 0; i < nn.inputNeurons; i++ {
sum += input[i] * nn.weights1[i][j]
}
sum += nn.biases1[j]
hidden[j] = sigmoid(sum)
}
output := make([]float64, nn.outputNeurons)
for j := 0; j < nn.outputNeurons; j++ {
sum := 0.0
for i := 0; i < nn.hiddenNeurons; i++ {
sum += hidden[i] * nn.weights2[i][j]
}
sum += nn.biases2[j]
output[j] = sum
}
probs := softmax(output)
dOutput := make([]float64, nn.outputNeurons)
for j := 0; j < nn.outputNeurons; j++ {
if label == float64(j) {
dOutput[j] = probs[j] - 1
} else {
dOutput[j] = probs[j]
}
}
dHidden := make([]float64, nn.hiddenNeurons)
for j := 0; j < nn.hiddenNeurons; j++ {
sum := 0.0
for k := 0; k < nn.outputNeurons; k++ {
sum += nn.weights2[j][k] * dOutput[k]
}
dHidden[j] = hidden[j] * (1 - hidden[j]) * sum
}
for i := 0; i < nn.inputNeurons; i++ {
for j := 0; j < nn.hiddenNeurons; j++ {
nn.weights1[i][j] -= learningRate * input[i] * dHidden[j]
}
}
for j := 0; j < nn.hiddenNeurons; j++ {
for k := 0; k < nn.outputNeurons; k++ {
nn.weights2[j][k] -= learningRate * hidden[j] * dOutput[k]
}
}
for j := 0; j < nn.hiddenNeurons; j++ {
nn.biases1[j] -= learningRate * dHidden[j]
}
for j := 0; j < nn.outputNeurons; j++ {
nn.biases2[j] -= learningRate * dOutput[j]
}
}The `backprop` method takes an input vector, a label, and a learning rate. It first performs the forward pass to compute the probabilities of the classes. Then, it computes the gradient of the loss function with respect to the weights and biases using the chain rule of differentiation. Finally, it updates the weights and biases using the gradient descent algorithm.
Training
To train the neural network, we need to repeatedly feed the training examples to the network, compute the loss and the gradients using the `forward` and `backprop` methods, and update the weights and biases using the gradient descent algorithm. We will also measure the accuracy of the network on the validation set at each epoch to detect overfitting.
func (nn *neuralNetwork) train(xTrain, yTrain, xVal, yVal [][]float64, epochs int, learningRate float64) {
for epoch := 1; epoch <= epochs; epoch++ {
// train
for i, input := range xTrain {
label := yTrain[i][0]
nn.backprop(input, label, learningRate)
}
// evaluate
trainAcc := nn.evaluate(xTrain, yTrain)
valAcc := nn.evaluate(xVal, yVal)
fmt.Printf("Epoch %d: train_acc=%.3f, val_acc=%.3f\n", epoch, trainAcc, valAcc)
}
}
func (nn *neuralNetwork) evaluate(x, y [][]float64) float64 {
correct := 0.0
for i, input := range x {
label := y[i][0]
output := nn.forward(input)
prediction := argmax(output)
if prediction == int(label) {
correct++
}
}
return correct / float64(len(x))
}The train method takes the training and validation sets, the number of epochs, and the learning rate. It iterates over the training set and calls the backprop method for each example. Then, it evaluates the accuracy of the network on the training and validation sets using the evaluate method. The evaluate method feeds each example to the network, computes the output, and compares the prediction with the label. It returns the fraction of correct predictions.
Conclusion
In this tutorial, we have implemented a simple neural network with one hidden layer using the Go programming language. We have shown how to define the architecture of the network, initialize the weights and biases, perform the forward pass, compute the gradients using the backpropagation algorithm, and update the weights and biases using the gradient descent algorithm. We have also shown how to train the network on a classification task and measure its accuracy on a validation set. The implementation can be extended to handle more complex tasks and architectures, and can serve as a starting point for further research in deep learning with Go.