Radial Basis Function Networks (RBFNs)

Introduction

In this article, we will talk about one of the algorithms that belong to the deep learning algorithms, RBFNs, as they are a special type of feeder neural network that use radial basis functions as activation functions. It has an input layer, a hidden layer, and an output layer and is mostly used for classification, regression, and time-series prediction.

What is a radial basis function network?

Radial basis function (RBF) networks are a common type of use in artificial neural networks for function approximation problems. Radial-based function networks are distinguished from other neural networks due to their global approximation and fast learning speed. and RBF neural networks are also a type of feed-forward network trained using a supervised training algorithm. The main advantage of the RBF network is that it has only one hidden layer and uses the radial basis function as the activation function. These functions are very powerful in approximation.

How Do RBFNs Work?

• RBFNs perform classification by measuring the input’s similarity to examples from the training set.
• RBFNs have an input vector that feeds to the input layer. They have a layer of RBF neurons.
• The function finds the weighted sum of the inputs, and the output layer has one node per category or class of data.
• The neurons in the hidden layer contain the Gaussian transfer functions, which have outputs that are inversely proportional to the distance from the neuron’s center.
• The network’s output is a linear combination of the input’s radial-basis functions and the neuron’s parameters.

Input vector

The input vector is the n-dimensional vector you are trying to classify. The entire input vector is shown for each of the RBF neurons.

RBF. neurons

Each RBF neuron stores a ‘prototype’ vector, just one of the vectors from the training set. Each RBF neuron compares the input vector to its prototype and outputs a value between 0 and 1 which is a measure of similarity. If the input is equal to the prototype, the output of the RBF neuron will be 1. As the distance between the input and the prototype increases, the response decreases exponentially towards 0. The shape of the response of the RBF neuron is a bell curve, as shown in the network geometry diagram. The value of a neuron’s response is also called the “activation” value. The prototype vector is also often called the “center” of a neuron since it is the value at the center of the bell curve.

What is RBFNNoutput?

The input layer transforms the data into hidden neurons that contain radial basis activation functions. This function generally calculates the distance between the network inputs and the centers of the hidden layer. The output assembly of the hidden layers is provided with some weight as the output of the RBFNN.

What is a Radial Basis Function?

The Output Nodes

The output of the network consists of a set of nodes, one per category that we are trying to classify. Each output node computes a sort of score for the associated category. Typically, a classification decision is made by assigning the input to the category with the highest score.

The score is computed by taking a weighted sum of the activation values from every RBF neuron. By weighted sum, we mean that an output node associates a weight value with each of the RBF neurons, and multiplies the neuron’s activation by this weight before adding it to the total response. Because each output node is computing the score for a different category, every output node has its own set of weights. The output node will typically give a positive weight to the RBF neurons that belong to its category, and a negative weight to the others.

Each RBF neuron computes a measure of the similarity between the input and its prototype vector (taken from the training set). Input vectors which are more similar to the prototype return a result closer to 1. There are different possible choices of similarity functions, but the most popular is based on the Gaussian. Below is the equation for a Gaussian with a one-dimensional input.

Where x is the input, mu is the mean, and sigma is the standard deviation. This produces the familiar bell curve is shown below, which is centered at the mean, mu (in the below plot the mean is 5 and sigma is 1).

The radial basis function is a function whose value depends only on the distance from the origin. In fact, the function should contain only real values. The alternate forms of radial basis functions are defined as the distance from another point denoted C, called the center.

What are the advantages and disadvantages of the RBF network?

The main advantage of the RBF network is that it has only one hidden layer and it uses the radial basis function as the activation function. These functions are very powerful in approximation.

Conclusion

In this article, we talked about one of the algorithms for deep learning, as it is one of the best algorithms, and we talked about its definition, characteristics, the function it works on, and its advantages. I hope the article has achieved what you are asking for.

Mohamed B Mahmoud. Data Scientist.