An Overview of Artificial Neural Networks:Part 3 Activation Functions

This paper presents the concepts of Activation Functions in Artificial Neural Networks (ANNs). The activation function is essential in ANN to map relationship between input and output.  The nonlinear property between input and output in ANN is added by the Nonlinear Activation Functions. The different types of Threshold, Linear and Nonlinear activation function can be used in ANN. This paper aims to explain concept of Nonlinear Activation Functions i.e. Sigmoidal Activation Function and Hyperbolic Tangent Activation Function. How to select Activation Functions in ANN is discussed. The effect of type of Activation Function and slope of activation function on Mean Square Error is analyzed with the help of nonlinear input and output data. The algorithm and MATLAB Code is also added to help researcher.