Oriental Journal of Computer Science and Technology

Open Access Open Access  Restricted Access Subscription Access

Architecture Selection in Neural Networks by Statistical and Machine Learning


Affiliations
1 Department of Statistics, Hacettepe University, Turkey
 

One of the biggest problems in using artificial neural networks is to determine the best architecture. This is a crucial problem since there are no general rules to select the best architecture structure. Selection of the best architecture is to determine how many neurons should be used in the layers of a network. It is a well-known fact that using a proper architecture structure directly affect the performance of the method. Therefore, various approaches ranging from trial and error method to heuristic optimization algorithms have been suggested to solve this problem in the literature. Although there have been systematical approaches in the literature, trial and error method has been widely used in various applications to find a good architecture. This study propose a new architecture selection method based on statistical and machine learning. The proposed method utilizes regression analysis that is a supervised learning technique in machine learning. In this new architecture selection approach, it is aimed to combine statistical and machine learning to reach good architectures which has high performance. The proposed approach brings a new perspective since it is possible to perform statistical hypothesis tests and to statistically evaluate the obtained results when artificial neural networks are used. The best architecture structure can be statistically determined in the proposed approach. In addition to this, the proposed approach provides some important advantages. This is the first study using a statistical method to utilize statistical hypothesis tests in artificial neural networks. Using regression analysis is easy to use so applying the proposed method is also easy. And, the proposed approach saves time since the best architecture is determined by regression analysis. Furthermore, it is possible to make inference for architectures which is not examined. The proposed approach is applied to three real data sets to show the applicability of the approach. The obtained results show that the proposed method gives very satisfactory results for real data sets.

Keywords

Architecture Selection, Artificial Neural Networks, Machine Learning, Regression Analysis, Statistical Hypothesis Tests, Time Series.
User
Notifications
Font Size

  • Aladag C.H., Egrioglu E., Gunay S. A new architecture selection strategy in solving seasonal autoregressive time series by artificial neural networks. Hacettepe Journal of Mathematics and Statistics, 2008, 37(2): 185–200.

  • Aladag C.H. Using tabu search algorithm in the selection of architecture for artificial neural networks. PhD thesis, Hacettepe University, 2009, Institute for Graduate School of Science and Engineering.

  • Aladag C.H. A new architecture selection method based on tabu search for artificial neural networks. Expert Systems with Applications, 2011, 38: 3287–3293.

  • Aladag C.H. A new candidate list strategy for architecture selection in artificial neural networks. In Robert W. Nelson (ed) New developments in artificial neural networks research Nova Publisher, 2011, pp 139-150, ISBN: 978-1-61324-286-5.

  • Aladag C.H. An architecture selection method based on tabu search. In Aladag CH and Egrioglu E (ed) Advances in time series forecasting, Bentham Science Publishers Ltd., 2012, pp. 88-95, eISBN: 978-1-60805373-5.

  • Aladag C.H., Kayabasi A., Gokceoglu C. Estimation of pressuremeter modulus and limit pressure of clayey soils by various artificial neural network models. Neural Computing & Applications, 2013, 23(2): 333339.

  • Aladag C.H., Egrioglu E., Yolcu U. Robust multilayer neural network based on median neuron model. Neural Computing & Applications, 2014, 24: 945-956.

  • Arriandiaga A., Portillo E., Sanchez J.A., Cabanes I., Pombo I. A new approach for dynamic modelling for energy consumption in the grinding process using recurrent neural networks. Neural Computing & Applications, 2015, 27(6): 1-16.

  • Balestrassi P.P., Popova E., Paiva A.P., Marangon L.J.W. Design of experiments on nn training for nonlinear time series forecasting, Neurocomputing, 2009, 72 (4-6): 1160-1178.

  • Buhamra S., Smaoui N., Gabr M. The Box–Jenkins analysis and neural networks: Prediction and time series modelling. Applied Mathematical Modeling, 2003, 27: 805–815.

  • Dam M., Saraf D.N. Design of neural networks using genetic algorithm for on-line property estimation of crude fractionator products. Computers and Chemical Engineering, 2006, 30: 722–729.

  • Durbin B., Dudoit S., Van der Laan M.J. A deletion/substitution/addition algorithm for classification neural networks, with applications to biomedical data. Journal of Statistical Planning and Inference, 2008, 138:464–488.

  • Egrioglu E., Aladag C.H., Gunay S. A new model selection strategy in artificial neural network. Applied Mathematics and Computation, 2008, 195: 591-597.

  • Gunay S., Egrioglu E., Aladag C.H. Introduction to single variable time series analysis. Hacettepe University Press, 2007, ISBN: 978-975-491-242-5.

  • Gundogdu O., Egrioglu E., Aladag C.H., Yolcu U. Multiplicative neuron model artificial neural network based on Gaussian activation function. Neural Computing & Applications, 2016, Volume 27(4): 927–935.

  • Krenker J., Bešter Kos A. Introduction to the artificial neural networks. In Prof. Kenji Suzuki (ed) Artificial neural networks methodological advances and biomedical applications, 2011, ISBN: 978-953-307243-2, InTech, Available from: http://www.intechopen.com/books/artificial-neural-networksmethodologicaladvances-and-biomedical-applications/introduction-to-the-artificial-neural-networks.

  • Leahy P., Kiely G., Corcoran G. Structural optimisation and input selection of an artificial neural network for river level prediction. Journal of Hydrology, 2008, 355: 192–201.

  • Murata N., Yoshizawa S., Amari S. Network information criterion determining the number of hidden units for an artificial neural network model. IEEE Transaction on Neural Networks, 1994, 5: 865–872.

  • Rathbun T.F., Rogers S.K., DeSimo M.P., Oxley M.E. MLP iterative construction algorithm. Neurocomputing, 1997, 17: 195– 216.

  • Roy A., Kim L.S., Mukhopadhyay S. A polynomial time algorithm for the construction and training of a class of multilayer perceptrons. Neural Networks, 1993, 6: 535–545.

  • Siestema J., Dow R. Neural net pruning: why and how? In Proceedings of the IEEE international conference on neural networks, 1988, (1): 325–333.

  • Solomatine D., See L.M., Abrahart R.J. DataDriven Modelling: Concepts. In RJ Abrahart, LM See, DP Solomatine (ed) Approaches and experiences, practical hydroinformatics, Part I, 2008, Springer Berlin Heidelberg, pp 17-30. doi: 10.1007/978-3-540-79881-1_2.

  • Talaee Hosseinzadeh P. Multilayer perceptron with different training algorithms for streamflow forecasting. Neural Computing & Applications 24:695-703.

  • Yaseen Z.M., El-Shafie A., Afan HA , Hameed M, Wan Hanna Melini Wan Mohtar WHMWM, Hussain A. RBFNN versus FFNN for daily river flow forecasting at Johor River, Malaysia. Neural Computing & Applications, 2016, Volume 27(6): 1533–1542.

  • Ye J., Qiao J., Li M., Ruan X. A tabu based neural network learning algorithm. Neurocomputing, 2007, 70: 875–882.

  • Yuan H.C., Xiong F.L., Huai X.Y. A method for estimating the number of hidden neurons in feed-forward neural networks based on information entropy. Computers and Electronics in Agriculture, 2003, 40: 57–64.

  • Zhang G., Patuwo B.E., Hu Y.M. Forecasting with artificial neural networks: the state of the art. International Journal of Forecasting, 1998, 14: 35-62.

  • Zeng J., Guo H., Hu Y. Artificial neural network model for identifying taxi gross emitter from remote sensing data of vehicle emission. Journal of Environmental Sciences, 2007, 19: 427–431.


Abstract Views: 664

PDF Views: 0




  • Architecture Selection in Neural Networks by Statistical and Machine Learning

Abstract Views: 664  |  PDF Views: 0

Authors

Cagdas Hakan Aladag
Department of Statistics, Hacettepe University, Turkey

Abstract


One of the biggest problems in using artificial neural networks is to determine the best architecture. This is a crucial problem since there are no general rules to select the best architecture structure. Selection of the best architecture is to determine how many neurons should be used in the layers of a network. It is a well-known fact that using a proper architecture structure directly affect the performance of the method. Therefore, various approaches ranging from trial and error method to heuristic optimization algorithms have been suggested to solve this problem in the literature. Although there have been systematical approaches in the literature, trial and error method has been widely used in various applications to find a good architecture. This study propose a new architecture selection method based on statistical and machine learning. The proposed method utilizes regression analysis that is a supervised learning technique in machine learning. In this new architecture selection approach, it is aimed to combine statistical and machine learning to reach good architectures which has high performance. The proposed approach brings a new perspective since it is possible to perform statistical hypothesis tests and to statistically evaluate the obtained results when artificial neural networks are used. The best architecture structure can be statistically determined in the proposed approach. In addition to this, the proposed approach provides some important advantages. This is the first study using a statistical method to utilize statistical hypothesis tests in artificial neural networks. Using regression analysis is easy to use so applying the proposed method is also easy. And, the proposed approach saves time since the best architecture is determined by regression analysis. Furthermore, it is possible to make inference for architectures which is not examined. The proposed approach is applied to three real data sets to show the applicability of the approach. The obtained results show that the proposed method gives very satisfactory results for real data sets.

Keywords


Architecture Selection, Artificial Neural Networks, Machine Learning, Regression Analysis, Statistical Hypothesis Tests, Time Series.

References





DOI: https://doi.org/10.13005/ojcst12.03.02