Artificial Neural Network Solution of Ordinary Differential Equations using BFGS Optimization: Activation Function Comparison

Abstract

Ordinary Differential Equations (ODEs) are fundamental tools in science and engineering. Existing numerical methods such as Euler’s method and Runge-Kutta require discretization and repeated computations. This study develops Artificial Neural Network (ANN) with the Broyden-Fletcher-Goldfarb-Shanno (BFGS) optimization for solving ODEs. Second aim is to perform numerical simulations using ANN with BFGS using hyperbolic tangent function and sigmoid to solve ODEs and compares the efficiency and accuracy of ANN with existing numerical methods. the neural network created a trial solution to satisfy the initial conditions and is trained using the BFGS optimization that updates weights by approximating the inverse Hessian. After training, the ANN with BFGS solutions were compared with existing numerical results obtained from Euler’s method and RK4 method. The findings show that the ANN with BFGS method give accuracy better than Euler and similar with RK4 methods. The hyperbolic tangent activation function generally shows better approximation accuracy than sigmoid.