Performance of Classic Numerical Methods in Unrestricted Optimization Problems

Abstract

Minimizing a real function, with many different characteristics, has been one of the main mathematical  challenges in the Optimization field. Among the problems, the minimization of unrestricted functions, known as unrestricted optimization, has become a specific field of study under which mathematical and computational strategies have been developed that, under certain conditions, ensure the identification of critical points – possible global minimizer candidates. The present study aims to analyze the potentialities and weaknesses of five of these iterative methods: Gradient method, Newton’s method, Quasi-Newton BFGS method, Trust Region method and Nonlinear Conjugate Gradient method. For this, besides the theoretical analysis and computational implementation of each method, we sought to examine its performance in a set of ten test-functions proposed by Moré, Garbow and Hillstrom (1981). The results, obtained through comparison, indicate the advantages and disadvantages of each studied method, according to the problem, the convergence, and possible difficulties that can be found during the optimization process.