In this paper, we present a modified spectral memoryless quasi-Newton method for solving unconstrained optimization problems. The proposed method is based on the symmetric rank-one (SR1) update, which introduces a rank-one correction to the inverse Hessian approximation. To enhance efficiency and robustness, the method employs a non-quadratic spectral parameter derived from gradient information. This parameter approximates curvature information without the need to store or update full matrices, significantly reducing computational cost and making the method suitable for large-scale problems. A key feature of the proposed algorithm is its ability to preserve the descent property of the search direction at each iteration. This is ensured by incorporating a line search procedure that satisfies the strong Wolfe conditions, thereby improving stability and convergence reliability. Theoretical analysis demonstrates that, under standard assumptions, the algorithm converges globally to a stationary point. Extensive numerical experiments conducted on benchmark test functions show that the proposed method is competitive with, and often superior to, several state-of-the-art optimization methods in terms of convergence speed, robustness, and accuracy.
Keywords
Quasi-Newton SR1Spectral ParameterMemoryless AlgorithmsStrong Wolfe Criteria Line SearchDolan and Moré Performance Profiles.
References
J. Lu, Y. Li, and H. Pham, “A Modified Dai-Liao Conjugate Gradient Method with a New Parameter for Solving Image Restoration Problems,” Mathematical Problems in Engineering, vol. 2020, 2020.
B. Ivanov, G. V. Milovanović, P. S. Stanimirović, A. M. Awwal, L. A. Kazakovtsev, and V. N. Krutikov, “A Modified Dai-Liao Conjugate Gradient Method Based on a Scalar Matrix Approximation of Hessian and Its Application,” Journal of Mathematics, vol. 2023, 2023.
A. Yusuf, N. H. Manjak, H. Mohammad, A. I. Kiri, and A. B. Abubakar, “A Solution Method for Nonlinear Monotone Equations via Hybrid Spectral Conjugate Gradient and Signal Recovery Problems,” Operations Research Forum, vol. 5, no. 2, pp. 0-14, 2024.
F. N. Jardow and G. M. Al-Naemi, “A new parameter to enhance three-term conjugate gradient method with inexact line search,” 2025.
I. M. Sulaiman, P. Kaelo, R. Khalid, and M. K. M. Nawawi, “A Descent Generalized RMIL Spectral Gradient Algorithm for Optimization Problems,” International Journal of Applied Mathematics and Computer Science, vol. 34, no. 2, pp. 225-233, 2024.
J. Frédéric Bonnans, J. Charles Gilbert, C. Lemaréchal, and C. A. Sagastizábal, “Numerical optimization: Theoretical and practical aspects,” Numerical Optimization: Theoretical and Practical Aspects, pp. 1-494, 2006.
J. Barzilai and J. M. Borwein, “Two-point step size gradient methods,” IMA Journal of Numerical Analysis, vol. 8, no. 1, pp. 141-148, 1988.
G. M. Al-Naemi, “Modules With Chain Conditions On δ-Small Submodules,” Iraqi Journal of Science, vol. 55, no. 1, pp. 202-217, 2014.
W. R. Boland, E. R. Kamgnia, and J. S. Kowalik, “A conjugate-gradient optimization method invariant to nonlinear scaling,” Journal of Optimization Theory and Applications, vol. 27, no. 2, pp. 221-230, Feb. 1979.
A. Tassopoulos and C. Storey, “A conjugate-direction method based on a nonquadratic model,” Journal of Optimization Theory and Applications, vol. 43, no. 3, pp. 371-381, Jul. 1984.
G. M. Al-Naemi, “A Modified Hestenes-Stiefel Conjugate Gradient Method and its Global convergence for unconstrained optimization,” Iraqi Journal of Science, vol. 55, no. 1, pp. 202-217, 2014.
G. M. R. Al-Naemi, “New multi-version extended conjugate gradient methods for non-linear optimization,” 1993.
J. C. Gilbert, “Global convergence properties of conjugate gradient methods for optimization,” SIAM Journal on Optimization, vol. 2, no. 1, pp. 21-42, Feb. 1992.
S. S. Djordjević, “New hybrid conjugate gradient method as a convex combination of LS and CD methods,” Filomat, vol. 31, no. 6, pp. 1813-1825, 2017.
E. D. Dolan and J. J. Moré, “Benchmarking optimization software with performance profiles,” Mathematical Programming, vol. 91, no. 2, pp. 201-213, 2002.