HMC-PSO: A Hamiltonian Monte Carlo and Particle Swarm Optimization-Based Optimizer
Document Type
Conference Article
Publication Title
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Abstract
We introduce the Hamiltonian Monte Carlo Particle Swarm Optimizer (HMC-PSO), an optimization algorithm that reaps the benefits of both Exponentially Averaged Momentum PSO and HMC sampling. The coupling of the position and velocity of each particle with Hamiltonian dynamics in the simulation allows for extensive freedom for exploration and exploitation of the search space. It also provides an excellent technique to explore highly non-convex functions while ensuring efficient sampling. We extend the method to approximate error gradients in closed form for Deep Neural Network (DNN) settings. We discuss possible methods of coupling and compare its performance to that of state-of-the-art optimizers on the Golomb’s Ruler problem and Classification tasks(HMC-PSO code and additional results are in the Github repository: https://github.com/rtdsouza/torchswarm ).
First Page
212
Last Page
223
DOI
10.1007/978-3-031-30105-6_18
Publication Date
1-1-2023
Recommended Citation
Vaidya, Omatharv Bharat; DSouza, Rithvik Terence; Dhavala, Soma; Saha, Snehanshu; and Das, Swagatam, "HMC-PSO: A Hamiltonian Monte Carlo and Particle Swarm Optimization-Based Optimizer" (2023). Conference Articles. 604.
https://digitalcommons.isical.ac.in/conf-articles/604