Publications

---

All papers are listed below by year of submission before they are published, or year of publication.

2024

• Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance,
  Dimitris Oikonomou, Nicolas Loizou
  

• Enhancing Vision-language Models for Medical Imaging: Bridging the 3D Gap with Innovative Slice Selection,
  Yuli Wang, Peng jian, Yuwei Dai, Craig Jones, Haris I. Sair, Jinglai Shen, Nicolas Loizou, Jing Wu, Wen-Chi Hsu, Maliha Rubaiyat Imami, Zhicheng Jiao, Paul J Zhang, Harrison Bai
  Thirty-Eighth Conference on Neural Information Processing Systems (NeurIPS), D&B Track, 2024
  

• Remove that Square Root: A New Efficient Scale-Invariant Version of AdaGrad,
  Sayantan Choudhury, Nazarii Tupitsa, Nicolas Loizou, Samuel Horvath, Martin Takac, Eduard Gorbunov
  Thirty-Eighth Conference on Neural Information Processing Systems (NeurIPS), 2024
  

• Communication-Efficient Gradient Descent-Accent Methods for Distributed Variational Inequalities: Unified Analysis and Local Updates,
  Siqi Zhang, Sayantan Choudhury, Sebastian U Stich, Nicolas Loizou
  12th International Conference on Learning Representations (ICLR), 2024
  

• Stochastic Extragradient with Random Reshuffling: Improved Convergence for Variational Inequalities,
  Konstantinos Emmanouilidis, Rene Vidal, Nicolas Loizou
  27th International Conference on Artificial Intelligence and Statistics (AISTATS), 2024
  

• Dissipative Gradient Descent Ascent Method: A Control Theory Inspired Algorithm for Min-max Optimization,
  Tianqi Zheng, Nicolas Loizou, Pengcheng You, Enrique Mallada
  IEEE Control Systems Letters [Journal] , 2024

• Locally Adaptive Federated Learning via Stochastic Polyak Stepsizes,
  Sohom Mukherjee, Nicolas Loizou, Sebastian U Stich
  Transactions on Machine Learning Research, 2024
  

2023

• Single-Call Stochastic Extragradient Methods for Structured Non-monotone Variational Inequalities: Improved Analysis under Weaker Conditions,
  Sayantan Choudhury, Eduard Gorbunov, Nicolas Loizou
  Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS), 2023
  

• A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Games,
  Samuel Sokota, Ryan D'Orazio, J Zico Kolter, Nicolas Loizou, Marc Lanctot, Ioannis Mitliagkas, Noam Brown, Christian Kroer
  11th International Conference on Learning Representations (ICLR), 2023
  short version: Deep Reinforcement Learning Workshop, NeurIPS 2022

• Stochastic Gradient Descent-Ascent: Unified Theory and New Efficient Methods,
  Aleksandr Beznosikov, Eduard Gorbunov, Hugo Berard, Nicolas Loizou
  26th International Conference on Artificial Intelligence and Statistics (AISTATS), 2023
  short version: Optimization for Machine Learning Workshop, NeurIPS 2022

• Stochastic Mirror Descent: Convergence Analysis and Adaptive Variants via the Mirror Stochastic Polyak Stepsize,
  Ryan D'Orazio, Nicolas Loizou, Issam Laradji, Ioannis Mitliagkas
  Transactions on Machine Learning Research, 2023 [Journal]
  short version: Beyond First-order Methods in ML Systems Workshop, ICML 2021.

• Unified Analysis of Stochastic Gradient Methods for Composite Convex and Smooth Optimization,
  Ahmed Khaled, Othmane Sebbouh, Nicolas Loizou, Robert M. Gower, Peter Richtárik,
  Journal of Optimization Theory and Applications (2023): 1-42 [Journal]

• AI-SARAH: Adaptive and Implicit Stochastic Recursive Gradient Methods,
  Zheng Shi, Abdurakhmon Sadiev, Nicolas Loizou, Peter Richtárik, Martin Takáč,
  Transactions on Machine Learning Research, 2835-8856, 2023 [Journal]

2022

• ProxSkip for Stochastic Variational Inequalities: A Federated Learning Algorithm for Provable Communication Acceleration,
  Siqi Zhang, Nicolas Loizou.
  Optimization for Machine Learning Workshop, NeurIPS 2022

• Dynamics of SGD with Stochastic Polyak Stepsizes: Truly Adaptive Variants and Convergence to Exact Solution,
  Antonio Orvieto, Simon Lacoste-Julien, Nicolas Loizou
  Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS), 2022
  

• Stochastic Extragradient: General Analysis and Improved Rates,
  Eduard Gorbunov, Hugo Berard, Gauthier Gidel, Nicolas Loizou
  25th International Conference on Artificial Intelligence and Statistics (AISTATS), 2022
  

• Extragradient Method: O(1/K) Last-Iterate Convergence for Monotone Variational Inequalities and Connections With Cocoercivity,
  Eduard Gorbunov, Nicolas Loizou, Gauthier Gidel
  25th International Conference on Artificial Intelligence and Statistics (AISTATS), 2022
  

• On the Convergence of Stochastic Extragradient for Bilinear Games with Restarted Iteration Averaging,
  Chris Junchi Li, Yaodong Yu, Nicolas Loizou, Gauthier Gidel, Yi Ma, Nicolas Le Roux, Michael I Jordan
  25th International Conference on Artificial Intelligence and Statistics (AISTATS), 2022
  short version: Optimization for Machine Learning Workshop, NeurIPS 2021 (Oral Talk)

2021

• Stochastic Gradient Descent-Ascent and Consensus Optimization for Smooth Games: Convergence Analysis under Expected Co-coercivity,
  Nicolas Loizou, Hugo Berard, Gauthier Gidel, Ioannis Mitliagkas, Simon Lacoste-Julien
  Thirty-fifth Conference on Neural Information Processing Systems (NeurIPS), 2021
  

• SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolation,
  Robert M. Gower, Othmane Sebbouh, Nicolas Loizou
  24th International Conference on Artificial Intelligence and Statistics (AISTATS), 2021
  short version: Optimization for Machine Learning Workshop, NeurIPS 2020

• Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence,
  Nicolas Loizou, Sharan Vaswani, Issam Laradji and Simon Lacoste-Julien.
  24th International Conference on Artificial Intelligence and Statistics (AISTATS), 2021
  short version: Optimization for Machine Learning Workshop, NeurIPS 2020 (Spotlight Talk)

• Revisiting Randomized Gossip Algorithms: General Framework, Convergence Rates and Novel Block and Accelerated Protocols,
  Nicolas Loizou, Peter Richtarik
  IEEE Transactions on Information Theory 67.12 (2021): 8300-8324. [Journal]

2020

• Stochastic Hamiltonian Gradient Methods for Smooth Games,
  Nicolas Loizou, Hugo Berard, Alexia Jolicoeur-Martineau, Pascal Vincent, Simon Lacoste-Julien and Ioannis Mitliagkas.
  Proceedings of the 37th International Conference on Machine Learning (ICML), 2020.

• A Unified Theory of Decentralized SGD with Changing Topology and Local Updates,
  Anastasia Koloskova, Nicolas Loizou, Sadra Boreiri, Martin Jaggi and Sebastian U. Stich.
  Proceedings of the 37th International Conference on Machine Learning (ICML), 2020.

• Convergence Analysis of Inexact Randomized Iterative Methods,
  Nicolas Loizou, Peter Richtarik
  SIAM Journal on Scientific Computing (SISC) 42.6 (2020): A3979-A4016. [Journal]

• Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methods ,
  Nicolas Loizou, Peter Richtarik.
  Computational Optimization and Applications (COAP) 77.3 (2020): 653-710. [Journal]
  2020 COAP Best Paper Award [Link]
  

2019

• Randomized Iterative Methods for Linear Systems: Momentum, Inexactness and Gossip
  Nicolas Loizou
  PhD Dissertation, The University of Edinburgh, 2019.
  OR Society’s Doctoral Award (runner-up)
  

• SGD: General Analysis and Improved Rates,
  Robert Mansel Gower, Nicolas Loizou, Xun Qian, Alibek Sailanbayev, Egor Shulgin and Peter Richtarik.
  Proceedings of the 36th International Conference on Machine Learning (ICML), pages 5200-5209, 2019.

• Stochastic Gradient Push for Distributed Deep Learning,
  Mahmoud Assran, Nicolas Loizou, Nicolas Ballas and Michael Rabbat
  Proceedings of the 36th International Conference on Machine Learning (ICML), pages 344-353, 2019.
  short version: Systems for ML workshop, NeurIPS 2018.

• Provably Accelerated Randomized Gossip Algorithms ,
  Nicolas Loizou, Michael Rabbat and Peter Richtarik.
  IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019

2018

• A Privacy Preserving Randomized Gossip Algorithm via Controlled Noise Insertion,
  with Filip Hanzely, Jakub Konecny, Nicolas Loizou, Peter Richtarik, Dmitry Grishchenko.
  Privacy Preserving Machine Learning Workshop-NeurIPS, 2018

• Accelerated Gossip via Stochastic Heavy Ball Method ,
  Nicolas Loizou, Peter Richtarik.
  56th Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2018 [poster]

2017

• Linearly convergent stochastic heavy ball method for minimizing generalization error ,
  Nicolas Loizou, Peter Richtarik.
  NIPS Workshop on Optimization for Machine Learning 2017 [poster]

• Privacy Preserving Randomized Gossip Algorithms,
  Filip Hanzely, Jakub Konecny, Nicolas Loizou, Peter Richtarik, Dmitry Grishchenko

2016 or earlier

• A New Perspective on Randomized Gossip Algorithms ,
  Nicolas Loizou, Peter Richtarik.
  IEEE Global Conference on Signal and Information Processing (GlobalSIP), pp.440-444, 2016 [poster]

• Distributionally Robust Games with Risk-averse Players,
  Nicolas Loizou
  Proceedings of the 5th International Conference on Operations Research and Enterprise Systems (ICORES), ISBN 978-989-758-171-7, pages 186-196, 2016

• Distributionally Robust Game Theory
  Nicolas Loizou
  MSc Thesis, Imperial College London, 2015.