Skip to Main content Skip to Navigation
New interface
Conference papers

Factored Reinforcement Learning for Auto-scaling in Tandem Queues

Abstract : As today’s networking systems utilize more virtualisation, efficient auto-scaling of resources becomes increasingly critical for controlling both the performance and energy consumption. In this paper, we study the techniques to learn the optimal auto-scaling policies in a distributed network when parts of the system dynamics are unknown. Reinforcement Learning methods have been applied to solve auto-scaling problems. However they can run into computational and convergence issues as the problem scale grows. On the other hand, distributed networks have relational structures with local dependencies between physical and virtual resources. We can exploit these structures to overcome the convergence issues by using a factored representation of the system.We consider a distributed network in the form of a tandem queue composed of two nodes. The objective of the auto-scaling problem is to find policies that have a good trade-off between quality of service (QoS) and operating costs. We develop a factored Reinforcement Learning algorithm, named FMDP online, to find the optimal auto-scaling policies. We evaluate our algorithm with a simulated environment. We compare it with existing Reinforcement Learning methods and show its relevance in terms of policy efficiency and convergence speed.
Complete list of metadata

https://hal-cnrs.archives-ouvertes.fr/hal-03811176
Contributor : Emmanuel Hyon Connect in order to contact the contributor
Submitted on : Tuesday, October 11, 2022 - 4:55:49 PM
Last modification on : Friday, October 21, 2022 - 3:34:31 AM

Identifiers

Citation

Thomas Tournaire, Yue Jin, Armen Aghasaryan, Hind Castel-Taleb, Emmanuel Hyon. Factored Reinforcement Learning for Auto-scaling in Tandem Queues. NOMS 2022-2022 IEEE/IFIP Network Operations and Management Symposium, Apr 2022, Budapest, Hungary. pp.1-7, ⟨10.1109/NOMS54207.2022.9789809⟩. ⟨hal-03811176⟩

Share

Metrics

Record views

0