Publikation
MuS-Q-Le: On the fly Multi-State Q-Learning for Large-Scale State and Action Spaces on GPU’s
Julian Groß
In: Parallel and Distributed Computing, Applications and Technologies. International Conference on Parallel and Distributed Computing, Applications and Technologies (PDCAT-2025), 26th International Conference, PDCAT 2025, Gold Coast, Australia, November 22–24, 2025, Proceedings, November 22-24, Gold Coast, QLD, Australia, Springer, 2025.
Zusammenfassung
In modern optimization systems, Reinforcement Learning plays a fundamental role in improving existing processes. In particular in large-scale environments, determination of optimized results is still an extremely challenging task. This task becomes much more sophisticated if the underlying environment is dynamically changing leading to time-consuming recalculation steps. The presented method shows a
novel algorithm which allows an on-the-fly Q-Learning for large-scale state and action space scenarios. The algorithm leverages Q-State permutations to determine and process different sub-matrices of a global Q-Matrix iteratively. The approach eliminates shared memory restrictions using a smart reloading technique that is perfectly aligned for the GPU system design to run efficiently in parallel. Extending this method
to a multi-state approach to enable multi goal investigation leads to a notable increase in learning space exploration capabilities. Our method shows significant performance improvements in processing large-scale Q-Learning matrices on a GPU compared to similar optimized methods on CPU, GPU and GPU-based neural network methods.