MCRL2: Multi-resource Cross-attention-based Representation Learning-augmented Reinforcement Learning for Cloud Microservice Scheduling

Summary

The paper proposes MCRL2, a soft actor-critic scheduler for cloud microservices whose actor and critic consume representations from MCRL, a multi-resource cross-attention encoder over node resource-utilization vectors, fused with microservice-instance features. Motivated by Alibaba production-trace analysis of dynamic imbalance, cross-resource coupling, and heterogeneous demand, the authors add a maximum-entropy objective, twin Q-networks, target networks, and a convergence argument (Lemmas 1-2, Theorem) that assumes stable representations. Experiments run in a simulated 15-node cluster (three VM types) driven by Alibaba microservice traces v2021, with 2,000 training instances and two sequential evaluation patterns (MS-2000, MS-4000; instance durations Poisson, lambda=600 s). Against Round Robin, Random, DDQN, PPO, and MLP-SAC, MCRL2 reports scheduling success rates of 0.995/0.988 (stated gains of 5.85%/5.56% over the best baseline), average completion times of 603.33/607.28 s, and the lowest cumulative memory imbalance (DCLB 63.13/97.97), at roughly 1.4-2.4x the DRL baselines' decision-time cost. Main limitations: results come from what appear to be single runs without error bars or significance tests; the environment is simulated despite 'real production cluster' phrasing; the cross-node attention claim is not supported by the mechanism, whose softmax (Eq. 15) attends only across resources within each node; finer ablations and attention/GNN-based baselines are absent; and the convergence proof sidesteps representation non-stationarity. A code URL is provided but could not be verified here.