W-RPC: A Weighted Reducer Placement and Coflow Scheduling Scheme

Abstract

Coflow scheduling and reducer placement are key to minimizing job completion times in data-parallel clusters. The RPC framework jointly addresses these tasks but assumes all coflows have equal importance, neglecting priority differentiation in practical workloads. This paper extends RPC by introducing a weighted scheduling mechanism that computes a score for each coflow based on its waiting time and priority, enabling priorityaware placement and bandwidth allocation. An efficient online algorithm minimizes these scores to favor high-priority coflows. Simulation results show that our approach significantly improves completion times for critical coflows and enhances overall fairness compared to baseline RPC.