Dynamic production scheduling and maintenance planning under opportunistic grouping

Abstract

• A novel two-level deep reinforcement learning framework is proposed to address the joint flexible job-shop scheduling and maintenance planning. • The maintenance planning method is developed based on multi-component and multi-machine considerations simultaneously. • The proposed framework considers economic grouping, allowing for cost and time savings by performing maintenance on multiple components of a machine simultaneously. • The proposed framework can solve the dynamic flexible job shop of different sizes. Maintenance and production scheduling are intertwined activities that should be addressed simultaneously to uphold production systems’ reliability and production efficiency. The digital twin is an emerging technology that advances industrial digitalization, as it embeds a “virtual” image of reality that runs in line with the real system, enabling evaluation, optimization, and prediction of the physical system’ state. On the other hand, deep reinforcement learning (DRL) can provide real-time decision-making by leveraging real-time data from the digital twin such as condition monitoring and production progress data. This research addresses the joint flexible job shop scheduling and maintenance planning problem, considering new job insertions and multi-component machines with economic dependencies among components. The objective is to minimize both the expected total tardiness and maintenance cost, considering opportunistic grouping of maintenance activities on components and breakdown costs associated with failure risk. To achieve this joint decision-making, we develop a hierarchical architecture with two interconnected hierarchies. The upper-level hierarchy determines whether to switch the decision-making process between production scheduling and maintenance planning. Subsequently, the lower-level hierarchy selects an action through the corresponding agent: a multi-head Deep Q-Network (DQN) if the maintenance option is chosen, and a DQN with seven dispatching rules for the production option. The computational experiment results reveal that the proposed scheduling method can learn a high-quality dispatching policy, outperforming the non-hierarchical DRL approach and individual dispatching rules in solution quality, CPLEX in runtime efficiency, and the genetic algorithm and particle swarm optimization in both solution quality and runtime efficiency.