The “dual carbon” goal accelerates the rapid development of new power system with new energy as the main part. It is urgent to realize the cooperative control of distributed multi-region power grids in new power system. From the perspective of automatic generation control (AGC), a cooperative control reinforcement learning algorithm for power grid with new energy is proposed to obtain the cooperative control of distributed multi-regional power grid. The proposed algorithm solves the problem that the action values are overestimated or underestimated in the classical Markov Q learning and its derivative algorithms through the weighted thought, and uses the delayed update strategy to further accelerate the convergence speed. The improved two-area load frequency control model incorporating electric vehicles and the distributed five-region interconnected grid model with new energy are simulated to verify the effectiveness of the proposed algorithm. Compared with the other existing control algorithms, the proposed algorithm has better control performance and faster convergence speed.