The large-scale access of renewable energy to the power grid has intermittent and strong random characteristics, which is easy to cause significant frequency fluctuations. For multi-area power grids, the existing multi-agent cooperative algorithms based on neural networks often have unsatisfactory control performance due to gradient explosion and gradient disappearance in the face of strong random disturbances. To this end, this paper proposes an adaptive weight residual proximal policy optimization method for automatic power generation control. This method connects the layers of the agent policy neural network through residual connections, and adds a trainable adaptive weight to the residual connection to achieve the optimal residual ratio and alleviate the problem of gradient explosion and gradient disappearance of the neural network, so that the training process of the deep network is more stable and efficient, and then the multi-area cooperative optimal solution under strong random disturbance is obtained faster to eliminate the frequency fluctuation caused by strong random disturbance. The effectiveness of the proposed algorithm is verified by simulation tests on IEEE two-area and three-area load frequency control systems. Compared with various reinforcement learning algorithms, the proposed algorithm has faster convergence and stronger stability, higher frequency stability and control performance.