As renewable energy penetration continues to increase, the strong randomness of wind and solar renewable energy output leads to instability of grid frequency and deterioration of control performance. To address this, a multi-agent reinforcement learning approach is explored from the perspective of automatic generation control, that is the high dimensional cooperative soft actor-critic algorithm. The proposed algorithm encourages agents to engage in random exploration within a maximum entropy framework to address the inability of Q-learning and its derivative algorithm to rapidly update the Q-table in response to environmental changes. It also utilizes a Gaussian distribution strategy to generate continuous action values, enabling the algorithm to find collaborative optimal solutions in high dimensional continuous state spaces, thus solving the traditional reinforcement learning curse of dimensionality of high dimensionality "state-action". Frequency instability and declining control performance caused by the strong randomness of renewable energy outputs are effectively addressed. Through simulation experiments on an improved IEEE standard two-area load frequency control model and the Central China three-area load frequency control model, the effectiveness of the algorithm is validated. And the proposed algorithm has superior control performance and frequency stability compared to other algorithms.