Open-loop mode resonance theory is the research result to explain the mechanism of power system oscillation caused by wind power integration in which the reliability of the off-line time-domain simulation results based on the average model in the theoretical research is questionable. Firstly, to improve the reliability of engineering guidance for the theory, based on hardware in the loop (HIL) simulation, a phisical experimental platform is established. Secondly, a small signal model of the grid-connected system of doubly-fed induction generator (DFIG) wind farm is built, and a general method of open-loop mode analysis is proposed to assess the oscillation risk of the system. Finally, two examples are used to analyze and experiment the phenomenon of open-loop mode resonance. It is concluded that the experiment results are consistent with the results of open-loop mode analysis. Under the condition of open-loop mode resonance, the stability of power system decreases. The residue method can accurately predict the closed-loop mode of the system, and the reasonable tuning of parameters can reduce the impact of open-loop mode resonance.