The problem of inrush current becomes more prominent by changing the arrangement order of the inner windings of the transformer with built-in high voltage winding. The zero-sequence over-current protection of bus tie circuit-breaker is faced with a high risk of maloperation because of the high amplitude zero-sequence inrush current due to the unloaded transformer energizaiton, which threatens the safe operation of the power grid. Combining with the setting principle of zero-sequence overcurrent protection of bus tie circuit-breaker, the maloperation mechanism of the protection under inrush current conditions of unloaded energization of the transformer with built-in high voltage winding is revealed in this paper. Based on the difference of waveform characteristics between zero-sequence inrush current and fault zero-sequence current, and considering the influence of saturation of current transformer (CT), the restraint criterion of zero-sequence overcurrent protection against maloperations applying the variation characteristics of symbol sequence of zero-sequence current is proposed. The maloperation mechanism of the protection and the validity of the proposed criterion are verified by simulation tests.