This paper proposes an electric vehicle(EV) charging and discharging scheduling strategy that takes into account the user responsiveness in the scheduling process of traditional EV clusters, which fails to fully consider the impact of user responsiveness and its influencing factors on the schedulable capacity. Firstly, the charging load model of EV clusters based on user travel data is built. Secondly, an EV user responsiveness model is established based on Weber-Fechner's law, and the influence of the charging and discharging price set by the aggregator and the vehicle state of charge (SOC) on user charging and discharging responsiveness is comprehensively considered. Finally, the charging and discharging price set by aggregators and the charging and discharging power of EVs are taken as decision variables. Overall considering of the benefit of the power grid, aggregators and EV users, the EV charging and discharging optimal scheduling strategy model is designed to minimize the load fluctuation of the distribution network and the charging cost of the user, maximize the revenue of the aggregator. And the optimal problem is solved by particle swarm optimization(PSO) algorithm. The example results show that the model can achieve peak-shaving and valley-filling while ensuring the benefits of aggregators and EV users.