High precision wind power prediction plays an important role in the safe and stable operation of power system and the optimal allocation of energy system. In this paper, in order to extract the hidden information from the multi-dimensional wind power historical data, accurately select the features highly related to the prediction target, and overcome the short-term memory problem of time series, an ultra-short-term wind power forecasting model is proposed which is based on convolutional neural network and bidirectional long short term memory network (CNN-BiLSTM), combined with dual-stage attention mechanism and Bayesian optimization algorithm. Firstly, in order to autonomously mine the data association between input features and wind power data, and further highlight the impact of important high-dimensional features, the attention mechanism is added to the common CNN to build a feature attention module. Secondly, the attention mechanism is introduced into the output of the BiLSTM network to form a temporal attention module, which enhances the long-term memory ability and strengthens the influence of important historical information. Finally, the Bayesian optimization algorithm is used to optimize the hyperparameters of the proposed model, and the optimal hyperparameters can be selected to show the best performance of the model. The verification experiment is conducted with actual data from a wind farm in Northwest China, accuracy of single-step prediction model is 95.58%, and accuracy of 4 hours ahead ultra-short-term multi-step prediction model combined with NWP information is 90.44%. The experimental results show that the proposed model is more accurate than others.