Due to the complex backgrounds, small defect targets, and significant size variations among different detection objects of insulator defects in drone aerial inspection images, current object detection algorithms often lose defect features when identifying such targets, resulting in low accuracy and high misidentification rates. To address this, an improved YOLOv8n-based method for detecting insulator defects in transmission lines is proposed. Firstly, to tackle the feature loss issue of small insulator defects, a PCFocalNeXt-C2f module is designed and used to optimize the Backbone of YOLOv8n. Secondly, to mitigate feature information loss or degradation during multi-level transmission, an asymptotic feature pyramid network （AFPN） supporting direct interaction across non-adjacent layers is employed to enhance the Neck of YOLOv8n. Finally, the Sigmoid-CIoU loss function is introduced to optimize the loss function in YOLOv8n, and Sigmoid-CIoU NMS is adopted to obtain detection results, improving the model's robustness. Experimental results show that compared to the original YOLOv8n, the proposed algorithm achieves a 3.3 % increase in F1-score, with mAP@0.5 and mAP@0.5-0.95 improving by 3.8 % and 6.1%, respectively, while reducing the parameter count by 23.9 %, demonstrating superior detection performance.