Due to the diverse and complex processes in the manufacturing of Carbon Fiber-Reinforced Thermoplastic Prepreg(CFRTP), its surface is prone to generating defects, such as yarn feathers and wrinkles, leading to performance degradation. Currently, it is still a challenge to accurately and effectively detect these common surface defects. In this work, we propose DD-Net, a visual inspection-based defect detection model designed for CFRTP surface quality assessment. Specifically, an Efficient Re-parameter Aggregation Module (ERAM) is introduced to enhance feature extraction and inference speed, while a lightweight multi-scale pooling module (CCSPPF) is designed to improve multi-scale feature fusion efficiency. In addition, an attention-based downsampling module (DS-A) is proposed to strengthen small defect perception. Finally, a lightweight decoupled detection head is proposed to balance detection accuracy and speed by improving localization and classification precision. Extensive experiments demonstrate that DD-Net achieves superior performance compared with mainstream detection methods, reaching an mAP@0.5 of 95.2% on the CFRTP dataset. Furthermore, comprehensive interpretability and ablation analyses validate the effectiveness of each proposed module and provide deeper insights into how the model captures and distinguishes key defect characteristics.