Abstract: Objectives: Under the coupling action of environment and loads, dam underwater structures suffer from defects, affecting the safety, stability, and functional performance of the project. Underwater robots like Remotely Operated Vehicles (ROVs) equipped with visible light cameras can realize the high-resolution spatial information in a non-contact form for underwater damage. However, it is still a challenging task that needs to be solved urgently to efficiently extract effective information from massive image and video data. Methods: Based on this, this paper proposes a real-time multi-class defect automatic identification framework for dam underwater structures. Specifically, the single-stage object detection network YOLOv5s is utilized as the base model to develop the damage detector. Then, the model sparsity and pruning strategies are combined to change the batch layer weight distribution and remove model redundant parameters. Next, transfer learning and knowledge distillation are combined to recover the accuracy degradation caused by model pruning and compression. Results: Take the underwater detection of a high dam as an example. The effectiveness of the proposed method was validated in complex underwater scenes like obstacle occlusion, low visibility, and uneven illumination. Conclusions: Experimental results indicated that the proposed method can effectively overcome the interference of complicated underwater imaging environments and accurately identify different types of defects. Moreover, it also achieves the inference speed of processing 100 defect images per second, indicating its real-time detection capability.