The offshore energy industry faces challenges in maintaining ageing infrastructure, with over half of North Sea platforms past their 25-year design life. This creates a need for scalable inspection methods beyond traditional manual reviews. We present a collaborative Human-AI framework that assists in detecting structural defects whilst preserving expert oversight under challenging marine conditions. Our two-stage system first employs a lightweight classifier to filter video frames by risk level. High-risk frames are then analysed by a modified Pyramid Attention Network that performs precise defect localisation. Experts validate the results at both stages, ensuring the system's continuous improvement. To better identify rare but critical flaws, we design Enhanced Tversky, a composite loss function to mitigate severe class imbalance by explicitly prioritising rare yet safety-critical defects like cracks. Evaluation on 5,525 classified frames and 1,013 segmented images demonstrates F2 score of 87.59% and a mean IoU of 78.73%, with crack detection reaching a crucial 89.62% F2 score. The framework reduces expert review time by 3.88 x whilst maintaining safety standards, offering a practical approach to scaling offshore inspection capabilities through Human-AI collaboration.