Breast cancer is now considered as one of the leading causes of deaths among women all over the world. Aiming to assist clinicians in improving the accuracy of diagnostic decisions, computer-aided diagnosis (CAD) system is of increasing interest in breast cancer detection and analysis nowadays. In this paper, a novel computer-aided diagnosis scheme with human-in-the-loop is proposed to help clinicians identify the benign and malignant breast tumors in ultrasound. In this framework, feature acquisition is performed by a user-participated feature scoring scheme that is based on Breast Imaging Reporting and Data System (BI-RADS) lexicon and experience of doctors. Biclustering mining is then used as a useful tool to discover the column consistency patterns on the training data. The patterns frequently appearing in the tumors with the same label can be regarded as a potential diagnostic rule. Subsequently, the diagnostic rules are utilized to construct component classifiers of the Adaboost algorithm via a novel rules combination strategy which resolves the problem of classification in different feature spaces (PC-DFS). Finally, the AdaBoost learning is performed to discover effective combinations and integrate them into a strong classifier. The proposed approach has been validated using a large ultrasounic dataset of 1,062 breast tumor instances (including 418 benign cases and 644 malignant cases) and its performance was compared with several conventional approaches. The experimental results show that the proposed method yielded the best prediction performance, indicating a good potential in clinical applications.