Development and Open-Source Release of a Multi-Cancer Module for Structuring Pathology Reports

<title>Abstract</title> <p>Background Pathology reports contain clinically critical information; however, their semi-structured or free-text format substantially limits secondary use in large-scale cancer research. Scalable, reproducible methods are therefore needed to transform these reports into structured datasets across institutions. This study aimed to develop, externally validate, and publicly release a scalable NLP (Natural Language Processing) framework for structuring multi-cancer pathology report and supporting reproducible, institutionally transferable research workflows. Methods We developed a Clinical BERT-based NLP framework for the automated extraction of structured variables from pathology reports across five cancer types. The pipeline integrated data preprocessing, annotation, model training using a question-answering approach, and a public release of the trained models. The trained models were applied to independent datasets from two external institutions without retraining, enabling direct assessment of cross-institutional. Results In the internal validation, the model demonstrated consistently strong performance across cancer types, with mean F1 scores ranging from 0.945 to 0.977 and median F1 scores exceeding 0.99. In external validation, performance declined substantially across institutions, with mean F1 scores declining to 0.623 and 0.427 for breast cancer and similar reductions observed in kidney, thyroid, liver, and colorectal cancers (ΔF1 ranging from − 0.209 to − 0.518). Despite this decline, the model maintained clinically meaningful, moderate performance across external datasets. Conclusion We developed and validated a scalable and reusable NLP framework for multi-cancer pathology report across institutional settings. By publicly releasing trained models and processing pipelines, this study advances reproducible, deployable NLP infrastructure for clinical data integration and multicenter cancer research.</p>