Analisis Perbandingan Pipelines Pemodelan Topik Berbasis Transformer untuk Literatur Ilmiah

The exponential growth of scientific literature poses a significant challenge for manually identifying thematic trends, necessitating automated analysis methods. This study aims to determine an optimal topic modeling pipeline by conducting a comparative analysis between two transformer-based approaches to maximize the coherence of topics extracted from scientific research. Two distinct pipelines were implemented and evaluated on a corpus of 20,972 scientific article abstracts. These included a custom pipeline combining SBERT, UMAP, and HDBSCAN, and the integrated BERTopic model. Performance evaluation, quantitatively benchmarked using the C_v coherence score, revealed that the integrated BERTopic model achieved the highest score of 0.7012. This result significantly surpassed the custom SBERT-UMAP-HDBSCAN pipeline, which scored 0.6079. The findings demonstrate that an integrated, purpose-built model like BERTopic is superior for generating highly coherent and interpretable thematic structures from scientific text. This research provides a validated directive for researchers, demonstrating that integrated models offer a more robust and efficient solution for uncovering thematic structures in large-scale literature analysis compared to custom modular pipelines.