Research Paper Quality Recognition Through Textual Feature Analysis

arXiv:2608.20368v1 Announce Type: new Abstract: Knowledge and innovations are shaped by using the quality and credibility of the scientific research. Yet, distinguishing between impactful, high-quality work and flawed studies remains a challenge. This paper introduces a benchmark for classifying research papers into two categories: good (highly cited) and non-good (retracted), using only textual features from titles and abstracts. We evaluate multiple embedding techniques, including SBERT, Word2Vec, FastText, USE, and TF-IDF, combined with classifiers such as Support Vector Machines (SVM), Random Forests, and Neural Networks. Our contributions include: (1) hyperparameter transparency, (2) feature space visualizations using t-SNE, (3) model interpretability analysis with SHAP, and (4) detailed examination of error cases. Experimental results show that a neural network with SBERT embeddings achieves 87.22% accuracy, while FastText combined with SVM reaches 91.12%. These findings highlight the value of textual information in assessing research quality, with ethical considerations for deployment. This work contributes toward the development of academic integrity tools that promote trustworthy scholarship.

Related

• CORTEX: High-Quality Cross-Domain Organization of Web-Scale Corpora through Ontological Corpus Graph
• LitTraceQA: A Benchmark for Multi-Stage Grounding and Verification in Scientific Question Answering
• Triples and Knowledge-Infused Embeddings for Clustering and Classification of Scientific Documents

Source: arXiv cs.CL | 2026-08-24