Design efficiency overcomes scale in reducing AI hallucinations | Scientific Reports - Nature

Download PDF Abstract Language model development prioritizes scaling to ever-larger sizes despite massive computational and environmental costs, yet whether size alone determines reliability remains unclear. Through systematic evaluation of seven models from one to seven billion parameters, this study reveals that non-scale factors—encompassing architectural design, training data quality, and alignment methodology—account for nearly one-quarter of performance variation, sufficient for modern three-billion-parameter models to match seven-billion-parameter predecessors. This advantage at moderate scale equals gains from doubling model size while requiring only half the parameters and associated computational resources. However, fundamental limits persist: one-quarter of carefully designed prompts defeat all tested models regardless of scale or architecture, with questions requiring multi-step reasoning and temporal tracking proving universally problematic. Efficiency analysis shows returns diminish severely beyond three billion parameters, with marginal gains collapsing up to sixteenfold across this heterogeneous cross-family model set. These findings challenge the bigger-is-better paradigm, showing that architectural optimization at moderate scales offers superior cost-effectiveness, yet fundamental training limitations remain resistant to both approaches. Subjects Mathematics and computing Physics Acknowledgements We thank the anonymous reviewers for their constructive feedback. We acknowledge the developers of PyTorch, Transformers, and other open-source tools used in this work. Code and materials will be made available upon reasonable request to the corresponding author. Funding This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors. Author information Authors and Affiliations Westcliff University, Irvine, CA, USA Nur Nahar Rimi, Md Tanvir Rahman Tarafder & Nipa Akter International American University, Los Angeles, USA Abu Sayed & Anika Mahjabin Authors Nur Nahar Rimi View author publications Search author on: PubMed Google Scholar Md Tanvir Rahman Tarafder View author publications Search author on: PubMed Google Scholar Nipa Akter View author publications Search author on: PubMed Google Scholar Abu Sayed View author publications Search author on: PubMed Google Scholar Anika Mahjabin View author publications Search author on: PubMed Google Scholar Corresponding author Correspondence to Md Tanvir Rahman Tarafder . Ethics declarations Competing interests The authors declare no competing interests. Additional information Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Supplementary Information Supplementary Information. (download PDF ) Rights and permissions Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ . Reprints and permissions About this article Cite this article Rimi, N.N., Tarafder, M.T.R., Akter, N. et al. Design efficiency overcomes scale in reducing AI hallucinations. Sci Rep (2026). https://doi.org/10.1038/s41598-026-64549-x Download citation Received : 29 December 2025 Accepted : 27 July 2026 Published : 23 August 2026 DOI : https://doi.org/10.1038/s41598-026-64549-x Share this article Anyone you share the following link with will be able to read this content: Get shareable link Sorry, a shareable link is not currently available for this article. Copy shareable link to clipboard Provided by the Springer Nature SharedIt content-sharing initiative