Performance of ChatGPT-5.5, Gemini 3, and copilot 2026 in answering chemistry course questions

Download PDF Abstract Chemistry, environmental health, and occupational health have historically developed in close collaboration. This study aimed to evaluate the performance of LLMs, in responding to a chemistry course questionnaire used in environmental and occupational health department for educational purposes. This was a cross-sectional study conducted in 2026. The study used a 50-item multiple-choice questionnaire, primarily in Persian with limited English terms and formulas, which was administered to LLMs (ChatGPT-5.5, Gemini 3, and Copilot 2026). Performance was evaluated using confusion matrix analysis. Cochran`s Q test, McNemar`s test for pair-wise comparisons, and Point-biserial correlation were also conducted to further evaluate the study data. The LLMs presented mean sensitivity, specificity, and accuracy of 0.81, 0.29, and 0.68, respectively. Gemini 3 presented a high level of performance, achieving the highest sensitivity (0.85), specificity (0.56), and accuracy (0.80). ChatGPT-5.5 had the lowest sensitivity (0.76), while Copilot showed the lowest specificity (0.07) and an accuracy of 0.62. No significant correlations were found between performance and questionnaire characteristics; however, all models performed poorly on negatively framed items requiring identification of incorrect options. The study presented some preliminary insights on the performance of LLMs in chemistry-related questions. Further research is warranted. Explore related subjects Discover the latest articles and news in related subjects. Health care Medical research Author information Authors and Affiliations Department of Environmental Health Engineering, School of Health, Social Determinant of Health Research Center, Birjand University of Medical Sciences, Birjand, Iran Hadigheh Dorri Faculty of Health, Birjand University of Medical Sciences, Birjand, Iran Morteza Zahedy Gol Pharmaceutical Sciences Research Center, Birjand University of Medical Sciences, Birjand, Iran Karim Naraki Social Determinants of Health Research Center, Birjand University of Medical Sciences, Birjand, Iran Mohsen Khosravi Authors Hadigheh Dorri View author publications Search author on: PubMed Google Scholar Morteza Zahedy Gol View author publications Search author on: PubMed Google Scholar Karim Naraki View author publications Search author on: PubMed Google Scholar Mohsen Khosravi View author publications Search author on: PubMed Google Scholar Corresponding author Correspondence to Mohsen Khosravi . Ethics declarations Competing interests The authors declare no competing interests. Additional information Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Supplementary Information Below is the link to the electronic supplementary material. Supplementary Material 1 (download DOCX ) Supplementary Material 2 (download DOCX ) Supplementary Material 3 (download DOCX ) Rights and permissions Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ . Reprints and permissions About this article Cite this article Dorri, H., Gol, M.Z., Naraki, K. et al. Performance of ChatGPT-5.5, Gemini 3, and copilot 2026 in answering chemistry course questions: a cross-sectional study. Sci Rep (2026). https://doi.org/10.1038/s41598-026-73546-z Download citation Received : 02 July 2026 Accepted : 23 September 2026 Published : 10 October 2026 DOI : https://doi.org/10.1038/s41598-026-73546-z Share this article Anyone you share the following link with will be able to read this content: Get shareable link Sorry, a shareable link is not currently available for this article. Copy shareable link to clipboard Provided by the Springer Nature SharedIt content-sharing initiative Keywords Environmental health Occupational health Chemistry Digital health Large language models