Private, offline and yours: how to search the literature with local AI systems | Nature

Email Bluesky Facebook LinkedIn Reddit Whatsapp X Save article View saved research Illustration: The Project Twins When he wants to scratch an intellectual itch, Ian Gaudet does what people are increasingly doing on the Internet: he asks a chatbot. But when it comes to his research data, Gaudet, a neuroscientist at Florida Atlantic University in Boca Raton, prefers to stay offline. “I do not want Claude to start telling people my research results,” he explains. AI co-scientists are revolutionizing how research is done Cloud-based large language models (LLMs), such as those underlying the chatbots Claude (Anthropic), Gemini (Google) and ChatGPT (OpenAI), represent the cutting edge of artificial-intelligence performance. However, these systems might use what is typed into them for training purposes. For researchers such as Gaudet, state-of-the-art performance isn’t worth the risk to data protection, reproducibility or budgets. Instead, they turn to a growing collection of tools, such as Ollama and LM Studio , that enable researchers to download LLMs to their computer and use them offline . These programs can help researchers to sort and query their own literature libraries to gather insights, for instance. But two fresh barriers have replaced the programming and set-up issues that once kept researchers from using them: the hardware needed to run them has become much more expensive, and the tools’ output still requires scrutiny to assess its veracity. Going local Ask researchers why they want to run an AI model on their desktop rather than in the cloud, and the answer almost always starts with data. Eric Böhnisch-Volkmann, president of DEVONtechnologies in Coeur d’Alene, Idaho, hears it constantly from users of the firm’s document-management software DEVONthink, which integrates LLMs. “Medical records is one thing, lawsuits is one other thing, research is another,” he says. “They just want to keep their hands on [their data] and not give it to someone else they don’t trust.” Privacy concerns are well founded. A 2025 analysis found that the six AI developers studied — Amazon, Anthropic, Google, Meta, Microsoft and OpenAI — trained their models on users’ input data by default ( J. King et al. Preprint at arXiv https://doi.org/g967b4; 2025 ). AI for research: the ultimate guide to choosing the right tool Ahmed Farooq, a computer scientist at Tampere University in Finland, built a local system called TAUCHI-GPT, so that papers, reports and study data protected under the European Union’s General Data Protection Regulation could be searched without leaving the university’s network. Reproducibility is another benefit: cloud-based AI providers update their models without notice, so a query that works today might return different results next month. Casey Greene, a biomedical informatician at the University of Colorado Anschutz School of Medicine in Aurora, notes that if a local model converts his documents into a searchable index using the same settings each time, the results will always be identical. “If you want to use that for a scientific workflow, being able to know that is important,” he says. Another benefit of local LLMs is economic. Jayashree Kalpathy-Cramer, who leads ophthalmology AI research at Anschutz, used Qwen (developed by Alibaba Cloud) and NVIDIA Nemotron through Ollama to index a decade’s worth of open-access publications on her machines. The corpus is public, so she could send it to a remote, commercial AI system. But local inference is free, and that changes how she works. Her system sends a daily briefing to her department’s chair highlighting new papers worth reading. The same query is run through three or four AI models, before another model reconciles the outputs, and every summarized finding is checked against the passage it came from. Because all of the field’s records sit in one searchable index, she can cluster them, map them and ask the LLM where any research gaps are. The search for meaning Preliminary findings of a survey by Wiley, a scientific publisher in Hoboken, New Jersey, indicate that 68% of some 2,400 researchers worldwide had used AI for literature searches or reviews (see go.nature.com/4rpuvwj ). The tasks that were embraced fastest include reviewing large collections of studies, monitoring the latest publications and assisting with writing. Those are jobs that local models already do well, says Kalpathy-Cramer. Greene built a local database of embeddings — numerical representations of each document’s overall meaning — to process the public abstracts of grants awarded to his university. When a federal-funding call for proposals in HIV research came through, he searched his database for potential collaborators. A keyword search for HIV highlighted a group researching the virus. But a semantic search of the same database, which looks at meaning rather than exact words, also came up with a researcher whose grants did not mention HIV in the title but who studied antiretroviral therapies, he says. He then asked that researcher about collaborating. Neuroscientist Ian Gaudet at Florida Atlantic University in Boca Raton prefers to keep his data offline. Credit: Julianna Cross Dmitrijs Lvovs, a bioinformatician at the Institute for Genome Sciences at the University of Maryland School of Medicine in Baltimore, uses a computational pipeline to analyse spatial data from tumour tissue, producing hundreds of tables containing gene activity data, cell-type interaction information and records of treatment responses. To make sense of those data, he and his colleagues paste them into Microsoft Copilot, the cloud-based AI service that their university provides. The tool then searches the published literature and suggests what any patterns might mean. But Lvovs wants to bring that step in-house. So, he started using a tool called llmize to feed the tables into Google Gemma 4 through Ollama, in a fully offline environment. Because the model is not connected to the Internet, Gemma cannot search the literature in the way that Copilot can. But the tables represent people’s data, and Lvovs would rather accept a ‘weaker’ model than send such data sets to the cloud. Even if study participants have given their consent for data collection, “it feels wrong to be concentrating all that data in the hands of a few companies”, he says. Getting started In the world of LLMs, ‘weaker’ is a moving target, and the gap between local and cloud-based models is shrinking fast, Lvovs says. “The competition is now more about the harness, the actual program that uses the model as a back end, not the model itself.” Enjoying our latest content? Log in or create an account to continue Access the most recent journalism from Nature's award-winning team Explore the latest features & opinion covering groundbreaking research Access through your institution or Sign in or create an account Continue with Google Continue with ORCiD Nature 658 , 830-831 (2026) doi: https://doi.org/10.1038/d41586-026-03227-4 Related Articles AI co-scientists are revolutionizing how research is done AI for research: the ultimate guide to choosing the right tool Forget ChatGPT: why researchers now run small AIs on their laptops Chatbots in science: What can ChatGPT do for you? Who gets credit in the AI era? OpenAI maths bombshell sparks debate AI tool turns any paper into an ‘agent’ that can collaborate and answer complex queries NatureTech hub Subjects Technology Machine learning Research management Information technology Latest on: Technology Machine learning Research management Hidden signals in sleep can predict future health News & Views 12 OCT 26 UAE completes Arab world's first solo space probe – and sets course for an asteroid News 08 OCT 26 Why we must advocate for proper care on the frontlines of the Ebola outbreak Spotlight 07 OCT 26 Award-winning microscopy image disqualified over AI use News 12 OCT 26 Hidden signals in sleep can predict future health News & Views 12 OCT 26 I told ChatGPT to ‘continue’ — it went too far Career Column 12 OCT 26 How we ditched presentations and embraced silence in our lab meetings Career Column 07 OCT 26 AI could undermine scientific independence in subtle ways Correspondence 06 OCT 26 How to mobilize the agile science needed in a crisis Editorial 06 OCT 26 Jobs Division Chief, Reproductive Endocrinology Lead UC San Diego’s nationally recognized REI program, advancing IVF, fertility research, academic medicine, and faculty development. San Diego, California AMN Healthcare Associate professor / professor Molecular Translational Research Professor in Molecular Translational Research to translate fundamental discoveries to applications that benefit human health Diepenbeek (Commune), Hasselt (BE) Biomedical Research Institute - Hasselt University Associate or Senior Editor, Nature Communications (Machine Learning) Job Title: Associate or Senior Editor, Nature Communications (Machine Learning) Locations: Shanghai, Beijing, Nanjing, Pune or New Delhi (Hybrid) A... Shanghai, Beijing, Nanjing, Pune or New Delhi (Hybrid) Springer Nature Ltd Associate or Senior Editor, Nature Materials Title: Associate or Senior Editor, Nature Materials Organization: Nature Portfolio Locations: Shanghai, Beijing, Milan or Pune – hybrid working mod... Shanghai, Beijing, Milan or Pune – hybrid working model Springer Nature Ltd Faculty Position in School of Biomedical Engineering, Nanjing University Join us! Suzhou, Jiangsu (CN) Nanjing University