Many users want to run large language models (LLMs) locally for more privacy and control, and without subscriptions, but until recently, this meant a trade-off in output quality. Newly released open-weight models, like OpenAI’s gpt-oss and Alibaba’s Qwen 3, can run directly on PCs, delivering useful high-quality outputs, especially for local agentic AI.
This opens up new opportunities for students, hobbyists and developers to explore generative AI applications locally. NVIDIA RTX PCs accelerate these experiences, delivering fast and snappy AI to users.
Getting Started With Local LLMs Optimized for RTX PCs
NVIDIA has worked to optimize top LLM applications for RTX PCs, extracting maximum performance of Tensor Cores in RTX GPUs.
One of the easiest ways to get started with AI on a PC is with Ollama, an open-source tool that provides a simple interface for running and interacting with LLMs. It supports the ability to drag and drop PDFs into prompts, conversational chat and multimodal understanding workflows that include text and images.
NVIDIA has collaborated with Ollama to improve its performance and user experience on GeForce RTX GPUs. The most recent developments include:
- 50% performance improvements on OpenAI’s gpt-oss-20B model
- 60% performance improvements on the new Gemma 3 270M and EmbeddingGemma models for hyper-efficient RAG
- Improved model scheduling system to maximize and accurately report memory utilization
- Stability enhancements to reduce the number of crashes
Ollama is a developer framework that can be used with other applications. For example, AnythingLLM — an open-source app that lets users build their own AI assistants powered by any LLM — can run on top of Ollama and benefit from all of its accelerations.
Enthusiasts can also get started with local LLMs using LM Studio, an app powered by the popular llama.cpp framework. The app provides a user-friendly interface for running models locally, letting users load different LLMs, chat with them in real time and even serve them as local application programming interface endpoints for integration into custom project


Ollama Gets a Major Performance Boost on RTX
Llama.cpp and GGML Optimized for RTX
Project G-Assist Update Rolls Out
Windows ML With NVIDIA TensorRT for RTX Now Generally Available
NVIDIA Nemotron Powers AI Development 

