Running Local LLMs With Docker Model Runner — From Setup to Private AI Assistant

Docker Model Runner works with a wide range of open weight models. For a personal AI assistant use case, the instructor recommends starting with models in the 3B to 8B parameter range — they run comfortably on 16GB RAM and provide good response quality. In this tutorial you will try several models and learn to evaluate them for your specific hardware.

Local LLM inference speed depends on your hardware. On a modern laptop with 16GB RAM, you can typically expect 10 to 30 tokens per second with a 7B parameter model on CPU. The instructor covers performance expectations for different hardware configurations and model sizes during this tutorial.

Docker Model Runner significantly simplifies local LLM setup. In this tutorial you will have a local LLM running within the first hour of the 4-hour session. The instructor walks through every step including Docker setup, model pulling, and API configuration.

Modern open weight models have improved dramatically. For personal assistant tasks — answering questions, summarising content, helping with writing, and conversational tasks — models available through Docker Model Runner perform very well. The instructor covers model selection to help you choose the best fit for your use case.

Docker Model Runner makes switching between local LLMs straightforward. This tutorial covers how to pull multiple models, how to configure OpenClaw to use a specific model, and how to switch models without disrupting your assistant setup.

Yes. All local LLM inference through Docker Model Runner happens entirely on your own machine. No conversation data is sent to any external server. Your private AI assistant powered by a local LLM is completely air-gapped from cloud AI providers.