How to Run Open Weight LLMs on Your Own Machine — the Complete Guide

To run open weight LLMs on your own machine comfortably, 16GB of RAM is recommended. The instructor covers model selection for machines with different specs — including options that run on 8GB RAM machines at reduced but still useful performance levels. No dedicated GPU is required for the models used in this workshop.

For a typical developer laptop with 16GB RAM, models in the 3B to 8B parameter range offer the best balance of quality and performance. Phi-3 Mini, Mistral 7B, and Llama 3 8B are all excellent choices. The instructor covers performance benchmarks for each during the live session.

The impact on your other work depends on the model size and your hardware. Smaller models (3B to 4B parameters) have a minimal impact on system performance. The instructor covers how to configure Docker Model Runner resource limits to ensure your local LLM does not interfere with your other workloads.

You can run open weight LLMs without Docker using tools like Ollama or direct Python inference libraries. This workshop uses Docker Model Runner because it provides the cleanest integration with OpenClaw and the most straightforward setup for developers already in the Docker ecosystem.

The instructor covers model evaluation during the workshop — testing different models for the personal AI assistant use case and showing you how to compare their responses. This gives you a practical framework for choosing the right model for your specific needs.

When you close Docker Desktop or stop the Docker Model Runner, your local LLM stops running. The workshop covers how to configure automatic startup and how to deploy your setup to a VPS for always-on availability — so your private AI assistant keeps running even when your laptop is off.