I decided to try hosting an ai model locally. It’s easier than you think and doesn’t require more than a typical PC. Of course, the more powerful the PC the faster the model operates. I initially hosted the model on my laptop.

The benefits of hosting a model locally is that you use open source models rather than commercial versions that charge. And your chats are private. No data is stored or transmitted on/to an external server. You can even network your hosted model to serve all of your devices by hosting it on a server if you have one set up.

I’m using LM Studio to host the open source Mistral 7b-instruct-v0.3 model. LM Studio is free and easy to use. I can add my own documents to LM Studio to provide context for the model and connect to any MCP server to access Internet services for tasks. Changing models is easy as well. Just search for one and download it.

To network the model so anyone on my local network can access it using their PC or mobile, LM Studio has an API server and can provide a localhost URL. Then you can use something like Open-WebUI or LMStudioWebUI to chat with the model.