How to Run Large Language Models Locally on a Windows Machine Using WSL and Ollama

prerequisite

Install WSL

https://learn.microsoft.com/en-us/windows/wsl/install?source=post_page-----9d53151e254a--------------------------------

Install curl

sudo apt install curl

Install Ollama via curl

curl https://ollama.ai/install.sh | sh

Run Ollama

In this case, we will try to run Mistral-7B.
If you want to try another model, you can pick from the following site.
https://ollama.ai/library

ollama serve

Open another Terminal tab and run the following command. The following command will pull a model.

ollama run mistral

If everything works properly, you will see something like below.
My machine has a GPU, RTX3070. So Ollama is using the GPU.

Terminate Ollama

If you want to exit Ollama, you need to type the following.

/exit

Then ctrl + c in a terminal what you ran ollama serve.