How to run Meta's LLaMA 3 on your PC

Meta, the company formerly known as Facebook, has recently unveiled LLaMA 3, the latest iteration of its large language model. This advanced model is available in two versions: an eight billion (8B) parameter version and a 70 billion (70B) parameter version. In this article, we will explore how to run the 8B parameter version of LLaMA 3 locally, a more feasible option for standard desktops or laptops that may struggle to run the larger 70B version.

LLaMA 3’s performance overview

LLaMA 3 is an impressive large language model. The 8B parameter version, trained using 1.3 million hours of GPU time, outperforms its predecessor, LLaMA 2, in several ways. For instance, it is 34% better than the 7 billion parameter version of LLaMA 2 and 14% better than the 13 billion parameter version. Remarkably, the 8B parameter version of LLaMA 3 even surpasses the performance of the 13 billion parameter version of LLaMA 2. It only falls short by 8% when compared to the 70B parameter version of LLaMA 2, making it an impressive model for its size.

The 8B parameter version of LLaMA 3 has a knowledge cutoff date of March 2023, while the 70B version extends to December 2023. As a result, there will be a real-world information discrepancy between the two.

How to run LLaMA 3 on your PC

To run LLaMA 3 on Windows, we will use LM Studio.

1. To get started, visit lmstudio.ai and download the appropriate LM Studio version for your system.
2. Once downloaded, install LM Studio.
3. Open LM Studio.
4. Once inside LM Studio, ensure that the LLaMA 3 model is downloaded. If it’s already installed, it should display on the home page. If it isn’t, use the search bar or search icon on the left-hand vertical bar to find it and then download it.
5. Once LLaMA 3 is installed, click the AI Chat icon on the left-hand vertical bar within LM Studio. This will open a chat interface similar to ChatGPT.
6. Click Select a model to load at the top of the page. Select the LLaMA 3 LLM you just downloaded. LM Studio will load the model, which could take a few seconds.
7. You can now chat to LLaMA 3 on your device.

Notably, if you aren’t using a Windows machine, LM Studio can also be used by Mac owners running new M processors (M1, M2, and M3). There is a beta version available for Linux, too.

How to run LLaMA 3 using OLLaMA

Alternatively, you can run LLaMA 3 locally using the OLLaMA project. This method will appeal to those who want to run the LLM on a Raspberry Pi.

For Mac and Windows, visit oolama.com and download the version compatible with your operating system. If you’re running Linux (or if you want to install LLaMA 3 on your Raspberry Pi), simply copy and paste the provided line to download and run the install script. Once installed, you can run LLaMA 3 directly from the terminal app of your choice.