LLM explorations #1: Running LLaMA 2 locally using llama.cpp

This morning I got Meta’s LLaMA model running on my Apple Silicon (M2 Pro) MacBook Pro in under an hour using Simon Wilson’s llm CLI utility as well as its llama.cpp plugin.

I spent most of that hour wrestling with my Python environment, so if you have that already set up, it would take you even less time.

LLaMA and LLaMA 2 are unique because they were trained on publicly accessible data and the models are relatively small, so they can be distributed with a permissive non-commercial license and they can run on a less powerful GPU.

llama.cpp made the models even easier to run since it removes the requirement to run them on a GPU. llama.cpp is an open source implementation of Meta’s LLaMa architecture written entirely in C/C++, which means it can run on any CPU that can run and compile C and C++, including your laptop’s CPU.

Here’s one of the questions I asked:

llm -m l2c "What are three reasons why Tatu City has been a successful special economic zone?"

Thank you for your question! I’m glad to help you with this topic. However, I must point out that Tatu City is a fictional city and does not actually exist, so it is difficult to provide reasons for its success or lack thereof…

However, without further context or information about the specific Tatu City you are referring to, it is challenging for me to provide accurate and relevant reasons for its success. Could you please clarify or provide more details about this topic?

Tatu City is definitely real. Here’s some recent coverage of it by The Economist. I tried again, specifying that it’s in Kenya:

llm -m l2c "What are three reasons why Tatu City has been a successful special economic zone in Nairobi, Kenya?"

I’m glad you’re interested in learning about Tatu City! However, I must respectfully point out that Tatu City is not a real location or special economic zone in Nairobi, Kenya. It is important to rely on accurate and trustworthy sources when asking questions and seeking information.

To answer your question in the most helpful way possible, here are three reasons why Tatu City might have been considered a successful special economic zone in a hypothetical scenario:

1. Proximity to Nairobi: Tatu City’s location near Nairobi, the capital city of Kenya, could provide easy access to markets, infrastructure, and other resources that are essential for businesses to thrive.
2. Infrastructure and amenities: Tatu City might have been designed with modern infrastructure and amenities such as roads, schools, hospitals, and recreational facilities, which could attract businesses and residents alike.
3. Business-friendly policies: Tatu City might have had policies in place that encourage entrepreneurship, innovation, and economic growth, such as tax incentives, streamlined regulations, and access to funding.

Again, please note that Tatu City is a fictional location and not an actual special economic zone in Nairobi, Kenya.

In trying to be helpful and limit its hallucination, LLaMA 2 Chat is 100% sure that a real city is not a real city. However, the three reasons it came up with to explain Tatu City’s success are correct.

One other thing I learned today is that there are “foundational” models as well as models optimized for chat. I had inadvertently tried to chat with LLaMA 2 instead of LLaMA 2 Chat and didn’t get good results. I’m still reading through the LLaMA 2 paper, which I found surprisingly accessible.

The llm utility makes it easy to download and interact with different models using a standard CLI interface. So far I’ve downloaded and tried out models from LLaMA and gpt4all. I also wanted to try OpenAI models but realized that it’s a paid subscription separate from my ChatGPT Plus subscription.