How to run an LLM locally on your iPhone

Updated October 17, 2026 · 10 min read

Yes, a recent iPhone can run a ChatGPT-style AI with no internet connection. You need a compact model (1 to 4 billion parameters, compressed to 4-bit) and ideally 8 GB of RAM, meaning an iPhone 15 Pro or newer. Answers are fast and private, but less knowledgeable than online AI.

What is a local LLM?

An LLM (large language model) is the program behind assistants like ChatGPT. In practice it is a large file of numbers, called parameters, plus an engine that uses them to predict the most likely next word, over and over.

With ChatGPT, Gemini or Claude, that file lives on the company’s servers: your message travels over the internet, gets processed there, and the answer comes back. A local LLM flips this around. The model is downloaded to your phone once, and every calculation runs on its chip. Your message never leaves the device.

Why run AI on your iPhone?

• Privacy. What you type is not sent to any server. No account, no history stored by a third party, no conversation that could be reviewed or used to train a model.
• Offline use. Once the model is downloaded, the assistant works in airplane mode, on the subway, abroad without a data plan, or anywhere without signal.
• No mandatory subscription. Your phone does the work, not a data center that has to be paid for every request.
• Always available. No queue, no server outage, no hourly message cap. The answer starts appearing almost instantly.

The trade-off: a phone has neither the memory nor the power of a server. The models that fit are far smaller than the ones behind ChatGPT, and it shows (more on that below).

Which iPhone do you need?

RAM matters most

To generate text, the model has to be loaded entirely into RAM. So memory, more than the chip, decides what is possible. And iOS only gives each app part of that memory, since the system and other apps need theirs. In practice, a model has to be noticeably smaller than the phone’s total RAM.

Treat these as rough guides, not hard limits: they depend on how the model is compressed, how long the conversation gets, and what else is running.

The chip and the GPU

Running an LLM is mostly huge matrix multiplications, which the iPhone’s GPU handles far faster than the main processor. Engines reach the GPU through Metal, Apple’s graphics and compute API. On Apple chips the CPU and GPU share the same unified memory, so the model doesn’t need to be copied between them. Newer chips write faster; from the iPhone 15 Pro (A17 Pro, 8 GB) onward, the experience feels genuinely smooth.

Storage

You need free space equal to the model size: a few hundred MB for the smallest, over 4 GB for the most capable. The download happens once, ideally on Wi-Fi.

How it works

Billions of parameters

Model size is counted in parameters: “1B” means one billion, “4B” four billion. More parameters mean more knowledge and better reasoning, but a heavier file. The large online models run to hundreds of billions; on an iPhone you generally stay under 5 billion.

Quantization: 16-bit, 4-bit, 2-bit

Each parameter is originally stored in 16 bits. Quantization rounds them to fewer bits, trading a little accuracy for size. According to Google’s documentation, Gemma 4 E2B takes about 11.4 GB at 16-bit but 2.9 GB at 4-bit; Gemma 4 E4B drops from 17.9 GB to 4.5 GB. Without this compression, none of these models would fit on an iPhone.

• 4-bit is the usual sweet spot, with a generally moderate loss of quality.
• 2–3-bit squeezes a model into less RAM, with a more noticeable loss. Apple compresses its own on-device model to 2 bits per weight, training it with that in mind (QAT, quantization-aware training).

A handy rule of thumb: at 4-bit, a model weighs a little over half a byte per parameter, so a 3 billion parameter model is roughly 2 GB.

GGUF and inference engines

• llama.cpp is the most widely used open-source engine. Its documentation calls Apple silicon “a first-class citizen”, optimized through Metal, and it supports 1.5- to 8-bit quantization. Its models use the GGUF format, a single file holding the weights and their metadata, built for fast loading. Thousands of GGUF models are published on Hugging Face.
• MLX is Apple’s machine learning framework for its own chips, designed around unified memory, with a Swift API used by some iOS apps.
• Core ML is Apple’s long-standing framework for on-device models, able to use the CPU, GPU and Neural Engine.

What about Apple Intelligence?

Apple already runs an LLM on the iPhone. The Apple Intelligence foundation model has about 3 billion parameters, compressed to 2 bits per weight in its 2025 version, and developers can use it free of charge through the Foundation Models framework. It only runs on Apple Intelligence devices (iPhone 15 Pro and later), and it is a single model chosen by Apple: you can’t swap in another one.

Which models fit on an iPhone?

Several companies publish open-weight models designed for mobile devices. Here are the main ones, with their approximate size at around 4-bit.

Exact sizes vary by file and compression method. Gemma 4 “E” models (for effective) are built for devices: they activate about 2 or 4 billion parameters at a time, though the full file is larger. To pick one for your phone, the DontKeep model list shows the recommended RAM for each.

What to expect

Speed

Speed is measured in tokens per second (a token is a piece of a word). In benchmarks published by the llama.cpp community, an iPhone 15 Pro generates about 57 tokens/s with a 1 billion parameter model at 4-bit, about 25 tokens/s with a 2.8 billion model, and about 9 tokens/s with a 7 billion model. An iPhone Air (A19 Pro) reaches roughly 87 and 36 tokens/s on the first two. Apple, for its part, reports about 30 tokens/s for its 3 billion parameter model on the iPhone 15 Pro.

The takeaway: with a 2–4 billion parameter model on a recent iPhone, text appears faster than you can read it. Real-world numbers depend on the model, the compression and how warm the phone is.

Quality compared with ChatGPT

To be clear, a 4 billion parameter model is not a match for the best online models. It is very good at drafting, rewording, summarizing, translating and explaining, or as a thinking partner. It is weaker at multi-step reasoning, math, complex code and niche questions.

Frozen knowledge, and mistakes

• A local model doesn’t browse the web. It only knows what it learned up to its training cutoff (August 2024 for Gemma 3, according to Google), so it knows nothing about recent news.
• Like every LLM, it can hallucinate: state a made-up figure, quote or reference with confidence. Google itself notes its models are not a knowledge base. The smaller the model, the higher the risk, so check anything important.

Context length

Context is how much text the model can “keep in mind” during a conversation. On paper, small models advertise 32,000 to 128,000 tokens. On a phone, every token of context uses memory on top of the model, so apps often cap it well below those limits. In a very long chat, the beginning eventually gets forgotten.

Heat and battery

Running an LLM works the GPU hard. Over long sessions the phone warms up, iOS may slow the chip to cool it down, and the battery drains faster than with a messaging app. For occasional use, it’s barely noticeable.

In practice: which apps?

There is no “offline ChatGPT” on the App Store, but several apps download and run an open model on your iPhone:

• DontKeep runs Google’s Gemma models with llama.cpp (GGUF, Metal acceleration), with no account. Conversations are never written to disk: they disappear when the app closes, and at the latest after an adjustable delay (one hour by default), unless you choose to keep them, encrypted.
• PocketPal AI: an open-source app (MIT license) built on llama.cpp that can load any GGUF model from Hugging Face. Great for experimenting.
• Private LLM: a paid app (one-time purchase) for iPhone, iPad and Mac, with a large model catalog and Shortcuts integration.
• Enclave: free with in-app purchases, several model families (Llama, Gemma, Qwen, Phi…), Shortcuts and Siri integration.
• LLM Farm: a pioneering open-source project, though its repository says it is temporarily unavailable on the App Store and TestFlight.

Before installing, check the “App Privacy” section of the App Store listing, then test in airplane mode: if the assistant still answers, the work really is happening on your iPhone.

What still stays on the iPhone

A local LLM keeps your messages off servers, but the iPhone itself records some things, whatever the app: that it is installed and has been opened (Screen Time, battery usage), the data used to download a model, and the model file itself. Depending on the app, conversations may also be stored on the phone, sometimes included in backups. That’s why DontKeep erases by default anything you haven’t explicitly kept.

Local doesn’t mean invisible. It means the content of your conversations isn’t sent to a server.

FAQ

Can you really have an offline ChatGPT on an iPhone?

Yes, in the sense of a chat assistant that answers without a connection. Not in the sense of the same model: the models that fit on an iPhone are much smaller than OpenAI’s, so they know less and have no access to current news.

What’s the best iPhone for running a local LLM?

The one with the most RAM. The iPhone 18 Pro, 18 Pro Max, 17 Pro, 17 Pro Max and Air (12 GB) run models of around 4 billion parameters. With 8 GB (iPhone 15 Pro and later), 2–4 billion parameter models work well.

Can an iPhone 12 or 13 run local AI?

Yes, with small models (1 billion parameters or fewer) or heavily compressed ones. Answers will be simpler, but still useful for rewording or summarizing.

Does a local LLM need the internet?

Only to download the model, once. After that, everything works in airplane mode.

Will it damage my battery?

A long session uses more power and warms the phone, like a 3D game. It isn’t dangerous: iOS manages temperature automatically.

Are local models free?

Open models like Gemma, Llama and Qwen can be downloaded for free under their respective licenses. The apps that run them may be free, paid or offer optional upgrades.