On-device AI

Breadstick Pro. Experimental.

On-device AI gives Linux programs your device's own AI model, Gemini Nano. Anything that talks to Ollama, or to OpenAI's chat API, can use it as it is: a chat app, an editor plugin, a script, curl. The model runs on the device, and no data leaves it.

It is experimental. It may change, and Google can change the model or how Android offers it.

Which devices have it

Only devices whose Android offers Gemini Nano to apps: Googlebooks, and the Pixel and Galaxy phones on Google's list. On every other device Breadstick shows nothing for it, in Settings or on the Pro screen.

Turning it on

It is under Settings > Advanced, in On-device AI (Experimental).

1. If Model shows Download, tap it. Android downloads the model once, and the row shows how far it has got.
2. Turn on On-device AI.

The section then shows:

• Model: Gemini Nano and its version, such as nano-v3
• Address on this device: http://127.0.0.1:11434, with Copy
• Port: 11434 unless you change it. If you also run Ollama itself in Linux, give one of them another port.
• Key: None, or a key of your own, at least 16 characters with no spaces. With a key set, every request has to carry Authorization: Bearer followed by the key. Other apps on this device can reach the address too, and a key keeps them out.
• Report a response

Breadstick Settings inside Linux has the same section. Turned on from Linux, it asks on Android first.

Using it from Linux

While it is on, Linux programs find it through OLLAMA_HOST, which Breadstick sets for the desktop, Terminal tabs and the apps it opens. In a shell that was open before you turned it on, run eval "$(bsl-ai env)". bsl-ai on its own says whether it is on and answering.

The model is called gemini-nano:

curl http://127.0.0.1:11434/api/chat -d '{
  "model": "gemini-nano",
  "messages": [{"role": "user", "content": "Explain what a symlink is in two sentences."}]
}'

For tools that speak OpenAI's API, set the base URL to http://127.0.0.1:11434/v1 and the model to gemini-nano. If a tool insists on an API key, give it your key, or any text when you set none.

What	Where
Chat and completion	/api/chat and /api/generate, streamed unless you ask otherwise
OpenAI's chat API	/v1/chat/completions, streamed or not, and /v1/models
The model list and details	/api/tags, /api/show, /api/ps, /api/version
Downloading the model	/api/pull, so ollama pull gemini-nano works too
Images	Sent with the request, where the device's model takes them. Links to images are refused

Limits

• Breadstick has to be in front. Android lets only the app on screen use the model. A Linux window counts: the desktop, a floating window, the Terminal. While you are in an Android app, requests are refused, and a program that tries again gets its answer once Breadstick is back in front.
• Short input. About 4,000 tokens per request at most, less where the device's model takes less. In a long chat, the oldest turns are dropped to fit.
• One request at a time. Up to four more wait their turn.
• No embeddings, no tool calling and no JSON mode yet. A program that asks for them gets an error that says so.
• Android's own limits. Android rations how much each app can use the model, in a short burst and over a day. When it says no, programs are told to try again later.
• One model. It answers like Ollama, but other models cannot be added.

No data leaves this device

What you send the model and what it answers stay on the device. The model runs in Android's own AI service, on the device, and Breadstick serves it only on the device, at 127.0.0.1. The only download is the model itself, from Google, when you ask for it.

Breadstick's build carries no usage reporting for this feature.

Report a response

If the model says something offensive or harmful, Report a response opens an email to us with only the subject filled in. You write the rest, and it is sent only if you send it.