A small menu bar app called oMLX has fixed one of the biggest problems with running local coding agents on Apple Silicon Macs.
Coding agents can sometimes get stuck on a Mac when they have to process the same code and conversation again and again. A prompt may take 30 to 90 seconds to respond, which can make users give up and switch back to cloud based AI services.
A developer known as jundot created oMLX to solve this problem. The goal is to reduce those long delays to just a few seconds.
It’s about memory, not speed.
Problem is not really the performance of Apple Silicon. Problem is how coding agents handle memory.Every time an agent works on a project, it sends the conversation history, file information, and tool commands along with the new request. This can create a very large amount of information that the model needs to process.
Local AI tools such as Ollama and LM Studio normally keep this information in RAM. When the context changes, the cache can be cleared, forcing the model to process thousands or even hundreds of thousands of tokens again.
That is where the long waiting time comes from. It is not really generating the answer. It is reading the context again.
How oMLX changes things
oMLX runs on Apple’s mlx lm but uses a different way of handling memory. Instead of throwing away old cache data, oMLX moves it from RAM to the Mac’s SSD. Frequently used data stays in RAM, while older data is stored on the SSD.
When the coding agent needs that information again, oMLX can load it from the SSD instead of rebuilding everything from the beginning. This can make repeated requests much faster. Benefit also continues after the server restarts, so users don’t have to start from scratch every time. oMLX also supports continuous batching, which allows it to handle multiple requests at the same time. It can load language, vision, embedding, and reranking models. App is built natively in Swift, so it runs quietly from the Mac menu bar instead of using a heavy Electron based interface.
It was actually tested
Writer for Mac O’Clock tested oMLX against the standard mlx lm server.
For simple repeated prompts, both performed almost the same, taking around 3.5 seconds. This is because mlx lm already keeps the most recent prompt in its cache.
Real difference appears when the cache has been cleared, something that happens regularly when coding agents modify files and their context changes.
There are also some trade offs. oMLX may take longer when loading a model for the first time, and its response handling is different from standard token by token streaming.
Who is this for?
oMLX uses the same model folder as LM Studio, so users don’t need to download their large models again. It also supports OpenAI and Anthropic APIs, making it easier to connect tools such as Claude Code and Cursor. It supports tool calling, MCP, common chat templates, and is distributed under the Apache 2.0 license.
For Mac users who want to run coding agents locally without paying high cloud AI costs or waiting 30 to 90 seconds for responses, oMLX could be a very useful solution.
It isn’t trying to replace every local AI tool. It simply fixes an important problem that has made local coding agents frustrating to use.