PrismML releases Bonsai 27B, claims it is the first large AI model of its size suitable for iPhone

PrismML today announced the release of its Bonsai 27B model, which it claims is the first model of its size to run directly on iPhone, iPad, and Mac. The model uses 1-bit quantization, allowing it to occupy only 4 GB of memory and fit within the limits offered by Apple devices.

Performance and availability

Bonsai 27B achieves up to 163 tokens/s in 1-bit mode on NVIDIA GeForce RTX 5090 and 87 tokens/s on M5 Max. The model runs natively on Apple devices via MLX and on NVIDIA GPUs via CUDA. It is available under the Apache 2.0 license, and PrismML also offers a free API for developers.

Collaboration with Apple

PrismML CEO Babak Hassibi said in an interview with CNBC that Apple is evaluating their technology. The discussions are reportedly in early stages, but 'things are progressing well.' Apple has not yet responded to a request for comment. Analysts note that PrismML may use this information to bolster its reputation.

The Bonsai 27B model is the first large language model that can be run directly on an iPhone without needing a cloud connection. This opens up new possibilities for privacy and processing speed.