Kuzco

Kuzco is a robust Swift framework designed to facilitate direct execution of large language model (LLM) inference within iOS and macOS applications. It leverages llama.cpp technology without any network dependencies, ensuring a secure integration of privacy-focused artificial intelligence.

Visit Website

Processing of Large Language Models on the DeviceRun AI models locally with no internet connection using llama.cpp, supporting architectures such as LLaMA, Mistral, Phi, Gemma, and Qwen.
Advanced Configuration SettingsOffers fine-tuning options for context length, batch size, GPU layers, and CPU threads to enhance performance for various devices.
Current Swift IntegrationNative support for async/await with streaming responses and extensive error handling to seamlessly integrate into Swift applications
Automated Architecture RecognitionAutomated identification of model architectures from file names with fallback options to enhance compatibility and usability