nanochat

nanoChat is an open-source, full-stack implementation of a ChatGPT-like language model that can be trained for just $100 in 4 hours on an 8xH100 GPU node, offering a clean, minimal, and hackable codebase with complete pipelines from tokenization to deployment.

Visit Website

Full-Chain Training WorkflowEnd-to-end implementation from tokenization to web serving, with all components integrated into a single codebase that can be executed using simple scripts.
Affordable LearningProvides basic ChatGPT-like capabilities for around $100 worth of computing power (equivalent to 4 hours of training on 8 X100 GPUs), making it suitable for individual researchers and smaller teams.
Low DependenciesA clean, hackable codebase with few external dependencies, which makes it straightforward to comprehend and adjust.
Scale-Out ArchitectureSupports training of variously budgeted large models, ranging from $100 for basic models to $1000 for more advanced versions.