Jamba

Jamba is a family of large-scale models from AI21 built on a hybrid Transformer + Mamba architecture using Mixture-of-Experts (MoE). This design combines the strengths of classic transformers (accuracy, reasoning quality) and Mamba-SSM (linear handling of long sequences), delivering high speed and low memory consumption while preserving the quality of top-tier LLMs. In Jamba and Jamba 1.5, the context window reaches 256K tokens (hundreds of pages of text), and the larger versions have up to hundreds of billions of parameters with ~94B active per token while remaining deployable on a limited GPU fleet. Technically, Jamba is a decoder-only LLM with alternating Transformer and Mamba blocks, and some layers include MoE experts, which increases model capacity without explosive compute growth. The architecture is designed to process 256K context with high throughput, supports instruction tuning, and includes optimizations such as ExpertsInt8, allowing long requests to be served on a cluster of several GPUs without quality loss. Jamba can be used to build systems that “read” massive volumes of text: corporate RAG platforms over internal documentation, assistants for analyzing contracts, logs, and reports, powerful chatbots and research assistants, as well as copilot solutions for developers and analysts where long dialogue history and deep contextual understanding matter. The FreeBlock team delivers the full project cycle based on Jamba: we will select the optimal model version, tune it on your data, build the RAG architecture, and integrate the solution into your existing products and infrastructure. If you want to use Jamba to build assistants, analytical systems, or long-context AI platforms, order AI project development with the Jamba model from FreeBlock.