Mistral AI, a company known for its innovative AI models, has recently released two significant models: Mistral 7B and Mixtral 8x7B. Mistral 7B is the first dense model released by Mistral AI, with 7.3 billion parameters, while Mixtral 8x7B is a high-quality sparse mixture of experts model (SMoE) with 46.7 billion total parameters, but it only uses 12.9 billion parameters per token during inference. Mixtral 8x7B is designed to be highly efficient, processing input and generating output at the same speed and cost as a 12.9B model. It is pre-trained on data extracted from the open web and supports multiple languages, including French, German, Spanish, Italian, and English. Mixtral 8x7B has been shown to match or outperform other models such as Llama 2 and GPT-3.5 on various benchmarks. Mistral AI has made these models available for download and open-sourced them to empower users to test and refine their applications. The company's focus on open and accessible AI solutions positions it as a significant player in the rapidly evolving AI technology landscape
These models have shown impressive performance, with Mixtral 8x7B matching or outperforming other models like Llama 2 and GPT-3.5 on various benchmarks. Additionally, Mixtral 8x7B has been optimized for instruction following and has a high MT-Bench score, making it a top-performing open-source model
