this post was submitted on 27 Sep 2023
1 points (100.0% liked)

LocalLLaMa

29 readers
1 users here now

Magazine to talk about LLaMA (large language model created by Meta AI) and any related Open Source LLMs. Inspired by Reddit's /r/LocalLLaMA/ subreddit.

founded 1 year ago
 

From their website

Mistral AI team is proud to release Mistral 7B, the most powerful language model for its size to date.

Mistral 7B in short

Mistral 7B is a 7.3B parameter model that:

  • Outperforms Llama 2 13B on all benchmarks
  • Outperforms Llama 1 34B on many benchmarks
  • Approaches CodeLlama 7B performance on code, while remaining good at English tasks
  • Uses Grouped-query attention (GQA) for faster inference
  • Uses Sliding Window Attention (SWA) to handle longer sequences at smaller cost

We’re releasing Mistral 7B under the Apache 2.0 license, it can be used without restrictions.

Mistral 7B is easy to fine-tune on any task. As a demonstration, we’re providing a model fine-tuned for chat, which outperforms Llama 2 13B chat.

no comments (yet)
sorted by: hot top controversial new old
there doesn't seem to be anything here