The Allen Institute for AI (Ai2) has recently launched a new family of language models, OLMo 2, which aims to enhance natural language understanding and generation. These models, available in 7 billion and 13 billion parameter versions, have been trained on a diverse range of datasets to improve the performance of AI systems in various applications. The development was announced by Ai2 on their official Twitter account, stating that OLMo 2 is the best fully open language model to date, with a family of models trained on up to 5 trillion tokens.
The new OLMo 2 models have been trained on OLMo-mix-1124 and Dolmino-mix-1124 datasets, giving them an edge over the original OLMo 7B model. Additionally, Ai2 has also released instruction-tuned versions of the OLMo 2 models, which have been optimized to handle more structured and goal-oriented tasks. These models have shown significant improvements in benchmarks such as MATH, GSM8K, and IFEval, demonstrating their ability to tackle complex reasoning tasks and respond effectively to inputs.
One of the key highlights of the OLMo 2 release is its emphasis on open access. Ai2 has made the models and training data publicly available under the Apache 2.0 license, along with all code and intermediate checkpoints. This aligns with Ai2’s commitment to promoting transparency and reproducibility in AI research, allowing other researchers to build upon their work and contribute to further advancements in language modelling.
The OLMo 2 models have been trained on a massive amount of text data, up to 5 trillion tokens, which has enabled them to achieve high performance in various natural language processing tasks. With the application of their state-of-the-art Tülu 3 post-training recipe, Ai2 has also developed OLMo 2 Instruct models, which are competitive with the best open-weight models in the market. These models have outperformed Qwen 2.5 14B instruct, Tülu 3 8B, and Llama 3.1 8B Instruct models, showcasing their superiority in performance.
In conclusion, the release of OLMo 2 by Ai2 marks a significant advancement in the field of language modelling. These models have shown promising results and have the potential to compete with leading frontier models. With their focus on open access, Ai2 has set a precedent for transparency and collaboration in AI research.
