AI2 Releases Olmo-core 3 Framework for Efficient Mixture-of-Experts LLM Development

VERA Brief

AI-generated. Grounded in the article and its cited sources.

Allen Institute for AI has released Olmo-core 3, a framework for developing large mixture-of-experts language models. This framework aims to enable trillion-parameter scale MoE models while maintaining computational efficiency and lower costs.

Key facts

• Allen Institute for AI has launched Olmo-core 3.
• Olmo-core 3 is a development framework for large language models.
• The framework is designed to streamline the training of mixture-of-experts (MoE) large language models.
• Olmo-core 3 aims to enable trillion-parameter scale MoE models.
• The framework seeks to maintain computational efficiency and lower costs during MoE training.