Summary:
- Researchers from Moonshot AI and UCLA have released Moonlight, a 3B-16B parameter Mixture of Experts (MoE) model trained on 5.7T tokens using the Muon optimizer.
- Moonlight is a large language model designed for general-purpose tasks and can be used for a variety of applications, including natural language processing, generation, and understanding.
- The model's performance is said to be competitive with state-of-the-art large language models, and it is available for research and development purposes.