Our long-term mission is to advance AI for humanity. We are dedicated to pioneering a new paradigm of AI, focusing on paradigm-shifting technologies and next-generation frontier AI models.
Building Frontier Efficiency Model [03/2025 - ]
A New Paradigm of AI [01/2025 - ]
We have published and open-sourced pioneering and high-impact research works and models on Foundation Models and General AI. Examples include,
- Model Architecture: BitNet (v1 | b1.58 | a4.8 | v2) / 1-bit LLMs, YOCO / Decoder-Decoder Architecture, DIFF / Differential Transformer, ReSa / Rectified Sparse Attention, MH-MoE / 1-bit MoE, RetNet, LongNet, DeepNet
- Learning Paradigm: RPT / Reinforcement Pre-Training, RRM / Reward Reasoning Model, Adaptive Thinking Model, MiniLLM
- Multimodality: LatentLM / Latent Language Modeling, (M)VoT, Kosmos-*, VALL-E*
- Data: SynthLLM, GLAN
- System & Hardware: BitNet.cpp / 1-bit AI Infra
- OSS Models on GitHub/Hugging Face: BitNet b1.58 2B4T, (m)E5, Kosmos-2, WavLM, LaytoutLM(-1/2/3)
- Large-scale Pre-trained Models: BEiT-3, BEiT, WavLM, LayoutLM*, XLM-E, InfoXLM, MiniLM*, UniLM
We are hiring at all levels (including FTE researchers and interns)! If you are interested in working with us on Foundation Models and General AI, NLP, MT, Speech, Document AI and Multimodal AI,
please send your resume to fuwei@microsoft.com.