Our long-term mission is to advance AI for humanity. We advance the frontier of AI research and build next-generation AI models, systems, and experiences.
Building Next-Generation AI Models and Systems
Frontier AI Research
We published and open-sourced our research and models on Foundation Models and General AI.
- Model Architecture: BitNet (v1 | b1.58 | a4.8 | v2) / 1-bit LLMs, YOCO / Decoder-Decoder Architecture (YOCO | YOCO-U | YOCO-S), DIFF / Differential Transformer, ReSa / Rectified Sparse Attention, MH-MoE / 1-bit MoE, RetNet, LongNet, DeepNet
- Learning Paradigm: TPT / Thinking Augmented Pre-Training, RPT / Reinforcement Pre-Training, RRM / Reward Reasoning Model, Adaptive Thinking Model
- Online/Continual Learning: On-Policy Context Distillation, Online Experiential Learning, LLM-as-a-Coach / Experiential Learning, Learning to Coach
- RL in LLMs: GMPO, OPO
- Multimodality: VibeVoice, LatentLM / Latent Language Modeling, (M)VoT, Kosmos-*, VALL-E*
- Multiagent: LLM-in-Sandbox, Agentic Organization / Asynchronous Thinking
- Distillation: GAD (Black-Box On-Policy Distillation), MiniLLM (On-Policy Distillation)
- Synthetic Data: QUEST, SynthLLM, GLAN
- System & Hardware: BitNet.cpp / 1-bit AI Infra
- Agent Harness: Agensh (Agens Harness)
- OSS Models on GitHub/HuggingFace: VibeVoice-ASR-Streaming, VibeVoice-ASR-BitNet, BitEmbed (BitNet Embedding Model), Harrier (Embedding Model), VibeVoice-ASR, VibeVoice-Realtime-0.5B, VibeVoice, BitNet b1.58 2B4T, (m)E5, Kosmos-2, WavLM, BEiT-3, BEiT, LayoutLM(-1/2/3), InfoXLM, MiniLM
- Large-Scale Pre-trained Models: BEiT-3, BEiT, WavLM, LayoutLM*, XLM-E, InfoXLM, MiniLM*, UniLM
We are hiring at all levels (including FTE researchers and interns)! If you are interested in working with us on Foundation Models and General AI, NLP, Speech, and Multimodal AI,
please send your resume to fuwei@microsoft.com.