Blog

The Second Curve of Scaling Law

Scaling Law
2024
How Microsoft’s next-gen BitNet architecture is turbocharging LLM efficiency (by VentureBeat)

BitNet (a4.8)
Microsoft’s Differential Transformer cancels attention noise in LLMs (by VentureBeat)

DIFF Transformer
1-bit LLMs Could Solve AI’s Energy Demands (by IEEE Spectrum)

BitNet | 1-bit LLMs
The Era of 1-bit LLMs

BitNet (b1.58)
TrOCR | Qualcomm® AI Hub
Kosmos-2 | NVIDIA NGC

Kosmos-2 MLLM
2023
人工智能基础创新的第二增长曲线 (By Furu Wei)
2023
AI voice synthesis tech 🤝 Impromptu (By Reid Hoffman)
2023
Turing Bletchley v3 - A Vision-Language Foundation Model

(Multilingual) BEiT-3
2023
Retentive Network: A Successor to Transformer for Large Language Models
2023
LongNet: Scaling Transformers to 1,000,000,000 Tokens
2023
Kosmos-2: Grounding Multimodal Large Language Models to the World

Kosmos-2 MLLM
2023
Achieving Zero-COGS with Microsoft Editor Neural Grammar Checker

LLMOps EdgeFormer LLM Accelerator
2023
Kosmos-1: A Multimodal Large Language Model (MLLM)

Kosmos-1 MLLM
2023
让天下没有难训练的大模型，微软亚洲研究院开源TorchScale

#TorchScale DeepNet Magneto X-MoE
2023
Revolutionizing Document AI with Multimodal Document Foundation Models

Layout(X)LM
2023
After ChatGPT and DALL-E, meet VALL-E - the text-to-speech AI that can mimic anyone’s voice
2022
Microsoft Turing Universal Language Representation model, T-ULRv6, tops both XTREME and GLUE leaderboards with a single model
2022
文档基础模型引领文档智能走向多模态大一统

Layout(X)LM MarkupLM
2022
通用多模态基础模型BEiT-3：引领文本、图像、多模态预训练迈向“大一统”
BEiT(-3)
2022
文档智能多模态预训练模型LayoutLMv3：兼具通用性与优越性
2021
通用模型、全新框架，WavLM语音预训练模型全解
2021
WMT 2021冠军来了！重建巴别塔之多语言翻译模型
2021
Multilingual translation at scale: 10000 language pairs and beyond
2021
Microsoft Turing Universal Language Representation model, T-ULRv5, tops XTREME leaderboard and trains 100x faster

XLM-E InfoXLM
2021
The science behind semantic search: How AI from Bing is powering Azure Cognitive Search

UniLM MiniLM
2020
Microsoft Turing Universal Language Representation model, T-ULRv2, tops XTREME leaderboard
2019
Microsoft’s UniLM AI achieves state-of-the-art performance on summarization and language generation