Star ...
RetNet

BitNet: 1-bit Transformers and LLMs
RetNet

Retentive Network: Revolutionizing Transformers for Large Language Models
RetNet

Kosmos-G: Generating Images in Context with Multimodal Large Language Models
RetNet

Kosmos-2.5: A Multimodal Literate Model
RetNet

Large Language Model for Science: A Study on P vs. NP
LongNet

LongNet: Scaling Transformers to 1,000,000,000 Tokens
KOSMOS-2

Kosmos-2: Grounding Multimodal Large Language Models (MLLMs) to the World
KOSMOS-1

Kosmos-1: A Multimodal Large Language Model (MLLM)
VALL-E

WavMark: Watermarking for Audio Generation
VALL-E

VALL-E (X): Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers
LLMA

PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training
LLMA

AdaLLM: Adapting Large Language Models via Reading Comprehension
MiniLLM

MiniLLM: Knowledge Distillation of Large Language Models
LongMem

Large Language Models with Long-Term Memory
LLMA

LLM Accelerator: Lossless Acceleration of Large Language Models
XPos

A Length-Extrapolatable Transformer
ICL

Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta Optimizers
Promptist

Promptist: Optimizing Prompts for Text-to-Image Generation
Structured Prompting

Structured Prompting: Scaling In-Context Learning to 1,000 Examples

TorchScale

TorchScale: Transformers at (Any) Scale

Magneto: A Foundation Transformer
BEiT-3

BEiT-3: A General-Purpose Multimodal Foundation Model

Language Models are General-Purpose Interfaces

DeepNet: Scaling Transformers to 1,000 Layers

BEiT: BERT Pre-Training of Image Transformers

XLM-E: Efficient Multilingual Language Model Pre-training

UniLM: Unified Language Model Pre-training