Featured

• Blog

  Parallel constrained decoding for typed JSON (Qwen-2.5-1B-RLCD)

  Prefill once, score each closed JSON field in parallel, and assemble schema-valid output in code.

• Paper

  Tulu 3: Pushing Frontiers in Open Language Model Post-Training

  Tulu 3 names RLVR: reinforce a model only when a verifier can check that the answer is correct.

• Blog

  Rebuilding Netflix Video Processing Pipeline with Microservices

  Netflix split video encoding into Cosmos microservices so ladder design and encoding can change on their own.

• Blog

  Techniques for training large neural networks

  A field guide to data, pipeline, and tensor parallelism when one training step has to span many GPUs.

• Paper

  FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

  FlashAttention computes exact attention faster by cutting GPU memory traffic, without approximating the math.

• Paper

  Training language models to follow instructions with human feedback

  InstructGPT fine-tunes GPT-3 on human demonstrations and preferences so outputs follow what people ask for.

• Paper

  Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

  Showing a few worked examples with intermediate steps makes large models better at multi-step reasoning.

• Blog

  Introducing Triton: Open-source GPU programming for neural networks

  Triton is a Python-like language for writing GPU kernels that stay fast without hand-tuned CUDA.

• Paper

  LoRA: Low-Rank Adaptation of Large Language Models

  LoRA adapts a frozen model by training a small pair of low-rank matrices instead of every weight.

• Blog

  Scaling Kubernetes to 7,500 nodes

  OpenAI describes what broke, and what they changed, while running research jobs on a 7,500-node Kubernetes cluster.

• Paper

  Language Models are Few-Shot Learners

  GPT-3 shows a large language model can pick up a new task from a handful of examples in the prompt.

• Paper

  Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

  RAG looks up documents at generation time so the model can answer from sources, not only its weights.

• Blog

  Open-Sourcing Metaflow, a Human-Centric Framework for Data Science

  Netflix open-sourced Metaflow so data scientists can move a project from a laptop to production with less glue code.

• Paper

  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

  BERT pretrains a Transformer that reads both directions, then fine-tunes with one new layer.

• Blog

  AVA: The Art and Science of Image Discovery at Netflix

  AVA ranks frames inside a title so editors are not hunting through millions of images by hand.

• Paper

  Attention Is All You Need

  The Transformer drops recurrence and trains sequence models with attention alone.

• Paper

  Deep Residual Learning for Image Recognition

  Residual connections let very deep image networks train by learning what to add to the input.