Archive - Interconnects

Llama 3: Scaling open LLMs to AGI

Llama 3 shows that scaling won't be a limit for open LLM progress in the near future.

Apr 18 •

Stop "reinventing" everything to solve alignment

Integrating some non-computing science into reinforcement learning from human feedback (RLHF) can give us the models we want.

Apr 17 •

The end of the “best open LLM”

Modeling the compute versus performance tradeoff of many open LLMs.

Apr 15 •

We disagree on what open-source AI should mean

How to read what multiple people mean by the word openness and see through the PR speak.

Apr 3 •

March 2024

DBRX: The new best open model and Databricks’ ML strategy

Databricks’ new model is surpassing the performance of Mixtral and Llama 2 70B while still being in a size category that's reasonably accessible.

Mar 28 •

Evaluations: Trust, performance, and price (bonus, announcing RewardBench)

Evaluation is not only getting harder with modern LLMs, it’s getting harder because it means something different.

Mar 20 •

Model commoditization and product moats

Where moats are tested now that so many people have trained GPT4 class models. Claude 3, Gemini 1.5, Inflection 2.5, and Mistral Large are here to…

Mar 13 •

The koan of an open-source LLM

A proposal for a new definition of an “open-source” LLM and why no definition will ever just work.

Mar 6 •

Interviewing Louis Castricato of Synth Labs and Eleuther AI on RLHF, Gemini Drama, DPO, founding Carper AI, preference data, reward models…

An interview I've wanted to bring you for a while.

Mar 4 •

1:26:27

February 2024

How to cultivate a high-signal AI feed

Basic tips on how to assess inbound ML content and cultivate your news feed.

Feb 28 •

Google ships it: Gemma open LLMs and Gemini backlash

Google rejoins the open model party and gets some backlash for a frequent problem for generative AI.

Feb 22 •

10 Sora and Gemini 1.5 follow-ups: code-base in context, deepfakes, pixel-peeping, inference costs, and more

The cutting edge technical discussions beneath the wow factor.

Feb 19 •

#nojs-banner { position: fixed; bottom: 0; left: 0; padding: 16px 16px 16px 32px; width: 100%; box-sizing: border-box; background: red; color: white; font-family: -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-size: 13px; line-height: 13px; } #nojs-banner a { color: inherit; text-decoration: underline; } This site requires JavaScript to run correctly. Please turn on JavaScript or unblock scripts