Library

We are often asked about the resources we can recommend. This is a non-exhaustive list of resources that we have found useful. If you have any suggestions, please raise a PR.

Prompting

• Learn Prompting, docs page
• The Prompt Report: A Systematic Survey of Prompting Techniques, paper on arxiv, summary in tweets

LLMOps

• What We Learned from a Year of Building with LLMs, by Eugene Yan, Bryan Bischof, Charles Frye, Hamel Husain, Jason Liu and Shreya Shankar, Part 1, Part 2, Part 3
• Traceability and Observability in Multi-Step LLM Systems, webinar
• Data Flywheels for LLM Applications, blog post, by Shreya Shankar
• Latency optimization, cookbook, by OpenAI

Evaluation

• Evaluating the Effectiveness of LLM-Evaluators (aka LLM-as-Judge), blog post, by Eugene Yan

LLMs

• Intro to Large Language Models talk by Andrej Karpathy

News / Research

• Last Week in AI podcast, weekly summary AI news and research
• Latent Space podcast, deep dives and interview episodes

Libraries / Tools

• Instructor, library for structured outputs with LLMs.
• Vercel AI SDK, TypeScript toolkit designed to help developers build AI-powered applications with React, Next.js, Vue, Svelte, Node.js, and more.
• LiteLLM: Use 100+ LLMs as a drop in replacement for OpenAI GPT models, either library or proxy.
• Popular frameworks to get started quickly with RAG/Chains/Agents: LangChain, LlamaIndex, Haystack

Misc

• Stratechery newsletter/podcast, tech/business strategy deep dives and news, many episodes related to AI/Labs