Charles Frye
Building useful technology out of large neural networks
Actions
Charles teaches people to build data, ML, and AI applications. He got his PhD from the University of California, Berkeley, in 2020 for work on the geometry of neural network optimization. He has since worked as an educator and evangelist for neural network applications at Weights & Biases, Full Stack Deep Learning, and now Modal Labs.
What every AI engineer needs to know about GPUs
Every programmer needs to know a few things about hardware, like processors, memory, and disks. Due to AI systems' extreme demand for mathematical processing power, AI engineers need to know a few things about GPUs -- the world's most popular high-throughput mathematical co-processor.
In this talk, I will explain the fundamental engineering constraints and design decisions that shape GPUs and trace those up to some counter-intuitive facts about the performance characteristics of AI systems, with actionable insights for their deployers and consumers.
The low-latency inference engineering playbook
Inference optimization is a grind. The big wins like quantization affect model quality, triggering numerous, expensive quality evals. Both host and accelerator performance improvement is a game of inches. And the engines have more flags than an Erasmus alumni meetup in Brussels.
In this talk, I will present the playbook we've developed at Modal for optimizing inference deployments, like our Kimi-K3 Shared Endpoint (available via OpenRouter) and application-specific endpoints with and for customers like Decagon.
We focus in particular on speculative decoding, which uses inference to speed up inference. I'll explain the core intuition, the latest techniques, and the relevance of the bitter lesson, with reference to our work on state-of-the-art DFlash speculators.
How fast are LLM inference engines anyway?
Open weights models and open source inference servers have made massive strides in the year since we last got together at AIE World's Fair.
Where once we had only pirated LLaMA 2 weights and Transformers, we now have an embarrassment of riches. In fact, we have too many choices! What's an AI engineer looking to self-host inference to do?
In this session, we'll share our benchmarking results from hundreds of runs across models, frameworks, and hardware. We'll also share tips and tricks from working with teams deploying LLM inference at scale.
DIY LLMs
In this talk, AI Engineer Charles Frye will discuss the stack for running your own LLM inference services, from low-level hardware to high-level wrapper software.
Charles Frye
Building useful technology out of large neural networks
Actions
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top