Luke Francl
GitHub
San Francisco, California, United States
Actions
Luke Francl works on code search at GitHub. He's excited to build software that makes developers happier and more productive. Prior to joining GitHub, he worked at a search-as-a-service startup, as a freelance developer, and spent a lot of time doing XML sit-ups in the Java world. He lives in San Francisco with his family.
Lessons from building GitHub code search
In this talk, I'll share some lessons we learned building a high-performance code search engine, designed to meet GitHub's large scale. GitHub code search is the world's largest publicly available code search engine, with more than 60 million repositories and over 160 TB of content indexed. To build it, we had to turn the unique content-addressable nature of Git repositories to our advantage. I'll cover the key strategies we used, including using deduplication and repository similarity to reduce indexing workload, full index compaction to remove deleted documents, multiple levels of sharding, and load balancing. Come discover how we turned code search from a frustrating experience to a powerful feature for our users.
Integrating Rust and Go for GitHub code search
GitHub's new code search engine, Blackbird, is written in Rust because it provides safety and high performance. But GitHub's backend services are mostly written in Go, so we implemented the document ingest and query API in Go to take advantage of existing infrastructure. Learn how we bridged the gap between Go and Rust, starting with RPCs, then using FFI to share core code written in Rust. We'll share our experience using Rust code from Go, and how we plan to move more and more of our code into Rust over time.
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top