Search Infrastructure and Software Engineering at Shopify: Index

I’m an engineering intern at Shopify, on the Sidekick and CX R&D team in Toronto since May 2026. We take Sidekick, Shopify’s AI assistant, and build the help tooling around it: a support copilot, and the software and search infrastructure it runs on. I spend most of my time on the search-infrastructure and software side, with some of the model-tuning work too.

These pages are my working notes, written as I learn the material. They lean into the parts I find most interesting, so they read like someone thinking out loud rather than teaching from a podium. Everything here is generalizable; nothing internal or under NDA.

Search and Retrieval

The deep one, and closest to my day-to-day. A ground-up map of how production search works, from a document on disk to a ranked result: the inverted index, TF-IDF and BM25, the IDF variants and the tokenization everything quietly depends on, then semantic search, hybrid fusion, reranking, the indexing pipeline, and serving at scale. I lean hardest on the infrastructure side, the batch jobs and the memory and the staging-to-serving handoff, since that’s where most write-ups stop and where the work actually lives. Most pages are backed by a small experiment over a real corpus, because I trust a measured number more than a formula I’ve only read.

Applied LLM Engineering

The model side: getting the assistant to behave for the help-tooling job, which is prompting, context design, and evaluation. Two patterns I worked through from the team’s public talks: returning tool instructions just-in-time with the tool result instead of stuffing them in the system prompt, and calibrating an unreliable LLM-as-judge into a signal you can actually trust.

Index

• Search and Retrieval. A ground-up map of how production search works: a document on disk to a ranked result in front of a user. Lexical and semantic retrieval, hybrid fusion, reranking, the indexing pipeline, and serving at scale.
• Applied LLM Engineering. Working notes on production LLM patterns: returning tool instructions just-in-time with the tool result, and calibrating an LLM-as-judge into a signal you can trust. Worked through from the Sidekick team's public talks.