Search Infrastructure and Software Engineering at Shopify: Index

I’m an engineering intern at Shopify, on the Sidekick and CX R&D team in Toronto since May 2026. We take Sidekick, Shopify’s AI assistant, and build the help tooling around it: a support copilot, and the software and search infrastructure it runs on. I spend most of my time on the search-infrastructure and software side, with some of the model-tuning work too.

These pages are my working notes, written as I learn the material. They lean into the parts I find most interesting, so they read like someone thinking out loud rather than teaching from a podium. Everything here is generalizable; nothing internal or under NDA.

Search and Retrieval

The deep one, and closest to my day-to-day. A ground-up map of how production search works, from a document on disk to a ranked result: the inverted index, TF-IDF and BM25, the IDF variants and the tokenization everything quietly depends on, then semantic search, hybrid fusion, reranking, the indexing pipeline, and serving at scale. I lean hardest on the infrastructure side, the batch jobs and the memory and the staging-to-serving handoff, since that’s where most write-ups stop and where the work actually lives. Most pages are backed by a small experiment over a real corpus, because I trust a measured number more than a formula I’ve only read.

Applied LLM Engineering

The model side: getting the assistant to behave for the help-tooling job, which is prompting, context design, and evaluation. Two patterns I worked through from the team’s public talks: returning tool instructions just-in-time with the tool result instead of stuffing them in the system prompt, and calibrating an unreliable LLM-as-judge into a signal you can actually trust.

Index

• Search and Retrieval. How to stand up a modern search system: the two clocks that govern every design decision, lexical and semantic retrieval and why you need both, fusion and reranking, the build pipeline that produces what serving reads, and how to know whether a change helped.
• Applied LLM Engineering. Working notes on production LLM patterns: returning tool instructions just-in-time with the tool result, and calibrating an LLM-as-judge into a signal you can trust. Worked through from the Sidekick team's public talks.