views this week
Home
James
Triathlon
Fueling a six-hour ride
Getting better sleep
Sarnia to Barrie
Tracking load with CTL, ATL, and form
Zones, and why mine were wrong
Projects and Writing
Implementing a Go Engine in Rust with MCTS, RAVE, and Heuristic Rollouts
Engineering and Research
ML Infra, Neuralink
Imaging
Research, Firefox
Additional Work
Mozilla Engineering Talks + Networking
Instrumenting Firefox's Notification Permission Lifecycle with Glean
Removing 2,645 Lines of the Old Clear Data Dialog
SmartBlock Shims: Click-to-Load Placeholders for Blocked Tracker Content
Tracking-Parameter Stripping in Firefox's Copy Clean Link
Building Firefox's New-Tab Privacy Surface
Privacy Metrics Service
Privacy Metrics Component
Top Trackers Database
Nova Redesign & New Tab Integration
Flushing Live ContentBlockingLogs on Anti-Tracking DB Query
ETP Infrastructure: A Build-Time Pipeline for Firefox Privacy Configuration
Android ETP Pipeline
Sphinx ETP Extension
Private Browsing Cell Clarification
Pref Comment Cleanup
Strict Mode Fix
Gecko Security Patches
Defensive Range Checks in a Browser-Side Compression Decoder
Marking Browser Clipboard Writes as Sensitive in Private Contexts
Embed Link Preservation
Machine Learning Research: Cost-Aware Tracker Blocking in Firefox
Data Exploration and Visualization
Hand-Crafted Features and Tree Models
Learned URL Representations
Firefox ONNX Inference
Search, Shopify
Applied LLM Engineering
Calibrating an LLM Judge: Cohen's Kappa from 0.02 to 0.61
Just-in-Time Context: Moving Tool Instructions Out of the System Prompt
Search and Retrieval
Build Time and Query Time
Building the Index
Fusion and Reranking
Lexical Retrieval
Measuring a Relevance Change
Ranking on Evidence
Semantic Retrieval
Open Source Contributions
MarkUs
Killing an N+1 in the Grading Interface
Scheduled Assignment Visibility, End-to-End
Model-Layer Invariants and Tests as Contracts
PythonTA
AST Manipulation, Custom Checkers, and a Student-Facing Reporter
Course Notes
Machine Learning
Deep Learning
Backpropagation
Neural Networks
Transformers and Attention
Foundations
Bias-Variance Tradeoff and Bagging
Feature Engineering and Embeddings
Gaussian Discriminant Analysis
Gradient Boosting
Loss Functions and Optimization
Naive Bayes
Logistic Regression and Regularization
Decision Trees
Decision Trees
Experiment: Heart Disease Prediction
K-Nearest Neighbors
K-Nearest Neighbors
Experiment: MNIST Digit Classification
Linear Regression
Linear Regression
Gradient Descent
Experiment: Gradient Descent
Practice Problems
Large Language Models
Large Language Models
Retrieval Augmented Generation
Fine-Tuning and Parameter-Efficient Methods
Fine-Tuning and Parameter-Efficient Methods
Linguistics and Tokenization
Privacy and ML
Differential Privacy
Federated Learning
On-Device Inference
Privacy Attacks on ML Models
Projects
Training a Flux LoRA of My Face
Painting Classifier
Experiment: Painting Classifier
Feature Engineering Pipeline
Model Comparison: Logistic Regression vs Random Forest vs MLP
Search and Retrieval
Counting Words Smarter: TF, Length Normalization, and IDF
The Whole Pipeline: From Query to Ranked Results
What Search Is
Systems and Practice
ML Systems Design
Model Evaluation and Experiment Design
General Staff
Socratic Learning System
Building with AI
Linear Algebra
1: Re-imagining Matrices
2: Solving Linear Systems
3: Vectors in Euclidian Space
4: Matrix Operations
5: Matrix Transformations as Functions
6: Subspaces
7: Kernel and Image
8: Orthogonality and Projections
9: The Determinant
10: Eigenvalues and Eigenvectors
11: Diagonalization and Similarity
12: Orthogonal Diagonalization and the Spectral Theorem
13: Singular Value Decomposition
14: Portfolio Optimization
15: PCA in Finance
16: Covariance Estimation and Regularization
Statistics
STA238 Probability Statistics and Data Analysis II
Confidence intervals
Estimating a parameter
Hypothesis testing
Judging an estimator
Data Structures and Algorithms
Binary Search
Deques
Dictionary compositions
Dicts, defaultdict and Counter
Graphs and traversal
Heaps
PlainSpec
PlainSpec Practice
Tries
Philosophy
PHL217: Introduction to Continental Philosophy
Lukács
Recognition
Sartre
1,662 contributions in the last year @lxyhan
jameshan.net / Search, Shopify / Applied LLM Engineering /

Applied LLM Engineering

The model side of the team’s work: getting the Sidekick assistant to behave for the help-tooling job through prompting, context design, and evaluation. Two patterns I worked through from the team’s public talks, the counterpart to the search-infrastructure side I spend most of my time on.

Posts

Just-in-Time Context: Moving Tool Instructions Out of the System Prompt

Calibrating an LLM Judge: Cohen’s Kappa from 0.02 to 0.61

1 The model side of the [team's work](/shopify): getting the Sidekick assistant to behave for the help-tooling job through prompting, context design, and evaluation. Two patterns I worked through from the team's public talks, the counterpart to the [search-infrastructure](/shopify/search) side I spend most of my time on.
2
3 ## Posts
4
5 ### [Just-in-Time Context: Moving Tool Instructions Out of the System Prompt](/shopify/applied-llm-engineering/jit_context)
6
7 ### [Calibrating an LLM Judge: Cohen's Kappa from 0.02 to 0.61](/shopify/applied-llm-engineering/llm_as_judge)