Dictionary and key terms

520 key terms from every track, defined in a sentence, with related terms and the problems that teach them.

520 terms

1cycleLearning Rate: The 1cycle policy (fast.ai, `OneCycleLR`) warms the learning rate up from `max_lr / div` to `max_lr` over the first `pct_start` of training, then anneals it down to `max_lr / (div * final_div)` with a cosine.; One-Cycle Learning Rate Policy →
20 tokens per parameterScaling Laws: The Chinchilla result says compute-optimal training uses roughly 20 tokens per parameter.; Related:Chinchilla; Training Compute and Chinchilla Scaling →
3 per pagePagination: An API lists orders newest first (by `order_date` descending, then `id` descending), 3 per page.; Paginating Results →
30 daysWindow Functions: Return `name`, `first_order`, `second_order` (dates) and `days_between` (a whole number of days) for every customer whose second order was within 30 days (inclusive) of their first.; Related:second order within 30 days; Customers Who Came Back Within 30 Days →
30 or moreDates: Using the latest order date in the data as "today", return the `name` and `days_quiet` of buying customers whose last order was 30 or more days before it.; Customers Quiet for 30+ Days →
accumulateCNNs: With overlapping windows (stride < kernel), one input can win several windows and must accumulate their gradients.; Max-Pooling Backward Pass →
accumulated attentionInference: It keeps the tokens with the highest accumulated attention, plus a window of the most recent tokens.; Related:H2O; Heavy-Hitter KV Cache Eviction (H2O) →
Activation checkpointingMemory: Activation checkpointing stores activations only at every k-th layer and recomputes the rest during the backward pass, trading compute for memory.; Plan Activation Checkpointing →
AdaBoostEnsembles: AdaBoost trains weak learners in sequence, each focusing on the examples the previous ones got wrong by reweighting them.; Related:reweighting; AdaBoost with Decision Stumps →
AdadeltaOptimizers: Adadelta removes Adagrad's ever-shrinking step and even the learning rate.; Adadelta →
AdamWOptimization: AdamW (Loshchilov & Hutter, 2019) decays the weights directly instead.; AdamW: Decoupled Weight Decay →
adjacentDynamic Programming: You can't take from two adjacent houses.; House Robber (DP on a Line and a Circle) →
adjusted Rand indexClustering: The adjusted Rand index compares two clusterings, correcting for chance: random labelings score about 0 and identical ones score 1 (label names don't matter).; Adjusted Rand Index →
agentAgents: An agent is a loop: the model either calls a tool or gives a final answer, tool results are fed back, and the loop repeats until it's done.; Build a Mini Agent Loop →
aggressiveOnline Learning: The passive-aggressive algorithm learns from one example at a time: if the example is classified with margin ≥ 1 it stays passive; otherwise it makes the smallest update that fixes it (aggressive), capped by `C` (PA-I).; Related:passive passive-aggressive; Passive-Aggressive Online Classifier →
all alignmentsSpeech: CTC (Graves et al., 2006) sums the probability over all alignments that collapse to the target: merge repeated symbols, then remove blanks.; Related:CTC; CTC Loss: The Forward Algorithm →
ALSRecommenders: ALS alternates: fix V and solve a ridge regression for each user, then fix U and solve for each item.; Recommendations with Alternating Least Squares →
anagramsDicts: Two words are anagrams if they use exactly the same letters, like `"listen"` and `"silent"`.; Group Anagrams →
ancestorsInference: Each draft token may only attend to the verified prefix, its ancestors in the tree, and itself.; Related:tree; Tree Attention for Speculative Drafts →
anchor boxesObject Detection: Two-stage and SSD-style detectors place anchor boxes at every feature-map cell, in several sizes and aspect ratios, then predict offsets from them.; Generating Anchor Boxes →
any number of timesBacktracking: Given distinct positive `candidates` and a `target`, return all unique combinations that sum to the target, where each candidate may be used any number of times.; Related:unique combinations; Combination Sum →
append-only ledgerClasses: Model a bank account properly: money uses `decimal.Decimal` (never floats), and the balance is derived from an append-only ledger of transactions, not stored separately.; Bank Account Ledger →
AprioriAssociation Rules: Apriori grows itemsets one item at a time, using the fact that every subset of a frequent set must itself be frequent.; Frequent Itemsets with Apriori →
articulation pointGraphs: In a network, a bridge is a link whose failure disconnects part of the graph, and an articulation point is a node whose failure does.; Related:bridge low-link; Bridges and Articulation Points →
as they arriveStreaming: When a model streams JSON, the UI wants to show fields as they arrive, but `json.loads` fails on every incomplete prefix.; Related:prefix; Parse Partial JSON While Streaming →
at equal lengthEvaluation: AlpacaEval 2's length-controlled win rate fits a logistic model of the judge's preference on the length difference, then reports the predicted win rate at equal length.; Related:length-controlled win rate longer; Length-Controlled Win Rates →
at least 150Joins: A big spender has total spend of at least 150.; Big Spenders per Country, Including Zeros →
at least 2 ordersGroup By: Return `name`, `order_count` and `total_spent` for every customer with at least 2 orders, sorted by `total_spent` from highest to lowest.; Repeat Customers →
at least 3Window Functions: For each country with at least 3 orders, drop its single smallest and single largest order (by `total`, ties by `id`) and average the rest.; Trimmed Mean per Country →
at least oneAggregation: Return the `name` of customers who have at least one order and whose every order total is above 75.; Customers Whose Every Order Exceeds 75 →
at least twiceAggregation: What percentage of customers who have ordered went on to order at least twice?; Repeat Purchase Rate →Longest Repeated Substring with Rolling Hashes →
at least twoWindow Functions: Find customers with at least two orders where every order (by date, ties by `id`) is strictly larger than their previous one.; Related:strictly larger; Average Days Between Orders →Customers Whose Orders Keep Growing →Countries with More Than One Customer →
attention maskData Loading: A collate function pads them and builds the attention mask so the model ignores padding.; Pad and Collate Variable-Length Batches →
autocorrelation function (ACF)Time Series: The autocorrelation function (ACF) measures how correlated a series is with a lagged copy of itself: (the standard estimator, dividing by the full-series variance).; Autocorrelation and Seasonality Detection →
average country revenueWindow Functions: Return each country's total `revenue` and `pct_vs_avg`: how far it is above or below the average country revenue, as a percentage rounded to 1 decimal place.; Each Country Against the Average →
average order valueAOVStatistics: Did the new checkout flow change average order value?; Related:Welch's t-test; Welch's t-Test for Two Means →
Average order value (AOV)Aggregation: Average order value (AOV) is one of the first numbers an e-commerce team looks at.; Average Order Value by Country →
AWQQuantization: AWQ (Lin et al., 2024) scales those weight columns up before quantising and divides the activations by the same factor, which is mathematically a no-op in full precision but protects the important weights from rounding error.; Related:inputs are large; Activation-Aware Weight Quantisation (AWQ) →
backTwo Pointers: Fill from the back so you never overwrite values you still need.; Related:in place; Merge Two Sorted Arrays In Place →
backtrackingBacktracking: This is the classic backtracking problem.; N-Queens →
BanditsExperimentation: Bandits shift traffic towards the winner as evidence comes in.; Related:UCB1; Multi-Armed Bandit: UCB1 →
Batch-hard miningMetric Learning: Batch-hard mining (Hermans et al., 2017) uses, for each anchor in the batch, its farthest positive and closest negative.; Related:triplet loss; Triplet Loss with Batch-Hard Mining →
Beam searchLLM: Beam search keeps the `beam_width` best partial sequences at every step.; Beam Search Decoding →
bias²Model Selection: The expected test error of a model splits into bias² (how far the average prediction is from the truth), variance (how much predictions move between training sets) and irreducible noise.; Related:noise variance; Bias–Variance Decomposition by Simulation →
bidirectionalRNN: A bidirectional RNN runs one vanilla RNN left-to-right and another right-to-left, then concatenates their hidden states at each step, so every position sees the whole sequence.; Bidirectional RNN Forward Pass →
binary cross-entropyClassification: Train a binary logistic regression with gradient descent on the mean binary cross-entropy.; Logistic Regression from Scratch →
Binary quantisationEmbeddings: Binary quantisation keeps just the sign of each dimension, giving 32× less memory, and compares vectors by Hamming distance, which CPUs compute very fast.; Related:Hamming distance rescore sign; Binary Embeddings and Hamming Search →
binary search over the capacityBinary Search: Feasibility is monotonic (bigger capacity → fewer days), so binary search over the capacity.; Related:Example in order smallest daily capacity; Binary Search on the Answer: Batch Capacity →
bipartiteGraphs: That's asking whether a graph is bipartite, i.e. two-colourable.; Two-Colouring a Graph (Bipartite Check) →
BLEUEvaluation: BLEU scores machine translation (and other generation tasks) by n-gram overlap with references.; BLEU Score from Scratch →
blocking pairMatching: Such a pair is a blocking pair, and a matching without one is stable.; Related:stable; Stable Matching (Gale–Shapley) →
blocksInference: vLLM's PagedAttention stores each sequence's KV cache in fixed-size blocks, like virtual memory pages, so memory isn't wasted on reserved-but-unused space.; Related:vLLM's PagedAttention; Paged KV-Cache Block Allocator →
Bloom filterData Structures: A Bloom filter answers "have I seen this before?" using a tiny bit array.; Build a Bloom Filter →
bootstrap samplesEnsembles: Bagging trains many models on bootstrap samples (drawn with replacement) and averages them, which reduces variance.; Bagging: Bootstrap Aggregation →
bridgeGraphs: In a network, a bridge is a link whose failure disconnects part of the graph, and an articulation point is a node whose failure does.; Related:articulation point low-link; Bridges and Articulation Points →
Byte-Pair EncodingBPETokenization: GPT-style tokenisers are trained with Byte-Pair Encoding: repeatedly find the most frequent adjacent pair of symbols and merge it into a new symbol.; Byte-Pair Encoding: One Merge Step →Train a BPE Tokenizer →
CalibrationEvaluation: Calibration measures whether predicted probabilities match reality.; Model Calibration: Expected Calibration Error →
cancelledHeaps: Background job runners (and LLM request schedulers) need a queue where the most urgent task runs first, where tasks of equal priority run in arrival order, and where queued tasks can be cancelled.; Priority Task Queue with Cancellation →
cascadeRouting: A cascade sends every query to a cheap model first and escalates to an expensive one only when the cheap model isn't confident.; Cheap-to-Expensive Model Cascades →
causalAttention: Batched transformers combine two masks: a causal mask (no peeking ahead) and a padding mask (ignore pad tokens in each sequence).; Related:padding; Building Attention Masks →
centreEmbeddings: Word2Vec learns word vectors by training each centre word to predict its context words.; Related:context negative sampling; Word2Vec Skip-Gram with Negative Sampling →
centroidRouting: A cheap way to route user messages ("billing", "tech support", "sales") without an LLM call: embed a few example queries per route, average them into a centroid, and send each new query to the nearest centroid.; Route Queries by Embedding Centroids →
changesCausal Inference: A marketing campaign should target people it changes, not people who'd buy anyway.; Related:Qini curve Uplift models; Uplift Modelling and the Qini Curve →
chi-square test of independenceStatistics: The chi-square test of independence checks whether two categorical variables are associated, using a contingency table of counts.; Chi-Square Test of Independence →
ChinchillaScaling Laws: The Chinchilla result says compute-optimal training uses roughly 20 tokens per parameter.; Related:20 tokens per parameter; Training Compute and Chinchilla Scaling →
chosenRLHF: In RLHF, a reward model learns from human preference pairs: it should give the chosen response a higher score than the rejected one.; Related:rejected reward model; Reward Model Training Loss (Bradley–Terry) →
churns in month MPandas: It churns in month M if `end` falls within M.; Monthly Churn Rate from Subscriptions →
circuit breakerReliability: A circuit breaker stops calling it for a while, then cautiously tries again.; Circuit Breaker →
claimsRAG: RAG evaluators (RAGAS, TruLens) split an answer into claims, score how strongly each source passage supports each claim (with an NLI model or an LLM judge), and aggregate.; Faithfulness from a Claim–Source Support Matrix →
Cohen's dStatistics: Cohen's d is the difference in means measured in pooled standard deviations: with sample variances (`ddof=1`).; Related:Effect size; Effect Size: Cohen's d →
ColBERTRetrieval: ColBERT keeps one embedding per token and scores with MaxSim: for each query token, find its best-matching document token, and add up those maxima.; Related:MaxSim per token; Late Interaction Scoring (ColBERT MaxSim) →
Conformal predictionUncertainty: Conformal prediction wraps any model with intervals that have a guaranteed coverage of at least 1 − α, assuming only that the data points are exchangeable.; Split Conformal Prediction →
consecutiveHashing: Given an unsorted list of integers, find the length of the longest run of consecutive values (`[100, 4, 200, 1, 3, 2]` → `[1, 2, 3, 4]` → 4).; Longest Consecutive Sequence →
consecutive calendar daysWindow Functions: Find runs of consecutive calendar days with at least one order (an "island").; Gaps and Islands: Consecutive Order Days →
constrained decodingStructured Output: To guarantee an LLM outputs one of a fixed set of labels (`"positive"`, `"negative"`, `"neutral"`), constrained decoding masks out every token that can't lead to a valid answer.; Constrained Decoding with a Token Trie →
contextEmbeddings: Word2Vec learns word vectors by training each centre word to predict its context words.; Related:centre negative sampling; Word2Vec Skip-Gram with Negative Sampling →
Continuous batchingInference: Continuous batching (vLLM, TGI) adds new requests as soon as any slot frees up.; Simulate Continuous Batching →
controlCausal Inference: To estimate the effect of a treatment on the treated (ATT) from observational data, match each treated unit to the control unit with the closest propensity score (with replacement; ties to the lower index), then average the outcome differences.; Nearest-Neighbour Propensity Score Matching →
copies spans from the promptInference: When an LLM edits code or summarises a document, its output often copies spans from the prompt.; Prompt Lookup Decoding →
copyPandas: Given a DataFrame `df` with columns `region`, `product` and `sales`, implement `add_shares(df)` that returns a copy with two new columns: Keep the original row order.; Share of Group Total with transform →
cosine decayTraining: Almost every modern LLM is trained with a linear warmup followed by cosine decay: start small so early updates don't blow up, peak, then glide down.; Related:linear warmup; Learning Rate: Warmup + Cosine Decay →
coverageEvaluation: A model that can say "I don't know" trades coverage (how often it answers) for risk (how often its answers are wrong).; Related:risk; Abstain When Unsure: Risk–Coverage Curves →
cross-correlationCNNs: Implement one with stride 1 and no padding (actually a cross-correlation, as in every deep learning library): for `x: (N, C_in, L)`, `W: (C_out, C_in, K)` and `b: (C_out,)`, giving `y: (N, C_out, L - K + 1)`.; 1-D Convolution Forward and Backward →
CTCSpeech: CTC (Graves et al., 2006) sums the probability over all alignments that collapse to the target: merge repeated symbols, then remove blanks.; Related:all alignments; CTC Loss: The Forward Algorithm →
cumulativeOperations: Classify products by their share of revenue: sort by revenue (descending, ties by name), compute each product's cumulative share including itself, then: The first product is always `A`.; ABC Inventory Analysis →
Cumulative gainMetrics: Cumulative gain at the top `q` fraction is the share of all positives captured there; lift is gain divided by `q`.; Related:lift; Cumulative Gains and Lift →
Cursor paginationAPIs: Cursor pagination instead says "give me items after this one".; Cursor-Based Pagination →
CUSUMMonitoring: CUSUM (cumulative sum control charts) detects small, persistent shifts in a process mean much faster than checking each point against a threshold.; Detect Shifts with CUSUM →
data pointClustering: k-medoids is like k-means, but each cluster centre must be an actual data point (a medoid).; Related:k-medoids; k-Medoids Clustering →
datamarkingPrompt Injection: With datamarking, every run of whitespace in the untrusted text is replaced with a special marker character, and the system prompt explains that marked text is data, never instructions.; Related:Spotlighting; Spotlighting Untrusted Text with Datamarking →
date spineRecursive CTE: Build a date spine: every month from the first order's month to the last order's month, with revenue `0` for months with no orders.; Monthly Revenue with a Date Spine →
Davies-Bouldin indexClustering: The Davies-Bouldin index rates a clustering by how compact clusters are relative to how far apart they are.; Davies-Bouldin Index →
day of monthParsing: The fields are minute (0–59), hour (0–23), day of month (1–31), month (1–12) and day of week (0–6, with 0 = Sunday).; Related:day of week hour minute month; Does This Cron Schedule Fire Now? →
day of weekParsing: The fields are minute (0–59), hour (0–23), day of month (1–31), month (1–12) and day of week (0–6, with 0 = Sunday).; Related:day of month hour minute month; Does This Cron Schedule Fire Now? →
Day-N retentionRetention: Day-N retention: of users who signed up on day 0, what fraction were active exactly N days later?; N-Day Retention Curve →
DBSCANClustering: DBSCAN finds clusters of any shape and labels outliers as noise, using just two parameters: a radius `eps` and `min_samples`.; Related:noise; DBSCAN Density-Based Clustering →
decision stumpsEnsembles: Build a tiny random forest whose trees are decision stumps (one split each), to see the two sources of randomness.; Random Forest of Stumps →
decryptsStrings: Also implement `crack(ciphertext)`: try all 26 shifts and return the shift (0–25) that decrypts the text into the version with the most occurrences of the letter `e` (case-insensitive); ties go to the smallest shift.; Caesar Cipher →
dedupeAPIs: Batch a corpus to use as few requests as possible while respecting both limits, and dedupe identical texts so you don't pay twice.; Batch Texts for an Embeddings API →
dependency injection containerDesign Patterns: A dependency injection container builds objects and their dependencies for you.; A Small Dependency Injection Container →
depthsLong Context: The needle-in-a-haystack test hides one fact (the needle) at different depths inside contexts of different lengths, then asks the model to retrieve it.; Related:lengths; Needle-in-a-Haystack Evaluation →
depthwiseCNNs: A depthwise convolution gives each input channel its own k × k filter and doesn't mix channels.; Related:pointwise; Depthwise and Pointwise Convolutions →
diameterTrees: The diameter is the number of edges on the longest path between any two nodes (it need not pass through the root).; Related:edges; Diameter of a Binary Tree →
differentInference: Find pairs of different orders placed on the same date.; Serve Many LoRA Adapters in One Batch →Orders Placed on the Same Day →
disagreeEvaluation: What matters is the questions where they disagree.; Is Model B Really Better? (McNemar's Test) →
don'tPandas: A meeting ending at 10 and one starting at 10 don't overlap.; Related:minimum number of rooms; pandas: Anti-Joins and Mismatched Keys →Minimum Meeting Rooms →
downDynamic Programming: A robot starts at the top-left of a grid and can only move right or down to reach the bottom-right.; Related:right; Unique Grid Paths with Obstacles →
DPODirect preference optimisationAlignment: DPO aligns a language model with human preferences without training a separate reward model.; Related:policy reference; Direct Preference Optimization (DPO) Loss →
DropoutRegularization: Dropout randomly zeroes activations during training so the network can't rely on any single unit.; Inverted Dropout →
Dutch national flagTwo Pointers: Dijkstra's Dutch national flag problem: rearrange a list in place, in one pass, so that every value below a pivot comes first, then values equal to it, then values above it.; Related:in place; Three-Way Partitioning (Dutch National Flag) →
each fieldEvaluation: Whole-record exact match is too harsh: score each field.; Evaluate Structured Extraction Field by Field →
earliest qualifyingProduct Analytics: Each step must happen at or after the previous step's earliest qualifying time.; Related:reaches step k; Funnel Conversion from Event Logs →
Early stoppingTraining: Early stopping watches the validation loss and stops once it hasn't improved for a while.; Related:Example; Early Stopping Tracker →
edgesTrees: The diameter is the number of edges on the longest path between any two nodes (it need not pass through the root).; Related:diameter; Diameter of a Binary Tree →
edit distanceDynamic Programming: The edit distance between two strings is the minimum number of single-character insertions, deletions or substitutions needed to turn one into the other.; Related:insertions, deletions or substitutions; Edit Distance (Levenshtein) →
Effect sizeStatistics: Effect size tells you whether it's big enough to matter.; Related:Cohen's d; Effect Size: Cohen's d →
Elo ratingsEvaluation: Chatbot Arena ranks LLMs from head-to-head human votes using Elo ratings.; Elo Ratings for a Model Arena →
embeddingsRAG: Retrieval-Augmented Generation (RAG) starts by finding the documents whose embeddings are closest to the query embedding.; Related:semantic cache; Embedding Retrieval with Cosine Similarity →Semantic Cache for LLM Responses →
emitsDesign: Decouple parts of an app with events: the code that emits `"user_signed_up"` doesn't need to know who's listening.; Event Emitter (Pub/Sub) →
endLinked Lists: Remove the `n`-th node from the end of a linked list in one pass and return the head.; Related:one pass; Remove the N-th Node from the End →
end to endBackprop: Train a tiny network end to end with your own forward pass, backward pass and gradient descent.; Train an MLP on XOR with Manual Backprop →
evenStrings: A palindrome needs every character to appear an even number of times, except at most one (the middle).; Palindrome Permutations →
every ticket exactly onceGraphs: Given airline tickets `(from, to)`, reconstruct the itinerary that uses every ticket exactly once, starting from `"JFK"`.; Related:lexicographically smallest; Reconstructing an Itinerary (Eulerian Path) →
EWMATime Series: An EWMA smooths a noisy series by giving recent points more weight: Missing values: if `x[t]` is `NaN`, carry the previous smoothed value forward (`s_t = s_{t-1}`).; Related:Missing values:; Exponentially Weighted Moving Average →
exact prefixPrompt Caching: Provider prompt caches (and vLLM's prefix cache) only reuse an exact prefix.; Lay Out Prompts for Cache Hits →
exactly oneAggregation: Return the `country` and that customer's `name` for countries where exactly one customer has placed orders, sorted by `country`.; Countries with Exactly One Buying Customer →
exactly zeroRegularization: The L1 penalty `λ‖w‖₁` drives many weights to exactly zero, which does feature selection.; Sparse Logistic Regression with Proximal Gradient →
ExampleTraining: Example (patience 2): losses `1.0, 0.8, 0.85, 0.9` → stop on the 4th call; `best = 0.8`, `best_epoch = 1`.; Related:binary search over the capacity Early stopping in order smallest daily capacity; Early Stopping Tracker →Binary Search on the Answer: Batch Capacity →
Expand around centresStrings: Expand around centres: every palindrome has a centre that's either a single character (odd length) or between two characters (even length).; Longest Palindromic Substring →
Expectation–MaximisationClustering: It's fitted by Expectation–Maximisation.; Related:Gaussian mixture model; Gaussian Mixture Model: One EM Step →
exponential moving averageTraining: Keeping an exponential moving average (EMA) of model weights during training gives a smoother, often better model for evaluation.; Exponential Moving Average of Weights →
failure functionStrings: KMP is O(n + m): it precomputes, for each prefix of the pattern, the length of the longest proper prefix that is also a suffix (the failure function, or LPS array), so it never re-examines text characters.; Related:KMP; Knuth–Morris–Pratt String Search →
falling fastestOptimization: Leslie Smith's LR range test (popularised by fast.ai's `lr_find`) trains briefly while raising the learning rate exponentially, then picks a rate where the loss is falling fastest.; Related:LR range test; Learning-Rate Range Test →
February 2024Filtering: Return the `id`, `order_date` and `total` of every order placed in February 2024 (both ends inclusive), sorted by `order_date`, then `id`.; Orders in a Date Range →
Fenwick treeData Structures: A Fenwick tree supports point updates and prefix sums in O(log n) with just an array and some bit tricks: `i & -i` is the lowest set bit of i.; Fenwick Tree (Binary Indexed Tree) →
filterLLM: Before sampling the next token, LLM decoders usually filter the probability distribution.; Top-k & Top-p (Nucleus) Filtering →
final answerEvaluation: Math benchmarks like GSM8K grade only the final answer buried at the end of a long reasoning trace.; Extract the Final Answer from Chain-of-Thought →
first few tokensInference: A sliding-window KV cache that drops the oldest tokens makes LLM quality collapse, because models dump a lot of attention onto the first few tokens ("attention sinks").; Related:StreamingLLM; Streaming KV Cache with Attention Sinks →
first orderWindow Functions: How many customers had placed their first order by the end of each month?; Cumulative Customer Count by Month →
first order within 30 daysCohorts: What share of each month's sign-ups placed their first order within 30 days of signing up?; 30-Day Conversion by Signup Cohort →
flatDicts: Environment variables, CSV exports and metrics systems need flat keys like `db.host`, while code wants nested dicts.; Flatten and Unflatten Nested Dicts →
float16Mixed Precision: Training in float16 is fast, but small gradients underflow to zero.; Related:scale underflow; Dynamic Loss Scaling for Mixed Precision →
for that customerWindow Functions: Which orders were unusually large for that customer?; Related:greater than; Orders Above the Customer's Own Average →
FP8Quantization: Hopper and Blackwell GPUs train and serve models in FP8.; Simulate FP8 (E4M3) Quantisation →
Gaussian mixture modelGMMClustering: A Gaussian mixture model gives soft ones: each point belongs to each cluster with some probability.; Related:Expectation–Maximisation; Gaussian Mixture Model: One EM Step →
Gaussian processBayesian: A Gaussian process puts a prior over functions, and conditioning on data gives a posterior mean and variance at every input, so you get calibrated uncertainty for free.; Related:mean and variance; Gaussian Process Regression →
generatorRecursion: Write a generator `flatten(items)` that yields every non-list element of an arbitrarily nested list, in order.; Flatten Arbitrarily Nested Lists →
Gini coefficientStatistics: The Gini coefficient is twice the area between that curve and the line of perfect equality: 0 means everyone spends the same, and values near 1 mean one customer has nearly everything.; Related:Lorenz curve; Inequality: Gini Coefficient and Lorenz Curve →
global average poolingCNN: Modern CNNs and Vision Transformers end with a tiny head: global average pooling collapses each feature map to one number, then a linear layer produces class scores.; Global Average Pooling Classification Head →
Gradient accumulationTraining: Gradient accumulation sums gradients over micro-batches and steps the optimiser once.; Gradient Accumulation →
Gradient boostingBoosting: Gradient boosting builds a strong model from many weak ones: each new model fits the residuals, the errors left by the models so far.; Related:residuals; Gradient Boosting with Decision Stumps →
Gradient clippingTraining: Gradient clipping rescales all gradients together when their combined size is too large.; Gradient Clipping by Global Norm →
grand total rowUnion: Return revenue by `country` and `month` (`YYYY-MM`), plus a subtotal row per country (month shown as `'All'`) and a grand total row (country and month both `'All'`), with columns `country`, `month`, `revenue`.; Related:subtotal row per country; Subtotals and a Grand Total →
greater thanWindow Functions: Return `id` (order id), `name`, `total` and `customer_avg` (that customer's average order total, rounded to 2 decimal places) for every order whose total is greater than its customer's average.; Related:for that customer; Orders Above the Customer's Own Average →
green listSafety: (2023) watermark nudges an LLM to prefer a pseudo-random green list of tokens at each step, chosen by hashing the previous token.; Related:z-test; Detect an LLM Text Watermark →
Group-wiseQuantization: Group-wise quantisation (as in GPTQ, AWQ and llama.cpp's Q4 formats) gives every group of, say, 32 consecutive weights in a row its own scale.; Related:own scale; Group-Wise INT4 Quantisation →
grouped-query attentionTransformers: In grouped-query attention, many query heads share fewer key/value heads.; Grouped-Query Attention (GQA) →
GroupNormNormalization: GroupNorm (Wu & He, 2018) normalises each sample on its own, over groups of channels.; Related:groups of channels; Group Normalisation →
groups of channelsNormalization: GroupNorm (Wu & He, 2018) normalises each sample on its own, over groups of channels.; Related:GroupNorm; Group Normalisation →
GRPORLHF: GRPO, popularised by DeepSeek-R1 for training reasoning models, drops PPO's learned value network.; GRPO Group-Relative Advantages →
GRUGated recurrent unitRNN: The GRU is a lighter alternative to the LSTM, with two gates and no separate cell state.; GRU Cell Forward Pass →
H2OInference: H2O (Zhang et al., 2023) keeps the KV cache small by observing that a few "heavy-hitter" tokens receive most of the attention.; Related:accumulated attention; Heavy-Hitter KV Cache Eviction (H2O) →
Hamming distanceEmbeddings: Binary quantisation keeps just the sign of each dimension, giving 32× less memory, and compares vectors by Hamming distance, which CPUs compute very fast.; Related:Binary quantisation rescore sign; Binary Embeddings and Hamming Search →
hashing trickFeatures: The hashing trick maps each feature straight to one of `n_features` columns with a hash, with no vocabulary needed.; Related:sign; The Hashing Trick →
height-balancedTrees: A binary tree is height-balanced if, at every node, the heights of the left and right subtrees differ by at most 1.; Check if a Tree Is Height-Balanced →
heuristicGraphs: A\* finds shortest paths faster than Dijkstra by using a heuristic: an estimate of the remaining distance that never overestimates it.; A* Pathfinding on a Grid →
hidden Markov modelHMMSequence Models: A hidden Markov model has hidden states that follow a Markov chain and emit observations.; Hidden Markov Models: Forward and Viterbi →
highest firstWindow Functions: Return each order's `id`, `total` and `price_rank`: its `DENSE_RANK` by total, highest first, so equal totals share a rank and no ranks are skipped.; Dense Rank of Order Totals →
highest total log-probabilityTokenizers: SentencePiece's Unigram tokenizer (used by T5, ALBERT and XLNet) gives each vocabulary piece a log-probability, and tokenizes a string by choosing the segmentation with the highest total log-probability.; Related:Unigram Viterbi; Unigram Tokenization with Viterbi →
HNSWHierarchical navigable small worldVector Search: HNSW (hierarchical navigable small world) is the index behind most vector databases.; Search an HNSW Graph →
Holt–WintersTime Series: Holt–Winters (triple exponential smoothing) tracks a level, a trend and an additive seasonal pattern of period m.; Holt–Winters Seasonal Forecasting →
hourParsing: The fields are minute (0–59), hour (0–23), day of month (1–31), month (1–12) and day of week (0–6, with 0 = Sunday).; Related:day of month day of week minute month; Does This Cron Schedule Fire Now? →
how many users per variantExperimentation: Before launching an A/B test, you need to know how many users per variant you need to reliably detect the effect you care about.; A/B Test Sample Size →
Huber lossRegression: The Huber loss is quadratic for small residuals and linear for large ones: Fit it with iteratively reweighted least squares: start from the OLS solution, then repeat: compute residuals, set weights `w = 1` where `|r| <= delta` and `delta / |r|` elsewhere, and solve the weighted least-squares problem.; Related:iteratively reweighted least squares; Robust Regression with Huber Loss (IRLS) →
Huffman codingGreedy: Huffman coding builds an optimal prefix-free code: frequent symbols get short codes.; Huffman Coding →
ICEInterpretability: ICE (individual conditional expectation) curves keep the per-row predictions, which reveals interactions that averaging hides.; Related:Partial dependence; Partial Dependence and ICE Curves →
idsVector Search: Product quantisation (PQ) compresses it to a handful of bytes: split each vector into `m` sub-vectors, learn a small codebook of `k` centroids per sub-space, and store just the centroid ids.; Related:Product quantisation (PQ); Product Quantisation for Vector Compression →
im2colCNNs: Frameworks instead unfold every receptive field into a row of a matrix (im2col), so the whole convolution becomes a single matrix multiplication.; Related:unfold; Fast Convolution with im2col →
immutableSQL: Build a small immutable query builder that always uses `?` placeholders.; A Safe SQL Query Builder →
in date orderAggregation: Return each customer with orders: `name` and `order_ids`, a comma-separated list of their order ids (no spaces) in date order (ties by id).; Related:with orders; List Each Customer's Orders in One Row →
in orderBinary Search: You have jobs with sizes `sizes` that must be processed in order, packed into batches over `days` days (one batch per day, consecutive jobs per batch).; Related:binary search over the capacity Example smallest daily capacity; Binary Search on the Answer: Batch Capacity →
in parallelAgents: Run everything that's ready in parallel, wave by wave.; Related:Jacobi decoding; Run Dependent Tool Calls in Parallel Waves →Parallel Jacobi Decoding →
in placeTwo Pointers: Given a sorted list, remove duplicates in place so each value appears once, keeping order.; Related:back Dutch national flag lexicographic unique; Three-Way Partitioning (Dutch National Flag) →Remove Duplicates from a Sorted List In Place →Move Zeroes →
in the whole storeWindow Functions: Return each order's `id`, `order_date` and `days_since_prev`: days since the previous order in the whole store (by date, ties by id), or `NULL` for the first.; Days Since the Previous Order (Store-Wide) →
InertiaClustering: Inertia is the sum of squared distances from each point to its assigned centroid.; K-Means Inertia and the Elbow →
infixStacks: Also implement `to_rpn(tokens)`: convert an infix token list with `+ - * /` and parentheses to RPN using the shunting-yard algorithm (all operators left-associative, `* /` bind tighter).; Related:Reverse Polish Notation shunting-yard truncates toward zero; Evaluate Reverse Polish Notation →
information gainDecision Trees: Decision trees like ID3 and C4.5 choose splits by information gain: how much a split reduces label entropy.; Entropy and Information Gain →
information valueFeature Engineering: Credit scorecards encode a categorical feature by its weight of evidence and rank features by information value.; Related:weight of evidence; Weight of Evidence and Information Value →
inputs are largeQuantization: Quantising weights to 4 bits hurts most where the inputs are large: a few activation channels carry outliers.; Related:AWQ; Activation-Aware Weight Quantisation (AWQ) →
insertions, deletions or substitutionsDynamic Programming: The edit distance between two strings is the minimum number of single-character insertions, deletions or substitutions needed to turn one into the other.; Related:edit distance; Edit Distance (Levenshtein) →
Instance normNormalization: Instance norm normalises each sample's each channel over its spatial positions only, so the style (contrast) of one image doesn't depend on others in the batch.; Instance Normalisation →
interleavingDynamic Programming: Return whether `s3` can be formed by interleaving `s1` and `s2`: merging them while keeping each string's characters in order.; Interleaving Strings →
Inverse propensity weightingCausal Inference: Inverse propensity weighting reweights the observed users so the treated and untreated groups look alike on the confounders.; Inverse Propensity Weighting for Causal Effects →
inversionDivide and Conquer: An inversion is a pair `i < j` with `a[i] > a[j]`.; Count Inversions with Merge Sort →
IQR ruleStatistics: The IQR rule flags values far outside the middle 50% of the data.; Outlier Detection with the IQR Rule →
Isolation ForestAnomaly Detection: Isolation Forest finds anomalies by noticing they're easy to separate: random splits isolate an outlier in only a few steps, while normal points sit deep in the tree.; Anomaly Scores with Isolation Trees →
itemsRecommenders: Compare items by the users who rated them, then predict a user's rating for an item from their ratings of similar items.; Item-Item Collaborative Filtering →
iteratively reweighted least squaresRegression: The Huber loss is quadratic for small residuals and linear for large ones: Fit it with iteratively reweighted least squares: start from the OLS solution, then repeat: compute residuals, set weights `w = 1` where `|r| <= delta` and `delta / |r|` elsewhere, and solve the weighted least-squares problem.; Related:Huber loss; Robust Regression with Huber Loss (IRLS) →
IVF indexVector Search: An IVF index (as in FAISS) first clusters the vectors into `n_lists` buckets, then at query time only searches the `n_probe` buckets whose centroids are closest.; IVF Vector Index (Inverted File) →
jackknifeResampling: The jackknife estimates the standard error of any statistic by leaving out one observation at a time.; Jackknife Standard Error →
Jacobi decodingInference: Jacobi decoding guesses all n tokens at once and refines them in parallel: on each iteration, every position i is recomputed from the prefix plus the *current guesses* for positions before it.; Related:in parallel; Parallel Jacobi Decoding →
JSON LinesJSON: Event logs are often stored as JSON Lines (one JSON object per line).; Aggregating JSON Lines →
JSON SchemaValidation: LLM structured outputs and API requests are validated against JSON Schema.; Mini JSON Schema Validator →
k largestHeaps: You're reading millions of scored items, such as search results or log events, and need the k largest.; Top-k from a Stream with a Heap →
k-fold cross-validationModel Selection: Pick hyperparameters by trying every combination and scoring each with k-fold cross-validation.; Grid Search with Cross-Validation →
k-medoidsClustering: k-medoids is like k-means, but each cluster centre must be an actual data point (a medoid).; Related:data point; k-Medoids Clustering →
Kadane's algorithmDynamic Programming: Kadane's algorithm does it in one pass: at each position, either extend the best run ending at the previous position or start fresh.; Related:largest sum; Maximum Subarray (Kadane's Algorithm) →
Kaplan–Meier estimatorSurvival Analysis: The Kaplan–Meier estimator handles this correctly.; Related:still subscribed; Kaplan–Meier Survival Curve →
Kendall's tauCorrelation: Kendall's tau counts how many pairs of observations are ordered the same way by two variables.; Kendall's Tau-b →
kernelKernels: Kernel methods (SVMs, Gaussian processes, kernel ridge) compare every pair of points with a kernel.; RBF Kernel Matrix (Vectorised) →
keyword searchRAG: They run keyword search (BM25) and vector search side by side, then merge the ranked lists.; Related:vector search; Hybrid Search with Reciprocal Rank Fusion →
KMPKnuth-Morris-PrattStrings: KMP is O(n + m): it precomputes, for each prefix of the pattern, the length of the longest proper prefix that is also a suffix (the failure function, or LPS array), so it never re-examines text characters.; Related:failure function; Knuth–Morris–Pratt String Search →
Knowledge distillationDistillation: Knowledge distillation trains a small *student* model to mimic a large *teacher*.; Knowledge Distillation Loss →
Kruskal's algorithmGraphs: Kruskal's algorithm: sort edges by weight, then add each edge unless it would form a cycle, checked with union-find.; Minimum Spanning Tree (Kruskal) →
KS testHypothesis Testing: The KS test compares two whole distributions, not just their means.; Kolmogorov–Smirnov Two-Sample Test →
Label propagationSemi Supervised: Label propagation spreads the known labels along a similarity graph, so unlabelled points take the label of the region they sit in.; Semi-Supervised Label Propagation →
LAMBOptimizers: LAMB trains BERT with batch sizes of 64k by scaling each layer's Adam update by a trust ratio.; Related:trust ratio; LAMB: Layer-wise Adaptive Large-Batch Optimisation →
lapsedDates: A customer has lapsed if their last order was more than 30 days before the most recent order date in the whole table (use that as "today" so the query works on historical data).; Related:more than 30 days; Find Lapsed Customers →
largest sumDynamic Programming: Find the contiguous run of numbers with the largest sum, such as the best stretch of daily profit and loss.; Related:Kadane's algorithm; Maximum Subarray (Kadane's Algorithm) →
last knownWindow Functions: For reporting, carry the last known total forward in date order (ties by `id`).; Forward-Fill Missing Values →
laterPandas: Implement `latest(df)` returning one row per `id`: the one with the latest `updated_at` (if two rows tie, keep the one that appears later in the input).; Latest Record per Key in pandas →
latest quote at or beforePandas: You need the latest quote at or before each trade.; Point-in-Time Joins with merge_asof →
lazyGenerators: Build a pipeline of small generator functions that process log lines lazily (one at a time, never building lists): The pipeline must be lazy: pulling one result must not consume the whole input.; A Lazy Log-Processing Pipeline →
lazy deletionHeapq: The standard trick is lazy deletion: push a new entry and mark the old one as removed.; A Priority Queue with Priority Updates →
LCP arrayStrings: Together with the LCP array (the longest common prefix of each adjacent pair), it answers many string questions quickly.; Related:suffix array; Suffix Array and LCP Array →
leading digitFraud Detection: In many natural datasets (invoice amounts, populations, transaction sizes) the leading digit isn't uniform: 1 leads about 30% of the time and 9 under 5%.; Benford's Law Check →
leaksValidation: Scaling the whole dataset before cross-validation leaks test-fold statistics into training.; Leakage-Free Cross-Validation →
learning curveModel Selection: A learning curve plots training and validation error as the training set grows.; Diagnose Bias vs Variance from Learning Curves →
least recently usedLRUDesign: Ties are broken by evicting the least recently used of those keys.; Related:LFU (least frequently used); LFU Cache in O(1) →
length-controlled win rateEvaluation: AlpacaEval 2's length-controlled win rate fits a logistic model of the judge's preference on the length difference, then reports the predicted win rate at equal length.; Related:at equal length longer; Length-Controlled Win Rates →
lengthsLong Context: The needle-in-a-haystack test hides one fact (the needle) at different depths inside contexts of different lengths, then asks the model to retrieve it.; Related:depths; Needle-in-a-Haystack Evaluation →
levelCausal Inference: Did a launch change a metric's level or trend?; Related:trend; Interrupted Time Series (Segmented Regression) →
lexicographicBacktracking: Return all strings of `n` pairs of balanced parentheses, in lexicographic order (`'('` before `')'`).; Related:in place unique; Generate Balanced Parentheses →Unique Permutations →
lexicographically smallestGraphs: If several itineraries work, return the lexicographically smallest one (as a list of airports).; Related:every ticket exactly once; Reconstructing an Itinerary (Eulerian Path) →
LFU (least frequently used)Design: An LFU (least frequently used) cache evicts the key that has been accessed the fewest times.; Related:least recently used; LFU Cache in O(1) →
liftMetrics: Cumulative gain at the top `q` fraction is the share of all positives captured there; lift is gain divided by `q`.; Related:Cumulative gain; Cumulative Gains and Lift →
linearStatistics: Pearson correlation only measures linear relationships and is thrown off by outliers.; Related:ranks Spearman correlation; Spearman Rank Correlation →
linear warmupTraining: Almost every modern LLM is trained with a linear warmup followed by cosine decay: start small so early updates don't blow up, peak, then glide down.; Related:cosine decay; Learning Rate: Warmup + Cosine Decay →
linkageClustering: The linkage defines the distance between two clusters: Also return the merge history: implement `merge_distances(X, linkage)`, the list of linkage distances of the n − 1 merges, in order.; Related:merges the two closest clusters; Hierarchical Agglomerative Clustering →
listParsing: `parse_qs(query)` returns a dict mapping each key to a list of values in order: Also implement `build_qs(params)`: the inverse, taking a dict of lists, percent-encoding everything except letters, digits and `-_.~`, and encoding spaces as `+`.; Parse a URL Query String →
log-oddsBoosting: Gradient boosting for binary classification works in log-odds space.; Gradient Boosting for Classification →
log-softmaxLosses: Cross-entropy is usually computed as log-softmax followed by NLL, which avoids taking the log of a tiny probability.; Related:NLL; Log-Softmax and Negative Log-Likelihood →
logit biasDecoding: LLM APIs let you nudge or forbid specific tokens with a logit bias: add a value to a token's logit before sampling.; Logit Bias and Banned Tokens →
logitsLLM: Every LLM turns raw logits into a probability distribution over the vocabulary with softmax.; Softmax with Temperature →
longerEvaluation: LLM judges prefer longer answers, so a model can climb a leaderboard just by being verbose.; Related:at equal length length-controlled win rate; Length-Controlled Win Rates →
longest common subsequenceEvaluation: ROUGE-L scores summaries by their longest common subsequence of tokens with a reference.; Related:ROUGE-L; ROUGE-L for Summaries →Longest Common Subsequence & Diff →
LookaheadOptimization: Lookahead (Zhang et al., 2019) wraps any optimiser.; The Lookahead Optimiser Wrapper →
Lorenz curveStatistics: The Lorenz curve plots the cumulative share of revenue against the cumulative share of customers, sorted from smallest to largest.; Related:Gini coefficient; Inequality: Gini Coefficient and Lorenz Curve →
low-linkGraphs: Tarjan's algorithm finds both in one DFS using low-link values: `low[v]` is the earliest discovery time reachable from v's subtree using at most one back edge.; Related:articulation point bridge; Bridges and Articulation Points →
lower idSelf Join: List each pair once, with `customer_a` being the one with the lower id.; Related:pair; Pairs of Customers in the Same Country (Self-Join) →
LOWESSRegression: LOWESS fits a separate weighted straight line around every point, giving a smooth curve without choosing a global model.; LOWESS Local Regression →
lowest common ancestorLCATrees: In a BST, the lowest common ancestor of two values `p` and `q` (both present) is the first node, walking down from the root, whose value lies between them (inclusive).; Related:value; Lowest Common Ancestor in a BST →
lowest rankTokenizers: Starting from single characters, repeatedly find the adjacent pair with the lowest rank (earliest learned) and merge every occurrence of it, scanning left to right without overlaps.; Related:merge list; Encode Text with Learned BPE Merges →
LR range testOptimization: Leslie Smith's LR range test (popularised by fast.ai's `lr_find`) trains briefly while raising the learning rate exponentially, then picks a rate where the loss is falling fastest.; Related:falling fastest; Learning-Rate Range Test →
LuhnValidation: The Luhn algorithm catches typos in card and account numbers.; Related:second; Luhn Checksum for Card Numbers →
Luhn checkGuardrails: Luhn check: starting from the rightmost digit, double every second digit (subtracting 9 if the result exceeds 9), sum everything, and the total must be divisible by 10.; Redact PII Before Sending to an LLM →
Mann–Whitney U testStatistics: The Mann–Whitney U test compares two groups using only ranks.; Related:ranks; Mann–Whitney U Test →
mapLong Context: To summarise a document far bigger than the context window: summarise groups of chunks (map), then summarise the summaries (reduce), repeating until one call can see everything.; Related:reduce; Plan a Map-Reduce Summarisation →
March 2024Recursive CTE: Return every date in March 2024 (as `day`) on which no orders were placed, sorted by date.; Days in March Without Any Orders →
matricesOptimization: Standard practice when training transformers (GPT, BERT, ViT) is to apply weight decay to weight matrices but not to biases, normalisation parameters or embeddings (decaying a LayerNorm scale towards 0 just hurts).; Optimizer Parameter Groups: What Not to Decay →
MatryoshkaEmbeddings: Matryoshka embedding models (OpenAI's `text-embedding-3`, Nomic) are trained so the first dimensions carry the most information.; Truncate Matryoshka Embeddings →
Max poolingCNN: Max pooling shrinks feature maps in CNNs by keeping the largest value in each window.; 2-D Max Pooling →
max-heap of size kHeap: Use a max-heap of size k (via `heapq` with negated keys) so it runs in O(n log k).; K Closest Points to the Origin →
MaxSimRetrieval: ColBERT keeps one embedding per token and scores with MaxSim: for each query token, find its best-matching document token, and add up those maxima.; Related:ColBERT per token; Late Interaction Scoring (ColBERT MaxSim) →
mean and varianceBayesian: A Gaussian process puts a prior over functions, and conditioning on data gives a posterior mean and variance at every input, so you get calibrated uncertainty for free.; Related:Gaussian process; Gaussian Process Regression →
Mean shiftClustering: Mean shift finds clusters without being told how many there are: every point climbs uphill on the estimated density until it reaches a peak (a mode), and points that reach the same mode form a cluster.; Related:mode; Mean-Shift Clustering →
medianWindow Functions: The median is more robust, but SQLite (like many databases) has no `MEDIAN` function.; Related:two heaps; Median Order Value (No MEDIAN Function) →Running Median of a Stream →
merge listTokenizers: Once a BPE tokenizer is trained, its merge list (ranked pairs) defines how to encode any new word.; Related:lowest rank; Encode Text with Learned BPE Merges →
merges the two closest clustersClustering: Agglomerative clustering starts with every point in its own cluster and repeatedly merges the two closest clusters.; Related:linkage; Hierarchical Agglomerative Clustering →
million tokensLLM: LLM APIs charge per million tokens, with different prices for input and output, and a discount for input tokens served from the prompt cache.; Related:prompt cache; LLM API Cost Estimator →
Min-max normaliseRAG: Min-max normalise each retriever's scores to [0, 1] first, then blend: A document missing from one retriever's results gets 0 for that retriever.; Hybrid Search: Normalise and Blend Scores →
Min-pSampling: Min-p adapts: keep only tokens whose probability is at least `min_p` times the top token's probability.; Related:top; Min-p Sampling →
MinHashDedup: MinHash estimates the Jaccard similarity of two documents' shingle sets from short signatures, without comparing the sets directly.; Related:signatures; Near-Duplicate Detection with MinHash →
Mini-batch k-meansClustering: Mini-batch k-means updates centroids from small random batches with a per-centroid learning rate.; Mini-Batch K-Means →
minimum number of roomsHeaps: Given meetings as `[start, end)` intervals, find the minimum number of rooms needed so that no two overlapping meetings share a room.; Related:don't; Minimum Meeting Rooms →
minuteParsing: The fields are minute (0–59), hour (0–23), day of month (1–31), month (1–12) and day of week (0–6, with 0 = Sunday).; Related:day of month day of week hour month; Does This Cron Schedule Fire Now? →
Missing values:Time Series: An EWMA smooths a noisy series by giving recent points more weight: Missing values: if `x[t]` is `NaN`, carry the previous smoothed value forward (`s_t = s_{t-1}`).; Related:EWMA; Exponentially Weighted Moving Average →
modeClustering: Mean shift finds clusters without being told how many there are: every point climbs uphill on the estimated density until it reaches a peak (a mode), and points that reach the same mode form a cluster.; Related:Mean shift; Mean-Shift Clustering →
MomentumOptimization: Momentum keeps a running velocity so consistent directions speed up and oscillations cancel out.; SGD with Momentum and Nesterov →
monthParsing: The fields are minute (0–59), hour (0–23), day of month (1–31), month (1–12) and day of week (0–6, with 0 = Sunday).; Related:day of month day of week hour minute; Does This Cron Schedule Fire Now? →
more thanPandas: Web analytics groups each user's events into sessions: a new session starts when the gap since that user's previous event is more than `timeout` seconds (Google Analytics uses 30 minutes).; Related:sessions; Sessionise a Clickstream →
more than 30 daysDates: A customer has lapsed if their last order was more than 30 days before the most recent order date in the whole table (use that as "today" so the query works on historical data).; Related:lapsed; Find Lapsed Customers →
more than halfArrays: Find the element that appears in more than half of a list using O(1) memory: keep one candidate and a counter, adding 1 when you see the candidate and subtracting 1 otherwise, switching candidates when the counter hits 0.; Boyer–Moore Majority Vote →
most ordersAggregation: Return the `month` (`YYYY-MM`) with the most orders and its `orders` count.; The Busiest Month →
most recentLLM: The usual fix: keep the system prompt, then keep as many of the most recent messages as fit.; Fit Chat History in the Context Window →Deduplicate Records, Keeping the Latest →
Multinomial Naive BayesClassification: Multinomial Naive Bayes is a fast, surprisingly strong baseline for text classification such as spam detection or ticket routing.; Naive Bayes Text Classifier →
Mutual informationFeature Selection: Mutual information measures how much knowing a feature reduces uncertainty about the label.; Feature Selection with Mutual Information →
n-gram overlapsEvaluation: The common check: look for long n-gram overlaps between each test item and the training corpus (GPT-3 used 13-grams).; Check a Benchmark for Training-Data Contamination →
NadamOptimizers: Nadam applies Nesterov's look-ahead to Adam's first moment.; Nadam: Adam with Nesterov Momentum →
negativePrefix Sums: Values can be negative, so a sliding window doesn't work.; Count Subarrays Summing to k →
negative edge weightsGraphs: Dijkstra fails with negative edge weights (rebates, arbitrage, energy gains).; Related:still; Bellman–Ford and Negative Cycles →
negative samplingEmbeddings: Computing a softmax over the whole vocabulary is expensive, so negative sampling instead asks: can you tell the real context word from a few random ones?; Related:centre context; Word2Vec Skip-Gram with Negative Sampling →
new recordWindow Functions: In date order (ties by `id`), which orders set a new record, being strictly larger than every earlier order?; Record-Breaking Orders (Running MAX) →
NLLLosses: Cross-entropy is usually computed as log-softmax followed by NLL, which avoids taking the log of a tiny probability.; Related:log-softmax; Log-Softmax and Negative Log-Likelihood →
NMSObject Detection: NMS keeps the highest-scoring box, removes every remaining box that overlaps it by more than `iou_threshold`, and repeats.; Non-Maximum Suppression →
noiseClustering: DBSCAN finds clusters of any shape and labels outliers as noise, using just two parameters: a radius `eps` and `min_samples`.; Related:bias² DBSCAN variance; DBSCAN Density-Based Clustering →Bias–Variance Decomposition by Simulation →
non-negativeSliding Window: Given non-negative costs and a `budget`, return the length of the longest contiguous subarray whose sum is at most `budget`.; Longest Subarray Within a Budget →
O(1)Dynamic Programming: Return the minimum total cost to reach the top, in O(n) time and O(1) space.; Minimum Cost Climbing Stairs →
O(log(min(m, n)))Binary Search: Given two sorted lists `a` and `b`, find the median of their combined elements in O(log(min(m, n))) time, without merging them.; Median of Two Sorted Arrays →
O(n)Monotonic Stack: Given bar heights of width 1, find the area of the largest rectangle that fits under the histogram, in O(n) using a monotonic stack of indices with increasing heights.; Largest Rectangle in a Histogram →
on at test timeUncertainty: Keeping dropout on at test time and averaging several stochastic passes approximates a Bayesian ensemble: the spread of the predictions estimates uncertainty.; Monte Carlo Dropout Uncertainty →
one letter at a timeBFS: Transform `begin` into `end` by changing one letter at a time, where every intermediate word must be in `words`.; Word Ladder →
one passLinked Lists: Remove the `n`-th node from the end of a linked list in one pass and return the head.; Related:end; Remove the N-th Node from the End →
one repeated characterSliding Window: What's the longest substring you can make consist of one repeated character?; Longest Run After k Replacements →
one-hot encodedPreprocessing: Models need numbers, so categorical features like `"red"`, `"green"`, `"blue"` get one-hot encoded: one column per category, with a 1 in the matching column.; Related:wasn't in the training data; One-Hot Encoder with Unseen Categories →
only ifDecorators: Write a decorator `memoize` that caches results by both positional and keyword arguments, where `f(1, b=2)` and `f(b=2, a=1)` hit the same cache entry only if they bind to the same parameters.; Memoise Functions with Keyword Arguments →
orderDynamic Programming: The order of multiplication changes the total cost hugely.; Matrix Chain Multiplication Order →
order mattersSampling: Real samplers chain several steps, and order matters.; The Full Sampling Pipeline →
out-of-foldEnsembles: The meta-model must be trained on out-of-fold predictions (each row predicted by a model that never saw it); otherwise it learns to trust overfit models.; Related:Stacking; Stacking with Out-of-Fold Predictions →
overallStatistics: A treatment can win in every subgroup yet lose overall.; Related:Simpson's paradox The classic example; Spot Simpson's Paradox →
overconfidentCalibration: Fine-tuned models are often overconfident: they say 99% but are right 80% of the time.; Related:Temperature scaling; Calibrate Confidence with Temperature Scaling →
overflowMixed Precision: Small gradients underflow to zero; large activations overflow to infinity.; Related:underflow; Half-Precision Range and Loss Scaling →
own scaleQuantization: Group-wise quantisation (as in GPTQ, AWQ and llama.cpp's Q4 formats) gives every group of, say, 32 consecutive weights in a row its own scale.; Related:Group-wise; Group-Wise INT4 Quantisation →
paddingAttention: Batched transformers combine two masks: a causal mask (no peeking ahead) and a padding mask (ignore pad tokens in each sequence).; Related:causal; Building Attention Masks →
pairSelf Join: Return every pair of different customers from the same country: `customer_a`, `customer_b` and `country`.; Related:lower id; Pairs of Customers in the Same Country (Self-Join) →
parentRAG: Parent document retrieval searches over small chunks, then returns the larger parent sections they came from.; Related:Parent document retrieval; Small-to-Big: Parent Document Retrieval →
Parent document retrievalRAG: Parent document retrieval searches over small chunks, then returns the larger parent sections they came from.; Related:parent; Small-to-Big: Parent Document Retrieval →
partial correlationCorrelation: The partial correlation of `x` and `y` given controls `Z` is the correlation of their residuals after regressing each on `Z` (with an intercept).; Partial Correlation →
Partial dependenceInterpretability: Partial dependence sets that feature to each grid value for every row and averages the predictions.; Related:ICE; Partial Dependence and ICE Curves →
passiveOnline Learning: The passive-aggressive algorithm learns from one example at a time: if the example is classified with margin ≥ 1 it stays passive; otherwise it makes the smallest update that fixes it (aggressive), capped by `C` (PA-I).; Related:aggressive passive-aggressive; Passive-Aggressive Online Classifier →
passive-aggressiveOnline Learning: The passive-aggressive algorithm learns from one example at a time: if the example is classified with margin ≥ 1 it stays passive; otherwise it makes the smallest update that fixes it (aggressive), capped by `C` (PA-I).; Related:aggressive passive; Passive-Aggressive Online Classifier →
pastPandas: Forecasting models need features from the past, such as yesterday's sales and last week's average, computed separately for each store and without peeking at the future.; Related:without peeking at the future; pandas: Lag and Rolling Features →
patchesVision Transformers: A Vision Transformer treats an image as a sequence of patches.; Split Images into ViT Patches →
pathAgents: Agent evals often check the path, not just the final answer: did the agent call the right tools, in a sensible order?; Score an Agent's Tool-Call Trajectory →
path-traversalStrings: Turning `"/srv//app/./logs/../data/"` into `"/srv/app/data"` is everyday path handling, and getting it wrong causes path-traversal security bugs, where `../../etc/passwd` escapes the folder you meant to allow.; Normalise a File Path →
per tokenRetrieval: ColBERT keeps one embedding per token and scores with MaxSim: for each query token, find its best-matching document token, and add up those maxima.; Related:ColBERT MaxSim; Late Interaction Scoring (ColBERT MaxSim) →
perceptronClassification: The perceptron (1958) is the ancestor of every neural network.; The Perceptron →
Permutation importanceInterpretability: Permutation importance answers it for any model: shuffle one feature column, breaking its link to the target, and measure how much the score drops.; Permutation Feature Importance →
permutation testHypothesis Testing: A permutation test needs no distributional assumptions: if the treatment did nothing, the group labels are arbitrary, so shuffling them shows how big a difference chance alone produces.; Permutation Tests →
PerplexityEvaluation: Perplexity is the standard intrinsic metric for language models.; Perplexity from Token Log-Probs →
point updatesTrees: Build a structure over an array that supports point updates and range queries in O(log n) each.; Related:range queries; Segment Tree for Range Queries →
pointwiseCNNs: A pointwise (1 × 1) convolution then mixes channels at each position.; Related:depthwise; Depthwise and Pointwise Convolutions →
policyAlignment: For each prompt you have a *chosen* (preferred) and a *rejected* response, and the summed log-probabilities of each under the policy being trained and a frozen reference model.; Related:DPO reference; Direct Preference Optimization (DPO) Loss →
polynomial featuresFeatures: A straight line can't fit a curve, but linear regression on polynomial features can.; Polynomial Features and Overfitting →
PowerA/B Testing: Power is the chance a test detects a real effect.; Statistical Power of an A/B Test →
power iterationRegularization: Computing an SVD every step is too slow, so it's estimated with power iteration: repeat `v = normalise(Wᵀu)`, `u = normalise(W v)`, then σ ≈ uᵀWv.; Related:spectral norm; Spectral Normalisation by Power Iteration →
PPORLHF: PPO, used in classic RLHF, improves a policy while preventing any single update from moving it too far.; PPO Clipped Policy Objective →
precision of at least 90%Metrics: A fraud team needs precision of at least 90%.; Pick the Threshold for a Target Precision →
predictablePrompting: LLMLingua-style compression drops the tokens a small language model finds predictable.; Compress Prompts by Self-Information →
prefixStreaming: Write a tolerant parser for a JSON prefix.; Related:as they arrive; Parse Partial JSON While Streaming →
preorderTrees: Use a preorder walk writing values separated by commas and `#` for missing children.; Serialise and Deserialise a Binary Tree →
Price elasticityEconomics: Price elasticity is the percentage change in quantity sold for a 1% change in price.; Estimate Price Elasticity →
Prim's algorithmGraphs: Prim's algorithm grows a minimum spanning tree from a start vertex, always adding the cheapest edge that connects a new vertex.; Minimum Spanning Tree with Prim's Algorithm →
Principal Component AnalysisPCADimensionality Reduction: Principal Component Analysis finds the directions of greatest variance in your data.; Related:singular value decomposition; PCA via SVD →
Product quantisation (PQ)Vector Search: Product quantisation (PQ) compresses it to a handful of bytes: split each vector into `m` sub-vectors, learn a small codebook of `k` centroids per sub-space, and store just the centroid ids.; Related:ids; Product Quantisation for Vector Compression →
prompt cacheLLM: LLM APIs charge per million tokens, with different prices for input and output, and a discount for input tokens served from the prompt cache.; Related:million tokens; LLM API Cost Estimator →
prompt templatesLLM: They use prompt templates with named slots, and fail fast when a variable is missing.; Prompt Templates with Validation →
QDAClassification: QDA fits a Gaussian with its own mean and covariance to each class and classifies by the highest posterior.; Quadratic Discriminant Analysis →
Qini curveCausal Inference: Uplift models estimate the individual treatment effect, and the Qini curve evaluates them on a randomised experiment.; Related:changes Uplift models; Uplift Modelling and the Qini Curve →
Rabin-KarpStrings: Find all start indices where `pattern` occurs in `text` using Rabin-Karp: compare rolling polynomial hashes (base 256, modulus `1_000_000_007`) and confirm candidate matches with a direct comparison (so hash collisions never produce false positives).; Rabin-Karp Rolling Hash Search →
radix sortSorting: Comparison sorts can't beat O(n log n), but radix sort sorts integers in O(d · (n + b)) by sorting digit by digit, least significant first, using a stable counting sort for each digit.; Related:stable; Radix Sort →
range queriesTrees: Build a structure over an array that supports point updates and range queries in O(log n) each.; Related:point updates; Segment Tree for Range Queries →
rankingEmbeddings: An embedding model is good if the ranking of its cosine similarities matches the human ranking, measured by Spearman correlation.; Related:Spearman; Evaluate Embeddings on Semantic Similarity (STS) →
ranksStatistics: The Mann–Whitney U test compares two groups using only ranks.; Related:linear Mann–Whitney U test Spearman correlation; Mann–Whitney U Test →Spearman Rank Correlation →
RANSACRobust Statistics: RANSAC fits a model to data with many outliers by trying random minimal samples and keeping the one most points agree with.; RANSAC Line Fitting →
reaches step kProduct Analytics: A user reaches step k if they did step 1, then step 2 at or after that time, and so on, in order.; Related:earliest qualifying; Funnel Conversion from Event Logs →
read-onlySafety: A text-to-SQL assistant must only ever run read-only queries.; Guardrails for LLM-Generated SQL →
realEmbeddings: To get a single sentence embedding for search, models like sentence-transformers average the token vectors, but only over real tokens, not padding.; Related:sentence embedding; Sentence Embeddings: Masked Mean Pooling →
receptive fieldCNN: The receptive field is how many input pixels influence one output unit.; Receptive Field of a CNN →
Recursive feature eliminationFeature Selection: Recursive feature elimination (RFE) fits a model, drops the least important feature, and repeats.; Related:standardised; Recursive Feature Elimination →
recursive splitterRAG: LangChain's popular recursive splitter tries the most natural boundaries first: split on paragraphs, and if a piece is still too big, split it on lines, then sentences, then words, then characters.; Recursive Character Text Splitter →
reduceLong Context: To summarise a document far bigger than the context window: summarise groups of chunks (map), then summarise the summaries (reduce), repeating until one call can see everything.; Related:map; Plan a Map-Reduce Summarisation →
referenceAlignment: For each prompt you have a *chosen* (preferred) and a *rejected* response, and the summed log-probabilities of each under the policy being trained and a frozen reference model.; Related:DPO policy; Direct Preference Optimization (DPO) Loss →
regression treeTrees: Build a complete regression tree from scratch.; Decision Tree Regressor →
rejectedRLHF: In RLHF, a reward model learns from human preference pairs: it should give the chosen response a higher score than the rejected one.; Related:chosen reward model; Reward Model Training Loss (Bradley–Terry) →
relative to the recent pastAnomaly Detection: Flag points that are unusual relative to the recent past.; Rolling z-Score Anomaly Detection →
repeatWindow Functions: How much revenue comes from customers' first orders versus their repeat orders?; First-Order vs Repeat Revenue →
requests per minuteAPIs: LLM APIs enforce two limits at once: requests per minute (RPM) and tokens per minute (TPM).; Related:tokens per minute; Schedule Requests Under RPM and TPM Limits →
reranksRAG: Production RAG retrieves cheaply first (BM25 or vectors), then reranks the top few candidates with an expensive cross-encoder that reads the query and document together.; Two-Stage Retrieval with a Reranker →
rescoreEmbeddings: You then rescore the shortlist with the full-precision vectors.; Related:Binary quantisation Hamming distance sign; Binary Embeddings and Hamming Search →
residualsBoosting: Gradient boosting builds a strong model from many weak ones: each new model fits the residuals, the errors left by the models so far.; Related:Gradient boosting; Gradient Boosting with Decision Stumps →
retry with exponential backoffDecorators: The standard fix is to retry with exponential backoff: wait 1s, then 2s, then 4s… `sleep` is a parameter so tests can pass a fake one instead of actually waiting.; Retry Decorator with Exponential Backoff →
Reverse Polish NotationRPNStacks: Evaluate an expression in Reverse Polish Notation given as a list of tokens: numbers (possibly negative) and operators `+ - * /`.; Related:infix shunting-yard truncates toward zero; Evaluate Reverse Polish Notation →
reward modelRLHF: In RLHF, a reward model learns from human preference pairs: it should give the chosen response a higher score than the rejected one.; Related:chosen rejected; Reward Model Training Loss (Bradley–Terry) →
reweightingEnsembles: AdaBoost trains weak learners in sequence, each focusing on the examples the previous ones got wrong by reweighting them.; Related:AdaBoost; AdaBoost with Decision Stumps →
RFMPandas: RFM scores customers on Recency (days since last order), Frequency (number of orders) and Monetary value (total spend), then turns the scores into segments marketing can act on.; RFM Customer Segmentation →
rightDynamic Programming: A robot starts at the top-left of a grid and can only move right or down to reach the bottom-right.; Related:down; Unique Grid Paths with Obstacles →
Ring all-reduceDistributed Training: Ring all-reduce does this with each worker talking only to its neighbour, so bandwidth per worker stays constant as you add workers.; Related:sum; Simulate Ring All-Reduce →
ring bufferData Structures: A ring buffer keeps the last N items in a fixed-size list, overwriting the oldest when full: ideal for "last 100 log lines" or a rolling metric window.; A Fixed-Size Ring Buffer →
riskEvaluation: A model that can say "I don't know" trades coverage (how often it answers) for risk (how often its answers are wrong).; Related:coverage; Abstain When Unsure: Risk–Coverage Curves →
RMSpropOptimization: RMSprop adapts the step size per parameter: it divides each gradient by a running root-mean-square of its recent magnitudes, so parameters with large, noisy gradients take smaller steps.; RMSprop Optimiser →
robust scalingPreprocessing: One extreme value squashes min-max scaling, while robust scaling (median and IQR) barely notices it.; Min-Max vs Robust Scaling →
ROC AUCEvaluation: ROC AUC summarises how well a model ranks positives above negatives, across every possible threshold.; ROC AUC from Scratch →
RoPERotary position embeddingTransformers: Llama, Mistral, Qwen and most modern LLMs encode token position with RoPE: instead of adding a position vector, they *rotate* each query and key vector by an angle that grows with position.; Rotary Position Embeddings (RoPE) →
ROUGE-LEvaluation: ROUGE-L scores summaries by their longest common subsequence of tokens with a reference.; Related:longest common subsequence; ROUGE-L for Summaries →
routerLLM: A router sends easy prompts to a cheap model and hard ones to a strong model.; Cost-Aware Model Router →
RulesEvaluation: Rules (these match scikit-learn's `KFold`); K-Fold Cross-Validation Splits →
Run-length encodingStrings: Run-length encoding compresses repeated characters: `"aaabccdddd"` → `"3a1b2c4d"`.; Run-Length Encoding →
same calendar monthJoins: Return the `id` and `name` of customers who placed at least one order in the same calendar month they signed up, sorted by `id`.; Customers Who Ordered in Their Signup Month →
same keyCaching: Caching LLM responses only works if logically identical requests produce the same key, and different requests never collide.; Prompt Cache Keys →
same month one year earlierPandas: Implement `yoy(df)` returning a DataFrame with columns `month`, `revenue` and `yoy_pct`: the percentage change versus the same month one year earlier, rounded to 1 decimal place, or `NaN` if that month isn't in the data.; Year-over-Year Growth by Month →
sample ratio mismatchSRMA/B Testing: A sample ratio mismatch check runs a chi-square test on the counts.; Sample Ratio Mismatch Check →
Scalar quantisationEmbeddings: Scalar quantisation maps each dimension's calibrated range to `uint8` (one byte), a 4× saving with little loss in retrieval quality.; Scalar Quantisation of Embeddings →
scaleMixed Precision: Mixed precision multiplies the loss by a large scale before backprop and divides the gradients afterwards.; Related:float16 underflow; Dynamic Loss Scaling for Mixed Precision →
scoresCalibration: SVMs and boosted trees output scores, not probabilities.; Platt Scaling for Classifier Scores →
secondValidation: From the rightmost digit, double every second digit (subtracting 9 if the result is over 9), sum everything, and the number is valid if the total is divisible by 10.; Related:Luhn; Luhn Checksum for Card Numbers →
second order within 30 daysWindow Functions: A customer who places a second order within 30 days of their first is far more likely to stick around.; Related:30 days; Customers Who Came Back Within 30 Days →
second-highest distinctSubqueries: A classic interview question: return a single row with one column, `second_highest`, holding the second-highest distinct order total.; Second-Highest Order Total →
Self-consistencyLLM: Self-consistency boosts LLM accuracy on reasoning tasks: sample several answers at a non-zero temperature, then take the most common final answer.; Self-Consistency: Majority Vote over Samples →
semantic cacheCaching: A semantic cache compares query embeddings and reuses an answer when a new query is similar enough.; Related:embeddings; Semantic Cache for LLM Responses →
SemDeDupData: SemDeDup groups examples whose embeddings are nearly identical and keeps one per group.; Semantic Deduplication of a Dataset →
sentence boundariesRAG: Chunk documents at sentence boundaries with a token budget and a little overlap, so a fact split across a boundary still appears whole in one chunk.; Sentence-Aware Chunking with Overlap →
sentence embeddingEmbeddings: To get a single sentence embedding for search, models like sentence-transformers average the token vectors, but only over real tokens, not padding.; Related:real; Sentence Embeddings: Masked Mean Pooling →
Sentence-window retrievalRAG: Sentence-window retrieval (LlamaIndex) retrieves sentences, then expands each hit to include its neighbours.; Sentence-Window Retrieval →
Server-Sent EventsStreaming: LLM APIs stream tokens as Server-Sent Events.; Parse a Streaming LLM Response (SSE) →
sessionsPandas: Web analytics groups each user's events into sessions: a new session starts when the gap since that user's previous event is more than `timeout` seconds (Google Analytics uses 30 minutes).; Related:more than; Sessionise a Clickstream →
setMetrics: In multi-label problems (tagging a support ticket as both "billing" and "urgent"), each sample has a set of labels.; Multi-Label Classification Metrics →
SGDRLearning Rate: SGDR decays the learning rate along a cosine curve, then jumps back up.; Cosine Annealing with Warm Restarts →
Shapley valuesInterpretability: SHAP explains a prediction by sharing out `f(x) - E[f]` among the features according to Shapley values from game theory.; Exact Shapley Values for a Prediction →
shiftedLanguage Models: Training a language model means predicting token t+1 from position t, so logits and labels are shifted by one.; Next-Token Loss with Padding and Prompt Masks →
shortDecoding: Raw beam search favours short outputs, because every extra token adds a negative log-probability.; Beam Search Scoring with Length Penalty →
shorterTwo Pointers: Water above a bar is limited by the shorter of the tallest bars to its left and right.; Trapping Rain Water →
shortest substringSliding Window: Given strings `s` and `t`, find the shortest substring of `s` that contains every character of `t`, including duplicates (if `t` has two `a`s, the window needs two).; Minimum Window Substring →
shunting-yardStacks: Also implement `to_rpn(tokens)`: convert an infix token list with `+ - * /` and parentheses to RPN using the shunting-yard algorithm (all operators left-associative, `* /` bind tighter).; Related:infix Reverse Polish Notation truncates toward zero; Evaluate Reverse Polish Notation →
signEmbeddings: A second hash picks a sign (+1 or −1), so collisions cancel out on average instead of piling up.; Related:Binary quantisation Hamming distance hashing trick rescore; Binary Embeddings and Hamming Search →The Hashing Trick →
sign testHypothesis Testing: The sign test makes no assumption about the shape of the data: under the null, each non-zero paired difference is equally likely to be positive or negative.; Exact Sign Test →
signaturesDedup: MinHash estimates the Jaccard similarity of two documents' shingle sets from short signatures, without comparing the sets directly.; Related:MinHash; Near-Duplicate Detection with MinHash →
silhouetteClustering: The silhouette compares how close each point is to its own cluster versus the nearest other cluster.; Silhouette Score for Clustering →
similarPrompting: Few-shot examples work best when they're similar to the user's input but not near-duplicates of each other.; Choose Few-Shot Examples by Similarity and Diversity →
Simpson's paradoxStatistics: This is Simpson's paradox, and it happens when the groups have very different sizes or base rates.; Related:overall The classic example; Spot Simpson's Paradox →
singular value decompositionSVDDimensionality Reduction: The numerically robust way to compute it is the singular value decomposition of the centred data.; Related:Principal Component Analysis; PCA via SVD →
sliding window logRate Limiting: Many APIs instead promise *"at most N requests in any 60-second window"*: a sliding window log.; Sliding-Window Rate Limiter per User →
smallest daily capacityBinary Search: What's the smallest daily capacity that finishes everything in time?; Related:binary search over the capacity Example in order; Binary Search on the Answer: Batch Capacity →
SMOTEImbalanced Data: SMOTE creates synthetic minority examples by interpolating between real ones and their nearest neighbours.; SMOTE: Synthetic Minority Oversampling →
sorted listAlgorithms: A simple correct approach keeps a sorted list of the current window and uses `bisect` to insert the new value and remove the old one, giving O(k) per step.; Rolling Median with Two Heaps →
sorted, non-overlappingIntervals: A calendar holds sorted, non-overlapping busy intervals `[start, end]`.; Insert an Interval into a Schedule →
sparseRetrieval: SPLADE-style models encode text as sparse vectors over the vocabulary, with learned term weights (including expansion terms that never appear in the text).; Sparse Learned Retrieval (SPLADE-Style) →
SpearmanEmbeddings: An embedding model is good if the ranking of its cosine similarities matches the human ranking, measured by Spearman correlation.; Related:ranking; Evaluate Embeddings on Semantic Similarity (STS) →
Spearman correlationStatistics: Spearman correlation is Pearson correlation applied to the ranks, so it captures any monotonic relationship.; Related:linear ranks; Spearman Rank Correlation →
Spectral clusteringClustering: Spectral clustering (Ng, Jordan & Weiss) clusters by graph connectivity instead, so it can separate two concentric rings.; Spectral Clustering →
spectral normRegularization: A weight matrix's spectral norm (its largest singular value) bounds how much the layer can stretch its input, so it controls the network's Lipschitz constant.; Related:power iteration; Spectral Normalisation by Power Iteration →
Speculative samplingInference: Speculative sampling keeps the *exact* output distribution of the big model when sampling.; Speculative Sampling (Probabilistic) →
SpotlightingPrompt Injection: Spotlighting (Microsoft, 2024) helps a model tell instructions apart from untrusted data (emails, web pages) by transforming the data visibly.; Related:datamarking; Spotlighting Untrusted Text with Datamarking →
squeeze-and-excitationCNN: A squeeze-and-excitation block lets a CNN reweight its channels using global context.; Squeeze-and-Excitation Block →
stableSorting: Such a pair is a blocking pair, and a matching without one is stable.; Related:blocking pair radix sort; Sorting by Multiple Keys →Stable Matching (Gale–Shapley) →Radix Sort →
StackingEnsembles: Stacking trains a meta-model on the predictions of several base models.; Related:out-of-fold; Stacking with Out-of-Fold Predictions →
StandardisationPreprocessing: Standardisation rescales each column to mean 0 and standard deviation 1.; Related:training data only; Standard Scaler Without Data Leakage →
standardisedFeature Selection: With a linear model on standardised features, importance is the absolute coefficient.; Related:Recursive feature elimination; Recursive Feature Elimination →
start and endRAG: "Lost in the Middle" (Liu et al., 2023) found that LLMs use information at the start and end of a long context far better than information in the middle.; Reorder Context for Long-Context Models →
stillGraphs: If any edge can still be relaxed after that, there's a negative cycle reachable from the source.; Related:negative edge weights; Bellman–Ford and Negative Cycles →
still subscribedSurvival Analysis: You can't just average subscription lengths, because customers who are still subscribed haven't churned yet (they're *censored*).; Related:Kaplan–Meier estimator; Kaplan–Meier Survival Curve →
stratifiedData Splitting: A stratified split keeps each class's share the same in both parts.; Stratified Train/Test Split →
stratified splitSampling: A stratified split keeps each class's proportion the same in train and test.; Stratified Train/Test Split →
StreamingLLMInference: StreamingLLM's fix: always keep the first `n_sink` tokens, plus a rolling window of the most recent ones.; Related:first few tokens; Streaming KV Cache with Attention Sinks →
strictly aboveWindow Functions: Among customers with orders, return `name`, `country`, `spend` and `country_avg` (the average spend of buying customers in the same country, rounded to 2 decimals) for those whose spend is strictly above their country's average.; Related:with orders; Customers Spending Above Their Country's Average →
strictly afterDatetime: Also implement `next_birthday(birth, on)` returning the ISO date of their next birthday strictly after `on`, using the same rule.; Exact Age, Including Leap Days →
strictly greaterTrees: Return `True` if a binary tree is a valid BST: every node's value is strictly greater than all values in its left subtree and strictly less than all in its right subtree.; Related:strictly less; Validate a Binary Search Tree →
strictly largerWindow Functions: Find customers with at least two orders where every order (by date, ties by `id`) is strictly larger than their previous one.; Related:at least two; Customers Whose Orders Keep Growing →
strictly lessTrees: Return `True` if a binary tree is a valid BST: every node's value is strictly greater than all values in its left subtree and strictly less than all in its right subtree.; Related:strictly greater; Validate a Binary Search Tree →
strongly connected componentGraphs: In a directed graph, a strongly connected component (SCC) is a maximal set of nodes where every node can reach every other.; Strongly Connected Components →
subdirectories firstRecursion: Implement `render_tree(name, node)` returning a string: Within a directory, list subdirectories first, then files, each group sorted alphabetically.; Print a Directory Tree →
subsequenceDynamic Programming: A subsequence keeps order but may skip characters.; Longest Palindromic Subsequence →
substringSliding Window: Return the longest substring of `s` with no repeated characters.; Longest Substring Without Repeats →
subtly wrongEmbeddings: Its backward pass is easy to get subtly wrong: when a token appears several times in a batch, its gradient must be the sum of the gradients at every position.; Related:sum; Embedding Layer Backward (Scatter-Add) →
subtotal row per countryUnion: Return revenue by `country` and `month` (`YYYY-MM`), plus a subtotal row per country (month shown as `'All'`) and a grand total row (country and month both `'All'`), with columns `country`, `month`, `revenue`.; Related:grand total row; Subtotals and a Grand Total →
suffix arrayStrings: A suffix array lists the starting positions of a string's suffixes in sorted order.; Related:LCP array; Suffix Array and LCP Array →
sumEmbeddings: Its backward pass is easy to get subtly wrong: when a token appears several times in a batch, its gradient must be the sum of the gradients at every position.; Related:Ring all-reduce subtly wrong; Embedding Layer Backward (Scatter-Add) →Simulate Ring All-Reduce →
summariseMemory: Instead of dropping old messages, summarise them: keep the recent messages verbatim and fold older ones into a running summary.; Conversation Memory with Rolling Summaries →
support vector machineSVMClassification: A linear support vector machine finds the separating line with the widest margin.; Linear SVM with Hinge Loss →
swappedEvaluation: The standard check: ask twice with the order swapped and see whether the verdict flips consistently.; Measure LLM-Judge Position Bias →
Temperature scalingCalibration: Temperature scaling (Guo et al., 2017) fixes this by dividing the logits by a single learned T > 1 before the softmax.; Related:overconfident; Calibrate Confidence with Temperature Scaling →
TF-IDFTerm frequency-inverse document frequencyNLP: TF-IDF turns documents into vectors where common words count for little and distinctive words count for a lot.; TF-IDF Vectoriser →
The classic exampleStatistics: Implement: The classic example (kidney stone treatments): A wins in both groups, but B wins overall.; Related:overall Simpson's paradox; Spot Simpson's Paradox →
their previous ordering month'sWindow Functions: Return `customer_id`, `month` (`YYYY-MM`), `spend` and `change`: the difference from their previous ordering month's spend (`NULL` for their first).; Customer Spend Change Month over Month →
Thompson samplingBandits: Thompson sampling balances exploring and exploiting by acting on a random draw from each arm's posterior.; Thompson Sampling for Bernoulli Bandits →
time to activationDates: Growth teams track time to activation: how long new customers take to place their first order.; Days from Signup to First Order →
Time-series splitsCross Validation: Time-series splits always train on earlier data and test on the block right after it.; Time-Series Cross-Validation Splits →
token trieInference: Inference servers like SGLang (RadixAttention) and vLLM keep the KV cache of previous prompts in a token trie, so a new request only computes the tokens after its longest cached prefix.; Prefix Caching with a Radix Tree →
Tokenising:RAG: Tokenising: lowercase, then split on whitespace.; BM25 Keyword Search →
tokens per minuteAPIs: LLM APIs enforce two limits at once: requests per minute (RPM) and tokens per minute (TPM).; Related:requests per minute; Schedule Requests Under RPM and TPM Limits →
topSampling: Return the `id` and `total` of orders in the top group, sorted by `total` descending, then `id`.; Related:Min-p; Min-p Sampling →Orders in the Top Quartile →
top 3 revenue ranksWindow Functions: Return the dates in the top 3 revenue ranks (using `DENSE_RANK`, so tied days share a rank and more than three days may appear), with `order_date`, `revenue` and `day_rank`.; Top Revenue Days with Ties →
topological sortGraphs: That ordering is a topological sort of a directed acyclic graph.; Task Ordering with Topological Sort →
TOTALUnion: Finance wants a report with a TOTAL line at the bottom.; Report with a Grand Total Row →
touchSorting: Intervals that only touch (like `[1, 3]` and `[3, 5]`) count as overlapping.; Merge Overlapping Intervals →
trainingPreprocessing: Scale every feature into `[0, 1]` using statistics learned on the training data only.; Min-Max Scaler →
training data onlyPreprocessing: The catch: the mean and standard deviation must come from the training data only.; Related:Standardisation; Standard Scaler Without Data Leakage →
transposed convolutionCNN: A transposed convolution ("deconvolution") upsamples: each input element scatters a scaled copy of the kernel into the output, with copies placed `stride` apart and overlaps summed.; Transposed Convolution 1D →
treeInference: Medusa and EAGLE propose a tree of draft continuations instead of a single chain, and verify the whole tree in one forward pass.; Related:ancestors; Tree Attention for Speculative Drafts →
trendCausal Inference: Did a launch change a metric's level or trend?; Related:level; Interrupted Time Series (Segmented Regression) →
trend + seasonal + residualTime Series: Split a series into trend + seasonal + residual, the classical additive decomposition (what `statsmodels.seasonal_decompose` does).; Classical Seasonal Decomposition →
trieWeb: Build a router backed by a trie of path segments.; A Trie-Based URL Router →Autocomplete with a Trie →
triplet lossMetric Learning: The triplet loss pushes an anchor closer to a positive than to a negative by a margin: Random triplets are mostly too easy.; Related:Batch-hard mining; Triplet Loss with Batch-Hard Mining →
truncates toward zeroStacks: Division truncates toward zero (so `-7 / 2` is `-3`).; Related:infix Reverse Polish Notation shunting-yard; Evaluate Reverse Polish Notation →
trust ratioOptimizers: LAMB trains BERT with batch sizes of 64k by scaling each layer's Adam update by a trust ratio.; Related:LAMB; LAMB: Layer-wise Adaptive Large-Batch Optimisation →
TTL cacheDecorators: A TTL cache remembers results for a limited time.; Memoize with a Time-to-Live →
two heapsHeaps: The classic solution uses two heaps: a max-heap holding the smaller half and a min-heap holding the larger half, kept balanced so their sizes differ by at most one.; Related:median; Running Median of a Stream →
two or moreUI Logic: Replace any gap of two or more hidden pages with `"…"`; if only one page would be hidden, show it instead.; Pagination Links with Ellipses →
type of the first argumentDecorators: `functools.singledispatch` picks an implementation based on the type of the first argument.; Build Your Own singledispatch →
UCB1Experimentation: UCB1 is the classic algorithm: be optimistic about arms you haven't tried much.; Related:Bandits; Multi-Armed Bandit: UCB1 →
underestimatesA/B Testing: Treating every view as an independent coin flip underestimates the variance and produces false wins.; A/B Testing Ratio Metrics with the Delta Method →
underflowMixed Precision: Training in float16 is fast, but small gradients underflow to zero.; Related:float16 overflow scale; Dynamic Loss Scaling for Mixed Precision →Half-Precision Range and Loss Scaling →
unfoldCNNs: Frameworks instead unfold every receptive field into a row of a matrix (im2col), so the whole convolution becomes a single matrix multiplication.; Related:im2col; Fast Convolution with im2col →
unhashableLists: It must work for unhashable items (like dicts or lists) when a `key` that returns something hashable is supplied, and run in linear time.; Remove Duplicates, Keep Order →
uniformly at randomRandomized: Pick `k` items uniformly at random from a stream too big to store, or whose length you don't know in advance, such as sampling log lines for review.; Reservoir Sampling →
UnigramTokenizers: SentencePiece's Unigram tokenizer (used by T5, ALBERT and XLNet) gives each vocabulary piece a log-probability, and tokenizes a string by choosing the segmentation with the highest total log-probability.; Related:highest total log-probability Viterbi; Unigram Tokenization with Viterbi →
Union-findGraphs: Union-find (disjoint set union) groups them almost in constant time per operation.; Union-Find: Grouping Duplicates →
uniqueBacktracking: Return all unique permutations of a list that may contain duplicates, in lexicographic order, without generating duplicates and filtering them afterwards.; Related:in place lexicographic; Unique Permutations →
unique combinationsBacktracking: Given distinct positive `candidates` and a `target`, return all unique combinations that sum to the target, where each candidate may be used any number of times.; Related:any number of times; Combination Sum →
unroundedWindow Functions: Return each order's `id`, `customer_id`, `total`, `customer_avg` (that customer's average order, rounded to 2 decimals) and `vs_avg`: `'above'`, `'below'` or `'equal'` comparing the order to the unrounded average.; Each Order Compared with the Customer's Average →
Uplift modelsCausal Inference: Uplift models estimate the individual treatment effect, and the Qini curve evaluates them on a randomised experiment.; Related:changes Qini curve; Uplift Modelling and the Qini Curve →
upsampleVision: Segmentation models, super-resolution and diffusion U-Nets all need to upsample feature maps.; Upsampling: Nearest and Bilinear →
valueTrees: Implement `lca(root, p, q)` returning that node's value, iteratively in O(height).; Related:lowest common ancestor; Lowest Common Ancestor in a BST →
varianceModel Selection: The expected test error of a model splits into bias² (how far the average prediction is from the truth), variance (how much predictions move between training sets) and irreducible noise.; Related:bias² noise; Bias–Variance Decomposition by Simulation →
vector searchRAG: They run keyword search (BM25) and vector search side by side, then merge the ranked lists.; Related:keyword search; Hybrid Search with Reciprocal Rank Fusion →
ViterbiTokenizers: Finding it is a shortest-path problem solved with the Viterbi algorithm.; Related:highest total log-probability Unigram; Unigram Tokenization with Viterbi →
vLLM's PagedAttentionInference: vLLM's PagedAttention stores each sequence's KV cache in fixed-size blocks, like virtual memory pages, so memory isn't wasted on reserved-but-unused space.; Related:blocks; Paged KV-Cache Block Allocator →
Wald's SPRTA/B Testing: Wald's SPRT is designed to be checked after every observation.; Sequential Probability Ratio Test →
wasn't in the training dataPreprocessing: The production trap is a category at prediction time that wasn't in the training data.; Related:one-hot encoded; One-Hot Encoder with Unseen Categories →
weight of evidenceFeature Engineering: Credit scorecards encode a categorical feature by its weight of evidence and rank features by information value.; Related:information value; Weight of Evidence and Information Value →
weightsStatistics: Survey respondents and logged events often carry weights: one respondent may represent 500 people, another 2,000.; Weighted Mean, Variance and Survey Weights →
Welch's t-testStatistics: For continuous metrics, compare means with Welch's t-test, which, unlike Student's t-test, doesn't assume the two groups have equal variance.; Related:average order value; Welch's t-Test for Two Means →
WhiteningPreprocessing: Whitening transforms features so they're uncorrelated with unit variance (covariance = identity).; PCA and ZCA Whitening →
WinsorisingOutliers: Winsorising caps them instead: values below the p-th percentile are raised to it, and values above the (100 − p)-th are lowered to it.; Winsorise Outliers →
with ordersAggregation: Return each customer with orders: `name` and `order_ids`, a comma-separated list of their order ids (no spaces) in date order (ties by id).; Related:in date order strictly above; List Each Customer's Orders in One Row →Customers Spending Above Their Country's Average →
within their own countryWindow Functions: Regional managers want a leaderboard of customers by total spend within their own country.; Rank Customers Within Each Country →
without consuming itIterators: Parsers often need to look at the next item without consuming it.; A Peekable, Pushback Iterator →
without peeking at the futurePandas: Forecasting models need features from the past, such as yesterday's sales and last week's average, computed separately for each store and without peeking at the future.; Related:past; pandas: Lag and Rolling Features →
worst caseDynamic Programming: Find the minimum number of drops needed in the worst case to determine the highest floor an egg survives.; The Egg Drop Problem →
z-testSafety: Watermarked text uses far more, which a z-test detects.; Related:green list; Detect an LLM Text Watermark →