Few-Shot Prompting and In-Context Learning Explained (With Code)

Imagine teaching a brand new skill to a model, and not changing a single one of its billions of weights. No gradient descent, no training run, no GPU cluster. You just show it a few examples, right in the prompt, and it gets the idea. That sounds like magic, but it is something you can use today.

• No gradient updates
• No fine-tuning
• Just examples in the prompt

In the last episode we covered prompt patterns and system prompts, which are about telling the model what you want. This episode is about showing the model what you want. We will see what in-context learning is, why researchers still argue about how it works, and how to build a few-shot prompt that is reliable and affordable.

• Last time: prompt patterns
• Today: examples in the prompt

Here is the plan. First the definitions. Then the theory, with honest uncertainty. Then what actually makes examples good or bad, a worked example with similarity based selection, a Python prompt builder, the cost trade-off, and the classic mistakes. We will not cover chain of thought here. That is the next episode.

• Zero, one, few-shot
• Why examples matter
• kNN selection and code
• Cost and mistakes

Zero-shot means you give the model an instruction and no examples. Classify the sentiment of this review, and here is the review. The model relies entirely on what it picked up in pretraining and instruction tuning. For common tasks this works surprisingly well, and it is always the baseline you should try first.

• Instruction only
• No examples
• Relies on pretraining

One-shot adds a single worked example before your real input. Few-shot adds several, typically somewhere between two and a few dozen. Each example is an input and the output you want, written in the same format as your final query. The model sees the pattern, and then it continues it for the new input.

• One-shot: one example
• Few-shot: a handful
• Same frozen model

A few-shot prompt has three parts. An instruction, a list of demonstrations, and the final query with the answer left blank. Review colon, text, label colon, positive. Then again, and again. Then the new review, and label colon with nothing after it. The most natural continuation is the label, so the model writes it.

• Instruction
• Demonstrations
• Final query, blank answer

In-context learning is the ability of a model to adapt its behavior to a task based only on examples in its input. Compare it with fine-tuning. Fine-tuning changes the weights, and the change is permanent. In-context learning changes nothing in the weights. The adaptation lives in the activations during one forward pass, and it disappears when the prompt does.

• Weights fixed
• Behavior adapts
• Temporary: lasts one prompt

This was a real surprise when large language models appeared. Pretraining only asks the model to predict the next token. Nobody wrote a loss term for learn a task from three examples. Yet sufficiently large models do exactly that. As in the ML course's lessons on pretraining, the objective is simple, and capabilities fall out of it.

• Trained only to predict next token
• Few-shot ability emerged at scale

One explanation is task location, sometimes called task recognition. The model already learned many tasks during pretraining. The examples simply tell it which one you mean, and what the output format and label set look like. Under this view, even examples with randomly shuffled labels can still help a lot, because they reveal the task and the format.

• Model already knows many tasks
• Examples pick which one
• Format and label space matter

A competing explanation says the model really does learn something new from the examples. Some research shows that transformers can implement simple learning procedures, like fitting a linear function, inside the forward pass, with attention doing the work. Other work suggests that larger models pay more attention to the actual input label mapping. Both views have evidence behind them.

• Forward pass mimics learning
• Attention as an algorithm
• Larger models use labels more

So which is right? This is still active research, and it is not settled. The truth is probably a mixture that depends on the model size and the task. Do not repeat either story as proven fact. What matters for engineering is the practical lesson: examples strongly steer the model, and the details of those examples matter.

• Likely a mix of both
• Depends on model and task
• Practical takeaway stays the same

The first lever is which examples you choose. Examples that look like your real inputs help the most. If your inputs are short customer chats with typos and your examples are polished formal paragraphs, the model sees a mismatch. Studies across tasks have shown that the choice of examples alone can swing accuracy substantially.

• Relevant examples help most
• Cover the real input distribution
• Bad examples can hurt

The second lever is order. The same examples in a different order can give different predictions. Models often lean on the examples closest to the end of the prompt, which is a form of recency bias. If your last example is negative, the model may lean negative on ambiguous inputs. So test several orderings, and do not place a single label at the end every time.

• Order changes outputs
• Last example weighs heavily
• Shuffle and test

The third lever is label balance. If five of your six examples are positive, the model learns that positive is common, and it will over predict it. This is majority label bias. A safe default is a balanced set of examples. If your real data is heavily skewed, choose that skew on purpose and measure the result, rather than letting it happen by accident.

• Skewed labels skew predictions
• Balance the label counts
• Match real class priors deliberately

The fourth lever is format. Every example should use exactly the same structure, the same delimiters, and the same label words. If one example says positive and another says pos, you are teaching the model that either is fine. Consistent format is also why few-shot prompting is so good at format adherence. The model copies the pattern you give it.

• Consistent delimiters
• Same structure every time
• Clear label vocabulary

How many examples do you need? There is no universal number. Typically the first few examples give the biggest gain, then the curve flattens, while your token cost keeps growing in a straight line. Start with three to five good examples, measure, and add more only if the measurements show a real improvement. Hard or ambiguous tasks may benefit from more.

• Gains are biggest early
• Diminishing returns
• Context and cost limits

Let us design examples for a real task. We need to label customer support tickets into three intents: refund, shipping, or account. Here is a tricky ticket. My package never arrived and I want my money back. Is that shipping, or refund? This is exactly the kind of case where example design matters.

• Labels: refund, shipping, account
• Incoming ticket to label

Start with three examples, one for each label, so the set is balanced. Write them in the style of real tickets. Charged twice, please return the extra payment, labeled refund. Tracking has not changed in a week, labeled shipping. Cannot log in after changing my email, labeled account. Now add a borderline case, because borderline cases show the model where the boundary sits.

• One per label
• Realistic wording
• One borderline case

But a fixed set of examples has a limit. Suppose you have a pool of two hundred labeled tickets. Your three fixed examples might be far from the incoming query. A smarter approach is dynamic selection: for each new input, find the labeled examples most similar to it, and put those in the prompt. This is called kNN example selection.

• Same examples for every input
• Pool may be hundreds of labeled items
• Better: choose per query

Here is how it works with numbers. Turn the query and every pool item into vectors. Compute cosine similarity between the query and each item. Say the best matches score zero point nine one, zero point eight four, and zero point seven seven, while an unrelated billing question scores only zero point one two. With k equal to three, we take the top three and drop the rest.

• Embed the query and the pool
• Score by cosine similarity
• Take the top k

Once selected, order matters. A common choice is to put the most similar example last, right before the query, which uses recency bias in your favor. Also look at the labels of the neighbors. If two are refund and one is shipping, the model sees the real ambiguity of our tricky ticket, and it can decide with relevant evidence in front of it.

• Most similar goes last
• Check label mix of the neighbors
• Fall back if all neighbors agree

Now the code, in plain Python, provider neutral. The builder has four steps. Store a pool of labeled examples. Turn texts into vectors. Pick the top k most similar examples for a query. Then format everything into one prompt string that you send to any LLM API.

• Example pool
• Vectorizer
• Select, format, assemble

For a simple version, use TF-IDF from scikit-learn. Fit the vectorizer on the texts in your pool and transform them into a matrix. If you prefer, write a function called embed that takes a string and returns a vector, and leave it as a stub. Later you can swap in a real embedding model from the upcoming embeddings episodes without touching the rest.

• TF-IDF with scikit-learn
• Or an embedding function stub
• Fit on pool texts only

The selection function does this. Transform the query with the same vectorizer. Compute cosine similarity against every row of the pool matrix. Sort the scores from high to low, keep the first k indices, and then reverse that list so the most similar example ends up last. Return the matching example dictionaries.

• Transform the query
• Cosine similarity to all
• argsort, take top k

Finally the formatter. Start with a one line instruction listing the allowed labels. Loop over the selected examples and append ticket colon text, newline, intent colon label. End with the new ticket and intent colon, left open. Send that string to your model with temperature zero for classification, and parse the first word of the reply.

• Instruction first
• Identical format per example
• End with open label

Few-shot is not free. Every example is input tokens, and you pay for them on every single request. Suppose each example is sixty tokens and you use eight. That is four hundred eighty tokens added to every call, before your query. At a million calls, that overhead is large, and longer prompts also add latency.

• Every example is input tokens
• Paid on every single call
• Latency grows too

Many providers support prompt caching. If the beginning of your prompt is identical across requests, the system can reuse the work already done for that prefix, typically at lower cost and latency. Details vary by provider, so check yours. The design rule is simple: put static content first, and dynamic content last.

• Reuse a shared prefix
• Cached prefix is cheaper and faster
• Put static content first

Here is a trade-off. Static examples cache well but are not tailored. Dynamic kNN examples are tailored but change the middle of the prompt, which breaks the cache after that point. A practical hybrid is a small fixed core of examples at the top that can be cached, then two or three retrieved examples, then the query.

• Static examples cache well
• kNN examples improve relevance
• Hybrid: fixed core plus retrieved extras

Where does few-shot shine? Classification and labeling, like our intent example. Style transfer, where two or three before and after pairs teach a brand voice better than a paragraph of adjectives. And format adherence, where examples show the exact layout of the output, so downstream code can parse it. Models copy patterns very well.

• Classification and labeling
• Style transfer
• Format adherence

Mistake number one is leaking test examples. If an item you evaluate on also appears in your prompt, even through kNN retrieval from a pool that includes the test set, the model can just copy the answer. Your scores look great and mean nothing. Split your data first, retrieve only from the training pool, and also remove exact duplicates.

• Test items inside the prompt
• Inflated evaluation scores
• Split pool and test set first

Mistake number two is unrepresentative examples. If you only show clean, easy cases, the model struggles on messy production text. If a rare class never appears in your examples, the model may never predict it. Sample examples from real traffic, include hard and borderline cases, and revisit them as your data drifts.

• Only easy, clean cases
• Missing rare classes
• Mirror production inputs

Mistake number three is label bias. Skewed label counts, a repeated label at the end, or a few examples that are simply wrong can all push predictions in the wrong direction. Review your examples by hand, balance them on purpose, shuffle the order across experiments, and always measure on held-out data rather than trusting a single prompt that looked good on three inputs.

• Skewed or wrongly labeled examples
• Wrong labels teach wrong rules
• Always evaluate on held-out data

A final practical note. Start with zero-shot, move to few-shot when you need more control, and consider fine-tuning only if you have lots of labeled data and the prompt gets too long or expensive. We will compare fine-tuning, retrieval, and prompting properly in episode twenty six. For now, few-shot is the cheapest experiment you can run.

• Hundreds of examples available
• Prompt too long or costly
• Consider fine-tuning later

Let us recap. Zero-shot has no examples, one-shot has one, few-shot has several. In-context learning adapts behavior with no weight updates, and why it works is still an active research question. Example choice, order, label balance and format all change results. Use kNN selection for relevance, watch your token cost, use caching, and avoid leaks, bias and unrepresentative examples.

• Zero, one, few-shot
• ICL: no weight updates
• Choice, order, balance, format
• kNN selection, cost, mistakes

Examples teach a model what the answer should look like. But some problems need more than a pattern to copy. They need the model to work through steps before it answers. That is the topic of the next episode: chain of thought and reasoning models.

• Examples show what to answer
• Next: how models think before answering