Advanced Prompt Engineering Techniques: From Few-Shot to Chain-of-Thought | FromZero2AI Blog

As Large Language Models (LLMs) grow in parameter size and training quality, prompt engineering has evolved from a simple art of querying to a structured methodology of programming in natural language. By structuring prompts carefully, we can steer model outputs, reduce hallucinations, and unlock reasoning capabilities.

In this guide, we will analyze advanced prompt engineering techniques: Zero-shot vs. Few-shot learning, Chain-of-Thought (CoT) prompting, and the Self-Consistency decoding strategy. We will also write a Python script simulating consistency voting.

Prompt Engineering Techniques: Flowchart mapping Few-Shot learning and Chain-of-Thought reasoning steps Prompt Engineering Techniques: Flowchart mapping Few-Shot learning and Chain-of-Thought reasoning steps

1. Zero-Shot vs. Few-Shot Prompting

LLMs are trained on massive text corpora, giving them broad zero-shot capabilities. However, when you require a specific formatting style, output constraint, or domain-specific logic, Few-Shot Prompting is the standard approach.

Zero-Shot Prompting

You ask the model to perform a task without giving it any examples.

Classify the following review as Positive or Negative:
"The battery life is decent, but the screen is prone to scratches."

Few-Shot Prompting

You provide one or more high-quality input-output examples (exemplars) in the prompt to demonstrate the desired behavior.

Classify the following reviews:

Review: "The camera is incredible, and the design is sleek."
Sentiment: Positive

Review: "It broke within two days of usage. Highly disappointed."
Sentiment: Negative

Review: "The battery life is decent, but the screen is prone to scratches."
Sentiment:

Providing structured exemplars establishes context, helps the model understand your constraints, and leads to more reliable classification.

2. Chain-of-Thought (CoT) Prompting

Introduced by Wei et al. (2022), Chain-of-Thought prompting dramatically improves performance on complex reasoning, mathematical, and symbolic tasks. Instead of asking the model to output the final answer directly, CoT prompts the model to generate intermediate reasoning steps.

The simplest way to invoke CoT is to append the phrase "Let's think step by step" to the end of the prompt.

Why CoT Works

Autoregressive LLMs predict the next token based on all previous tokens. If you ask a model to solve a complex math puzzle and output only the final number, the model has to perform the entire reasoning pathway in a single forward pass. By prompting the model to write out its reasoning steps, we give it "scratchpad" space to break down the problem, calculate intermediate values, and base its final answer on those step-by-step calculations.

       Standard Prompt                             Chain-of-Thought Prompt
[ Input Math Question ]                    [ Input Math Question ]
           |                                           |
           v                                           v
[ Instant Final Output (Error-prone) ]     [ Step-by-Step Reasonings (Scratchpad) ]
                                                       |
                                                       v
                                           [ Final Calculated Answer (Accurate) ]

3. Self-Consistency: Voting on the Best Path

Even with Chain-of-Thought prompting, a model can still make a minor mathematical or logical error in its reasoning path, leading to an incorrect final answer. Self-Consistency (Wang et al., 2022) mitigates this by:

Generating multiple different reasoning paths (e.g., using a temperature > 0).
Parsing the final answer from each reasoning path.
Taking a majority vote to select the most consistent final answer.

                                    +---> Path 1: "... So the answer is 42."
                                    |
[ Input Query with CoT Prompt ] ----+---> Path 2: "... Therefore, we get 42." ---> [ Majority Vote: 42 ]
                                    |
                                    +---> Path 3: "... Thus, it yields 18."

4. Simulating Self-Consistency in Python

Below is a Python demonstration showing how to programmatically implement a self-consistency checker. We mock LLM outputs for a math question and run a consensus algorithm.

from collections import Counter
import re

# Mocking LLM outputs generated with temperature = 0.7 for the question:
# "A toy store has 15 blocks. They sell 4, then buy 8. How many blocks do they have?"
mock_llm_outputs = [
    "Let's think step-by-step. The store starts with 15 blocks. Selling 4 means 15 - 4 = 11. Then they buy 8, which is 11 + 8 = 19. The answer is 19.",
    "First, we subtract 4 from 15, leaving 11 blocks. Next, we add 8 blocks because they bought more. 11 + 8 = 19. The final answer is 19.",
    "Starting with 15. Sold 4: 15 - 4 = 11. Bought 8: 11 + 8 = 18. So the answer is 18.", # Hallucination/calculation error
    "We have 15 blocks. 15 - 4 = 11. They add 8 blocks: 11 + 8 = 19. Therefore, the answer is 19."
]

def extract_final_number(text):
    """Regex helper to extract the final number from the LLM text output."""
    numbers = re.findall(r'\b\d+\b', text)
    return int(numbers[-1]) if numbers else None

def run_self_consistency(outputs):
    answers = []
    for out in outputs:
        answer = extract_final_number(out)
        if answer is not None:
            answers.append(answer)
            print(f"Path reasoning extracted answer: {answer}")

    # Count the frequencies of each answer
    consensus = Counter(answers)
    most_common_answer, count = consensus.most_common(1)[0]
    
    print(f"\nConsensus Votes: {dict(consensus)}")
    print(f"Final Selection: {most_common_answer} (Confidence: {count}/{len(outputs)})")
    return most_common_answer

# Run the consistency voting pipeline
final_answer = run_self_consistency(mock_llm_outputs)

Summary

Notice what the self-consistency script actually did: it ran the model four times to answer one arithmetic question. That is the part that gets skipped in most write-ups of the technique. Self-consistency is a 4x to 10x cost and latency multiplier applied to every query, which makes it appropriate for offline batch reasoning, high-stakes single decisions, or generating training data — and inappropriate for an interactive chat endpoint where the user is watching a spinner. Reach for it when a wrong answer is more expensive than five extra API calls, and not by default.

The naive majority vote also has a sharp edge worth knowing about. It works when errors are random and correct paths converge, which is the assumption behind the technique. When a model has a systematic misconception — it consistently misreads the problem the same way — all five paths agree on the same wrong answer, and voting returns it with high apparent confidence. Consensus is not calibration. Disagreement across paths is a genuinely useful signal that a question is hard; agreement is much weaker evidence than it feels like.

One more caveat that has aged into relevance: appending "let's think step by step" to a modern reasoning model is at best redundant and can be actively counterproductive, since those models already produce internal reasoning traces and explicit CoT instructions sometimes interfere with them. Chain-of-thought was a discovery about a specific generation of models, not a permanent law. Check whether it helps on the model you are actually calling — for a lot of current models it does not, and the extra tokens cost you.

If you take one habit from all of this, make it running an A/B test on your own examples rather than trusting a published technique. Prompt patterns that produce dramatic gains on GSM8K frequently produce nothing on a classification task with a well-specified output format. The exemplars in a few-shot prompt are usually worth more than the cleverness of the framing.