Prompt Engineering Guide for Data Analysts | by Olivia Tanuwidjaja | May, 2023

Prompt engineering is a growing field, with research on this topic rapidly increasing from 2022 onwards. Some of the state-of-the-art prompting techniques commonly used include n-shot prompting, chain-of-thought (CoT) prompting, and generated knowledge prompting.

A sample Python notebook demonstrating these techniques is shared under this GitHub project.

1. N-shot prompting (Zero-shot prompting, Few-shot prompting)

Known for its variation like Zero-shot prompting and Few-shot prompting, the N in N-shot prompting represents the number of “training” or clues given to the model to make predictions.

Zero-shot prompting is where a model makes predictions without any additional training. This works for common straightforward problems like classification (i.e. sentiment analysis, spam classification), text transformation (i.e. translation, summarizing, expanding), and simple text generation on which the LLM has been largely trained.

Zero-shot prompting: Straightforwardly ask the model on sentiment (Image by Author)

Few-shot prompting uses a small amount of data (typically between two and five) to adapt its output based on these small examples. These examples are meant to steer the model to better performance for a more context-specific problem.

Few-shot prompting: Give examples of how we expect the model output to be

2. Chain-of-Thought (CoT) prompting

Chain-of-Thought prompting was introduced by Google researchers in 2022. In the Chain-of-Thought prompting, the model is prompted to produce intermediate reasoning steps before giving the final answer to a multi-step problem. The idea is that a model-generated chain of thought would mimic an intuitive thought process when working through a multi-step reasoning problem.

Chain-of-Thought prompting helps in driving the model to break down problems accordingly

This method enables models to decompose multi-step problems into intermediate steps, enabling them to solve complex reasoning problems that are not solvable with standard prompting methods.

Some further variations of Chain-of Thought prompting include:

  • Self-consistency prompting: Sample multiple diverse reasoning paths and select the most consistent answers. By utilizing a majority voting system, the model can arrive at more accurate and reliable answers.
  • Least-to-Most prompting (LtM): Specify the chain of thought to first break a problem into a series of simpler subproblems and then solve them in sequence. Solving each subproblem is facilitated by the answers to previously solved subproblems. This technique is inspired by real-world educational strategies for children.
  • Active Prompting: Scaling the CoT approach by determining which questions are the most important and helpful ones for human annotation. It first calculates the uncertainty among the LLM’s predictions, then select the most uncertain questions, and these questions are selected for human annotation before being put into a CoT prompt.

3. Generated knowledge prompting

The idea behind the generated knowledge prompting is to ask the LLM to generate potentially useful information about a given question/prompt, and then leverage that provided knowledge as additional input for generating a final response.

For example, say you want to write an article about cybersecurity, particularly cookie theft. Before asking the LLM to write the article, you can ask it to generate some danger and protection against cookie theft. This will help the LLM write a more informative blog post.

Generated knowledge prompting: (1) Ask the model to generate some content
Generated knowledge prompting: (2) Use the generated content as input to the model

Additional tactics

On top of the above-specified techniques, you can also use these tactics below to make the prompting more effective

  • Use delimiters like triple backticks (“`), angle brackets (<>), or tags (<tag> </tag>) to indicate distinct parts of the input, making it cleaner for debugging and avoiding prompt injection.
  • Ask for structured output (i.e. HTML/JSON format), this is useful for using the model output for another machine processing.
  • Specify the intended tone of the text to get the tonality, format, and length of model output that you need. For example, you can instruct the model to formalize the language, generate not more than 50 words, etc.
  • Modify the model’s temperature parameter to play around the model’s degree of randomness. The higher the temperature, the model’s output would be random than accurate, and even hallucinate.

A sample Python notebook demonstrating these techniques is shared under this GitHub project.

Source link

Leave a Comment