Prompt Engineering

https://www.coursera.org/learn/chatgpt-prompt-engineering-for-developers-project/home/welcome

Prompt engineering is the practice of designing and refining the input you give a large language model so it produces the output you actually want. It took off after GPT-3 came out, and it’s basically the bridge between what a user wants and what the model can do.

Key papers:

  • Brown, T. B., et al. (2020). “Language Models are Few-Shot Learners.” NeurIPS. [link] — the GPT-3 paper, the first to really demonstrate the potential of prompt engineering
  • Wei, J., et al. (2022). “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.” NeurIPS. [link]

1. Prompting Principles

Principle 1: Write clear and specific instructions

Clarity and specificity are the two things that matter most when writing a prompt. Vague instructions tend to produce unpredictable outputs, while specific ones noticeably improve output quality.

  • Tactic 1: Use delimiters to clearly indicate distinct parts of the input

    Delimiters like ```, """, ---, or <> help the model tell different parts of your input apart, which avoids confusion — especially useful when you’re feeding in multiple chunks of text or code at once.

  • Tactic 2: Ask for a structured output

    Asking for structured output (JSON, HTML, a Markdown table) makes the result easy to parse downstream, which is basically a best practice for building anything on top of an LLM.

  • Tactic 3: Ask the model to check whether conditions are satisfied

    Having the model verify conditions before it acts cuts down on wrong outputs.

  • Tactic 4: “Few-shot” prompting — give examples

    Few-shot prompting gives the model a few examples so it can pick up the format and the kind of output you expect.

    Key papers:

    • Brown, T. B., et al. (2020). “Language Models are Few-Shot Learners.” NeurIPS. [link] — introduced the idea of few-shot learning
    • Min, S., et al. (2022). “Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?” EMNLP. [link]

Principle 2: Give the model time to “think”

This comes out of chain-of-thought research — letting the model reason step by step instead of jumping straight to an answer.

Key papers:

  • Wei, J., et al. (2022). “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.” NeurIPS. [link] — the paper that started it all

  • Kojima, T., et al. (2022). “Large Language Models are Zero-Shot Reasoners.” NeurIPS. [link] — where “let’s think step by step” comes from

  • Wang, X., et al. (2022). “Self-Consistency Improves Chain of Thought Reasoning in Language Models.” ICLR. [link]

  • Tactic 1: Specify the steps required to complete a task

    Spell out the steps so the model follows them in order.

  • Tactic 2: Instruct the model to work out its own solution before rushing to a conclusion

    Have the model work the problem out on its own before it commits to an answer, instead of guessing right away.

About Hallucinations

A hallucination is when the model outputs something that sounds plausible but is actually wrong or made up.

Key papers:

  • Maynez, J., et al. (2020). “On Faithfulness and Factuality in Abstractive Summarization.” ACL. [link]
  • Zhang, Y., et al. (2023). “Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models.” arXiv. [link]

Ways to reduce it:

  • Ask the model to find relevant information first and answer based on that information
  • Ask the model to cite its sources
  • Use a RAG system to feed it reliable context

2. Iterative Prompt Development

Prompt engineering is iterative — you don’t get it right the first time, you keep testing and tweaking.

Key methodology:

  • Zhou, Y., et al. (2022). “Large Language Models Are Human-Level Prompt Engineers.” ICLR. [link] — APE (Automatic Prompt Engineer)

The loop:

  • Try something (write an initial prompt)
  • Analyze where the result falls short
  • Clarify the instructions, give the model more room to think
  • Refine the prompt against a batch of examples

3. Summarizing

Summarization is one of the most common NLP tasks, and LLMs are quite good at it.

Key papers:

  • See, A., et al. (2017). “Get To The Point: Summarization with Pointer-Generator Networks.” ACL. [link]
  • Liu, Y., & Lapata, M. (2019). “Text Summarization with Pretrained Encoders.” EMNLP. [link]

Things worth knowing:

  • Summarize with a word/sentence/character limit
  • Summarize with a focus on a specific topic, like shipping and delivery
  • Try “extract” instead of “summarize” when you only care about one particular topic
  • Summarize across multiple product reviews at once

4. Inferring

LLMs are pretty good at extracting specific things out of a piece of text.

They’re especially good at pulling structured information out of unstructured text.

Key paper:

  • Radford, A., et al. (2019). “Language Models are Unsupervised Multitask Learners.” — the GPT-2 paper, which demonstrated zero-shot capability [link]

Use cases:

  • Sentiment: positive or negative

    Key paper: Socher, R., et al. (2013). “Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank.” EMNLP.

  • Identifying types of emotion: happy, grateful, angry, etc.

    Key paper: Mohammad, S. M., & Turney, P. D. (2013). “Crowdsourcing a Word-Emotion Association Lexicon.” Computational Intelligence.

  • Identifying a specific emotion, e.g. “is the writer expressing anger?”

  • Extracting product and company names from customer reviews (named entity recognition)

    Key paper: Nadeau, D., & Sekine, S. (2007). “A survey of named entity recognition and classification.” Lingvisticae Investigationes.

  • Inferring topics and having the LLM answer in JSON

    Key paper: Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). “Latent Dirichlet Allocation.” JMLR. — the paper that started topic modeling with LDA

  • Setting up a news alert for certain topics (zero-shot learning)

    Key papers:

    • Radford, A., et al. (2021). “Learning Transferable Visual Models From Natural Language Supervision.” ICML. — the CLIP paper [link]
    • Xian, Y., et al. (2017). “Zero-Shot Learning - A Comprehensive Evaluation of the Good, the Bad and the Ugly.” TPAMI. [link]

5. Transforming

LLMs are good at all kinds of text transformation — translation, tone shifts, format conversion, you name it.

Key paper:

  • Vaswani, A., et al. (2017). “Attention Is All You Need.” NeurIPS. [link] — the Transformer paper, the foundation of modern NLP

Types of transformation:

  • Language translation

    Key paper: Wu, Y., et al. (2016). “Google’s Neural Machine Translation System.” arXiv. [link]

  • Tone: informal to formal

  • Format conversion: JSON to HTML

  • Spelling/grammar checks, or matching a given style such as APA

    Key paper: Yuan, Z., et al. (2021). “Synthesizing Coherent Story with Generative Pre-trained Transformer.” IJCAI.

6. Expanding

Expanding is the opposite of summarizing — turning a short input into a longer, more detailed output.

Key paper:

  • Fan, A., et al. (2018). “Hierarchical Neural Story Generation.” ACL. [link]

Things worth knowing:

  • Customize an automated reply to a customer email
  • Remind the model to pull in details from the customer’s email
  • Use temperature to control the answer: more creative, or more stable

A note on temperature

Temperature controls how random an LLM’s output is, and the idea actually comes from statistical mechanics and simulated annealing.

Key papers:

  • Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985). “A Learning Algorithm for Boltzmann Machines.” Cognitive Science. — the theoretical basis for simulated annealing
  • Ficler, J., & Goldberg, Y. (2017). “Controlling Linguistic Style Aspects in Neural Language Generation.” EMNLP. [link]

7. Chatbots

Chatbots are the most direct application of LLMs.

Key papers:

  • Vinyals, O., & Le, Q. (2015). “A Neural Conversational Model.” ICML Deep Learning Workshop. [link]
  • Zhang, Y., et al. (2020). “DIALOGPT: Large-Scale Generative Pre-training for Conversational Response Generation.” ACL. [link]

Three Roles

Modern chatbot systems are usually built around three roles:

  • System: defines the assistant’s overall behavior and persona
  • User: what the user says
  • Assistant: what the model replies

Chatbots follow the system role first, so you can use it to lock in specific behavior.

## example
messages =  [  
{'role':'system', 'content':'You are an assistant that speaks like Shakespeare.'},    
{'role':'user', 'content':'tell me a joke'},   
{'role':'assistant', 'content':'Why did the chicken cross the road'},   
{'role':'user', 'content':'I don\'t know'}  ]

Context

Context: all the messages across the three roles that the chatbot needs in order to generate its next response.

Managing context well is one of the core engineering challenges in building a chatbot system — how do you make good use of the conversation history without it getting out of hand?

Key paper:

  • Bae, S., et al. (2022). “Keep Me Updated! Memory Management in Long-term Conversations.” EMNLP. [link]