Prompt Engineering

Last modified: August 27, 2024

Introduction

Prompt engineering is the action of designing the input text that will be send to the model. This typically contains input from the end-user or generated by the app which is enriched with instructions from the developer/administrator or enriched by the app. A prompt typically contains the following:

instructions on what the model should do
context and information that the model needs to follow the instructions
the relevant input data (from the end-user or passed from a microflow)
the requested output structure (e.g. tone of voice or a JSON format)

Prompt engineering is very important to instruct the model to do what you would like it to do. It is therefore the first step to take when interacting with generative AI. When implementing patterns like RAG and ReAct, you can still change the behavior of the system by modifying the prompt. You need to explain the system how to use the knowledge and functions that are provided, otherwise it might ignore them, or act in a different way.

Typical Components of a Prompt

A prompt typically contains four components.

Instructions

Explain what the model should do. The model follows the instructions more easily if you:

have a specific task in mind
break it down into clear steps and create instructions that can be followed
explain what persona or role the model should fulfill

When the input text is coming directly from the end-user, also include what not to do.

Context and Additional Information

After telling the model what to do, you can include additional context. This can include:

information about the end-user of your application – for example their language, role, department, specific database records
context information – for example, all data related to an object the end-user is looking at
knowledge coming from Retrieval Augmented Generation (RAG)

Tip: you can provide information in a JSON or XML structure to ensure the information is presented in a consistent way. From Mendix apps, you can use Export Mappings to create JSON structures and Export XML Documents to create XML structures.

Input Data

The actual input, usually from an end-user.

Output Style

You can instruct the model to format the output in a specific way. For example:

tell the model to specify its reasoning steps, or just give the answer
give examples of the output style you want, for example a JSON structure, if you want to use a structured response to generate data or get structured information about intermediate steps taken, or decisions made, in coming up with a final response to a prompt.
request that responses are in a particular tone of voice, target a specific audience, or have a specified content length
request the use (or not) of Markdown formatting
ask the model to skip or include a preamble

Use an Iterative Approach

Mendix recommends that you test your prompt against different scenarios. Writing a prompt is therefore similar to modeling a microflow.

You should do the following:

Start with a goal: what should the model do?
Think about your test and edge cases: what should the model do in a particular situation?
Write a first version of the prompt.
Test your prompt against your test cases.
Refine the prompt, by tweaking your variables and writing defensive statements against undesired behavior.

There is a difference between how models behave. For example newer models might interpret instructions slightly differently, or be more elaborate. You should therefore retest your prompt when you switch models (for example, after moving from ChatGPT-3.5 to ChatGPT-4o).

Tips for Better Prompting

There are some techniques which have been found to produce better responses from Gen AI models.

Be Specific

Specificity and clarity are important. Like humans, large language models (LLMs) require specific instructions and cannot guess what you want. There is a difference between:

Write a story about Mendix

and

Write a story about Mendix.
It should be 1000 words long.
It should be focused on the business impact low code can make to a company.
Make it exciting and focused on developers.

Tip: if you are unsure about whether a prompt is clear enough, ask a co-worker to interpret the prompt and see if they would follow the prompt and reach your desired outcome.

Explicitly Teach the Model to Solve the Problem

Instead of relying on the model to come up with the best strategy to solve a problem, break the larger problem down into smaller steps.

Provide the model with instructions on the steps to solve the problem. This is normally done by providing one or more examples. This encourages the model to follow those instructions. As an end result, the quality of the output will be higher compared to asking the LLM to come up with the answer right away.

When you want the model to respond in a specific manner or syntax that is hard to describe, it can be particularly useful to provide examples. This technique is known as One-Shot-Prompting (1 example) or Few-Shot-Prompting (multiple examples).

You are an classification assistant.
You're job is to classify user reviews based on their sentiment.

<examples>
User prompt: I love the product!
Response: positive

User prompt: It didn't meet my expectations
Response: negative

User prompt: It's the best thing I ever bought
Response: positive
</examples>

Allow the Model to Say “I don’t know”

A model will always try to follow the instructions and can therefore come up with a response that might not be what you expect, or worse made up. This is known as hallucination.

If your prompt includes instructions which allow the LLM to ask for more info, or respond that it does not know something, this will make it more effective.

Example instructions are:

If you are unsure how to respond, say “Sorry, I didn’t get that. Could you rephrase the question or provide more details?”

You are a barista that only talks about coffee.
If a user asks something about other topics, say:
    “Sorry, as a barista I cannot help you with that. Would you like some recommendations on how to brew coffee?”

Or, when using RAG:

You are a helpful assistant that tries to answer user questions based on chunks of topic-specific data.
If you cannot answer a question based on the provided information alone, you respond that you do not know.
For the current question, please base the answer on the following pieces of information:
<information>
...
</information>

Define the Role the Model Fulfills

You can prime the model by explaining what it does. This will create a bias in the model towards specific reasoning and increase the quality of their answer based on what you expect from the (stereotypical) persona.

Examples are:

You are a helpdesk assistant

You are a writer that is specialized in marketing content

Tell the Model How to Use Provided Tools

When using features like function calling, give the functions a descriptive name. Also, instruct the model what the functions can do and how they should be used. This will guide the LLM to call the functions at the right moment and use the response in the correct way.

For example, say you have a tool called GetTicketInformationForIdentifier which retrieves information from a specific support ticket in a database; you could add the following to the prompt:

Do not make assumptions about the Ticket Identifier.
Ask for clarification if you do not know this.
Only use the ticket information from the GetTicketInformationForIdentifier function for answering questions on ticket information.

Provide Structure

When the prompt becomes longer it can help to use XML-like tags to give more structure to the prompt. This will help the model interpret the different sections and their role in the prompt better.

For example, you could use something like:

<instructions>
Answer the question from the user.
Base the answer on the articles provided.
Provide a reference to the articles where relevant.
</instructions>
<article>{article 1}</article>
<article>{article 2}</article>
<input>{user input}</input>
<output_formatting>
Write in a lively tone of voice.
Do not exceed 200 words.
Skip the preamble.
</output_formatting>

Learn More

Showcases

Check out the OpenAI and Bedrock showcase apps in the Marketplace to see how you can apply prompt engineering in practice to let a model perform specific tasks.