Prompt Engineering
Introduction
Prompt engineering is the action of designing the input text that will be send to the model. This typically contains input from the end-user or generated by the app which is enriched with instructions from the developer/administrator or enriched by the app. A prompt typically contains the following:
- instructions on what the model should do
- context and information that the model needs to follow the instructions
- the relevant input data (from the end-user or passed from a microflow)
- the requested output structure (e.g. tone of voice or a JSON format)
Prompt engineering is very important to instruct the model to do what you would like it to do. It is therefore the first step to take when interacting with generative AI. When implementing patterns like RAG and ReAct, you can still change the behavior of the system by modifying the prompt. You need to explain the system how to use the knowledge and functions that are provided, otherwise it might ignore them, or act in a different way.
Typical Components of a Prompt
A prompt typically contains four components.
Instructions
Explain what the model should do. The model follows the instructions more easily if you:
- have a specific task in mind
- break it down into clear steps and create instructions that can be followed
- explain what persona or role the model should fulfill
When the input text is coming directly from the end-user, also include what not to do.
Context and Additional Information
After telling the model what to do, you can include additional context. This can include:
- information about the end-user of your application – for example their language, role, department, specific database records
- context information – for example, all data related to an object the end-user is looking at
- knowledge coming from Retrieval Augmented Generation (RAG)
Tip: you can provide information in a JSON or XML structure to ensure the information is presented in a consistent way. From Mendix apps, you can use Export Mappings to create JSON structures and Export XML Documents to create XML structures.
Input Data
The actual input, usually from an end-user.
Output Style
You can instruct the model to format the output in a specific way. For example:
- tell the model to specify its reasoning steps, or just give the answer
- give examples of the output style you want, for example a JSON structure, if you want to use a structured response to generate data or get structured information about intermediate steps taken, or decisions made, in coming up with a final response to a prompt.
- request that responses are in a particular tone of voice, target a specific audience, or have a specified content length
- request the use (or not) of Markdown formatting
- ask the model to skip or include a preamble
Use an Iterative Approach
Mendix recommends that you test your prompt against different scenarios. Writing a prompt is therefore similar to modeling a microflow.
You should do the following:
- Start with a goal: what should the model do?
- Think about your test and edge cases: what should the model do in a particular situation?
- Write a first version of the prompt.
- Test your prompt against your test cases.
- Refine the prompt, by tweaking your variables and writing defensive statements against undesired behavior.
There is a difference between how models behave. For example newer models might interpret instructions slightly differently, or be more elaborate. You should therefore retest your prompt when you switch models (for example, after moving from ChatGPT-3.5 to ChatGPT-4o).
Tips for Better Prompting
There are some techniques which have been found to produce better responses from Gen AI models.
Be Specific
Specificity and clarity are important. Like humans, large language models (LLMs) require specific instructions and cannot guess what you want. There is a difference between:
Write a story about Mendix
and
Write a story about Mendix.
It should be 1000 words long.
It should be focused on the business impact low code can make to a company.
Make it exciting and focused on developers.
Tip: if you are unsure about whether a prompt is clear enough, ask a co-worker to interpret the prompt and see if they would follow the prompt and reach your desired outcome.
Explicitly Teach the Model to Solve the Problem
Instead of relying on the model to come up with the best strategy to solve a problem, break the larger problem down into smaller steps.
Provide the model with instructions on the steps to solve the problem. This is normally done by providing one or more examples. This encourages the model to follow those instructions. As an end result, the quality of the output will be higher compared to asking the LLM to come up with the answer right away.
When you want the model to respond in a specific manner or syntax that is hard to describe, it can be particularly useful to provide examples. This technique is known as One-Shot-Prompting (1 example) or Few-Shot-Prompting (multiple examples).
You are an classification assistant.
You're job is to classify user reviews based on their sentiment.
<examples>
User prompt: I love the product!
Response: positive
User prompt: It didn't meet my expectations
Response: negative
User prompt: It's the best thing I ever bought
Response: positive
</examples>
Allow the Model to Say “I don’t know”
A model will always try to follow the instructions and can therefore come up with a response that might not be what you expect, or worse made up. This is known as hallucination.
If your prompt includes instructions which allow the LLM to ask for more info, or respond that it does not know something, this will make it more effective.
Example instructions are:
If you are unsure how to respond, say “Sorry, I didn’t get that. Could you rephrase the question or provide more details?”
You are a barista that only talks about coffee.
If a user asks something about other topics, say:
“Sorry, as a barista I cannot help you with that. Would you like some recommendations on how to brew coffee?”
Or, when using RAG:
You are a helpful assistant that tries to answer user questions based on chunks of topic-specific data.
If you cannot answer a question based on the provided information alone, you respond that you do not know.
For the current question, please base the answer on the following pieces of information:
<information>
...
</information>
Define the Role the Model Fulfills
You can prime the model by explaining what it does. This will create a bias in the model towards specific reasoning and increase the quality of their answer based on what you expect from the (stereotypical) persona.
Examples are:
You are a helpdesk assistant
You are a writer that is specialized in marketing content
Tell the Model How to Use Provided Tools
When using features like function calling, give the functions a descriptive name. Also, instruct the model what the functions can do and how they should be used. This will guide the LLM to call the functions at the right moment and use the response in the correct way.
For example, say you have a tool called GetTicketInformationForIdentifier
which retrieves information from a specific support ticket in a database; you could add the following to the prompt:
Do not make assumptions about the Ticket Identifier.
Ask for clarification if you do not know this.
Only use the ticket information from the GetTicketInformationForIdentifier function for answering questions on ticket information.
Provide Structure
When the prompt becomes longer it can help to use XML-like tags to give more structure to the prompt. This will help the model interpret the different sections and their role in the prompt better.
For example, you could use something like:
<instructions>
Answer the question from the user.
Base the answer on the articles provided.
Provide a reference to the articles where relevant.
</instructions>
<article>{article 1}</article>
<article>{article 2}</article>
<input>{user input}</input>
<output_formatting>
Write in a lively tone of voice.
Do not exceed 200 words.
Skip the preamble.
</output_formatting>
Learn More
Showcases
Check out the OpenAI and Bedrock showcase apps in the Marketplace to see how you can apply prompt engineering in practice to let a model perform specific tasks.