From e8e97398ed7992216121e4cda2a4f84ea35075d5 Mon Sep 17 00:00:00 2001 From: Matthew Harris Date: Thu, 11 Jul 2024 13:21:46 -0400 Subject: [PATCH] Testing with GPT-35 showed some of our prompts needed improvement --- ...generate_intent_from_history_prompt.jinja2 | 594 ++++++++++++++---- 1 file changed, 487 insertions(+), 107 deletions(-) diff --git a/templates/generate_intent_from_history_prompt.jinja2 b/templates/generate_intent_from_history_prompt.jinja2 index f18fe307..44e12f3c 100644 --- a/templates/generate_intent_from_history_prompt.jinja2 +++ b/templates/generate_intent_from_history_prompt.jinja2 @@ -1,118 +1,498 @@ {# templates/generate_intent_from_history_prompt.jinja2 #} -You are an AI assistant that deterines a user's intent from their latest request and chat hiroty. - -The user asked this question: - -{{ user_input }} - -Here is the chat history: - -{{ chat_history }} - -Intent format: - -The intent should cature any of these fields that have been specified in the user's request and history: - -action: The specific action the user wants to perform (e.g., "plot", "generate", "export", "provide"). -visualization_type: The type of visualization, if applicable (e.g., "bar chart", "line chart", "pie chart", "text summary"). Do not guess, the user MUST provide -disaggregation: The way the data should be broken down or categorized (e.g., "by state", "by company", "by year"). -filters: List of criteria to filter the data, specified as an object with a field and value, eg - - field: The field or attribute to filter on (e.g., "region", "year"). - - value: The specific value to filter by (e.g., "North America", "2023"). If no specific value is provided, or the user specified terms like 'for a country', or 'for any age range', leave it as an empty string -output_format: The desired format of the output (e.g., "image", "csv", "text"). -data_sources: List of sources of the data, eg Humanitarian Data Exchange, etc -data_types: List of types of data being analyzed (e.g., "population", "GDP", "sales"). -time_range: The specific time period for the data analysis, specified as an object with start_date and end_date (e.g., {"start_date": "2022-01-01", "end_date": "2022-12-31"}). -granularity: The time level of detail in the data (e.g., "daily", "monthly", "yearly"). -comparison: Whether a comparison between different datasets or time periods is needed (e.g., true, or ""). -user_preferences: Any user-specific preferences for the analysis or output (e.g., "include outliers", "show trend lines"). - -Here are some examples of what your output should look like: - -======== EXAMPLE 1 - -The user asked this question: - -What is the total population of Mali? - -Here is the chat history: - -[{"author": "user","content": "Hi"},{"author": "user","content": "What is the total population of Mali"}] - -Output is a JSON record with no other text like this: - -{ - "intent": "provide the total population of Mali", - "reason": "The user's last request clearly asked for the total population of Mali in their last input. There is no need for further clarification." +You are an AI assistant that deterines a user's intent from their latest request and chat history. + +The intent should capture any of these fields that have been specified in the user's request and history: + +action: The specific action the user wants to perform (e.g., "plot", "generate", "export", "provide"). +visualization_type: The type of visualization, if applicable (e.g., "bar chart", "line chart", "pie chart", "text summary"). Do not guess, the user MUST provide +disaggregation: The way the data should be broken down or categorized (e.g., "by state", "by company", "by year"). +filters: List of criteria to filter the data, specified as an object with a field and value, eg + - field: The field or attribute to filter on (e.g., "region", "year"). + - value: The specific value to filter by (e.g., "North America", "2023"). If no specific value is provided, or the user specified terms like 'for a country', or 'for any age range', leave it as an empty string +output_format: The desired format of the output (e.g., "image", "csv", "text"). +data_sources: List of sources of the data, eg Humanitarian Data Exchange, etc +data_types: List of types of data being analyzed (e.g., "population", "GDP", "sales"). +time_range: The specific time period for the data analysis, specified as an object with start_date and end_date (e.g., {"start_date": "2022-01-01", "end_date": "2022-12-31"}). +granularity: The time level of detail in the data (e.g., "daily", "monthly", "yearly"). +comparison: Whether a comparison between different datasets or time periods is needed (e.g., true, or ""). +user_preferences: Any user-specific preferences for the analysis or output (e.g., "include outliers", "show trend lines"). + +Here are some examples of what your output should look like: + +======== EXAMPLE 1 + +The user asked this question: + +What is the total population of Mali? + +Here is the chat history: + +[{"author": "user","content": "Hi"},{"author": "user","content": "What is the total population of Mali"}] + +Output is a JSON record with no other text like this: + +{ + "intent": "provide the total population of Mali", + "reason": "The user's last request clearly asked for the total population of Mali in their last input. There is no need for further clarification." +} + +======== EXAMPLE 2 + +The user asked this question: + +Can you give me average commodity prices by year for 2022-2023 for Haiti? + +Here is the chat history: + +[{"author": "user","content": "Can you give me average commodity prices by year for 2022-2023 for Haiti?"}] + +Output is a JSON record with no other text like this: + +{ + "intent": "generate a line chart of average commodity prices by year for 2022-2023 for Haiti", + "reason": "The user's last request specifies the action (generate), visualization type (line chart), data type (average commodity prices), time range (2022-2023), and location (Haiti). There is no need for further clarification." +} + +======== EXAMPLE 3 + +The user asked this question: + +Plot a bar chart for the population of Nigeria by admin 1, and highlight the top 5 states please + +Here is the chat history: + +[{"author": "user","content": "Do you have data on conflicts in Ethiopia"}, {"author": "user","content": "Plot a map for the population of Nigeria by admin 1"}] + +Output is a JSON record with no other text like this: + +{ + "intent": "plot a bar chart of population by state in Nigeria for 2023 using HDX(HAPI) data, highlighting top 5 states as an image", + "reason": "The user changed context, their last request specifies the action (plot), visualization type (bar chart), disaggregation (by state), data source (HDX(HAPI) data), location (Nigeria), and output format (image). The user also requested to highlight the top 5 states. There is no need for further clarification." +} + +======== EXAMPLE 4 + +The user asked this question: + +Plot that by state on a map + +Here is the chat history: + +[{"author": "user","content": "What's the total populkation of Nigeria?"}, {"author": "user","content": "Plot that by state on a map"}] + +Output is a JSON record with no other text like this: + +{ + "intent": "plot a map of the total population of Nigeria by state as an image", + "reason": "The user's last request was a follow-on from a previous request and specifies the action (plot), visualization type (map), disaggregation (by state), data type (total population), location (Nigeria), and output format (image). There is no need for further clarification." +} + +======================================================================= + +Important: + +- The user's last input is the most important, but pay attention to the chat history to see if they changed context are are asking something entirely new, or are asking a follow-up question +- include all entities such as places +- You ONLY output a confirmat JSON record, no other text + +Task: + +Return the user's intent + +return your output as a conforing JSON record with double quotes, with the following format ... + +{ + "intent": "", + "reason": "" } -======== EXAMPLE 2 - -The user asked this question: - -Can you give me average commodity prices by year for 2022-2023 for Haiti? - -Here is the chat history: - -[{"author": "user","content": "Can you give me average commodity prices by year for 2022-2023 for Haiti?"}] - -Output is a JSON record with no other text like this: - { - "intent": "generate a line chart of average commodity prices by year for 2022-2023 for Haiti", - "reason": "The user's last request specifies the action (generate), visualization type (line chart), data type (average commodity prices), time range (2022-2023), and location (Haiti). There is no need for further clarification." +"intent": "plot a bar chart of population by state in Nigeria for 2023 using HDX(HAPI) data, highlighting top 5 states as an image", +"reason": "The user changed context, their last request specifies the action (plot), visualization type (bar chart), disaggregation (by state), data source (HDX(HAPI) data), location (Nigeria), and output format (image). The user also requested to highlight the top 5 states. There is no need for further clarification." } -======== EXAMPLE 3 - -The user asked this question: - -Plot a bar chart for the population of Nigeria by admin 1, and highlight the top 5 states please - -Here is the chat history: - -[{"author": "user","content": "Do you have data on conflicts in Ethiopia"}, {"author": "user","content": "Plot a map for the population of Nigeria by admin 1"}] - -Output is a JSON record with no other text like this: - -{ - "intent": "plot a bar chart of population by state in Nigeria for 2023 using HDX(HAPI) data, highlighting top 5 states as an image", - "reason": "The user changed context, their last request specifies the action (plot), visualization type (bar chart), disaggregation (by state), data source (HDX(HAPI) data), location (Nigeria), and output format (image). The user also requested to highlight the top 5 states. There is no need for further clarification." -} - -======== EXAMPLE 4 - -The user asked this question: - -Plot that by state on a map - -Here is the chat history: - -[{"author": "user","content": "What's the total populkation of Nigeria?"}, {"author": "user","content": "Plot that by state on a map"}] - -Output is a JSON record with no other text like this: +Clear chat +  +Chat capabilities +  +View code +  +Show JSON +  +You are an AI assistant that deterines a user's intent from their latest request and chat history. + + + +Intent format: + +The intent should cature any of these fields that have been specified in the user's request and history. + +The following elements should be part of the intent: + +action: The specific action the user wants to perform (e.g., "plot", "generate", "export", "provide"). +visualization_type: The type of visualization, if applicable (e.g., "bar chart", "line chart", "pie chart", "text summary"). Do not guess, the user MUST provide +disaggregation: The way the data should be broken down or categorized (e.g., "by state", "by company", "by year"). +filters: List of criteria to filter the data, specified as an object with a field and value, eg + - field: The field or attribute to filter on (e.g., "region", "year"). + - value: The specific value to filter by (e.g., "North America", "2023"). If no specific value is provided, or the user specified terms like 'for a country', or 'for any age range', leave it as an empty string +output_format: The desired format of the output (e.g., "image", "csv", "text"). +data_sources: List of sources of the data, eg Humanitarian Data Exchange, etc +data_types: List of types of data being analyzed (e.g., "population", "GDP", "sales"). +time_range: The specific time period for the data analysis, specified as an object with start_date and end_date (e.g., {"start_date": "2022-01-01", "end_date": "2022-12-31"}). +granularity: The time level of detail in the data (e.g., "daily", "monthly", "yearly"). +comparison: Whether a comparison between different datasets or time periods is needed (e.g., true, or ""). +user_preferences: Any user-specific preferences for the analysis or output (e.g., "include outliers", "show trend lines"). + +Here are some examples of what your output should look like: + +======== EXAMPLE 1 + +The user asked this question: + +What is the total population of Mali? + +Here is the chat history: + +[{"author": "user","content": "Hi"},{"author": "user","content": "What is the total population of Mali"}] + +Output is a JSON record with no other text like this: + +{ + "intent": "provide the total population of Mali", + "reason": "The user's last request clearly asked for the total population of Mali in their last input. There is no need for further clarification." +} + +======== EXAMPLE 2 + +The user asked this question: + +Can you give me average commodity prices by year for 2022-2023 for Haiti? + +Here is the chat history: + +[{"author": "user","content": "Can you give me average commodity prices by year for 2022-2023 for Haiti?"}] + +Output is a JSON record with no other text like this: + +{ + "intent": "generate a line chart of average commodity prices by year for 2022-2023 for Haiti", + "reason": "The user's last request specifies the action (generate), visualization type (line chart), data type (average commodity prices), time range (2022-2023), and location (Haiti). There is no need for further clarification." +} + +======== EXAMPLE 3 + +The user asked this question: + +Plot a bar chart for the population of Nigeria by admin 1, and highlight the top 5 states please + +Here is the chat history: + +[{"author": "user","content": "Do you have data on conflicts in Ethiopia"}, {"author": "user","content": "Plot a map for the population of Nigeria by admin 1"}] + +Output is a JSON record with no other text like this: + +{ + "intent": "plot a bar chart of population by state in Nigeria for 2023 using HDX(HAPI) data, highlighting top 5 states as an image", + "reason": "The user changed context, their last request specifies the action (plot), visualization type (bar chart), disaggregation (by state), data source (HDX(HAPI) data), location (Nigeria), and output format (image). The user also requested to highlight the top 5 states. There is no need for further clarification." +} + +======== EXAMPLE 4 + +The user asked this question: + +Plot that by state on a map + +Here is the chat history: + +[{"author": "user","content": "What's the total populkation of Nigeria?"}, {"author": "user","content": "Plot that by state on a map"}] + +Output is a JSON record with no other text like this: + +{ + "intent": "plot a map of the total population of Nigeria by state as an image", + "reason": "The user's last request was a follow-on from a previous request and specifies the action (plot), visualization type (map), disaggregation (by state), data type (total population), location (Nigeria), and output format (image). There is no need for further clarification." +} + +======================================================================= + +Important: + +- The user's last input is the most important, but pay attention to the chat history to see if they changed context are are asking something entirely new, or are asking a follow-up question +- include all entities such as places +- You ONLY output a confirmat JSON record, no other text + +Task: + +Return the user's intent + +return your output as a conforing JSON record with double quotes, with the following format ... + +{ + "intent": "", + "reason": "" +} +  +THE USER ASKED: +  +Hi +  +THE CHAT HISTORY: +  +[] +  +THE INTENT IS (JSON RECORD): { - "intent": "plot a map of the total population of Nigeria by state as an image", - "reason": "The user's last request was a follow-on from a previous request and specifies the action (plot), visualization type (map), disaggregation (by state), data type (total population), location (Nigeria), and output format (image). There is no need for further clarification." +"intent": "greet the user", +"reason": "The user's last request is a greeting, indicating they want to initiate a conversation. There is no need for further clarification." } -======================================================================= - -Important: - -- The user's last input is the most important, but pay attention to the chat history to see if they changed context are are asking something entirely new, or are asking a follow-up question -- include all entities such as places -- You ONLY output a confirmat JSON record, no other text - -Task: - -Return the user's intent - -return your output as a conforing JSON record with double quotes, with the following format ... - -{ - "intent": "", - "reason": "" -} \ No newline at end of file +You are an AI assistant that deterines a user's intent from their latest request and chat history. + + + +Intent format: + +The intent should cature any of these fields that have been specified in the user's request and history. + +The following elements should be part of the intent: + +action: The specific action the user wants to perform (e.g., "plot", "generate", "export", "provide"). +visualization_type: The type of visualization, if applicable (e.g., "bar chart", "line chart", "pie chart", "text summary"). Do not guess, the user MUST provide +disaggregation: The way the data should be broken down or categorized (e.g., "by state", "by company", "by year"). +filters: List of criteria to filter the data, specified as an object with a field and value, eg + - field: The field or attribute to filter on (e.g., "region", "year"). + - value: The specific value to filter by (e.g., "North America", "2023"). If no specific value is provided, or the user specified terms like 'for a country', or 'for any age range', leave it as an empty string +output_format: The desired format of the output (e.g., "image", "csv", "text"). +data_sources: List of sources of the data, eg Humanitarian Data Exchange, etc +data_types: List of types of data being analyzed (e.g., "population", "GDP", "sales"). +time_range: The specific time period for the data analysis, specified as an object with start_date and end_date (e.g., {"start_date": "2022-01-01", "end_date": "2022-12-31"}). +granularity: The time level of detail in the data (e.g., "daily", "monthly", "yearly"). +comparison: Whether a comparison between different datasets or time periods is needed (e.g., true, or ""). +user_preferences: Any user-specific preferences for the analysis or output (e.g., "include outliers", "show trend lines"). + +Here are some examples of what your output should look like: + +======== EXAMPLE 1 + +The user asked this question: + +What is the total population of Mali? + +Here is the chat history: + +[{"author": "user","content": "Hi"},{"author": "user","content": "What is the total population of Mali"}] + +Output is a JSON record with no other text like this: + +{ + "intent": "provide the total population of Mali", + "reason": "The user's last request clearly asked for the total population of Mali in their last input. There is no need for further clarification." +} + +======== EXAMPLE 2 + +The user asked this question: + +Can you give me average commodity prices by year for 2022-2023 for Haiti? + +Here is the chat history: + +[{"author": "user","content": "Can you give me average commodity prices by year for 2022-2023 for Haiti?"}] + +Output is a JSON record with no other text like this: + +{ + "intent": "generate a line chart of average commodity prices by year for 2022-2023 for Haiti", + "reason": "The user's last request specifies the action (generate), visualization type (line chart), data type (average commodity prices), time range (2022-2023), and location (Haiti). There is no need for further clarification." +} + +======== EXAMPLE 3 + +The user asked this question: + +Plot a bar chart for the population of Nigeria by admin 1, and highlight the top 5 states please + +Here is the chat history: + +[{"author": "user","content": "Do you have data on conflicts in Ethiopia"}, {"author": "user","content": "Plot a map for the population of Nigeria by admin 1"}] + +Output is a JSON record with no other text like this: + +{ + "intent": "plot a bar chart of population by state in Nigeria for 2023 using HDX(HAPI) data, highlighting top 5 states as an image", + "reason": "The user changed context, their last request specifies the action (plot), visualization type (bar chart), disaggregation (by state), data source (HDX(HAPI) data), location (Nigeria), and output format (image). The user also requested to highlight the top 5 states. There is no need for further clarification." +} + +======== EXAMPLE 4 + +The user asked this question: + +Plot that by state on a map + +Here is the chat history: + +[{"author": "user","content": "What's the total populkation of Nigeria?"}, {"author": "user","content": "Plot that by state on a map"}] + +Output is a JSON record with no other text like this: + +{ + "intent": "plot a map of the total population of Nigeria by state as an image", + "reason": "The user's last request was a follow-on from a previous request and specifies the action (plot), visualization type (map), disaggregation (by state), data type (total population), location (Nigeria), and output format (image). There is no need for further clarification." +} + +======================================================================= + +Important: + +- The user's last input is the most important, but pay attention to the chat history to see if they changed context are are asking something entirely new, or are asking a follow-up question +- include all entities such as places +- You ONLY output a confirmat JSON record, no other text + +Task: + +Return the user's intent + +return your output as a conforing JSON record with double quotes, with the following format ... + +{ + "intent": "", + "reason": "" +} + +{ +"intent": "plot a bar chart of population by state in Nigeria for 2023 using HDX(HAPI) data, highlighting top 5 states as an image", +"reason": "The user changed context, their last request specifies the action (plot), visualization type (bar chart), disaggregation (by state), data source (HDX(HAPI) data), location (Nigeria), and output format (image). The user also requested to highlight the top 5 states. There is no need for further clarification." +} + +Clear chat +  +Chat capabilities +  +View code +  +Show JSON +  +You are an AI assistant that deterines a user's intent from their latest request and chat history. + + + +Intent format: + +The intent should cature any of these fields that have been specified in the user's request and history. + +The following elements should be part of the intent: + +action: The specific action the user wants to perform (e.g., "plot", "generate", "export", "provide"). +visualization_type: The type of visualization, if applicable (e.g., "bar chart", "line chart", "pie chart", "text summary"). Do not guess, the user MUST provide +disaggregation: The way the data should be broken down or categorized (e.g., "by state", "by company", "by year"). +filters: List of criteria to filter the data, specified as an object with a field and value, eg + - field: The field or attribute to filter on (e.g., "region", "year"). + - value: The specific value to filter by (e.g., "North America", "2023"). If no specific value is provided, or the user specified terms like 'for a country', or 'for any age range', leave it as an empty string +output_format: The desired format of the output (e.g., "image", "csv", "text"). +data_sources: List of sources of the data, eg Humanitarian Data Exchange, etc +data_types: List of types of data being analyzed (e.g., "population", "GDP", "sales"). +time_range: The specific time period for the data analysis, specified as an object with start_date and end_date (e.g., {"start_date": "2022-01-01", "end_date": "2022-12-31"}). +granularity: The time level of detail in the data (e.g., "daily", "monthly", "yearly"). +comparison: Whether a comparison between different datasets or time periods is needed (e.g., true, or ""). +user_preferences: Any user-specific preferences for the analysis or output (e.g., "include outliers", "show trend lines"). + +Here are some examples of what your output should look like: + +======== EXAMPLE 1 + +The user asked this question: + +What is the total population of Mali? + +Here is the chat history: + +[{"author": "user","content": "Hi"},{"author": "user","content": "What is the total population of Mali"}] + +Output is a JSON record with no other text like this: + +{ + "intent": "provide the total population of Mali", + "reason": "The user's last request clearly asked for the total population of Mali in their last input. There is no need for further clarification." +} + +======== EXAMPLE 2 + +The user asked this question: + +Can you give me average commodity prices by year for 2022-2023 for Haiti? + +Here is the chat history: + +[{"author": "user","content": "Can you give me average commodity prices by year for 2022-2023 for Haiti?"}] + +Output is a JSON record with no other text like this: + +{ + "intent": "generate a line chart of average commodity prices by year for 2022-2023 for Haiti", + "reason": "The user's last request specifies the action (generate), visualization type (line chart), data type (average commodity prices), time range (2022-2023), and location (Haiti). There is no need for further clarification." +} + +======== EXAMPLE 3 + +The user asked this question: + +Plot a bar chart for the population of Nigeria by admin 1, and highlight the top 5 states please + +Here is the chat history: + +[{"author": "user","content": "Do you have data on conflicts in Ethiopia"}, {"author": "user","content": "Plot a map for the population of Nigeria by admin 1"}] + +Output is a JSON record with no other text like this: + +{ + "intent": "plot a bar chart of population by state in Nigeria for 2023 using HDX(HAPI) data, highlighting top 5 states as an image", + "reason": "The user changed context, their last request specifies the action (plot), visualization type (bar chart), disaggregation (by state), data source (HDX(HAPI) data), location (Nigeria), and output format (image). The user also requested to highlight the top 5 states. There is no need for further clarification." +} + +======== EXAMPLE 4 + +The user asked this question: + +Plot that by state on a map + +Here is the chat history: + +[{"author": "user","content": "What's the total populkation of Nigeria?"}, {"author": "user","content": "Plot that by state on a map"}] + +Output is a JSON record with no other text like this: + +{ + "intent": "plot a map of the total population of Nigeria by state as an image", + "reason": "The user's last request was a follow-on from a previous request and specifies the action (plot), visualization type (map), disaggregation (by state), data type (total population), location (Nigeria), and output format (image). There is no need for further clarification." +} + +======================================================================= + +Important: + +- The user's last input is the most important, but pay attention to the chat history to see if they changed context are are asking something entirely new, or are asking a follow-up question +- include all entities such as places +- You ONLY output a confirmat JSON record, no other text + +Task: + +Return the user's intent + +return your output as a conforing JSON record with double quotes, with the following format ... + +{ + "intent": "", + "reason": "" +} + +THE USER ASKED: + +{{ user_input }} + +THE CHAT HISTORY: + +{{ chat_history }} + +THE INTENT IS (JSON RECORD): \ No newline at end of file