Gen AI (beta)
A simple and user-friendly way to harness the capabilities of LLMs in your conversations
The Gen AI cell (powered by OpenAI's ChatGPT) is a sophisticated tool to enhance and enrich your conversational flows in eva, allowing a more coherent, fluid, and custom interactions. This is a component that uses Generative AI technology to produce any content, suitable for multiple needs and scenarios.
This cell allows you to access the specified model during execution and generate content on the fly according to your configured settings.
Important: Enabling this feature may result in additional costs on each new request, both at the time of creation and during the execution of flows, as a new request is made on content generation.
Creating a Gen AI cell
To create a Gen AI cell, proceed to the "Advanced Options" on the cells library in the Dialog Manager or thought the repository.
The main fields in a Gen AI cell are:
We will get to know a little bit about each one further on.
Prompt
Prompts are very useful in automating language-based tasks. You can use it to generate content to match a desired tone, emotion, context, and user preferences, and even contribute to the virtual agent personality. Prompts can also support code creation, content summarization, information classification, and masking sensitive information.
Prompts are compatible with Dynamic Content. You may add any of it's variables, such as the most recent user input, environment and bot parameters, all of the three eva contexts and any other straight into the prompt.
To make the most of a Gen AI cell, first we have to understand what a prompt is and what the best practices in prompt engineering are. Check our guide on how to craft a good and effective prompt to assist you with this topic. 😉
Advanced parameters
The advanced parameters will help you to customize the language model behavior. You can set the size of the generated content, level of creativity combining two or more parameters, and control vocabulary variation and repetitions.
Check in the following table the description of each parameter:
PARAMETER
DESCRIPTION
Maximum tokens
The maximum number of tokens to be generated, counting both prompt (input) and preview (output). A token is roughly 3-5 characters long, but its exact length may vary.
Temperature
Control the randomness of the generated text. Lower values lead to more predictable outputs, while higher values lead to more diverse and creative outputs.
Top P
Control vocabulary variation. Lower values produce to most common and predictable words or phrases, while higher values produce less frequent or unexpected words.
Frequency Penalty
Use the Frequency Penalty parameter to control repetitions within the generated content. Lower values keep the model from repeating tokens, while higher values will result in a more frequent use of the same tokens.
Presence Penalty
The Presence Penalty parameter doesn't consider how often a token is used, but whether it exists in the text. A higher value will result in the model being more likely to generate tokens that have not yet been included in the generated text.
These advanced parameters come all pre-filled as market default. But If you want to fine tune them to test different results.
Variable (hidden context)
This field stores the content generated automatically for future use in the flows.
To do so, paste the value written in this field into the text field in other cells with the syntax $hiddenContext. You can then use this freely at any eva-context compatible cell.
Preview
You can preview the generation when editing a Gen AI cell and fine tune it before you save it. If you click the "Generate Content" button, a new preview will be generated with the current prompt + advanced settings parameters.
Note that if you have added Dynamic Content variables to your prompt, the test will fail. We recommend you to replace the variables with the expected value of those fields during the test.
Testing content also consumes tokens as if performing a conversation. The same is true to dialogues performed in eva's dialog simulator.
Use cases examples
While the Gen AI cell has an infinity of use cases, we have detailed examples of interesting use cases in the Practical examples page.
Was this helpful?