This custom visual has the following 2 main configuration sections:

Config

This section covers all the required configurations for the LLM endpoint, along with individual settings for the chatbot, including fine-tuning options for its three modes: General, File, and Dataset. The General mode functions solely as an interface to the LLM, without any additional logic applied.

LLM: This section contains the settings for the LLM endpoint.

Chatbot: This section contains the settings for defining and customizing the behaviors of the Chatbot.

File: This section contains the fine-tuning options for the File mode.

**Dataset:** This section contains the fine-tuning options for the Dataset mode.

LLM

LLM host: The host of the LLM endpoint, currently you can choose between OpenAI and Azure OpenAI, depending on this selection the following options change.

OpenAI

• OpenAI API key: The API key is used for authentication, check this out to see how to generate one.
• OpenAI model: You can choose between the predefined models or select “other”. This is useful, if newly published models are not yet in the list. you can provide name manually.
• OpenAI model other: The textbox for the name of the model

Azure OpenAI

• Azure OpenAI resource/instance: This is the name of the “Azure OpenAI” resource within Azure. Find more here.
• Azure OpenAI deployment: This is the name of the deployment within Azure OpenAI (Studio/Foundry). Find more here.
• Azure OpenAI API key: The API key is used for authentication and can either be found in the “Keys and Endpoint**”** of your Azure resource or within Azure OpenAI Studio/Foundry. You can use either key KEY1 or KEY2.

Chatbot

This section will be helpful to determine the behavior and appearance of the Chatbot. The

• Chatbot mode show: The Chatbot is designed to focus on a single mode while responding, enhancing its accuracy and performance. If you want your users to interact with only one mode—either Dataset or File—you can hide the mode selection option in the user interface for a more streamlined experience.
• Chatbot mode default: This is the mode the user will have preselected when opening the report.
• Chatbot name: This name is displayed at the top of the Chatbot visual. It can also be used to refer to the assistant
• Chatbot avatar: This is the icon next to the Chatbot name. It needs to be a an image encoded in Base64.
• Chatbot instructions: This is the message that the assistant will internally receive to adapt its response style and behavior.
• Chatbot initial message: This is simply the first message shown in the conversation.
• Chatbot response: This setting configures how precise the responses should be.
• Chatbot max tokens: The maximum number of tokens to generate a response. 0 means there is no hard limit.
• Chatbot streaming: This setting will allow responses to be displayed as they arrive, without waiting to have the complete answer. This is useful especially for long responses
• Chatbot verbose: This setting allows information to be displayed in the browser console, to get a better understanding of the steps involved, and how the information flow is processed

File

This section provides an overview of the file-mode behavior of the chatbot, which enables users to summarize and ask questions about uploaded files. No external network requests are made, and no data is sent to external entities other than the designated LLM endpoint.

• Embeddings model name: This refers to the name of the model used to create embeddings stored in memory. You can select from predefined options such as text-embedding-3-small, text-embedding-3-large, text-embedding-ada-002, or specify a custom model name of your choice using other

• Fine-tuning Embeddings: Enables customization of embedding creation

• Embedding Dimension: Specifies the size of the vector space, balancing accuracy and computational efficiency.

• Embeddings Batch Size: Defines the number of items processed per batch, optimizing memory usage and speed.

• Fine-tuning summarization: This allows you to fine-tune the creation of embeddings.

• Summarization disable: This allows you to disable the automatic summarization feature on file upload.

• Summarization chunk size: Defines the number of characters summarized per chunk before combining results (map-reduce approach).

• Summarization chunk overlap: Defines the number of characters overlapping per chunk.

• Fine-tuning Question-Answering: Enables customization of embeddings for more precise answers.

• Question-Answering Chunk Size: Specifies the number of tokens per embedding.

• Question-Answering Chunk Overlap: Defines the number of overlapping tokens between chunks/embeddings.

• Question-Answering Vector Search Limit: Sets the maximum number of vectors to query for answers.