Working with LLM

You can work with LLM using the following blocks:

• Send a message
• Create embeddings
• Document to text

In the parameters of the Send message and Create embedding blocks, you can select not only a specific model but also an entire model family.
This allows the system to automatically choose an available model from the family if the specified one cannot be used due to hardware limitations.

Send a Message

The block allows you to send a message with custom content and receive a response from the selected model in the output data.

Block parameters:

• Message
• Model or model family — choose from the list of available options
• Temperature — controls the accuracy of AI responses. The closer to 0, the more precise and predictable the answer will be. A higher temperature produces more diverse but less accurate results. Minimum: 0, Maximum: 1

Example:

1. Enter a query in the Message field and select an model from the list.
2. Test the block.
3. The response appears in the output data.

Create embeddings

The block converts text into vector format.

Block parameters:

• Message
• Model or model family — choose from the list of available options

Example:

1. Enter the message for which you want to create an embedding and select an model from the list.
2. Test the block.
3. An array of numbers appears in the output.

Document to text

The block converts documents into a text format.

Block parameters:

• Model or model family — choose from the list of available options
• File — file content in MD or PDF format using mapping
• Extension — either the extension itself or the file name with extension (for example, filename.md, .md, md) or the variable file_name
• Prompt — a format template for text extraction
  • Important: When working with PDF documents, you must use the <FILE_TEXT> tag in the query text to pass the file contents to the model. For example: "Extract text from <FILE_TEXT>"
• Structure the result — enable this to generate mapping fields from the output data in JSON format
• Extract page by page — enable this so that each page of a PDF file is processed separately with the specified query
• Alternative text recognition — enable this to recognize the document content as an image and extract text from it. This is suitable for files containing only images or tables, as well as documents with mixed data (text and images)

Example:

1. Get a file using the Download the contents of the file block.
2. Add the Document to text block.
3. In the block parameters, select the required model from the available list.
4. Use mapping to insert the file_content and file_name variables.
5. Test the entire script.
6. The block returns the extracted text in Markdown format.