Installing and Running the AI Agent

The AI agent is an optional module of the Operavix system designed to work with neural networks and perform LLM-related tasks. It is an optional component that is installed only when neural network capabilities are required within Operavix.

The AI agent enables the following functions:

• Optical Character Recognition (OCR)
• Automatic Speech Recognition (ASR)

Each function uses a separate AI agent distribution. Multiple AI agents can operate simultaneously within the Operavix system.

Operavix and AI agents communicate via the gRPC protocol, ensuring fast and secure data exchange.

All AI agent distributions are provided as Docker images.

Technical requirements are detailed in the section Technical Requirements for Server and Hardware Equipment.

Running the AI Agent

Preparing Hosts and Configuration Parameters

Before launching the AI agent, define the network addresses and configuration parameters.

As an example, consider the following placeholder hostnames, which you can replace with your actual system addresses:

• operavix.domain.local — the host address where the Operavix system is deployed
• agent.domain.local — the host address where the AI agent will run

When starting the AI agent with docker run, you must specify the environment variables that define its operational parameters.

The table below provides reference information on model names and their corresponding capabilities.

Preparing the Image and Certificates

Download the AI agent distribution image to a GPU-enabled server. Then, import the image into your local Docker registry:

$ gunzip operavix_app_llm-agent:dlagdlag241201-llama3.1-instruct-8B-Q4.tar.gz

# docker load < operavix_app_llm-agent:dlag241201-llama3.1-instruct-8B-Q4.tar

Prepare X.509 certificates for HTTPS on this node in PEM format. The certificate file (.crt) contains the public key, while the key file (.key) contains the private key used to encrypt the connection.

Place the certificates in a convenient directory on the host—e.g., /opt/agent/. In the same directory, also place the Operavix server certificate (.crt).

Starting the Container

Run the AI agent container using the following command (adjust values to match your infrastructure):

# docker run --name operavix-llama \
-d \
--gpus all \
--runtime=nvidia \
-e CL_NAME='agent' \
-e CL_PORT='7000' \
-e CL_REMOTE_NODES='operavix.domain.local:7000' \
-v /opt/agent/agent_cert.crt:/run/secrets/cluster_current.crt:ro \
-v /opt/agent/agent_key.key:/run/secrets/cluster_current.key:ro \
-v /opt/agent/operavix_cert.crt:/run/secrets/cluster_remote_node_agent.crt:ro \
-p 7000:7000 \
--restart=on-failure \
operavix/operavix_app_llm-agent:dlag241201-llama3.1-instruct-8B-Q4

Where:

• -v /opt/agent/agent_cert.crt:/run/secrets/cluster_current.crt:ro — mounts the current node's certificate into the container
• -v /opt/agent/agent_key.key:/run/secrets/cluster_current.key:ro — mounts the current node's private key into the container
• -v /opt/agent/operavix_cert.crt:/run/secrets/cluster_remote_node_agent.crt:ro — mounts the Operavix server certificate into the container
• operavix/operavix_app_llm-agent:dlag241201-llama3.1-instruct-8B-Q4 — image name (may vary by version)

If the AI agent container starts successfully, restart the Operavix server with the new configuration parameters.

Configuring AI Agent Integration with Operavix on Linux-Based Systems

To enable communication between the AI agent and Operavix, prepare an X.509 certificate for HTTPS on the Operavix node in PEM format (if you have a PFX file, convert it to PEM), along with the AI agent node’s certificate.

Place the certificates in the /opt/agent/ directory, then create Docker secrets based on them:

# docker secret create cluster_current.crt /opt/agent/operavix.crt
# docker secret create cluster_current.key /opt/agent/operavix.key
# docker secret create cluster_remote_node_agent.crt /opt/agent/agent.crt

Before starting with the new parameters, stop the existing Operavix service:

# docker service rm operavix-app

Then launch the service with the updated configuration:

docker service create --name operavix-app \
-e AI_LLM_MODEL='llama-3.1-instruct-8B-Q4' \
-e CL_NAME='operavix' \
-e CL_PORT='7000' \
-e CL_REMOTE_NODES='agent.domain.local:7000' \
--secret operavix_app_https_certificate \
--secret operavix_app_https_certificate_password \
--secret cluster_current.crt \
--secret cluster_current.key \
--secret cluster_remote_node_agent.crt \
--mount type=volume,src=operavix-app-data,target=/var/lib/operavix/data/ \
--mount type=volume,src=operavix-app-log,target=/var/log/operavix/ \
--publish published=443,target=8010,mode=host \
--publish published=7000,target=7000,mode=host \
--restart-max-attempts 5 \
--restart-condition "on-failure" \
-e JVM_MAX_MEMORY='4G' \
-e FE_URL="https://operavix.domain.local" \
operavix/operavix_app:d241202

The updated launch command includes:

• Environment variables (AI_LLM_MODEL, CL_NAME, CL_PORT, CL_REMOTE_NODES)
• Secrets (cluster_current.crt, cluster_current.key, cluster_remote_node_agent.crt)
• Host-level publication of an additional gRPC port (--publish published=7000,target=7000,mode=host)

All other parameters should match your previous Operavix launch configuration.

Verifying Successful Startup

If the startup is successful, open the web interface and confirm that the new features are functioning correctly.

For more details about the functionality, see the Working with LLM section.