APIs & Integrations

LLM Agents and Chatbots

The AI module lets an application talk to a large language model: a single question and answer, a chat that remembers what was said, a conversation stored per user, and an agent with its own system prompt and tools that it calls while it answers. It also creates embeddings, makes the content of files searchable by meaning and answerable with citations, and it can expose an agent to external AI clients over MCP.

Everything is configured as data - a provider configuration, agents, tools - and used from scripts with a small set of functions. The functions run as the user who calls them, and so does every tool an agent calls: an agent can do exactly what the user it talks to may do, and nothing more.

Overview

Providers

A provider configuration names the service and holds its settings. Create an LLMProviderConfiguration with a name and a providerClassFQCN, and add its settings as LLMProviderConfigurationEntry records (key, value) in entries.

The easiest way is the LLM Providers tab of the AI node in the Code area: choose the provider, and the form shows the settings it reads, filled in with its defaults (the Ollama host, the OpenAI base URL, the timeout, maxToolRounds). Below them are model, the model used when a caller names none (the Companion, MCP exposures), and maxTokens, which stays empty unless set: the providers use 2048 for a prompt and 16384 for an agent turn. API keys are stored encrypted and never shown again; leave the field empty to keep the stored key. Test asks the provider whether its host can be reached. The tab is for admin users and uses /structr/rest/_llm/providers.

Not every provider offers every function:

Every function takes the configuration node as its first argument. A script finds it by name:

const config = $.find('LLMProviderConfiguration', { name: 'default' })[0];

The model is not part of the configuration: it is passed on each call, as the provider names it.

A single prompt

{
    const config = $.find('LLMProviderConfiguration', { name: 'default' })[0];

    $.llmPrompt(config, 'llama3.2', 'Summarise this in one sentence: ' + $.this.description)
}

The prompt is a string and the answer comes back as a string. There is no conversation, no agent and no tool.

Chats

llmStartChat(config, model) starts a chat that lives for the current request. Send messages with llmPromptChat(config, chat, message); each answer takes the earlier messages into account.

{
    const config = $.find('LLMProviderConfiguration', { name: 'default' })[0];
    const chat   = $.llmStartChat(config, 'llama3.2');

    $.llmPromptChat(config, chat, 'My name is Alex.');
    $.llmPromptChat(config, chat, 'What is my name?')
}

llmPromptChat also takes attachments as a fourth argument: an Image or File node, a map { data: '<base64>', mediaType: 'image/png' }, or a list of them.

Conversations stored per user

A chat that has to survive the request - a chat window in the app - is stored as an LLMChatSession with its turns and messages, and belongs to the user who started it: a user sees and continues only their own sessions.

A typical chat window needs three methods: one that starts a session (llmNewChatSession) and returns its id, one that sends a message (llmPromptChat with that id) and returns the answer, and one that lists the user’s sessions and their messages to render the history.

Agents

An agent is an LLMAgent record:

An agent has no provider and no model: they are chosen where the agent is used, so the same agent can run on different models.

{
    const config  = $.find('LLMProviderConfiguration', { name: 'default' })[0];
    const session = $.llmStartAgentSession(config, 'supportAgent', 'claude-sonnet-5');

    $.llmPromptChat(config, session, $.methodParameters.question)
}

The agent can be given as its node or by name. For a conversation stored per user, use llmNewChatSession(config, model, 'supportAgent') or llmResumeAgentSession(config, 'supportAgent', model) instead.

Tools

A tool is something an agent may call while it answers. It is an LLMTool record linked to the agent:

The arguments the model passes arrive in the script as args:

{
    const args   = $.get('args');
    const events = $.find('Event', { name: $.predicate.contains(args.query) });

    events.map(e => e.name + ' on ' + $.dateFormat(e.startDate, 'dd.MM.yyyy')).join('\n')
}

The result is returned to the model as text. A result starting with “Error” is reported to the model as a failed call.

A tool does not call a schema method by itself: to use one, call it from the script, e.g. { $.myMethod({ id: $.get('args').id }) } for a global method.

The script runs as the user the agent is talking to. Every query sees only what that user may read, and every write needs that user’s permissions - the agent cannot do more than the user could do without it.

Tools of the built-in agents

Structr’s own agents use a set of developer tools: reading the server log, reading and creating files, fetching any URL, reading the app’s pages, source code and records. An agent you create does not get them. If one of them is right for your agent, add it by name in the agent’s tool configuration (see below) - and never give a tool that reads the server log, the source code or arbitrary URLs to an agent that visitors or ordinary users talk to.

The tool configuration

Which tools an agent gets can be changed without touching the agent: per agent, tools are added or removed by name.

PUT /structr/rest/_llm/config
{ "agents": { "supportAgent": { "add": ["lookup-docs"], "remove": [] } } }

GET /structr/rest/_llm/config returns the current configuration, GET /structr/rest/_llm/agents and GET /structr/rest/_llm/tools list every agent and tool with the tools each agent ends up with. All of these are for administrators only.

Embeddings

llmEmbed(config, model, text) returns the vector of a text - a list of numbers -, and llmEmbed(config, model, texts) one vector per text of a list. It needs an Ollama or OpenAI-compatible provider and an embedding model, such as nomic-embed-text or text-embedding-3-small. cosineSimilarity(a, b) returns the cosine similarity of two vectors: 1 for the same meaning, around 0 for unrelated texts (the search functions report similarity on another scale, (1 + cos) / 2, where unrelated is 0.5).

Some models expect a prefix that says what a text is: nomic-embed-text wants search_document: for a stored text and search_query: for a question, e5 models passage: and query:, mxbai-embed-large a sentence in front of a query. The prefix belongs to the model, not to the provider, so it is given where the model is: $.llmEmbed(config, 'nomic-embed-text', texts, { prefix: 'search_document: ' }), and for files on the content extraction configuration (embeddingDocumentPrefix, embeddingQueryPrefix), which gets the prefixes of a known model when it is linked. If a host cannot be reached, the Content Extraction dialog and the error messages say why, e.g. “unknown host” for a host name that does not resolve.

Your own records can be searched by meaning as well. Give the type a Double[] attribute with a vector index (the Vector Index button of the attribute in the schema editor, or its details in the Code area: the size of the model’s vectors, e.g. 768 for nomic-embed-text, and the similarity cosine, the default, which suits embeddings), store the vector of each record, and search with searchVector(), which uses the index and returns only records the current user can read:

{
    const config = $.find('LLMProviderConfiguration', { name: 'ollama' })[0];

    // when an article is saved
    article.embedding = $.llmEmbed(config, 'nomic-embed-text', article.title + '\n' + article.text);

    // when searching
    const query = $.llmEmbed(config, 'nomic-embed-text', $.methodParameters.question);

    $.searchVector('Article_embedding_vector', query, 5).map(hit => hit.node)
}

If you do not know the size of your model’s vectors: in Structr Enterprise, the wand next to the vector size detects it. Click it, choose the provider configuration and the embedding model, and Structr fills in the size, as the host or the model’s documentation reports it, or measured with one test embedding. Models that can shorten their vectors (OpenAI’s text-embedding-3 models) take a dimensions option, in the dialog and in llmEmbed(config, model, text, { dimensions: 512 }); both must use the same. A vector of another size than the index is refused when it is saved, so a wrong size shows up right away instead of as an empty search result.

cosineSimilarity(a, b) compares two vectors directly, e.g. to find duplicates among a few records. For the content of files, Structr does all of it - see the next section. The article Searching (Building Applications) compares all search functions and explains which to use when.

Searching and answering from files

The chunks of extracted files can be embedded and searched by meaning. How to enable it for a folder, the three ways to search ($.documents.fulltextSearch() for words, $.documents.vectorSearch() for meaning, with hybrid: true for both) and what happens after a change to the Community Edition are described in the article Content Extraction, section “Embeddings and Semantic Search”. The parts that concern the AI module:

• Provider and model. An Ollama or OpenAI-compatible provider configuration and an embedding model are linked to a folder’s content extraction configuration, in the dialog or with $.documents.linkEmbeddingProvider(folder, config, model). Linking embeds one probe text, so a wrong key or an unknown model is reported right away, and the size of the vectors is known before the first file is embedded. All folders use the same model; $.documents.resetEmbeddings({ model }) switches all of them and embeds everything again.
• Answers. $.documents.answer(question, { config }) combines the semantic search with a chat model: it finds the chunks with $.documents.vectorSearch() (hybrid by default) and has a chat model answer from them alone, citing them as [1], [2]; it returns answer and citations. Each call is one request to the chat model. The chat model is the default model of the provider configuration, or model if given; the configuration is read with the caller’s permissions, the embedding model is the one the admin linked.
• Agents. The shared tool knowledge-search searches the same way for an agent it is added to by name.
• Models. The dialog suggests the embedding models the provider reports: Ollama lists the installed models whose capabilities include embedding, an OpenAI-compatible host the models with “embed” in their name. A chat model such as llama3.2 is refused when it is linked, with the embedding models of the host in the message. Anthropic has no embeddings; use an Ollama or OpenAI-compatible provider for them.
• Long chunks. An embedding model reads a limited number of tokens. Ollama reports the limit of each model (256 for all-minilm), for OpenAI’s own models it is known (8191), and it is stored as Max. Input Tokens when the folder is linked. Longer chunks are cut and the file shows a warning; choose a chunk size the model can read, or change Max. Input Tokens.
• State. $.documents.embeddingStatus() and GET /structr/rest/_llm/embedding (admins only) show the configurations, the embedded chunks per model, failed files, the vector index and running jobs. $.documents.embedContent(fileOrFolder, { force, wait }) embeds files again.

{
    const result = $.documents.answer('Which ports does the server need?', { config: 'default', limit: 5 });

    return result.answer + '\n\n' + result.citations.map(c => '[' + c.number + '] ' + c.path + (c.pageStart ? ', page ' + c.pageStart : '')).join('\n');
}

Agents for external AI clients (MCP)

An agent can be offered to external AI clients - Claude Desktop, an IDE assistant - over the Model Context Protocol. Each exposed agent appears to the client as one tool taking a prompt and an optional session_id to continue a conversation; it runs as the user who signed in through the client, with that user’s permissions.

1. Enable the servlet: add MCPServlet to httpservice.servlets in the configuration. It serves /structr/mcp/*.
2. Allow the users who should connect: a resource access permission for _mcp.
3. Clients sign in with OAuth (dynamic client registration with PKCE). The settings of the “MCP Settings” group (mcp.oauth.*) control the number of clients, the registration rate and the login and consent pages.
4. Expose the agent: in the agent’s MCP tab in the Code area, or with PUT /structr/rest/_llm/mcp. An exposure names the agent, the provider configuration and the model, and can override the tool name and description the client sees.

Deployment

A deployment export writes the AI configuration next to the app: llm-provider-configuration.json, llm-agents.json, llm-tools.json, llm-agent-tool-configuration.json, llm-mcp-agent-exposures.json and llm-embedding-links.json (which content extraction configuration embeds with which provider configuration and model). Vectors are not exported: after an import, the files are extracted and embedded again. API keys are part of the provider configuration, so treat an export like a secret.