Search and Chat

Hybrid search

Ragty combines semantic and lexical retrieval:

Dense search embeds the query and searches the dataset's named vector space.
Sparse BM25 search preserves exact terms such as identifiers and acronyms.
Reciprocal Rank Fusion (RRF) merges both rankings deterministically.

If sparse retrieval fails, the request degrades to dense-only retrieval rather than failing the complete search. Datasets that use different embedding models are searched in their respective vector spaces and their results are fused.

Every vector query includes the resolved tenant scope. Workspace and dataset filters further restrict a request when the active scope or caller selects them. Re-parsing replaces the document's previous vectors, and retrieval prunes orphaned points without a document identifier. Authorization is resolved again at request time; dataset names or MCP tool descriptions are not authorization controls.

RAG orchestration

A chat request can:

condense recent conversation history into a standalone retrieval query;
retrieve and rank chunks from the dialog's datasets;
build a cited context for the configured system prompt;
stream optional reasoning, answer tokens, citations, and errors as Server-Sent Events (SSE).

When query condensation fails, Ragty falls back to the user's original question.

Public chat endpoint

Dialog clients use the public API boundary:

POST /api/v1/chat/completions
Authorization: Bearer YOUR_DIALOG_API_KEY
Content-Type: application/json

{
  "messages": [
    {"role": "user", "content": "What is the vacation policy?"}
  ],
  "stream": true,
  "temperature": 0.7,
  "max_tokens": 2048
}

Field	Type	Required	Description
`messages`	array	Yes	At least one message. Ragty uses the last message as the current question and accepts `user` and `assistant` roles for prior history.
`model`	string	No	Model override; an empty value uses the dialog configuration.
`stream`	boolean	No	Accepted for OpenAI compatibility; the endpoint currently returns SSE and defaults to `true`.
`temperature`	number	No	Generation-temperature override.
`max_tokens`	integer	No	Maximum output-token override.

The response uses Ragty event envelopes rather than OpenAI delta objects:

data: {"type":"citations","data":[...]}

data: {"type":"token","data":"The vacation policy"}

data: [DONE]

When thinking is enabled for the configured model, reasoning events may precede answer tokens. Failures that occur after streaming starts are returned as error events, so clients must inspect the SSE stream even when the HTTP response itself is successful.

Citations

Citation events contain document references and source-position metadata:

Field	Description
`document_id`	Source document identifier
`page_num`	Page number in the original document
`position`	Source bounding box when the parser provides one
`content_snippet`	Bounded source excerpt used for the answer

The frontend uses these fields to link an answer to its source and, for supported PDFs, highlight the corresponding page region.

Dialogs and model providers

A dialog binds an LLM model, system prompt, datasets, and retrieval settings such as result count, similarity threshold, and reranking. Dialog API keys allow an external client to use that fixed configuration without receiving administrative access.

Chat and embedding models are configured in Settings → Providers. Ragty calls their OpenAI-compatible HTTP endpoints directly; a LiteLLM deployment is optional and can itself be configured as an OpenAI-compatible provider.

23 August 2026