index4.ai Retrieval

POST /search is index4.ai's main retrieval endpoint. It runs hybrid (lexical + semantic) search across the project namespace and returns a RetrievalContext — a prompt-ready XML + Markdown block, not a list of ranked hits.

This page covers the request shape, how the four tuning parameters interact, and when to use searchInDoc instead.

Request

POST /v1/search HTTP/1.1
Authorization: Bearer <project-api-key>
Content-Type: application/json

{
  "query": "How did operating margin change year over year?",
  "doc_filter": null,
  "exclude_doc_ids": [],
  "return_assets": false,
  "return_page_images": false
}

Only query is required. The tuning knobs (min_relevance, target_tokens, max_tokens) are omitted above on purpose — their defaults are sized for the common case, so set them only when you want to deviate.

Field	Type	Default	Purpose
query (required)	string	—	Natural-language query. Hybrid retrieval splits this into a lexical term set and a vector embedding internally.
min_relevance	float 0..1	0.3	Relevance floor: hits scoring below it are dropped. Higher = stricter (return less, drop borderline hits); lower = more permissive. 0.3 is the balanced default.
target_tokens	integer	5000	Soft token budget — the typical answer size. Hits are admitted best-first (sandwich-aware) up to this. This is the knob to size a response. Can never exceed max_tokens (clamped down if it would).
max_tokens	integer	15000	Hard ceiling. index4.ai only expands past target_tokens toward it for tight multi-document clusters. Set it below target_tokens and the soft target is clamped down to match — the hard ceiling always wins; it never inflates a response on its own.
doc_filter	string	unset	Optional document id. When set, restricts retrieval to a single doc. (Use /searchInDoc if you want this required.)
doc_ids	array of string	unset	Restrict retrieval to this set of document ids.
exclude_doc_ids	array of string	unset	Doc ids to exclude from results (max 100). Useful in agent loops to avoid re-citing docs already shown.
return_assets	boolean	false	When true, the cited documents' figures (and tables, where available) are returned in an assets field keyed by doc_id. Images are base64 data URIs; resolved per document.
return_page_images	boolean	false	Deprecated / not available — no-op today; page screenshots aren't stored, so nothing is returned regardless of this flag (a dedicated endpoint is planned). Included in the sample request above because the API accepts it, not because it does anything yet.

Metadata filters

Filter on the metadata you attached at ingest (see Ingestion → Attributes & metadata) — typed attributes with attribute_filters, or plaintext strings with unencrypted_strings_match. If you still send meta_tags_any / meta_tags_all or meta_int_*_gte / _lte, see Migrating from labels / meta_int for the replacements.

Field	Type	Purpose
attribute_filters	array of {attr, op, value}	Filter on typed attributes. Clauses AND together; operators depend on the attribute's type — see below.
unencrypted_strings_match	object (string→string)	Match on the plaintext X-Unencrypted-Strings you set at ingest: a key→glob-pattern map. Each entry is a case-insensitive glob (* = any run, ? = one char; a pattern with no wildcard is an exact match). Multiple keys AND together.

All filters (attribute_filters, unencrypted_strings_match, doc_ids / exclude_doc_ids) combine via AND.

Filtering on typed attributes

attribute_filters is a list of {attr, op, value} clauses, combined with AND. To see which attribute names exist in the project and what type each has (and so which operators apply), call GET /attributes (MCP: index4ai_attributes) — see the API reference. A 503 from it means "retry", not "no attributes".

g.search(
    "q3 revenue",
    attribute_filters=[
        {"attr": "region",       "op": "eq",             "value": "emea"},
        {"attr": "published_at", "op": "gte",            "value": "2026-01-01T00:00:00Z"},
        {"attr": "case_notes",   "op": "has_all_tokens", "value": "chest pain"},
    ],
)

Which operators you can use depends on the attribute's type:

Type	Operators
string	eq ne in nin glob iglob
int, float, datetime	eq ne lt lte gt gte (ints also in / nin)
bool	eq ne
[]string	contains contains_any eq in
[]int, []float	contains contains_any any_lt any_gt
[]datetime	contains any_lt any_gt
[]bool	contains contains_any
encrypted_text, []encrypted_text	has_all_tokens has_any_tokens

An operator the type does not offer returns a 400 naming the ones it does.

Two behaviours worth knowing, because both are deliberate:

• A filter on an attribute your project has never used matches nothing. It is not ignored — a filter that silently matched everything would be worse than an empty result.
• encrypted_text matching is exact: the same tokenization runs on both sides, with no stemming and no stop-word removal, so "pains" does not find "pain". For Chinese, Japanese and Thai the text is split into character n-grams, so a whole word matches but an arbitrary mid-word fragment may not.

For wildcard strings, if you stored {"path": "legal/contracts/acme"} at ingest, then {"unencrypted_strings_match": {"path": "legal/contracts/*"}} keeps only documents whose path begins with legal/contracts/. Unlike encrypted_text attributes (HMAC'd, exact-token matching), unencrypted_strings values are stored unencrypted so they can be glob-matched — don't put secrets or PII in them.

Other controls

Field	Type	Purpose
min_top_relevance	float 0..1	Floor for the top hit only. If the best hit scores below this, the whole result set is empty — a guard against returning anything when nothing is a strong match.
expand_tightness	float 0..1	How aggressively the engine expands context around each hit.
retrieval_tier	string	standard (fusion only) or advanced (adds a reranker). See Retrieval tiers. REST only: the Python SDK and the MCP server don't expose it.
premium	boolean	Legacy flag — true equals retrieval_tier: "advanced". Prefer retrieval_tier.
format	string	prompt_ready (default) returns the XML+Markdown block; json returns structured ranked hits instead.

No top_k. Result count is bounded server-side by relevance + token budget. The recall stage isn't a tunable parameter; tune precision via min_relevance and answer size via target_tokens.

Response

{
  "context": "<doc id=\"...\">…</doc>",
  "result_count": 4,
  "tokens_estimated": 5820,
  "results_dropped": 2,
  "detected_lang": "english",
  "mode": "advanced",
  "search_units": 1
}

context is the field you actually use — index4.ai's contract is "ready to drop into a prompt", not "here is JSON for you to format". The siblings are metadata: result_count / results_dropped, tokens_estimated (rendered size), detected_lang, mode (the retrieval tier used), and search_units (billing). For the wrapper grammar (<doc>, inline [pN] / […] markers, sandwich order, citation attributes), see RetrievalContext format. To get structured per-hit JSON instead of the rendered block, set format: "json".

How the parameters interact

Think of retrieval as a two-stage pipeline:

                ┌──── filter ────┐  ┌──────── render ────────┐
query ─▶ hybrid search ─▶ score ≥ min_relevance ─▶ fill to target_tokens (sandwich, up to max_tokens) ─▶ context

• min_relevance controls precision. Anything below the threshold is discarded outright, even if it would have fit the budget. Useful for "I'd rather return less than return junk" queries (e.g. legal Q&A).
• target_tokens controls the answer size. After thresholding, hits are placed in sandwich order and admitted best-first up to this soft budget; if it overflows, the lowest-ranked hits are evicted first. Hits at the very top and very bottom of the order survive longest.
• max_tokens is a hard ceiling that only matters for tight multi-document clusters (several near-equally-relevant hits expand past the target toward it). Leave it at the default unless you want to allow — or forbid — that expansion.

A typical chat-with-knowledge-base setup needs no tuning at all — the defaults are sized for exactly this:

{ "query": "..." }

A "strict citations" assistant is where deviating pays off — raise the floor to drop borderline hits, and shrink the budget to keep answers short:

{ "query": "...", "min_relevance": 0.5, "target_tokens": 2500 }

Searching a single document

When you already know which document the answer must come from — "chat with this PDF", a per-document help bot, a contract Q&A flow — use POST /searchInDoc. It is the same shape as /search but doc_filter is required and rejected as 400 if missing.

curl -sS -X POST "https://api.index4.ai/v1/searchInDoc" \
  -H "authorization: Bearer $INDEX4AI_API_KEY" \
  -H "content-type: application/json" \
  -d '{
    "query": "What is the cancellation policy?",
    "doc_filter": "terms_of_service_v3"
  }'

Why a separate endpoint instead of just setting doc_filter on /search? Two reasons:

• Safety. searchInDoc makes the per-document scope a contract, not an option — you cannot accidentally drop the filter and silently retrieve from the entire namespace.
• Validation. The server rejects an empty doc_filter up front, so a typo that would otherwise become a "search the world" bug fails fast.

The response is the same RetrievalContext shape.

From the Python SDK

The SDK mirrors the endpoint split — g.search(...) for namespace-wide retrieval, g.search_in_doc(...) for per-document retrieval. The doc_filter argument is required positionally on search_in_doc so the safety guarantee carries through to Python.

from index4ai import Index4AI

g = Index4AI()

# Namespace-wide retrieval — defaults are sized for the common case
ctx = g.search("How did operating margin change year over year?")
print(ctx.context)

# Scoped to one document — doc_filter is positional and required
ctx = g.search_in_doc("What is the cancellation policy?", "terms_of_service_v3")

SearchContext.context is the same prompt-ready XML + Markdown block you'd get from curl. Full signature: Index4AI.search / Index4AI.search_in_doc.

Errors

Status	When	Notes
400	Missing query, missing doc_filter (on searchInDoc), or upstream validation error	Body is plain text.
401	Missing or invalid token	Use a project API key, or an account key / login token with X-Project-ID.
404	doc_filter references an unknown document	Confirm the doc exists with GET /docs.
502	Other upstream / proxy errors	Retry with backoff.
503	Search backend unreachable	Retry; if persistent, check status page.

Upstream errors are mapped to these statuses based on the upstream message, so you'll see a consistent contract regardless of which internal subsystem failed.

Tuning checklist

When retrieval feels off, work down this list:

1. No matches at all. Check the doc is in GET /docs. Lower min_relevance (e.g. 0.3 → 0.1).
2. Junk in the context. Raise min_relevance (e.g. 0.3 → 0.5).
3. Truncated context. Raise target_tokens, or narrow the query so fewer hits are needed.
4. Slow latency / heavy payload. Drop return_assets when you don't need it; lower target_tokens so less context is rendered. (return_page_images never adds payload today — see the parameter table above.)
5. Wrong document picked. If you know the answer is in one doc, switch to /searchInDoc — that removes the cross-document ambiguity entirely. To exclude already-cited docs in an agent loop, pass exclude_doc_ids.

Next

• RetrievalContext format — the grammar of the returned block.
• API reference — full request/response schemas.