Stop Trusting the LLM
Building Authorization-Aware RAG for Multi-Tenant Enterprise Assistants

Most enterprise RAG tutorials quietly put the security boundary in the wrong place. They retrieve documents first, stuff the results into the prompt, and then instruct the model not to reveal anything sensitive.

That is backwards.

User prompt
  -> retrieve documents
  -> send everything to LLM
  -> ask model not to reveal restricted data

Once unauthorized data enters the context window, the access-control failure has already happened. Logs may record it. Traces may retain it. Caches may reuse it. The model may leak it. Even if the final answer looks clean, the system already crossed the boundary.

The correct pipeline starts with identity and scope:

User identity
  -> resolve tenant and permissions
  -> retrieve only authorized context
  -> fetch permitted live records
  -> construct prompt
  -> generate answer

The Real Problem Was Not RAG

Imagine a dealership employee asks an assistant:

What is the current service status for VIN123?

A useful answer may need live vehicle records, service history, troubleshooting SOPs, warranty policies, and dealership-specific access rights. The hard part is not generating fluent text. The hard part is ensuring that an employee from dealership A cannot retrieve data for a VIN assigned to dealership B.

That constraint is not a prompt-engineering problem. It is a backend architecture problem. The assistant is just another application surface sitting on top of tenant data, operational records, document search, audit requirements, and production latency constraints.

Why Prompt-Based Security Fails

The naive version looks deceptively reasonable:

prompt = f"""
Answer using the context below.

Do not reveal records belonging to other dealerships.

Context:
{all_retrieved_records}
"""

The flaw is not subtle: restricted data already entered the prompt. Behavioral instructions can be bypassed by malicious prompts, misunderstood by the model, or ignored under conflicting instructions. Even when the model behaves, your logs, traces, observability pipeline, prompt cache, and response cache may now contain data the user was never allowed to access.

Authorization needs deterministic enforcement. The LLM is probabilistic. Those two responsibilities do not belong in the same layer.

The Architecture

This design separates static knowledge, live operational data, and relationship-based permissions instead of pretending every data source belongs in the vector store.

Model the Authorization Graph

A minimal relationship model can express the rule directly:

entity user {}

entity dealership {
  relation employee @user
  relation manager @user
}

entity vehicle {
  relation assigned_dealership @dealership

  permission view =
    assigned_dealership.employee
    or assigned_dealership.manager
}

Then the data becomes relationship tuples:

dealership:pune_01#employee@user:employee_421
vehicle:VIN_A123#assigned_dealership@dealership:pune_01

The runtime check is ordinary backend code:

def can_view_vehicle(user_id: str, vin: str) -> bool:
    return permify.check(
        entity_type="vehicle",
        entity_id=vin,
        permission="view",
        subject_type="user",
        subject_id=user_id,
    )

The important principle is where this check happens. It happens before PostgreSQL is queried and before retrieval results are added to the prompt.

Vehicle-level access is only one dimension. Service history, customer PII, warranty notes, diagnostic telemetry, and internal escalation comments may each need separate disclosure classes. Being allowed to view a VIN does not automatically mean being allowed to view every field attached to it.

Why Hybrid BM25 + FAISS

Hybrid retrieval is useful here, but it is not the innovation. Dealership workflows contain exact identifiers and natural-language symptoms, and each needs a different retrieval strength.

def hybrid_search(query: str):
    lexical_hits = bm25.search(query, top_k=20)
    semantic_hits = faiss.search(embed(query), top_k=20)

    return reciprocal_rank_fusion(
        lexical_hits,
        semantic_hits,
    )[:8]

BM25 handles exact tokens well. FAISS handles semantic similarity. Reciprocal-rank fusion keeps the implementation simple and avoids pretending one retrieval mode is universally better.

Apply Authorization Inside Retrieval

This is the section that matters most. Each indexed chunk carries metadata that makes authorization possible before prompt assembly:

{
  "chunk_id": "service_case_991",
  "scope_type": "vehicle",
  "scope_id": "VIN_A123",
  "dealership_id": "pune_01",
  "classification": "service_history"
}

A global SOP has a different scope:

{
  "chunk_id": "sop_bms_17",
  "scope_type": "global",
  "classification": "service_manual"
}

The simplest version is a two-step operation: search broadly, then filter deterministically.

def retrieve_authorized_context(
    user_id: str,
    query: str,
):
    hits = hybrid_search(query)

    return [
        hit
        for hit in hits
        if is_authorized(user_id, hit.metadata)
    ][:6]

You can push the filter down into BM25 metadata filters, FAISS post-filtering, or a retrieval gateway. The implementation detail matters less than the invariant: unauthorized context never reaches the prompt assembler.

There is a real recall tradeoff here. If the top 20 hits contain 18 unauthorized chunks, post-filtering may leave only two useful results even when authorized matches exist lower in the ranking. Production systems usually need one of four patterns: pre-filter the candidate space by tenant and data class, over-fetch and post-filter, maintain tenant-partitioned indexes, or combine pre-filtering with authorization-aware reranking.

Keep Live Data Out of the Vector Store

Do not embed frequently changing vehicle records into FAISS just because the assistant uses RAG. They will become stale, they will be hard to invalidate, and they will blur the boundary between documents and operational truth.

Fetch live data from the system of record after authorization:

def get_vehicle_snapshot(user_id: str, vin: str):
    if not can_view_vehicle(user_id, vin):
        raise ForbiddenError()

    return db.fetch_one("""
        SELECT
            vin,
            current_status,
            battery_soc,
            open_issue_count,
            last_telemetry_at
        FROM vehicle_snapshot
        WHERE vin = %s
    """, [vin])

The model receives controlled JSON, not raw SQL access:

{
  "vin": "VIN_A123",
  "current_status": "Inspection Required",
  "battery_soc": 37,
  "open_issue_count": 1
}

That small distinction prevents an agent from improvising database access. The grounding service owns the query shape, selected fields, redaction rules, and audit trail.

Add a Scope Token

A common production failure is the confused deputy: the gateway performs the correct authorization check, then an internal retrieval service accidentally executes with broader privileges. A short-lived signed scope token keeps downstream services honest.

{
  "user_id": "employee_421",
  "dealership_ids": ["pune_01"],
  "allowed_data_classes": [
    "service_manual",
    "vehicle_snapshot",
    "service_history"
  ],
  "expires_at": "2026-06-02T18:00:00Z",
  "policy_version": "v14"
}

Gateway
  -> Retrieval service
  -> PostgreSQL grounding service
  -> Prompt assembler
  -> Audit logger

Every downstream service validates the token and applies the same scope. This is not about trusting internal services less. It is about making privilege explicit at every hop where data can enter the prompt.

Where the Naive Version Broke

The easiest mistake is response caching. In a naive assistant, a dealership employee asks about a VIN, the system retrieves context, the model generates an answer, and the response is cached by normalized prompt text.

Naive:
Prompt -> cache lookup -> retrieve -> LLM

That breaks as soon as scope changes. Imagine a VIN is reassigned from dealership A to dealership B. The gateway may correctly resolve the new employee's dealership, but a prompt-only cache key can still make an old answer reusable across the wrong authorization envelope.

The fix is not another model instruction. The fix is to version authorization policy, include the privilege envelope in the cache key, and invalidate VIN mappings when reassignment happens.

Safer:
Identity -> scope token -> scoped cache lookup
         -> authorized retrieval
         -> live grounding
         -> prompt assembly
         -> LLM

This is the difference between treating GenAI as a chatbot and treating it as a production data surface. The model did not cause the leak. The cache boundary did.

Illustrative Latency Budget

These are budget numbers, not production trace results. They are useful as a planning target: security that blows the interaction budget will get bypassed, so a practical assistant needs deterministic authorization and acceptable response time.

The optimizations are boring, which is usually a good sign:

• Run BM25 and FAISS in parallel.
• Precompute embeddings.
• Index VIN and dealership columns.
• Cache permission decisions carefully.
• Include authorization scope in every cache key.
• Retrieve a small number of chunks.
• Use a denormalized live snapshot table.
• Stream the model response.

Failure Cases Worth Designing For

This architecture is useful because it forces the uncomfortable cases into the design instead of discovering them during an incident.

• VIN reassignment: invalidate old permissions when a vehicle moves between dealerships.
• Stale cache entries: cache by scope and policy version, not just by prompt text.
• Bulk queries: apply result-level authorization to questions like "show all battery failures."
• Hidden identifiers: extract unauthorized VINs from long prompts and deny or redact before grounding.
• Cross-tenant prompt injection: treat retrieved tenant content as untrusted input, even after authorization.
• Authorization outages: fail closed for restricted data.
• Restricted logs: avoid storing raw prompt context in observability systems by default.
• Response caching: never cache only by natural-language prompt.

A safer response-cache key includes the actual authorization envelope:

hash(
  user_scope
  + policy_version
  + prompt
  + retrieved_context
  + model_id
)

The reason is simple: two users can ask the same question and be entitled to different answers.

What This Does Not Solve

Authorization-aware retrieval is not a magic security layer. It narrows the trust placed in the model, but it does not remove every risk.

• It does not prevent leakage from knowledge memorized during fine-tuning.
• It does not solve every prompt-injection attack.
• It does not replace field-level redaction.
• It does not prove that an agent will use an authorized tool safely.
• It does not eliminate the need for audits.

This architecture does not make LLMs trustworthy. It reduces the amount of trust the system places in them.