Skip to main content
Glama
srnux

proptech-inquiry

by srnux

proptech-inquiry-agent

CI

An AI agent that answers questions about property listings from the record, and hands everything else to a human: viewings, negotiation, legal questions, complaints, anything the data does not cover. It works through an MCP server, so the same tools serve Claude Desktop, the MCP Inspector and the agent loop here, and checks in code reject any figure or citation the tools did not return. 49 eval cases measure it: 46 pass on claude-opus-5-5, and every hand-off goes where it should.

The inquiry desk answers whether heating is included, citing the listing sentence and the utilities policy, turns a viewing request into a ticket for a colleague instead of confirming it, then answers a German inquiry in German: pets only on request, heating included, the viewing handed on

Recorded three times the real speed: a fact question, a viewing request, and the same kind of inquiry in German. Left: the conversation, with a chip per source. Middle: every tool call. Right: tickets for a colleague.

All listing and policy data is synthetic.

Quick start

git clone https://github.com/srnux/proptech-inquiry-agent.git && cd proptech-inquiry-agent
corepack enable   # once, provides the pnpm version pinned in package.json
pnpm install
pnpm dev          # the inquiry desk on http://localhost:5173, the API on 127.0.0.1:3000

Click one of the example inquiries. The conversation fills on the left, every tool call the agent made in the middle (click a row for its arguments and result), and the tickets for a colleague on the right. Each citation in a reply is a chip that opens the passage it came from. Follow-ups work ("and the deposit?"): the page sends the last five exchanges with each inquiry, and "New conversation" starts over (DECISIONS.md 31).

Without ANTHROPIC_API_KEY (or the Bedrock settings) in .env, pnpm dev runs a rule-based demo model that drives the same tools and checks, and the page says so (DECISIONS.md 28). The first start downloads the two retrieval models (about 690 MB); EMBEDDER=hashing pnpm dev starts at once with weaker, English-only retrieval.

How the parts fit together, and where each check sits: ARCHITECTURE.md.

Related MCP server: Repliers MCP Server

The one design rule

The agent answers facts from the record and escalates everything else. Viewings, negotiation, contract and legal questions, complaints, personal data requests and anything the record does not cover go to hand_off_to_human. The boundary sits in the tool contract, not only in a prompt, so it can be tested.

Retrieval follows the same rule: when no passage is relevant, search_knowledge returns found: false instead of the closest weak match, and the tool description tells the model to hand off.

What broke

The first full eval run passed 41 of 49 cases. It found three problems, each fixed in its own commit and measured again (first run, after the fixes):

  • German questions could not find the commission policy. "Muss ich als Mieter eine Provision zahlen?" returned found: false: the page said only "commission", so neither keyword nor vector search collected it. The page now names "Provision" and "Maklerprovision", and golden-set recall@5 went from 96% to 100% (495f2e6, DECISIONS.md 37).

  • Follow-ups "corrected" answers that were right. Told that earlier turns are not evidence, the agent looked the fact up again, found it confirmed, and announced a correction anyway. It now corrects only what the record contradicts; a case whose earlier figure really is wrong still gets its correction (3212e5b, DECISIONS.md 38).

  • The judge counted hand-off wording as invented claims. "A colleague will contact you" is fine when a ticket exists; "your viewing is confirmed" is not. That fixed the evaluator, not the agent (c3c7a39).

Still open: the agent offered to add an email address to an existing ticket, which no tool can do. The guards check that every figure and citation is in the record; they cannot check that a promise is one the tools can keep. The other two failing cases are borderline inferences the judge flags ("so you would have to use the stairs").

Status

Slice

What it adds

State

1

MCP server: structured search, listing lookup, hand-off to a human

done

2

Hybrid retrieval over listing texts and policy pages, local embeddings

done

3

Agent loop, citation and number guards, POST /inquiries (SSE), Streamable HTTP MCP, pnpm ask

done, acceptance inquiry and live test passed on claude-opus-5-5 (Bedrock)

4

React UI: chat, tool-call trace, live hand-off queue, citation chips; runs without a key on a demo model

done

5

Eval suite: 49 cases, model judge, thresholds; pnpm eval writes a report with a trace per case

done, 46 of 49 cases pass on claude-opus-5-5 (report)

6

ARCHITECTURE.md, demo GIF, "What broke", CI on every push, eval subset on GitHub Actions by hand

done

7

pgvector behind the same store interface

optional, not scheduled

Details, tasks and open decisions per slice are in ROADMAP.md; the reasons behind each choice are in DECISIONS.md.

Tools and resources

Name

Kind

Purpose

search_listings

tool, read

Hard-criteria search. Returns summaries without description text.

get_listing

tool, read

One full record: structured fields and description.

search_knowledge

tool, read

Hybrid search over listing texts and policies. Scoped to one listing when given an id. Returns citable chunk ids, or found: false.

hand_off_to_human

tool, write

Creates a ticket with a fixed reason code. Rejects unknown listing ids.

policy://<slug>

resource

The policy pages (deposit, utilities, pets, viewings, ...) as markdown.

How retrieval works

Two stages, like a librarian who first pulls likely pages off the shelf and then reads them.

  1. Collect. Listing descriptions are split into sentences and policy pages into sections. A question is first limited to one listing plus the general policies, then the closest passages are collected twice: by meaning (multilingual-e5-small embeddings, local, German and English) and by keywords (BM25).

  2. Judge. A reranker (bge-reranker-v2-m3, local, multilingual) reads the question together with each collected passage and scores whether it answers it. Only passages above a threshold calibrated on a golden set are returned. If none is, the answer is found: false.

Measured on the golden set (26 questions with an answer, 8 without, German and English): the right passage is in the top 5 for all 26, and none of the questions without an answer returns anything. The last miss, a German commission question, was fixed by an eval finding (DECISIONS.md 37). About 1.4 seconds per question on a laptop CPU.

Why two stages, with the measurements that led there: DECISIONS.md 16 to 19.

The agent

pnpm ask "..." sends one inquiry through the loop in packages/core/src/agent/. The agent is an MCP client of this repo's own server, so it can only do what the tools allow. Each run returns the reply, the citations parsed from it, the hand-off tickets it created, a trace of every tool call and the token usage.

Two checks run in code on the final answer (DECISIONS.md 20): every cited id must have been returned by a tool in this run, and every price, area and percentage must appear in a tool result. A failed check gets one repair turn; a second failure, the turn limit, the token budget or a timeout ends in a hand-off ticket created by the code, not an exception.

cp .env.example .env            # then put your ANTHROPIC_API_KEY (or the Bedrock settings) in .env (git-ignored)
pnpm ask --trace "Ist die Heizung bei HH-1001 inklusive, und kann ich Samstag besichtigen?"
pnpm serve                            # POST /inquiries (server-sent events) and /mcp on 127.0.0.1:3000
curl -N localhost:3000/inquiries -H "content-type: application/json" -d "{\"inquiry\":\"Is HH-1001 still free?\"}"

The model is claude-opus-5-5 by default; ANTHROPIC_MODEL and ANTHROPIC_EFFORT change it (DECISIONS.md 23). To run on Amazon Bedrock instead, set MODEL_PROVIDER=bedrock, AWS_REGION and AWS_BEARER_TOKEN_BEDROCK (DECISIONS.md 25). Tests use a scripted model and need no key; packages/server/test/agent-live.test.ts runs against the real model when credentials are set in the environment.

Evals

pnpm eval runs the 49 cases in evals/cases.jsonl through the real agent loop: fact and policy questions, searches, every hand-off reason, follow-ups, the traps (pets "on request" is not a yes, the facade levy is not final, Staffelmiete is 3% a year, the Munich rent is all-inclusive, the Köln unit is not for living in, polite haggling is still a negotiation) and a listing whose text tells the agent to confirm viewings. Code checks the hand-offs, the listings and every figure; a different model (claude-sonnet-5-5) grades the facts against a written rubric. The report lands in evals/reports/<date>.md, and every case in it links to the full trace of its run (DECISIONS.md 32 to 38).

Metric

First run

After the fixes

Threshold

Cases passed

41 / 49

46 / 49

90%

Hand-off precision / recall

93.8% / 100%

100% / 100%

93% / 100%

Replies with every figure grounded (final / first draft)

100% / 100%

100% / 100%

100% / 100%

Correct, by the judge

83.7%

93.9%

90%

Golden-set recall@5, false positives

96%, 0

100%, 0

100%, 0

Cost, latency per inquiry

$0.016, 9.2 s (p95 14.7 s)

$0.014, 8.6 s (p95 13.8 s)

$0.02, p95 20 s

Both runs are committed, claude-opus-5-5 on Bedrock. What the first run found and how each problem was fixed: What broke.

The subset also runs on GitHub Actions when started by hand (Actions → Eval subset → Run workflow), with ANTHROPIC_API_KEY or AWS_BEARER_TOKEN_BEDROCK as a repository secret. The report table appears on the run page (DECISIONS.md 39).

pnpm eval                 # all cases; needs credentials for the agent and the judge; exits 1 below a threshold
pnpm eval --subset        # 12 cases, run locally or through the manual Actions workflow; about 90 s and $0.25
pnpm eval --case handoff-viewing --case trap-pets-hh1001
MODEL_PROVIDER=demo EMBEDDER=hashing pnpm eval --no-judge   # offline: demo model, no judge

Run it

corepack enable   # once, provides the pnpm version pinned in package.json
pnpm install
pnpm test         # offline, no model download
pnpm test:e2e     # Playwright against the demo model; once before: pnpm --filter @proptech/web exec playwright install chromium
pnpm index        # first run downloads both models (about 120 MB and 570 MB) into .models/, builds .index/
pnpm calibrate    # prints the scores of every golden question and proposes thresholds
pnpm test:model   # golden set against the real models
pnpm inspect      # MCP Inspector against the server

EMBEDDER=hashing (PowerShell: $env:EMBEDDER = "hashing") switches to a deterministic offline embedder without a reranker: no download, English only, weaker. The test suite uses it. RERANKER=none keeps e5 but drops the reranker.

After changing the model or the corpus, run pnpm calibrate. It prints the relevance signals for every golden question and proposes thresholds for retrieval.thresholds.json.

Claude Desktop

Run pnpm build and pnpm index once, so the server does not download the models while Claude Desktop is waiting for it. Then add to claude_desktop_config.json (--conditions=built makes the server load the compiled core package instead of its TypeScript source, DECISIONS.md 26):

{
  "mcpServers": {
    "proptech-inquiry": {
      "command": "node",
      "args": [
        "--conditions=built",
        "/path/to/proptech-inquiry-agent/packages/server/dist/mcp/stdio.js"
      ]
    }
  }
}

Layout

A pnpm workspace. packages/core has no MCP, HTTP or model-provider imports; packages/server wraps it. All scripts run from the repository root.

data/listings.json              synthetic catalogue, validated with zod at load
data/policies/*.md              synthetic policy pages, one topic each
evals/retrieval-golden.json     questions with the chunks that must be found, plus questions with no answer
evals/cases.jsonl               agent eval cases; rubric.md for the judge, thresholds.json, run.ts (`pnpm eval`)
evals/reports/                  one committed report per full run, with a trace file per case
evals/fixtures/                 test-only listings (the prompt-injection case), never served by the app
retrieval.thresholds.json       relevance thresholds per embedding model
packages/core/src/domain/       listing model, repository, hand-off queue
packages/core/src/retrieval/    chunker, BM25, embedders, vector store, reranker, search
packages/core/src/agent/        model interface, tool-use loop, guards, prompt
packages/core/scripts/          index, calibrate, experiments
packages/server/src/mcp/        tool and resource definitions, stdio and Streamable HTTP entry points
packages/server/src/agent/      Claude API and Bedrock adapter, in-process MCP client, `pnpm ask`
packages/server/src/api/        HTTP server: /inquiries (SSE), /mcp, and /handoffs, /sources for the web app
packages/*/test/                unit tests, retrieval tests, in-memory MCP client tests
apps/web/                       the inquiry desk: Vite, React, one stylesheet; Playwright tests in e2e/

License

MIT

Available Tools

3 tools
get_listingGet listingA
Read-only

Return the full record of one listing, including its description text. Answer only from fields present here. If the answer is not in the record, say so and hand off with reason not_answerable_from_listing instead of guessing.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesListing id, e.g. HH-1001

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, so safety is covered by structured data. The description adds genuinely useful behavioral policy not present in the annotations: answer strictly from returned fields and refuse/redirect rather than guess when data is absent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the purpose and followed by the grounding and fallback rules. Nothing is redundant and every sentence changes agent behavior.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With one required parameter, full schema coverage, and no output schema, the definition covers what an agent needs: what comes back ('full record' incl. description) and how to behave when the record lacks the answer. It could briefly note behavior for a missing/invalid id, which keeps it from a 5.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single id parameter is documented with an example ('HH-1001'), so the schema carries the full burden. The description adds no syntax, format, or validation detail beyond it, which matches the baseline 3 for a fully documented parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource ('Return the full record of one listing') plus what the record contains ('including its description text'). It implicitly distinguishes itself from sibling search_listings by scoping to a single record, though it never names the alternative explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a clear grounding rule ('Answer only from fields present here') and an explicit when-not path ('If the answer is not in the record... hand off with reason not_answerable_from_listing instead of guessing'), which routes the agent to the hand_off_to_human sibling. It does not state prerequisites for obtaining the listing id, but the context of use is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

hand_off_to_humanHand off to a human agentA

Create a ticket for the letting or sales team. You MUST use this, and not answer yourself, for: booking viewings, any price or rent negotiation, contract or legal questions, complaints, requests about personal data, and any question the listing record does not answer. Summarise the inquiry in one or two sentences so the human does not have to reread the thread.

ParametersJSON Schema
NameRequiredDescriptionDefault
reasonYes
summaryYes
listingIdYesListing the inquiry is about, or null
contactEmailYes

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this is a non-read-only, non-destructive, non-idempotent, closed-world write, and the description's "Create a ticket" is consistent with that. It adds useful behavioural context about how the summary is consumed ("so the human does not have to reread the thread"), but does not say whether the customer is notified or what happens after the ticket is filed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the action and then the escalation trigger list. The long MUST clause is dense but every item earns its place as a routing rule; only the trailing summary instruction could be tightened slightly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 4-required-parameter escalation tool with no output schema, the description covers routing, required intent categories, and summary style. It leaves contactEmail semantics and any post-creation behaviour (ticket id, notification, SLA) unspecified, which is a minor but real gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 25% (only listingId is documented in the schema). The enumerated use cases in the description map well onto the reason enum values, and the summary wording gives a length/style expectation, but contactEmail (whose address, required even when the listing is unrelated) and listingId's null case are never explained in the description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: "Create a ticket for the letting or sales team." That is unambiguous and clearly distinct from the sibling tools search_listings and get_listing, which read listing data rather than escalate an inquiry to a human.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit MUST-use list (viewing bookings, price/rent negotiation, contract or legal, complaints, personal data requests, and anything the listing record cannot answer) plus the exclusion "not answer yourself," which implicitly routes answerable questions to get_listing. When-to-use and when-not-to-use are both covered.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_listingsSearch listingsA
Read-only

Find listings by hard criteria (city, rent or sale, price ceiling, rooms, area, features, move-in date). Use this when the inquirer states requirements. Returns summaries sorted by price, cheapest first. Do not use it to answer questions about a specific listing's details; call get_listing for that.

ParametersJSON Schema
NameRequiredDescriptionDefault
cityNoCity name, e.g. Hamburg, Berlin, München, Köln
limitNo
hasPetNoTrue if the inquirer has a pet. 'on-request' listings are kept; never promise them.
featuresNoAll must match, e.g. balcony, lift, garden, accessible
maxPriceNoEUR per month for rent, EUR total for sale
minRoomsNo
offerTypeNo
minAreaSqmNo
availableByNoLatest acceptable move-in date, YYYY-MM-DD
propertyTypeNo

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is covered. The description adds return behavior not present in structured data: results are summaries sorted by price, cheapest first. It does not address pagination or how the limit truncates results, so it stops short of full transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences with zero filler. The filter criteria and the primary usage condition are front-loaded, and the exclusion/alternative follows immediately after.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only search tool with no output schema, the description supplies the return shape (summaries sorted by price) and the routing rule to get_listing. It omits the limit/default page size and never mentions propertyType, which is a minor but real gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 50%, so several parameters (offerType, minRooms, minAreaSqm, propertyType, limit) lack schema descriptions. The description compensates by naming rent-or-sale, price ceiling, rooms, area and move-in date as filter criteria, covering most undocumented params, though propertyType and limit remain unmentioned anywhere.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+resource ("Find listings") and enumerates the hard criteria it filters on, so the agent knows exactly what the tool retrieves. It also explicitly contrasts itself with get_listing, making it distinguishable from its sibling without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states the triggering condition ("Use this when the inquirer states requirements") and an explicit exclusion with the correct alternative ("Do not use it to answer questions about a specific listing's details; call get_listing for that"). Both when-to-use and when-not-to-use are covered.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 3 tool updatesv0.1.0
    • First observedget_listing
    • First observedhand_off_to_human
    • First observedsearch_listings

TDQS

A4.3/5.0

Scored across 3 tools

Disambiguation5/5

Each tool targets a distinct action: search by criteria, retrieve one record, or escalate to a human. The descriptions explicitly define boundaries (search_listings says do not use it for a specific listing; get_listing says answer only from the record), so misselection is unlikely.

Naming Consistency5/5

All three names follow a clear verb_noun pattern (search_listings, get_listing, hand_off_to_human) with consistent snake_case.

Tool Count4/5

Three tools is lean for the narrow inquiry-handling scope, and each earns its place, though it sits near the lower bound where coverage could feel thin for edge cases.

Completeness4/5

The read-plus-escalate lifecycle is well covered: criteria search, full record retrieval, and human handoff for bookings, negotiation, legal, and unanswerable questions. Only listing management/creation is absent, which is outside an inquiry agent's stated purpose.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    B
    quality
    D
    maintenance
    Enables users to access the Realty In Ca1 API to search for residential and commercial property listings, view detailed property information, and retrieve neighborhood demographics or statistics. It supports filtering by geographic location, price range, building type, and other real estate characteristics.
    MIT
  • A
    license
    Not graded
    quality
    B
    maintenance
    Provides AI assistants access to real-time MLS data via the Repliers API, enabling natural language property search, market statistics, and listing details.
    204 npm
    18
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    Enables AI assistants to operate property management systems via natural language, covering repair orders, owner info, payments, notices, and inspections. Features a full agentic workflow with human-in-the-loop and observability.
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    Enables AI assistants to look up real estate agents, search MLS listings, submit buyer/seller leads, and access proprietary seller-intent signals in SC and GA.
    MIT