sicop_mcp

SICOP data server (Costa Rica public procurement, open data 2020-2026) exposed as a REST API and as an MCP server for AI assistants.

• Data: Salidas/ from the SICOP package (~4.3M rows loaded into Postgres in 31 tables: 13 sets per year 2020-2026 + 18 derived gold tables).

• Stack: Django 6 + DRF + Celery + Postgres + Redis. Same pattern as mwt/consola-mwt-one.

• Domain rule: every business figure for a supplier declares its measurement level (capture = awards · execution = purchase orders · delivery = receipts).

Local startup

Requires: Python 3.12+ (tested on 3.14), local PostgreSQL 16/18, Redis (or whatever broker you use).

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt

# crear DB (una vez):
#   createuser -U postgres sicop -P
#   createdb -U postgres -O sicop sicop

python manage.py migrate
python manage.py load_sicop --sync     # carga los CSV de Salidas/ (SICOP_DATA_DIR en .env)
python manage.py runserver 127.0.0.1:8000

Load data via Celery (stack pattern)

celery -A config worker -l info        # worker
python manage.py load_sicop            # encola una tarea por archivo
python manage.py load_sicop --only contratos --force

MCP server

python -m sicop.mcp_server                            # stdio (para clientes MCP)
python -m sicop.mcp_server streamable-http --port 9010  # HTTP

MCP tools (18): sicop_ficha_proveedor (execution vs capture), sicop_mercado_familia, sicop_competencia_procedimiento, sicop_producto, sicop_producto_historia (price series per year), sicop_cara_a_cara (two suppliers), sicop_expediente, sicop_adjudicaciones, sicop_carteles_objetados, sicop_representantes, sicop_representante_competencia, sicop_excepciones, sicop_sanciones, sicop_precios_institucion, sicop_perdidas_baratas (bid lowest and lost), sicop_campo_buscar, sicop_regimen_evaluacion, sicop_resumen.

Every business response carries the envelope (plan §5.4): nivel_medicion, cobertura_cruce (0.626), moneda, and caveats.

REST API

Equality filters use the exact column name (CEDULA_PROVEEDOR, NRO_SICOP, ANO, ...) and ?search= on text resources.

Related MCP server: chile-procurement

PHASE 2 — daily cycle (cron 06:00)

python manage.py ciclo_diario               # corrida manual del ciclo
celery -A config worker -l info             # worker (ya en tu stack)
celery -A config beat -l info               # cron: ciclo-diario 06:00 · vigilancia 06:05 · consolidar 06:15

• Daily cycle: rewrite monitoring (current month + 3 closed + 2 rotating) → consolidate PENDING results → watchlist signals → prioritized queue → gold + tests-gate.

• resultado_decision (SCH_RESULTADO v1, /api/v1/resultados/, POST /api/v1/resultado-registrar/): grain (nro_sicop, nro_linea, decision_id), append-only, mandatory frozen context (build_id/snapshot_ts/modelo_version/features_hash), override as a key field.

• Signals (/api/v1/senales/): client_participates/awarded/lost, objected notice, new sanction, watched institution, lost by a little (watchlist.json).

• Monitoring (/api/v1/vigilancia/): ETag/Content-Length of target months vs ctl_mes_fuente.

MCP tools: sicop_registrar_resultado, sicop_resultado, sicop_senales, sicop_vigilancia, sicop_ciclo_diario, sicop_consolidar_resultados.

PHASE 3 — physical enforcement + two lanes + logging

• Enforcement (/api/v1/politica/): middleware that blocks with 403 any request with raw paths (/salidas/, .csv, .zip, file://, ..\) or secrets; policy tests (no free SQL, no currency mixing, no raw paths) → 5/5 PASS.

• Two lanes: SICOP_CARRIL=operacion (canonical) or SICOP_CARRIL=laboratorio (every response labeled NO_APTO_PARA_DECISION, decision_eligible:false). The laboratory lane adds the tool sicop_lab_sql (read-only SQL, only SELECT/WITH, max 200 rows, rejects DELETE).

• Response logging (/api/v1/registro/, tool sicop_registro): every MCP call and API request is recorded in registro_respuesta (agent, tool, params, build_id, count, lane, duration, status).

PHASE 4 — test (ESOSA sheet, backtest, holdout) + pending items P1-P11

python manage.py fase4 --json        # ficha ESOSA desde gold + backtest + holdout (gate de muerte)
python manage.py pendientes --json   # P1-P7

• ESOSA sheet (/api/v1/prueba-fase4/?solo=ficha, tool sicop_ficha_esosa): reproduced from the canonical layer — EXACT performance (98.6% / 577 lines), capture reproduces the pattern (2022 peak, 2023 collapse), competition/face-to-face limited to cross coverage (62.6%) until the recovered data loads.

• Backtest (tool sicop_backtest_invitaciones): replay of past invitations (discount needed to win).

• Holdout + death gate (tool sicop_holdout): train <=2024, test 2025-26; if the model does not beat the tender anchor it is discarded (memory + monitoring remain).

• Pending items: p1_conversion_cartera (implicit FX 460-690 CRC/USD, official BCCR gate pending) · p3_catalogo_familias (9,295 derived families) · p5_recurrente_vs_recurrido · p6_sanciones_vigencia (1 sanctioned supplier currently winning) · p7_tamano_historico (1.04% change → no SCD2) · p10_bronze_zip_miembro (bronze from zip, literal raw row). P11 resultado_decision closed in PHASE 2.

Plan extras — Atlas, CGR, BCCR

• Atlas (/atlas/): corpus navigation app. Plan decisions respected: no figure travels alone (every screen shows its envelope), quality comes first (/atlas/calidad/: derivation by year, tests-gate, runs, monitoring, traps), blocked traps not documented (the UI warns about comparing currencies and cross-year prices without CL), and the app puts a face on the harness (day's signals visible). Screens: dashboard, search, suppliers, supplier sheet, product (history), procedure (record+competition+regime+invitees+appeals), market by family.

• CGR search (/api/v1/cgr/?termino=, tool sicop_cgr_buscar): PDFs of resolutions with native text. DIRECTED USE, not bulk scanning; legal gate pending (CGR terms not read).

• BCCR FX (/api/v1/bccr-tc/?fecha=, tool sicop_bccr_tc): official BCCR (series 317/318) if BCCR_TOKEN/BCCR_EMAIL in .env; without a token it returns the source's implicit FX (annual CRC/USD median) marked as such.

• Invitations: 42.2M rows loaded + invitados_vs_ofertantes.

Docker (VPS)

docker compose up -d --build
# expone 8400 -> django (puerto libre, no choca con 8100 de consola-mwt-one), con Salidas montado en /data/salidas

PHASE 1 — canonical layer (bronze + silver + control)

python manage.py fase1                # bronze -> silver (6 hechos) -> tests-gate -> gold atomico
python manage.py fase1 --solo-tests   # solo correr los tests como gate
python manage.py recalcular_derivadas # producto_firma, recursos_desenlace, tiempos_por_etapa, precios_identicos, invitados_vs_ofertantes, regimen_evaluacion, ctl_deriva, catalogo_campo

• Bronze (/api/v1/bronze/): immutable raw row + HASH_FILA + CORRIDA_ID + month.

• Silver — 6 facts (correct grain, DECIMAL(18,4), currency trio, bitemporality OBSERVADO_DESDE/HASTA/ES_VIGENTE): /fact-requerimiento (notice, proc x line x item) · /fact-oferta (proc x offer x line) · /fact-adjudicacion (act x proc x line x supplier) · /fact-contrato-linea · /fact-orden (one row per NRO_ORDEN, TOTAL_ORDEN only once, only summable CRC) · /fact-recepcion.

• Control (/api/v1/ctl-*): run, source month (zip hash), schema, quarantine, tests as gate.

• catalogo_campo (/api/v1/catalogo-campo/): navigable data dictionary (type, fill rate, key, trap, unit, join rule).

• Atomic publication: gold is not published if a gate test fails (the previous version remains).

MCP tools: sicop_fact_requerimiento/oferta/adjudicacion/contrato/orden/recepcion, sicop_catalogo_campo, sicop_ctl_deriva, sicop_regimen, sicop_competencia_por_regimen, sicop_gold_status.

Data

• MEASUREMENT LEVEL: cartera compares MONTO_EJECUTADO_CRC (only colones, dedupe by NRO_ORDEN) against MONTO_ADJUDICADO_CRC. Measuring by awards underestimates up to 59x (SONDEL 2026 case: 64x).

• Currencies: orders carry 5 currencies (CRC/USD/EUR/JPY/GBP); only colones are summed.

• Coverage: competencia_por_linea covers 62.6% of the offer x bidder cross (documented in the package).

• Privacy: inhibiciones contains officials; do not publish aggregates without an explicit decision (Law 8968).

• Cleanup: invalid amount/date cells are loaded as NULL (counted in estado-carga).