# foodex2-encoder

Library and stdio MCP server that turns an English food description into an
EFSA FoodEx2 code (MTX catalogue): base term, facet descriptors, and labelled
free text for what the code cannot express.

This repository is the core. A hosted version is available at https://foodex2-encoder.foodnotes.org/ .

## Limitations

Check every code before you use it. In particular:

- **English input only**, including foreign food names commonly used in English.
- **Group names the catalogue splits are kept as free text.** When a description
  uses a name that the catalogue only has as several separate terms, that part
  goes to free text instead of the code. For example, *canned shellfish* is coded
  as *Canned seafood* with `source-commodity=shellfish` in free text, rather than
  *Canned seafood* made from crustaceans and molluscs. The code is not wrong, but
  it is broader than the description: *Canned seafood* also covers other marine
  animals, such as sea urchins.

## What you need

- Node.js 20+
- Python 3 (one-time catalogue build from the vendored EFSA `.ecf`)
- An OpenAI-compatible model endpoint and API key (OpenRouter is the default)

## Install

```bash
git clone https://github.com/FoodNotes-org/foodex2-encoder.git
cd foodex2-encoder
git submodule update --init
npm install
npm run build:catalogue
npm run build:embeddings
```

`data/` is generated locally (~25 MB) and is gitignored. Rebuild after updating
the `vendor/efsa-catalogues` submodule.

The coding rules in `src/rules/` (EFSA business rules, facet cardinality, our
own checks) are shared with the FoodEx2 Atlas validator. Their data comes from
`rules/cardinality.yaml`, `rules/conflicts.yaml` and
`vendor/efsa-business-rules/`; after editing those, regenerate the committed
`src/rules/rules-data.json` with `npm run build:rules`. Sources, licence and
limits are in `rules/efsa-business-rules.md`.

## Model endpoint

Copy `.env` (or export the same variables). OpenRouter needs only a key:

```bash
FOODEX2_LLM_API_KEY=sk-or-…          # or OPENROUTER_API_KEY
# optional:
# FOODEX2_LLM_BASE_URL=https://openrouter.ai/api/v1
# FOODEX2_MODEL=openai/gpt-6.1-sol
```

Any other OpenAI-compatible endpoint works the same way (`FOODEX2_LLM_BASE_URL`,
`FOODEX2_LLM_API_KEY`, `FOODEX2_MODEL`). The **server** calls that endpoint for
classify / select / residual steps; your chat client’s subscription is separate.

## Try it (CLI)

```bash
npm run cli -- encode "orange juice"
npm run cli -- encode "fried rice with chicken"
npm run cli -- traverse "eggplant"    # walk only (debug)
```

Lexical hits return in about a second. Descriptions that need a catalogue walk
typically take 15–25 seconds.

## Try it (MCP stdio)

```bash
npm run server
```

Point a client at that process. Cursor example (`mcp.json`):

```json
{
  "mcpServers": {
    "foodex2-encoder": {
      "command": "npx",
      "args": ["tsx", "/ABS/PATH/TO/foodex2-encoder/src/server.ts"],
      "env": {
        "FOODEX2_LLM_API_KEY": "sk-or-…"
      }
    }
  }
}
```

Tool: `encode`. It returns the code, base term, facets, free text, fit,
method, and a short `explanation`. Catalogue lookups, decoding, validation
and EFSA guidance are in the free
[FoodEx2 Atlas MCP](https://foodex2-atlas.foodnotes.org/mcp-service/).

## Tests and eval

```bash
npm test                 # unit tests (no model)
npm run lexical-eval     # lexical base-term cases (no model)
npm run encode-eval      # end-to-end gold set (needs a model key)
```

`eval/encode-base-term.yaml` is a small, focused regression set: one realistic
case per capability (catalogue synonym, model synonym, named dish, restraint,
and so on). Gold rules can require or forbid facets and free-text fragments;
`expected` may list several acceptable base codes (any-of).

`npm run encode-eval` records the model and MTX version, and on a complete run
writes `eval/results/encode-base-term.json` for publishing or sharing with the
public core repo. Always cite the model named in that file’s `_provenance`.

## Layout

```
src/encode/     lexical → traverse → select → residuals
src/search/     lexical + embedding retrieval
src/server.ts   stdio MCP
src/tools.ts    tool handlers (shared surface for a future Worker wrapper)
data/           generated catalogue + embeddings (local)
eval/           gold cases + results artifact
vendor/         EFSA catalogues (submodule)
```

## Licence

MIT — see [LICENSE](LICENSE). Copyright (c) 2026 e-accent BV.

That grant covers this software. EFSA catalogue files under `vendor/efsa-catalogues/`
(and generated `data/` derived from them) remain subject to EFSA’s terms.

### Attribution (eval inputs)

Some food names used as evaluation cases under `eval/` were drawn from national
food composition databases. We use food names only (not nutrient values). Those
databases remain under their publishers’ terms:

- **CIQUAL** (ANSES, France) — [ciqual.anses.fr](https://ciqual.anses.fr/).
  Cite as: Anses. Table de composition nutritionnelle des aliments Ciqual.
  Data are available under the [Etalab Open Licence 2.0](https://www.etalab.gouv.fr/licence-ouverte-open-licence/).
- **CoFID** (UK) — McCance and Widdowson’s Composition of Foods Integrated
  Dataset, [GOV.UK](https://www.gov.uk/government/publications/composition-of-foods-integrated-dataset-cofid).
  Contains public sector information licensed under the
  [Open Government Licence v3.0](https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/).