socrata-mcp-server

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond readOnlyHint and idempotentHint annotations, the description discloses that the tool fails with canvas_id_required if canvas is enabled but no ID is provided, and that it only works after a spill. This adds critical behavioral context not implied by annotations alone.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three focused sentences: first states purpose, second links to sibling tool, third gives usage context. No redundancies, each sentence earns its place. Information is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Description covers purpose, prerequisites, parameter semantics, and usage sequence. Since an output schema exists, not describing return values is acceptable. Minor gap: no mention of possible errors beyond the canvas_id failure, but the tool is simple enough that the description is largely complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, baseline score 3. Description adds extra meaning: explains that canvas_id is practically required despite being optional in schema, describes its origin from socrata_query_dataset, and warns about failure mode. This goes beyond the schema's own description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists registered tables in a DataCanvas session, specifying the info provided (schema, row count, columns, registration time). It distinguishes itself by linking to sibling socrata_dataframe_query, and gives a specific use case after socrata_query_dataset spills to canvas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear context: only meaningful when CANVAS_PROVIDER_TYPE=duckdb is set, and suggests using after socrata_query_dataset spills results. However, it does not explicitly contrast with siblings or state when not to use, missing full comparative guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already provide readOnlyHint and idempotentHint. The description adds substantial behavioral context: DuckDB infers types from spilled data (making numeric columns queryable), and DDL/DML/file-reading functions are rejected. This goes well beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is four sentences, each earning its place: primary action, type inference detail, environment requirement, and pointer to sibling tool. No redundant or unnecessary information. Front-loaded with the core purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the presence of output schema, full annotations, and 100% schema coverage, the description provides the missing pieces: how to use the tool effectively (type inference, environment, table discovery). It equips the agent to select and invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so parameters are fully documented. The description adds meaningful context: explains the type inference benefit ('numeric columns that SODA returned as strings become queryable') and tells the agent to use table names from socrata_dataframe_describe. This enhances semantic understanding beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Run SELECT-only SQL against a DataCanvas table populated by socrata_query_dataset.' It specifies the verb (run), the resource (DataCanvas table), and the constraint (SELECT-only). It differentiates from siblings like socrata_dataframe_describe (which shows schemas) and socrata_query_dataset (which populates).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context: 'Only works when CANVAS_PROVIDER_TYPE=duckdb is set' and 'Use socrata_dataframe_describe to see registered tables and their schemas.' This guides when to use and how to prepare. It doesn't explicitly say when not to use, but implies DDL/DML are not allowed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations include readOnlyHint, openWorldHint, idempotentHint. The description adds context about return format (abbreviated column lists, lack of type information) and behavior across all Socrata portals. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, concise and front-loaded with primary action. Every sentence adds value without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 8 optional parameters, strong annotations, and an output schema referenced, the description covers core behavior, output fields, limitations, and connection to sibling tool. Complete and sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline 3. The description adds minimal additional meaning beyond schema; it mentions domain scoping but schema already covers that. The note about column previews is about output, not parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it searches for datasets across Socrata portals with optional domain scoping, and lists return fields. It distinguishes itself from sibling tool socrata_get_dataset by noting that this returns preview-only column list without type info.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly advises to use socrata_get_dataset for full typed column schema before writing queries, and indicates optional domain scoping. It provides clear context for when to use this tool versus its sibling.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and idempotentHint=true, so the description's safety profile is reinforced. The description adds behavioral details: returns field names, data types, descriptions, row count, licensing, and warns about query syntax differences.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two efficient sentences: first states purpose, second provides critical usage guidance. No wasted words; every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a schema-fetching tool with output schema available, the description covers purpose, usage context, and a critical behavioral note about query syntax. No gaps given the tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with clear descriptions for both parameters (domain defaulting to env var, dataset_id pattern). The description adds value by mentioning the env var default and giving an example pattern, but doesn't significantly expand beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Fetch full metadata and column schema for a Socrata dataset by ID', specifying both the action (fetch schema) and resource (dataset). It distinguishes from sibling tools like socrata_query_dataset by indicating it should be called first to understand column types.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly instructs to 'Always call this before writing a socrata_query_dataset' and explains why: column types determine correct WHERE clause syntax. This provides clear usage context and distinguishes from alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare readOnlyHint, openWorldHint, idempotentHint. The description adds valuable behavioral details: the catalog is curated (40 portals), dataset counts are fetched from the Discovery API and cached for ~24 hours, and filtering is client-side. No contradictions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise (4 sentences), front-loads the main purpose, and every sentence contributes unique information. No redundant or vague statements.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers all key aspects: what is returned (domain, org name, approx dataset count), caching, filtering, and pagination. An output schema exists but is not provided; however, the description adequately explains the return content. Given the tool's complexity and annotations, it is complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for all params. The description adds value by explaining that 'query' performs case-insensitive substring matching on portal/organization names and that omitting it lists all. For limit and offset, it doesn't add much beyond the schema, but the baseline is 3 and it adds some useful context.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it lists known Socrata-powered open-data portals with domain, organization name, and approximate dataset count. It distinguishes itself from sibling tools by explicitly noting to use it first when unsure of the portal, then pass the domain to socrata_find_datasets.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidance: 'Use this first when you do not know which portal to target, then pass the domain to socrata_find_datasets.' It also explains filtering behavior (client-side substring match) and caching (24-hour cache).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (readOnlyHint, idempotentHint), the description adds significant behavioral details: all values are strings, returns the SoQL string, explains spill to DataCanvas under DuckDB, and warns about quoting rules based on dataType. No contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with a front-loaded purpose, followed by specific guidance and edge cases. It is moderately concise; every sentence adds value but could be slightly more streamlined.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (11 params, full schema coverage, output schema present, annotations), the description covers usage, quoting, distinct values, DuckDB spill, and learning the SoQL pattern. It leaves no major gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds extra context: for where it explains quoting, for search it clarifies $q, for select gives examples, and for limit/offset mentions pagination. This adds value, but not exhaustive for all 11 parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Execute a SoQL query against any dataset on any Socrata portal', using a specific verb (query) and resource (dataset). It distinguishes from siblings like socrata_get_dataset (metadata) and socrata_dataframe_query (dataframe operations) by specifying the SoQL nature and analytical control.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context: use search for quick full-text, combine select/where/group/having/order for analytical control, and check dataType for correct quoting. It gives a tip for distinct values. However, it lacks explicit exclusions or comparisons with sibling tools to say when not to use this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.