codebase_search

Instructions

Implementation Reference

• src/index.ts:145-182 (registration)

  Registration of the codebase_search tool via server.tool() with Zod schema definitions for query, projectPath, limit, fileFilter, languageFilter, minScore, and includeLinked parameters. Delegates to handleQueryTool('codebase_search', args).

  server.tool(
    "codebase_search",
    "Semantic search across an indexed codebase. Only use after codebase_index is complete (check codebase_status first). Returns relevant code chunks matching a natural language query.",
    {
      query: z.string().describe("Natural language search query (e.g. 'authentication middleware', 'database connection setup')."),
      projectPath: z
        .string()
        .describe("Absolute path to the project directory.")
        .optional(),
      limit: z
        .number()
        .min(1)
        .max(50)
        .describe("Maximum number of results to return. Default: 10 (override globally via SEARCH_DEFAULT_LIMIT env var).")
        .optional(),
      fileFilter: z
        .string()
        .describe("Filter results to a specific file path (relative).")
        .optional(),
      languageFilter: z
        .string()
        .describe("Filter results to a specific language (e.g. 'typescript', 'python').")
        .optional(),
      minScore: z
        .number()
        .min(0)
        .max(1)
        .describe("Minimum RRF score threshold (0-1). Results below this are filtered out. Default: 0.10 (override globally via SEARCH_MIN_SCORE env var). Set to 0 to disable filtering.")
        .optional(),
      includeLinked: z
        .boolean()
        .describe("When true, also search across linked projects defined in .socraticode.json or SOCRATICODE_LINKED_PROJECTS env var. Results include a project label showing which project each result came from. Default: false.")
        .optional(),
    },
    async (args) => ({
      content: [{ type: "text", text: await handleQueryTool("codebase_search", args) }],
    }),
  );

• src/tools/query-tools.ts:48-132 (handler)

  The handleQueryTool function handles the 'codebase_search' case (line 61). It ensures Qdrant and embedding provider readiness, parses args (query, limit, fileFilter, languageFilter, includeLinked, minScore), performs hybrid semantic+BM25 search via searchChunks/searchMultipleCollections from qdrant.ts, applies minScore threshold, and formats results with warnings about incomplete indexing or missing file watcher.

  export async function handleQueryTool(
    name: string,
    args: Record<string, unknown>,
  ): Promise<string> {
    const projectPath = (args.projectPath as string) || process.cwd();
    const resolvedPath = path.resolve(projectPath);
    const projectId = projectIdFromPath(resolvedPath);
    const collection = collectionName(projectId);
  
    // Auto-start watcher on any query/status interaction (fire-and-forget)
    ensureWatcherStarted(resolvedPath);
  
    switch (name) {
      case "codebase_search": {
        await ensureQdrantReady();
        // Only ensure Ollama infrastructure when using the Ollama embedding provider.
        // For OpenAI/Google providers, just ensure the provider is initialized.
        if (getEmbeddingConfig().embeddingProvider === "ollama") {
          await ensureOllamaReady();
        } else {
          await getEmbeddingProvider();
        }
  
        const query = args.query as string;
        const limit = (args.limit as number) || SEARCH_DEFAULT_LIMIT;
        const fileFilter = args.fileFilter as string | undefined;
        const languageFilter = args.languageFilter as string | undefined;
        const includeLinked = args.includeLinked as boolean | undefined;
  
        let allResults: SearchResult[];
        if (includeLinked) {
          const collections = resolveLinkedCollections(resolvedPath);
          allResults = await searchMultipleCollections(collections, query, limit, fileFilter, languageFilter);
        } else {
          allResults = await searchChunks(collection, query, limit, fileFilter, languageFilter);
        }
  
        // Apply minimum score threshold
        const minScore = (args.minScore as number) ?? SEARCH_MIN_SCORE;
        const results = minScore > 0
          ? allResults.filter((r) => r.score >= minScore)
          : allResults;
        const filteredCount = allResults.length - results.length;
  
        if (results.length === 0) {
          if (filteredCount > 0) {
            return `No results above score threshold ${minScore.toFixed(2)} for "${query}" in project ${resolvedPath}.\n${filteredCount} result${filteredCount === 1 ? " was" : "s were"} below the threshold. Try a broader query or lower the minScore parameter.`;
          }
          return `No results found for "${query}" in project ${resolvedPath}.\nMake sure the project has been indexed first using codebase_index.`;
        }
  
        const lines = [`Search results for "${query}" (${results.length} matches):\n`];
  
        if (isIndexingInProgress(resolvedPath)) {
          const progress = getIndexingProgress(resolvedPath);
          if (progress?.type === "full-index") {
            const pct = progress.filesTotal > 0
              ? `${Math.round((progress.filesProcessed / progress.filesTotal) * 100)}%`
              : "unknown";
            lines.push(`⚠ INCOMPLETE INDEX: A full index is currently in progress (${pct} done).`);
            lines.push("  These results are from the portion indexed so far and may be significantly incomplete.");
            lines.push("  Call codebase_status to check progress. Wait for indexing to complete for full results.\n");
          } else {
            lines.push("⚠ NOTE: An incremental index update is in progress. Results may be slightly stale.\n");
          }
        }
  
        if (!(await isWatchedByAnyProcess(resolvedPath))) {
          lines.push("\u26a0 WARNING: File watcher is not yet active for this project. Results may be stale.");
          lines.push("  The watcher is being started automatically. Run codebase_update to force an immediate catch-up.\n");
        }
  
        for (const r of results) {
          const projectTag = r.project ? ` [${r.project}]` : "";
          lines.push(`--- ${r.relativePath} (lines ${r.startLine}-${r.endLine}) [${r.language}]${projectTag} score: ${r.score.toFixed(4)} ---`);
          lines.push(r.content);
          lines.push("");
        }
  
        if (filteredCount > 0) {
          lines.push(`(${filteredCount} additional result${filteredCount === 1 ? "" : "s"} below score threshold ${minScore.toFixed(2)} omitted)`);
        }
  
        return lines.join("\n");
      }

• src/services/qdrant.ts:343-407 (helper)

  The searchChunks function (line 343) generates a query embedding and calls searchChunksWithVector for hybrid dense+BM25 search using Qdrant's RRF fusion. searchChunksWithVector (line 356) builds the query prefetch with dense vector and BM25 text, applies optional file/language filters, and maps results to SearchResult objects.

  export async function searchChunks(
    collectionName: string,
    query: string,
    limit: number = 10,
    fileFilter?: string,
    languageFilter?: string,
  ): Promise<SearchResult[]> {
    const queryVector = await generateQueryEmbedding(query);
    return searchChunksWithVector(collectionName, query, queryVector, limit, fileFilter, languageFilter);
  }
  
  /** Internal: hybrid search using a pre-computed dense embedding vector.
   * Avoids recomputing the same embedding when querying multiple collections. */
  async function searchChunksWithVector(
    collectionName: string,
    query: string,
    queryVector: number[],
    limit: number,
    fileFilter?: string,
    languageFilter?: string,
  ): Promise<SearchResult[]> {
    const qdrant = getClient();
  
    const filter: { must: Array<{ key: string; match: { value: string } }> } = { must: [] };
    if (fileFilter) {
      filter.must.push({ key: "relativePath", match: { value: fileFilter } });
    }
    if (languageFilter) {
      filter.must.push({ key: "language", match: { value: languageFilter } });
    }
  
    // Fetch more candidates per sub-query so RRF has enough to re-rank
    const prefetchLimit = Math.max(limit * 3, 30);
    const activeFilter = filter.must.length > 0 ? filter : undefined;
  
    const queryPayload = {
      prefetch: [
        { query: queryVector, using: "dense", limit: prefetchLimit, filter: activeFilter },
        {
          query: { text: query, model: "qdrant/bm25" },
          using: "bm25",
          limit: prefetchLimit,
          filter: activeFilter,
        },
      ],
      query: { fusion: "rrf" },
      limit,
      with_payload: true,
      filter: activeFilter,
    };
    const results = await withRetry(
      () => qdrant.query(collectionName, queryPayload),
      "Qdrant hybrid search",
    );
  
    return results.points.map((r) => ({
      filePath: r.payload?.filePath as string,
      relativePath: r.payload?.relativePath as string,
      content: r.payload?.content as string,
      startLine: r.payload?.startLine as number,
      endLine: r.payload?.endLine as number,
      language: r.payload?.language as string,
      score: r.score,
    }));
  }

• src/services/qdrant.ts:458-496 (helper)

  The searchMultipleCollections function supports the includeLinked option — queries multiple Qdrant collections in parallel with a shared dense embedding, then merges results using client-side Reciprocal Rank Fusion (mergeMultiCollectionResults) with deduplication by label::relativePath.

  export async function searchMultipleCollections(
    collections: Array<{ name: string; label: string }>,
    query: string,
    limit: number = 10,
    fileFilter?: string,
    languageFilter?: string,
  ): Promise<SearchResult[]> {
    if (collections.length === 0) return [];
    if (collections.length === 1) {
      const results = await searchChunks(collections[0].name, query, limit, fileFilter, languageFilter);
      return results.map((r) => ({ ...r, project: collections[0].label }));
    }
  
    // Compute the dense embedding once for all collections
    const queryVector = await generateQueryEmbedding(query);
  
    // Query all collections in parallel, requesting extra candidates for RRF re-ranking
    const perCollectionLimit = Math.max(limit * 2, 20);
    const collectionResults: Array<{ label: string; results: SearchResult[] }> = [];
  
    const allResults = await Promise.all(
      collections.map(async ({ name, label }) => {
        try {
          const results = await searchChunksWithVector(name, query, queryVector, perCollectionLimit, fileFilter, languageFilter);
          return { label, results };
        } catch (err) {
          logger.warn("searchMultipleCollections: collection query failed, skipping", {
            collection: name,
            error: err instanceof Error ? err.message : String(err),
          });
          return { label, results: [] as SearchResult[] };
        }
      }),
    );
  
    collectionResults.push(...allResults);
  
    return mergeMultiCollectionResults(collectionResults, limit);
  }

• src/constants.ts:79-90 (schema)

  Default configuration constants used by codebase_search: SEARCH_DEFAULT_LIMIT (default 10, env-configurable) and SEARCH_MIN_SCORE (default 0.10, env-configurable) which control result count and minimum relevance threshold.

  /** Default number of search results returned by codebase_search.
   *  Override via SEARCH_DEFAULT_LIMIT env var (1-50). */
  export const SEARCH_DEFAULT_LIMIT = Math.max(1, Math.min(50,
    parseInt(process.env.SEARCH_DEFAULT_LIMIT || "10", 10) || 10,
  ));
  
  /** Default minimum RRF score threshold.
   *  Results below this score are filtered out. 0 disables filtering.
   *  Override via SEARCH_MIN_SCORE env var (0-1). */
  export const SEARCH_MIN_SCORE = Math.max(0, Math.min(1,
    parseFloat(process.env.SEARCH_MIN_SCORE || "0.10") || 0,
  ));