Hubble MCP Server

Instructions

웹 페이지 크롤링
args:
    url_list: List[str], 크롤링할 웹 페이지 리스트
returns:
    dict[Any] | None: 크롤링 결과

Implementation Reference

• data_api.py:402-422 (handler)

  The main handler function for the 'crawl_web_page' tool, decorated with @mcp.tool() for registration. It asynchronously posts the list of URLs to the Hubble API's /web_crawl endpoint and returns the response.

  @mcp.tool()
  @async_retry(exceptions=(Exception), tries=2, delay=0.3)
  async def crawl_web_page(
          url_list: List[str]) -> dict[Any] | None:
      '''
      웹 페이지 크롤링
      args:
          url_list: List[str], 크롤링할 웹 페이지 리스트
      returns:
          dict[Any] | None: 크롤링 결과
      '''
      async with httpx.AsyncClient() as client:
          headers = {"X-API-Key": HUBBLE_API_KEY}
          response = await client.post(
              f"{HUBBLE_API_URL}/web_crawl",
              headers=headers,
              json={"urls": url_list},
              timeout=30.0)
          response.raise_for_status()
          return response.text

• data_api.py:402-402 (registration)

  The @mcp.tool() decorator registers the crawl_web_page function as an MCP tool.

  @mcp.tool()