Skip to main content
Glama
jlucasmcrell

Apify Public Data & Leads

mcp-name: io.github.jlucasmcrell/apify-scrapers

Apify Public Data Scrapers & Extractors

Python 3.10+ Node.js 18+ Apify Verified License: MIT

A curated collection of reliable, production-ready scrapers and public-data extractors hosted on the Apify Store.

Each actor is built with strict schema validation, deterministic field mapping, self-healing DOM selectors, and pay-per-event pricing starting at $0.0002 / start.


Quick Navigation


Related MCP server: Katzilla MCP

Available Extractors & Store Listings

Tool

Store Link

Key Output Fields

Best For

Google Maps Business Leads

captainhandsome/google-maps-business-search

Name, phone, website, rating, reviews, address, coordinates, hours

B2B lead generation, local agency prospecting

Glassdoor Jobs & Salaries

captainhandsome/glassdoor-jobs-scraper

Title, company, salary estimate, rating, location, job URL, posting date

Hiring intelligence, compensation benchmarking

Airbnb Vacation Rentals

captainhandsome/airbnb-listings-search

Title, room type, nightly price, rating, reviews count, listing URL

Real estate research, market rate tracking

SEC EDGAR Corporate Filings

captainhandsome/sec-edgar-filings-search

Ticker, CIK, form (10-K, 10-Q, 8-K), filing date, primary document URL

Financial diligence, equity research, compliance

USAspending Federal Awards

captainhandsome/usaspending-federal-awards

Recipient vendor, award amount, awarding agency, description, dates

Government contracting, procurement intel

LinkedIn Public Jobs

captainhandsome/linkedin-public-jobs-search

Job title, employer, location, direct apply URL, posting age

Recruitment, tech talent monitoring

Google Play App Reviews

captainhandsome/google-play-reviews-scraper

Review text, star score, thumbs up, date, reviewer name

App store sentiment, competitor feedback

YouTube Video Search

captainhandsome/youtube-search-scraper

Title, video URL, channel, views count, duration, publish date

Content tracking, creator outreach

Twitch Live Streams

captainhandsome/twitch-live-streams-scraper

Streamer username, title, viewer count, language, category

Esports analytics, live stream monitoring

US Contractor Licenses

captainhandsome/us-contractor-license-search

Contractor name, license number, classification, status, state

Trades verification, subcontractor diligence

US Business Entity Registries

captainhandsome/us-business-entity-search

Legal entity name, filing number, jurisdiction, status

Legal due diligence, corporate registration checks


Python Quickstart

1. Install dependencies

pip install apify-client pandas python-dotenv

2. Export 50 Google Maps Leads to CSV

import os
from apify_client import ApifyClient
import pandas as pd

# Get your API token from https://console.apify.com/account/integrations
client = ApifyClient(os.getenv("APIFY_TOKEN"))

# Run the actor
run = client.actor("captainhandsome/google-maps-business-search").call(run_input={
    "search_query": "commercial electricians",
    "location": "Dallas, Texas",
    "max_items": 50,
    "include_details": True,
})

# Fetch dataset items and export to CSV
items = list(client.dataset(run["defaultDatasetId"]).iterate_items())
df = pd.DataFrame(items)
df.to_csv("dallas_electricians.csv", index=False)
print(f"Exported {len(df)} leads to dallas_electricians.csv")

See examples/google_maps_leads_to_csv.py for the full script.


Node.js Quickstart

1. Install dependencies

npm install apify-client

2. Query SEC EDGAR Filings

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('captainhandsome/sec-edgar-filings-search').call({
  companies: ['AAPL', 'NVDA', 'MSFT'],
  forms: ['10-K'],
  max_items: 15,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach(filing => {
  console.log(`[${filing.ticker}] ${filing.form} (${filing.filing_date}): ${filing.primary_document_url}`);
});

See examples/sec_filings.js for the full script.


No-Code & Automation Workflows

If you automate via n8n, Make, Zapier, or Google Sheets, ready-to-import blueprints are included in workflows/:


Pre-Built Example Tasks (Zero Code)

If you prefer runnable web UI tasks without writing any code, each actor includes pre-configured tasks published on Apify Store:

Google Maps Leads

Glassdoor Jobs

Airbnb Rentals

YouTube & Google Play


Free Sample Datasets

Looking for clean data to benchmark, analyze, or train models? Verified sample bundles with metadata schemas are available in datasets/ and hosted publicly on Hugging Face Datasets:

  1. Phoenix HVAC Contractor Leads: datasets/phoenix_hvac_leads/ | Hugging Face Hub (20 verified HVAC contractor profiles with ratings, addresses, and phone numbers).

  2. California Licensed Contractors: datasets/california_solar_contractors/ | Hugging Face Hub (Active C-46 and B licensed solar installers with state verification numbers).

  3. Austin Software Engineer Postings: datasets/austin_software_jobs/ | Hugging Face Hub (Normalized job listings with estimated posting dates and salary ranges).


AI Agent & MCP Integration

All actors in this repository conform to OpenAPI and JSON Schema standards, making them directly callable by AI agents via the Model Context Protocol (MCP):

Option A: Hosted Apify MCP Server (Claude Desktop / Cursor)

Add this to your claude_desktop_config.json or Cursor MCP settings:

{
  \"mcpServers\": {
    \"apify\": {
      \"command\": \"npx\",
      \"args\": [\"-y\", \"@apify/mcp-server\"],
      \"env\": {
        \"APIFY_TOKEN\": \"YOUR_APIFY_API_TOKEN\"
      }
    }
  }
}

Option B: Local Lightweight Python MCP Server

For local agent workflows without Node.js dependencies, a direct Python MCP server is included:

export APIFY_TOKEN=\"your_token_here\"
python mcp_server.py

Inspect tools and capabilities via mcp.json.

Agent Prompts That Work Out-of-the-Box:

  • "Search Google Maps for 50 commercial roofers in Atlanta with phone numbers and websites."

  • "Retrieve Apple and Microsoft Form 10-K filings from SEC EDGAR for the last 2 years."

  • "Find the 30 newest reviews for Duolingo on Google Play and analyze negative feedback."


In-Depth Engineering Guides

Technical case studies and problem-solution writeups are located in articles/:


Repository Structure

apify-scrapers/
 README.md                                # Documentation and quickstart
 LICENSE                                  # MIT License
 requirements.txt                         # Python client dependencies
 package.json                             # Node.js dependencies
 mcp.json                                 # MCP tool registry specification
 mcp_server.py                            # Native Python stdio MCP server
 articles/                                # In-depth engineering case studies
    airbnb_playwright_pagination_guide.md
    glassdoor_posting_dates_guide.md
    reddit_community_responses.md        # Reference technical answers for forums
 datasets/                                # Sample benchmark datasets
    phoenix_hvac_leads/
    california_solar_contractors/
    austin_software_jobs/
 workflows/                               # No-code automation templates
    n8n_google_maps_to_sheets.json
    n8n_sec_edgar_to_slack.json
    README.md
 examples/                                # Standalone developer scripts
     google_maps_leads_to_csv.py
     sec_edgar_filings_downloader.py
     glassdoor_jobs_tracker.py
     airbnb_market_scraper.py
     usaspending_defense_awards.py
     twitch_live_stream_monitor.py
     google_maps_leads.js
     sec_filings.js

Author & Support

Maintained by Joseph McRell.


License

This project is licensed under the MIT License - see the LICENSE file for details.

Available Tools

6 tools
sec_edgar_filingsC

Retrieve corporate SEC filings (10-K, 10-Q, 8-K) by company name or stock ticker. Backed by captainhandsome/sec-edgar-filings-search.

ParametersJSON Schema
NameRequiredDescriptionDefault
tickerYese.g. 'AAPL' or 'NVDA'
form_typeNo10-K, 10-Q, or 8-K10-K
max_resultsNo

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It names the backing source ("captainhandsome/sec-edgar-filings-search"), which is useful provenance, but says nothing about auth requirements, rate limits, read-only nature, or what the response contains — significant gaps for a tool with zero annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly written sentences front-loading the core capability, with zero filler. The provenance note is compact and earns its place by pointing at the data source.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a three-parameter tool with no annotations and no output schema, the description should say something about what a filing record looks like, whether results are paginated or ranked, and any access constraints. None of that is present, leaving the agent unable to predict the return shape before calling.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 67%, so the schema already documents ticker (with examples) and form_type (with allowed values). The description reinforces the form types and frames the lookup key as a ticker, but adds no syntax or format detail, and max_results is undocumented in both the schema and the description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ("Retrieve") and resource ("corporate SEC filings") and names the supported form types (10-K, 10-Q, 8-K), so the agent knows exactly what it returns. Sibling differentiation is trivially satisfied since the sibling tools cover unrelated domains (maps, jobs, contracts, streams, listings). It loses a point because "by company name or stock ticker" overstates the interface — the schema only accepts a ticker.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to reach for this tool versus alternatives, nor on which form_type to choose for a given intent. The default of 10-K and default max_results of 5 are silently applied with no explanation of when to override them.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

twitch_live_streamsB

Scrape live streaming channels, viewer counts, and game categories from Twitch. Backed by captainhandsome/twitch-live-streams-scraper.

ParametersJSON Schema
NameRequiredDescriptionDefault
languageNoen
game_nameNoe.g. 'Fortnite' or 'Just Chatting'
max_resultsNo

TDQS

B3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden, and it discloses almost nothing: no rate limits, no auth or credential needs, no latency/availability caveats, no pagination or result-ordering behavior, and no note on what happens when game_name matches nothing. The provenance line ('Backed by captainhandsome/twitch-live-streams-scraper') is the only added context, which is minor.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, and the functional content is front-loaded ahead of the provenance sentence. Nothing is padded, though the second sentence contributes little to correct invocation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and no annotations, the description partially covers the return shape by naming channels, viewer counts, and game categories. It is still silent on result structure, pagination, and error behavior, and two of three parameters are undocumented, leaving the definition just barely sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 33% — game_name carries an example but language and max_results are bare types with defaults. The description does not compensate: it never mentions language filtering, result caps, or the meaning of the default values, so an agent must guess whether max_results is a hard cap or a hint.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb (scrape) and a specific resource (live streaming channels, viewer counts, and game categories) tied to a named source (Twitch). That is enough for an agent to know exactly what comes back, though it offers no differentiation from siblings — which here is moot since every sibling covers an unrelated domain (maps, jobs, SEC filings, contracts, rentals).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: 'online live streaming channels ... from Twitch' makes the triggering scenario reasonably obvious, but there is no explicit when-to-use statement, no prerequisites, and no guidance on defaults such as language='en' or max_results=10. No alternative tool is named or excluded.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

usaspending_contractsC

Search US federal procurement, defense awards, and prime agency obligations. Backed by captainhandsome/usaspending-federal-awards.

ParametersJSON Schema
NameRequiredDescriptionDefault
max_resultsNo
recipient_nameYese.g. 'Lockheed Martin' or 'Palantir'

TDQS

C2.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It does not disclose any behavioral traits like data freshness, coverage, rate limits, or result format. The only hint is 'Backed by captainhandsome/usaspending-federal-awards' which gives provenance but no operational behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loaded with the core purpose. The second sentence about the backend is less useful but not overly verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a search tool with no annotations, no output schema, and incomplete parameter coverage, the description is too sparse. It should specify what the results contain, pagination, data recency, or any limitations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 50%, with only recipient_name having a description. The description does not mention any parameters, so it does not compensate for the undocumented max_results parameter. It provides no syntax or format details for recipient_name beyond what the schema already offers.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Search) and resource (US federal procurement, defense awards, prime agency obligations), making the purpose clear. However it does not distinguish itself from siblings since none of the siblings are in the same domain, so no differentiation is needed but also not provided.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides no when-to-use guidance, no alternatives, and no exclusions. It only states what it searches, not when or why to use it over other tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 6 tool updatesv0.1.0
    • First observedairbnb_listings_search
    • First observedglassdoor_jobs_search
    • First observedgoogle_maps_search
    • First observedsec_edgar_filings
    • First observedtwitch_live_streams
    • First observedusaspending_contracts

TDQS

B3.2/5.0

Scored across 6 tools

Disambiguation5/5

Each tool targets a completely distinct data source (Google Maps, Glassdoor, SEC, USASpending, Twitch, Airbnb), so there is no realistic risk of misselection. The descriptions make the domain of each tool explicit.

Naming Consistency4/5

Names consistently use snake_case with a source-oriented prefix, which is predictable. Minor inconsistency: some end in '_search' (google_maps_search, airbnb_listings_search) while others use a noun form (sec_edgar_filings, twitch_live_streams, usaspending_contracts).

Tool Count5/5

Six tools is a well-scoped set for a multi-source data aggregator, with each tool clearly earning its place as a distinct data provider. No redundancy or filler.

Completeness3/5

Each source exposes only a single search operation, with no detail-lookup, filter, or export variants, and the 'leads' theme suggests a broader source list (e.g. people/company enrichment) is absent. The surface is usable but notably thin per domain.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers