robots_txt_inspect
Fetch and parse a domain's robots.txt file to uncover hidden directories and sitemaps for security analysis.
Instructions
Fetch and parse the robots.txt file for a given domain to reveal hidden directories and sitemaps.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| domain | Yes |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||