Crawlbase
Crawlbase provides APIs for crawling web pages, managing crawler queues, storing crawl results, and monitoring account usage.
Uses your own API key. Composio asks for it once and keeps it; Quilt never sees it.
Reads on its own5
Agents use these right away.
- Get Crawling UsageReturn current Crawling API usage and optionally previous-month statistics for the connected Crawlbase account.
- Get Storage CountReturn the number of pages currently held in Crawlbase Cloud Storage.
- Get Stored PageRetrieve the latest Crawlbase Cloud Storage page by exactly one request ID or original URL.
- Get Stored PagesRetrieve up to 100 Crawlbase Cloud Storage pages by request ID without deleting them.
- List Stored Page IDsReturn one page of Crawlbase Cloud Storage request IDs and a short-lived continuation cursor.
Asks someone first3
Anything that sends, creates, changes or deletes waits for a person to approve it in the session’s chat.
- Crawl URLFetch a public HTTP or HTTPS URL through Crawlbase using the connected Normal token and return its content with separate target and Crawlbase statuses.
- Delete Stored PagesIrreversibly delete explicitly selected pages from Crawlbase Cloud Storage by request ID and return each page's deletion outcome.
- Generate User AgentsGenerate one to ten realistic randomized User-Agent strings for a selected device class.