Olostep API
Olostep API
Tool
olostep_start_crawl
Starts a new crawl. You receive a `id` to track the progress. The operation may take 1-10 mins depending upon the site and depth and pages parameters.
Pricing
Per call
$0.10
Model
flat
Pay only for what you use. No subscriptions.
Inputs
include_urls
arraytop_n
numberwebhook_url
stringmax_depth
numberinclude_external
booleanstart_url *
stringinclude_subdomain
booleanexclude_urls
arraymax_pages *
numbersearch_query
stringtimeout
numberTry It
API
MCP Config
Input Parameters
URL path patterns to include in the crawl using glob syntax. Defaults to `/**` which includes all URLs. Use patterns like `/blog/**` to crawl specific sections (e.g., only blog pages), `/products/*.html` for product pages, or multiple patterns for different sections. Supports standard glob features like * (any characters) and ** (recursive matching).
An optional number to only crawl the top N most relevant links on every page as per search query.
An optional POST request endpoint called when this crawl is completed. The body of the request will be same as the response of this [`v1/crawls/{crawl_id}`](./info#response-created) endpoint.
Maximum depth of the crawl. Useful to extract only up to n-degree of links.
The starting point of the crawl.
URL path names in glob pattern to exclude. For example: `/careers/**`. Excluded URLs will supersede included URLs.
Maximum number of pages to crawl. Recommended for most use cases like crawling an entire website.
An optional search query to find specific links and also sort the results by relevance.
End the crawl after n seconds with the pages completed until then. May take ~10s extra from provided timeout.