ActionTavilyUpdated September 2026

How do I crawl a whole site with Tavily?

Short answer: You can crawl site in Tavily by hand from its own interface, but it won’t repeat itself. On TinyCommand, add the Tavily Crawl Site action to a workflow, map its 4 inputs from any upstream app, and it runs automatically every time the trigger fires. No code, and a free tier to start.

Crawl Site in Tavily — start free All 4 Tavily actions

Crawl Site in Tavily — start free

Inputs

The fields this action accepts.

Every field can be mapped from an upstream trigger, AI step, table row, or hard-coded literal.

Field	Type	Required	Description
Starting URL url	string	Required	Starting URL (required)
Max Depth max_depth	string	Optional	Max Depth
Max Breadth max_breadth	string	Optional	Max Breadth
Page Limit limit	string	Optional	Page Limit

Sample request

{
  "url": "e.g. https://example.com/path",
  "max_depth": "{{trigger.max_depth}}",
  "max_breadth": "{{trigger.max_breadth}}",
  "limit": "{{trigger.limit}}"
}

Returns

{
  "results": [
    {
      "url": "https://example.com/p",
      "raw_content": "..."
    }
  ],
  "base_url": "https://example.com",
  "response_time": 2.5
}

Use these fields in downstream nodes for routing, logging, or error handling.

Triggered by

Apps that pair well as the trigger for Crawl Site.

Any of these apps can fire this action as part of a workflow.

Google Sheets → Tavily

2 Google Sheets triggers

HubSpot → Tavily

18 HubSpot triggers

FAQ

Questions about Crawl Site.

What does the Crawl Site action do in Tavily?

Crawls a website starting from the given URL and returns clean Markdown for every reachable page. Best for RAG ingestion, knowledge-base imports, or competitive content scrapes.

What inputs does Crawl Site require?

Required: Starting URL. Every input accepts a static value or a variable from any upstream node in your workflow.

Can I use dynamic inputs from earlier workflow nodes?

Yes. Any field on this action can pull values from upstream nodes, whether that's a form response, a trigger payload, an AI output, or a lookup result.

What happens if Tavily returns an error?

The run history shows the failed step with its input and the error message Tavily returned, so you can fix the input and run the workflow again. Automatic retries are built into HTTP steps only, where you can also set your own retry count and delay.

Does Crawl Site support batch operations?

Run Crawl Site inside a loop to process each item in an array. TinyCommand does not throttle calls for you, so for large arrays add a Delay step if you need to stay within Tavily's rate limits.

More actions

Other Tavily actions.

Action

Extract URLs

Fetches a list of URLs and returns clean, structured Markdown for each, stripping nav, ads, and boilerplate. Use this in RAG pipelines or when you need to feed pages into an LLM context.

Action

Map Site

Discovers URLs on a site without downloading or parsing page content. Much faster than Crawl when you only need the URL graph (sitemaps, audits, route inventories).

Action

Web Search

Runs a web search through Tavily and returns fresh, AI-optimized results tuned for LLM agents and RAG. Returns ranked snippets with URLs so the model can cite sources.

Crawl Site in Tavily — start free