# Website Crawl (`web.crawl`)

Crawl up to 25 same-site pages from a start URL (robots.txt honoured); pay per page fetched.

- **Route:** `POST /v1/run/web.crawl`
- **Price:** $0.0125 per page ($12.50 per 1,000) · `cost_usd` on every response
- **Source:** GetAllAPIs fetcher (`web`) · failover: no
- **Measured latency:** not enough calls measured yet
- **Heavy response:** yes — use `fields`, `max_items` or `summary`

## Input

| Field | Type | Required | Default | Notes |
|---|---|---|---|---|
| `url` | string | yes |  | Start URL |
| `max_pages` | integer | no | 10 | Pages to fetch; you pay only for pages fetched |
| `include_text` | boolean | no | true |  |

## Output (`output` in the run envelope)

| Field | Meaning |
|---|---|
| `pages` | One scrape result per fetched page. |
| `count` | Pages fetched (billed). |
| `skipped_by_robots` | Links skipped because robots.txt disallows them. |

## Example

```bash
curl -s -X POST https://getallapis.com/v1/run/web.crawl \
  -H "Authorization: Bearer $GETALLAPIS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","max_pages":5}'
```

```python
from getallapis_client import Client
client = Client()  # reads GETALLAPIS_API_KEY
result = client.run('web.crawl', {'url': 'https://example.com', 'max_pages': 5})
print(result['cost_usd'], result['output'])
```

```bash
getallapis run web.crawl '{"url":"https://example.com","max_pages":5}'
```

Quote first (free, validates input): `POST /v1/quote/web.crawl` with the same body.
