> ## Documentation Index
> Fetch the complete documentation index at: https://docs.fetchfox.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Scrape = Crawl + Extract

> Scraping with FetchFox has two phases: crawl and extract

FetchFox scraping is two steps:

1. **Crawl** to find relevant URLs.
2. **Extract** to turn page content into structured items.

## Crawl for URLs

Use `/api/crawl` with a `pattern`:

* `*` matches any characters except `/`
* `**` matches any characters including `/`

Example:

<CodeGroup>
  ```bash curl theme={null}
  curl -X POST https://api.fetchfox.ai/api/crawl \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $FETCHFOX_API_KEY" \
  -d '{
      "pattern":"https://pokemondb.net/pokedex/*",
      "maxVisits": 50
  }'
  ```
</CodeGroup>

Typical response:

```json theme={null}
{
  "jobId": "5ooygvit1y",
  "results": {
    "hits": [
      "https://pokemondb.net/pokedex/all",
      "https://pokemondb.net/pokedex/archaludon",
      "https://pokemondb.net/pokedex/charizard",
      "https://pokemondb.net/pokedex/corviknight",
      "https://pokemondb.net/pokedex/dipplin",
      "https://pokemondb.net/pokedex/dragapult",
      "https://pokemondb.net/pokedex/dragonite",
      "https://pokemondb.net/pokedex/eevee",
      "https://pokemondb.net/pokedex/game/legends-arceus",
      "https://pokemondb.net/pokedex/game/scarlet-violet",
      "...more results..."
    ]
  },
  "metrics": { "...cost and usage metrics..." }
}
```

The URLs are returned in `results.hits`.

## Extract from URLs to get items

Use `/api/extract` with:

* `url` or `urls`
* `template`

Example:

<CodeGroup>
  ```bash curl theme={null}
  curl -X POST "https://api.fetchfox.ai/api/extract" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $FETCHFOX_API_KEY" \
  -d '{
     "urls": [
       "https://pokemondb.net/pokedex/bulbasaur",
       "https://pokemondb.net/pokedex/ivysaur",
       "https://pokemondb.net/pokedex/venusaur"
     ],
     "template": {
       "name": "Name of the pokemon",
       "number": "National pokedex number",
       "stats": "Base stats as a dictionary"
     }
  }'
  ```
</CodeGroup>

Typical response:

```json theme={null}
{
  "jobId": "j8rcgsnxq3",
  "results": {
    "items": [
      {
        "name": "Bulbasaur",
        "number": "0001",
        "stats": {
          "HP": 45,
          "Attack": 49,
          "Defense": 49,
          "Sp. Atk": 65,
          "Sp. Def": 65,
          "Speed": 45,
          "Total": 318
        },
        "_url": "https://pokemondb.net/pokedex/bulbasaur",
        "_htmlUrl": "https://ffcloud.s3.amazonaws.com/visit/html/4h0o70v9fh.html"
      },
      "...more results..."
    ]
  },
  "metrics": { "...cost and usage metrics..." }
}

```

## A single endpoint to crawl and extract

To run both phases in one request, use `/api/scrape`.

<CodeGroup>
  ```bash curl theme={null}
  curl -X POST "https://api.fetchfox.ai/api/scrape" \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer YOUR_API_KEY" \
    -d '{
       "pattern": "https://www.pokemon.com/us/pokedex/*",
       "template": {
         "name": "Name of the pokemon",
         "number": "National pokedex number"
       },
       "maxVisits": 50,
       "maxExtracts": 10
    }'
  ```
</CodeGroup>

Use `/api/scrape` when you want the convenience of one call. Use `/api/crawl` + `/api/extract` directly when you want fine-grained control over each phase.