HAR Scraper

Scrape real estate listings directly from HAR using structured input data. This task supports flexible filtering by city and price, and allows sorting the results by listing date or price.

Endpoint

POST https://api.crawlbyte.ai/api/tasks

Basic Configuration (Required)

{
  "type": "har",
  "input": [
    "{\"city\":\"Dallas\",\"priceMin\":10000,\"priceMax\":50000}"
  ],
  "sortBy": "CHEAPEST_FIRST"
}

Parameters

Field	Type	Description
type	string	Always "har"
input	array	Array of JSON-formatted strings with city and optional price filters
sortBy	string	Controls result order: • NEWEST_FIRST • OLDEST_FIRST • CHEAPEST_FIRST • EXPENSIVE_FIRST
multithread	boolean	Use true for faster processing with multiple threads

Input Object Format

"{\"city\":\"Houston\",\"priceMin\":100000,\"priceMax\":200000}"

• city (required): Target city name

• priceMin (optional): Minimum price

• priceMax (optional): Maximum price

You may use only priceMin or priceMax if desired.

Advanced Configuration (Optional)

{
  "type": "har",
  "input": [
    "{\"city\":\"Dallas\",\"priceMin\":10000,\"priceMax\":50000}"
  ],
  "sortBy": "CHEAPEST_FIRST",
  "user_agent_preset": "chrome",
  "user_agent_custom": "",
  "headers": "{\"X-Test\":\"abc\"}",
  "cookie": "session=xyz",
  "proxy": "username:password@ip:port"
}

Optional Parameters

Field	Type	Description
user_agent_preset	string	Preset user-agent. Options: chrome, firefox, edge, opera, safari, ios-safari, android-chrome, custom
user_agent_custom	string	Used if user_agent_preset is custom.
headers	string	JSON-formatted string of headers.
cookie	string	key=value;
proxy	string	username:password@ip:port

Pricing

• $0.005 per successful task This is a pay-as-you-go pricing model — you're only charged when a HAR task successfully returns listings.

You can view your current credit balance and usage history in the Crawlbyte Dashboard.

Response

The response contains metadata about the task. For har type, the most important fields are status and result.

{
  "id": "bd3e89ed-815e-4395-98a3-521ede71cc4d",
  "status": "completed",
  "result": {
    // all listings here
  }
}

• result is a JSON object, that's the final scraped results — no further polling is needed.

Status Types

Status	Meaning
queued	Task was accepted and added to the processing queue.
processing	Task is currently running.
completed	Task finished and listings were collected.
failed	Task encountered an error (invalid URL, etc.).

Polling

If status is queued or processing, continue polling the task until it's completed or failed.

GET https://api.crawlbyte.ai/api/tasks/:id

• You’ll receive the same structure with an updated status.

• Only poll until you receive completed or failed.

• Recommended interval: every 3–5 seconds.

SDK Usage

You can run this task using any official Crawlbyte SDK:

• Go SDK

• TypeScript SDK

• Python SDK

Each SDK provides a simple way to:

• Create the task

• Poll for status

• Handle the final result

Refer to the SDKs section for installation, examples, and setup instructions.

Notes

• Input values must be valid — misspelled cities or malformed price fields will result in empty results or failure.

• The input must always be a JSON string, even if it contains only the city field.

• sortBy is required and must exactly match one of the supported options.

• Crawlbyte automatically handles pagination, anti-bot protections, and request sequencing.

• multithread: true can be used in advanced settings for faster bulk processing.