Skip to main content

API Reference

This page documents the HTTP API used to run scraping tasks and (optionally) configure custom proxies.

Tools

You can use tools like Bruno or Postman to explore and test the APIs.

Authentication​

All endpoints require an API token:

  • Header: Authorization: Bearer ${token}

You should have received your token during your account creation. Please contact us if you need another one.

Common Concepts​

Scraping Task Lifecycle​

A scraping task goes through these phases:

  1. PENDING → task accepted and queued
  2. RUNNING → actively being executed
  3. SUCCESS → completed successfully (billed)
  4. BLOCKED / ERROR → recoverable failure, retried automatically (3 times)
  5. BLOCKED_TOO_MANY_TIMES / ERROR_TOO_MANY_TIMES → non-recoverable failure (not billed)
  6. CANCELLED → cancelled by client or by timeout (not billed)
  7. REPLACED → rare edge cases where a task is replaced by one or more derived tasks (e.g. a page is split into smaller pages)

Synchronous vs Asynchronous Execution​

The Run endpoints support two modes:

  • waitForCompletion=true (default): keep the HTTP connection open until tasks complete and return results directly.
  • waitForCompletion=false: return immediately. You can later fetch results using Read Scraping Tasks.
Long-lived connections

When waitForCompletion=true, if the HTTP connection closes prematurely, ongoing scraping tasks are cancelled.

Recommended for production

For production integrations, prefer waitForCompletion=false + polling/batching to avoid long-lived connections.

Scraping Tasks​

Run Scraping Tasks​

Run multiple requests in a single call.

HTTP Method and URL

POST https://continuous-scraper.common.chartedapi.com/scraping-tasks/${scraper}/run?autoCancelAfterSec=1200&waitForCompletion=false

Path Parameters

  • scraper: shopee, shopee-android or lazada

Query Parameters

  • waitForCompletion (default = true): wait for completion and return results directly.
  • autoCancelAfterSec (default = 600): execution timeout (only used when waitForCompletion=true).
  • includeAllFields (default = false): if false, only the most important fields are returned.
  • includeFields (optional): comma-separated list of fields to return (overrides includeAllFields). Example: includeFields=uuid,url,status
  • httpStatusOnError (optional): HTTP status code returned if at least one task ends in an error/blocked terminal status.
    • If not set, the API always returns 200 OK.
    • Example: httpStatusOnError=500

Request Headers

  • Authorization: Bearer ${token}

Request Body Example

{
  "requests": [
    {
      "url": "https://shopee.co.id/adidas-Runfalcon-3.0-Men's-Running-Shoes---Core-Black-i.234490784.25462132870?sp_atk=67836f0a-c0c8-40d4-a0a7-bd1998943cc9&xptdk=67836f0a-c0c8-40d4-a0a7-bd1998943cc9"
    },
    {
      "url": "https://shopee.co.id/SEPATU-SNEAKERS-CASUAL-ADIDS-SPEZIAL-HANDBALL-BNIB-i.197850298.20595626884?sp_atk=ff613749-9cf1-4b40-ae8e-f76dd2279ba8&xptdk=ff613749-9cf1-4b40-ae8e-f76dd2279ba8"
    },
    {
      "url": "https://shopee.sg/api/v4/microsite/batch_get_item_card",
      "method": "POST",
      "payload": {
        "shop_item_ids": [
          {
            "item_id": 40505798777,
            "shop_id": 1521408445
          },
          {
            "item_id": 40905798740,
            "shop_id": 1521408445
          },
          {
            "item_id": 42105798857,
            "shop_id": 1521408445
          },
          {
            "item_id": 43005798827,
            "shop_id": 1521408445
          },
          {
            "item_id": 27937932620,
            "shop_id": 1521408445
          }
        ]
      }
    }
  ],
  "cacheMaxAgeInSec": 3600
}

Request body fields

  • requests: array of request objects:
    • url: the API or page URL to scrape
    • method (optional): GET or POST
    • payload (optional): request body when method=POST
  • cacheMaxAgeInSec (default = 0): if a successful result exists for the same URL and is fresh enough, it will be returned from cache (not billed).

You can also pass scraper-specific parameters (see the scrapers section), for example:

{
  "requests": [
    {
      "url": "https://shopee.co.id/api/v4/search/search_items?keyword=tshirt"
    }
  ],
  "productSearch_enrichUrlQuery_pageSize": 60,
  "productSearch_crawlNextPages": true,
  "productSearch_crawlNextPages_maxPages": 3,
  "productSearch_crawlProductDetails": true
}

Response (when waitForCompletion=true)

[
  {
    "url": "https://shopee.co.id/SEPATU-SNEAKERS-CASUAL-ADIDS-SPEZIAL-HANDBALL-BNIB-i.197850298.20595626884?sp_atk=ff613749-9cf1-4b40-ae8e-f76dd2279ba8&xptdk=ff613749-9cf1-4b40-ae8e-f76dd2279ba8",
    "status": "SUCCESS",
    "responseBody": "..."
  },
  {
    "url": "https://shopee.co.id/adidas-Runfalcon-3.0-Men's-Running-Shoes---Core-Black-i.234490784.25462132870?sp_atk=67836f0a-c0c8-40d4-a0a7-bd1998943cc9&xptdk=67836f0a-c0c8-40d4-a0a7-bd1998943cc9",
    "status": "ERROR_TOO_MANY_TIMES",
    "responseBody": null,
    "errorMessage": "... explanation ..."
  },
  {
    "url": "https://shopee.sg/api/v4/microsite/batch_get_item_card",
    "method": "POST",
    "payload": {
      "shop_item_ids": [
        {
          "item_id": 40505798777,
          "shop_id": 1521408445
        },
        {
          "item_id": 40905798740,
          "shop_id": 1521408445
        },
        {
          "item_id": 42105798857,
          "shop_id": 1521408445
        },
        {
          "item_id": 43005798827,
          "shop_id": 1521408445
        },
        {
          "item_id": 27937932620,
          "shop_id": 1521408445
        }
      ]
    },
    "status": "BLOCKED_TOO_MANY_TIMES",
    "responseBody": "... message from scraped platform, if any ..."
  }
]

CURL Example

curl --header "Content-Type: application/json" \
  --header "Authorization: Bearer ${token}" \
  --request POST \
  --data '{"requests":[{"url": "https://shopee.co.id/adidas-Runfalcon-3.0-Mens-Running-Shoes---Core-Black-i.234490784.25462132870?sp_atk=67836f0a-c0c8-40d4-a0a7-bd1998943cc9&xptdk=67836f0a-c0c8-40d4-a0a7-bd1998943cc9"}, {"url": "https://shopee.co.id/SEPATU-SNEAKERS-CASUAL-ADIDS-SPEZIAL-HANDBALL-BNIB-i.197850298.20595626884?sp_atk=ff613749-9cf1-4b40-ae8e-f76dd2279ba8&xptdk=ff613749-9cf1-4b40-ae8e-f76dd2279ba8"}]}' \
  https://continuous-scraper.common.chartedapi.com/requests/shopee/run?autoCancelAfterSec=1200&includeAllFields=false

Run a Single Scraping Task​

Convenience endpoint to run a single request.

HTTP Method and URL

POST https://continuous-scraper.common.chartedapi.com/scraping-tasks/${scraper}/run-single?autoCancelAfterSec=1200

Parameters

Same as Run Scraping Tasks.

Request Body Example

{
  "url": "https://shopee.co.id/adidas-Runfalcon-3.0-Men's-Running-Shoes---Core-Black-i.234490784.25462132870?sp_atk=67836f0a-c0c8-40d4-a0a7-bd1998943cc9&xptdk=67836f0a-c0c8-40d4-a0a7-bd1998943cc9"
}

Request body fields

Same as Run Scraping Tasks, but fields that are normally located in the requests array are directly placed at the root (e.g. url, method, payload).

Response (when waitForCompletion=true)

{
  "url": "https://shopee.co.id/SEPATU-SNEAKERS-CASUAL-ADIDS-SPEZIAL-HANDBALL-BNIB-i.197850298.20595626884?sp_atk=ff613749-9cf1-4b40-ae8e-f76dd2279ba8&xptdk=ff613749-9cf1-4b40-ae8e-f76dd2279ba8",
  "status": "SUCCESS",
  "responseBody": "..."
}

CURL Example

curl --header "Content-Type: application/json" \
  --header "Authorization: Bearer ${token}" \
  --request POST \
  --data '{"url": "https://shopee.co.id/adidas-Runfalcon-3.0-Mens-Running-Shoes---Core-Black-i.234490784.25462132870?sp_atk=67836f0a-c0c8-40d4-a0a7-bd1998943cc9&xptdk=67836f0a-c0c8-40d4-a0a7-bd1998943cc9"}' \
  https://continuous-scraper.common.chartedapi.com/requests/shopee/run-single

Read Scraping Tasks​

Fetch tasks (and results) by UUID and/or filters.

HTTP Method and URL

GET https://continuous-scraper.common.chartedapi.com/scraping-tasks/${scraper}?limit=20&offset=0&uuids=398a3b4d-241c-4297-9e81-5c4c6ecce340,785ebeaf-ae2c-4453-908b-c7dac008fbf4

Path Parameters

  • scraper: shopee, shopee-android or lazada

Query Parameters

  • limit (optional, default = 20): max number of tasks to return
  • offset (default = 0): number of tasks to skip
  • uuids (optional): comma-separated list of task UUIDs
  • urls (optional): comma-separated list of task URLs
  • minPendingAt (optional): ISO 8601 datetime (inclusive)
  • maxPendingAt (optional): ISO 8601 datetime (exclusive)
  • includeAllFields (default = false)
  • includeFields (optional): overrides includeAllFields

Request Headers

  • Authorization: Bearer ${token}

Response Body Example

[
  {
    "uuid": "398a3b4d-241c-4297-9e81-5c4c6ecce340",
    "url": "https://shopee.co.id/SEPATU-SNEAKERS-CASUAL-ADIDS-SPEZIAL-HANDBALL-BNIB-i.197850298.20595626884?sp_atk=ff613749-9cf1-4b40-ae8e-f76dd2279ba8&xptdk=ff613749-9cf1-4b40-ae8e-f76dd2279ba8",
    "status": "SUCCESS",
    "responseBody": "..."
  },
  {
    "uuid": "785ebeaf-ae2c-4453-908b-c7dac008fbf4",
    "url": "https://shopee.co.id/adidas-Runfalcon-3.0-Men's-Running-Shoes---Core-Black-i.234490784.25462132870?sp_atk=67836f0a-c0c8-40d4-a0a7-bd1998943cc9&xptdk=67836f0a-c0c8-40d4-a0a7-bd1998943cc9",
    "status": "ERROR",
    "responseBody": null,
    "errorMessage": "... explanation ..."
  }
]

CURL Example

curl --header "Authorization: Bearer ${token}" \
  "https://continuous-scraper.common.chartedapi.com/scraping-tasks/shopee?limit=20&offset=0"

Count Scraping Tasks​

Count tasks by status and country.

HTTP Method and URL

GET https://continuous-scraper.common.chartedapi.com/scraping-tasks/${scraper}/count?statuses=PENDING,RUNNING,BLOCKED,ERROR

Path Parameters

  • scraper: shopee, shopee-android or lazada

Query Parameters

  • statuses (optional, default = all): comma-separated list of statuses

Example 1:

  • statuses=PENDING,RUNNING,BLOCKED,ERROR → pending/running counts

Example 2:

  • statuses=BLOCKED_TOO_MANY_TIMES,ERROR_TOO_MANY_TIMES,REPLACED,CANCELLED,SUCCESS → completed/non-recoverable counts

Request Headers

  • Authorization: Bearer ${token}

Response Body Example

[
  { "country": "ID", "status": "BLOCKED_TOO_MANY_TIMES", "count": 4846 },
  { "country": "ID", "status": "SUCCESS", "count": 452746545 },
  { "country": "MY", "status": "SUCCESS", "count": 3054565 }
]

CURL Example

curl --header "Authorization: Bearer $token" \
  "https://continuous-scraper.common.chartedapi.com/scraping-tasks/shopee/count?statuses=PENDING,RUNNING,BLOCKED,ERROR"

Cancel Scraping Tasks​

Cancel tasks (and sub-tasks) by UUID and/or country.

HTTP Method and URL

POST https://continuous-scraper.common.chartedapi.com/scraping-tasks/${scraper}/cancel

Path Parameters

  • scraper: shopee, shopee-android or lazada

Request Headers

  • Authorization: Bearer ${token}

Request Body Example

{
  "uuids": [
    "2199a83d-3ebe-442c-aa82-afd3a9903967",
    "1ce16c7b-26f1-4851-a281-e09f79ce2de9",
    "34d3449d-b5b9-46a5-a4e0-d524d2ab3222"
  ],
  "countries": ["ID", "MY", "PH", "SG", "TH", "VN"]
}

Request body fields

  • uuids: array of task UUIDs to cancel
  • countries: array of countries to cancel
info

Sub-scraping tasks are cancelled as well.

Response Body Example

{
  "uuids": [
    "2199a83d-3ebe-442c-aa82-afd3a9903967",
    "1ce16c7b-26f1-4851-a281-e09f79ce2de9",
    "34d3449d-b5b9-46a5-a4e0-d524d2ab3222",
    "37744b6d-35fb-47c0-9c73-e9b7feb9f089",
    "29d6b918-745b-4029-bdba-5f6489c43146"
  ],
  "countries": ["ID"]
}

CURL Example

curl --header "Content-Type: application/json" \
  --header "Authorization: Bearer ${token}" \
  --request POST \
  --data '{"uuids":["2199a83d-3ebe-442c-aa82-afd3a9903967", "1ce16c7b-26f1-4851-a281-e09f79ce2de9"}' \
  https://continuous-scraper.common.chartedapi.com/requests/shopee/cancel

Proxy Configuration​

By default, Charted Sea manages proxies for you. If you want to use your own proxy pool, you can configure custom proxy URLs.

Why configure your own proxies?

  • Expand your pool of IPs for higher throughput or improved success rate
  • Use your existing proxy provider deals
  • Control proxy routing behavior

If you use your own proxies, Charted Sea will not charge for managed proxy usage.

Sticky sessions required

Configured proxies must be sticky, because web browsers can open multiple TCP connections.

Update Proxy Configuration​

Replace the entire proxy configuration with a new set of proxy URLs.

HTTP Method and URL

PUT https://continuous-scraper.common.chartedapi.com/proxy-configs

Request Headers

  • Authorization: Bearer ${token}

Request Body Example

[
  {
    "scraper": "SHOPEE",
    "proxySettings": [
      {
        "type": "RESIDENTIAL",
        "urls": [
          "http://uuu__cr.${country(case=LOWER)}:ppp@gw.dataimpulse.com:${rand(range=[10000,19999])}",
          "http://uuu-residential-country_${country(case=UPPER)}-r_10m-s_${rand(len=10,use=[lowerCaseChar,upperCaseChar,digit])}:ppp@gate.nstproxy.io:24125"
        ]
      },
      {
        "country": "TW",
        "type": "RESIDENTIAL",
        "urls": [
          "http://uuu__cr.${country(case=LOWER)}:ppp@gw.dataimpulse.com:${rand(range=[10000,19999])}",
          "http://uuu-residential-country_${country(case=UPPER)}-r_10m-s_${rand(len=10,use=[lowerCaseChar,upperCaseChar,digit])}:ppp@gate.nstproxy.io:24125",
          "http://uuu:ppp_country-${country(case=LOWER)}_session-${rand(len=8,use=[lowerCaseChar,upperCaseChar,digit])}_lifetime-15m@geo.iproyal.com:12321"
        ]
      }
    ]
  },
  {
    "scraper": "SHOPEE_ANDROID",
    "proxySettings": [
      {
        "type": "RESIDENTIAL",
        "urls": [
          "http://uuu__cr.${country(case=LOWER)}:ppp@gw.dataimpulse.com:${rand(range=[10000,19999])}",
          "http://uuu-residential-country_${country(case=UPPER)}-r_10m-s_${rand(len=10,use=[lowerCaseChar,upperCaseChar,digit])}:ppp@gate.nstproxy.io:24125"
        ]
      },
      {
        "country": "TW",
        "type": "RESIDENTIAL",
        "urls": [
          "http://uuu__cr.${country(case=LOWER)}:ppp@gw.dataimpulse.com:${rand(range=[10000,19999])}",
          "http://uuu-residential-country_${country(case=UPPER)}-r_10m-s_${rand(len=10,use=[lowerCaseChar,upperCaseChar,digit])}:ppp@gate.nstproxy.io:24125",
          "http://uuu:ppp_country-${country(case=LOWER)}_session-${rand(len=8,use=[lowerCaseChar,upperCaseChar,digit])}_lifetime-15m@geo.iproyal.com:12321"
        ]
      }
    ]
  },
  {
    "scraper": "LAZADA",
    "proxySettings": [
      {
        "type": "DATACENTER",
        "urls": [
          "http://uuu__cr.${country(case=LOWER)}:ppp@gw.dataimpulse.com:${rand(range=[10000,19999])}"
        ]
      },
      {
        "type": "RESIDENTIAL",
        "urls": [
          "http://uuu__cr.${country(case=LOWER)}:ppp@gw.dataimpulse.com:${rand(range=[10000,19999])}"
        ]
      }
    ]
  }
]

Request body fields

  • scraper: SHOPEE, SHOPEE_ANDROID or LAZADA
  • proxySettings: array of objects:
    • type: DATACENTER or RESIDENTIAL
    • urls: array of proxy URLs (placeholders supported)
    • country (optional): if set, this setting is dedicated to that country

Response

Same as the request body.

Read Proxy Configuration​

HTTP Method and URL

GET https://continuous-scraper.common.chartedapi.com/proxy-configs

Request Headers

  • Authorization: Bearer ${token}

Response Body Example

Same as the request body in Update Proxy Configuration.

CURL Example

curl --header "Authorization: Bearer ${token}" \
  "https://continuous-scraper.common.chartedapi.com/proxy-configs"

Appendix: Proxy URL Placeholders​

Proxy URL placeholders are enclosed in ${...} and expanded server-side before use.
They allow you to dynamically generate proxy URLs (for example, to create sticky sessions or country-specific credentials).

Supported functions​

${rand(...)}​

Generates a random value.

Supported arguments:

  • len (number)
    Length of the generated string (required unless range is provided).

  • use (array)
    Character sets to include. Allowed values:

    • lowerCaseChar → abcdefghijklmnopqrstuvwxyz
    • upperCaseChar → ABCDEFGHIJKLMNOPQRSTUVWXYZ
    • digit → 0123456789
    • symbol → [],$&+:;=?@#|"'><.^*()%! -

  • subset (string)
    Custom set of allowed characters (alternative to use).
    The characters $, " and \ must be escaped as \$, \" and \\.

  • range (array [min, max])
    Generates a random number within the specified range (inclusive).

Examples:

${rand(len=8,use=[lowerCaseChar,digit])}
${rand(range=[10000,19999])}
${rand(len=12,subset="abcdef012345")}
${country(case=UPPER|LOWER)}​

Expands to the target country code associated with the scraping task.

Arguments:

  • case=UPPER → SG, ID, MY
  • case=LOWER → sg, id, my

Example:

${country(case=LOWER)}

Example Proxy URLs​

{
  "urls": [
    // Generate 9999 DataImpulse URLs
    "http://uuu__cr.${country(case=LOWER)}:ppp@gw.dataimpulse.com:${rand(range=[10000,19999])}",

    // Generate many LunaProxy URLs
    "http://user-uuu-region-${country(case=LOWER)}-sessid-${country(case=LOWER)}${rand(len=16,use=[lowerCaseChar,digit])}-sesstime-30:ppp@as.e4f4xh3i.lunaproxy.net:12233",

    // Generate many IPRoyal URLs
    "http://uuu:ppp_country-${country(case=LOWER)}_session-${rand(len=8,use=[lowerCaseChar,upperCaseChar,digit])}_lifetime-30m@geo.iproyal.com:12321",

    // Generate many BrightData URLs
    "http://brd-customer-hl_uuu-zone-zzz-country-${country(case=LOWER)}-session-${rand(range=[1,70000000])}:ppp@brd.superproxy.io:22225",

    // Generate many Oxylabs URLs
    "http://customer-uuu-cc-${country(case=UPPER)}-sessid-${rand(len=10,use=[lowerCaseChar,digit])}-sesstime-30:ppp@pr.oxylabs.io:7777",

    // Generate many Massive URLs (note how the characters @, ?, & and = are URI encoded)
    "http://john.doe%40example.com%3Fcountry%3D${country(case=UPPER)}%26session%3D${rand(len=10,use=[lowerCaseChar,upperCaseChar,digit])}:ppp@network.joinmassive.com:65534",

    // Generate many Nstproxy URLs
    "http://uuu-residential-country_${country(case=UPPER)}-r_10m-s_${rand(len=10,use=[lowerCaseChar,upperCaseChar,digit])}:ppp@gate.nstproxy.io:24125",

    // Generate many Mango Proxy URLs
    "http://uuu-zone-custom-region-${country(case=LOWER)}-session-${rand(len=9,use=[lowerCaseChar,digit])}-sessTime-30:ppp@p1.mangoproxy.com:2333",

    // Generate many Netnut URLs
    "http://uuu-res-${country(case=UPPER)}-sid-${rand(len=9,use=[digit])}:ppp@gw.ntnt.io:5959",

    // Generate many Zenrows URLs
    "http://uuu:ppp_country-${country(case=LOWER)}_ttl-30m_session-${rand(len=12,use=[lowerCaseChar,upperCaseChar,digit])}@superproxy.zenrows.com:1337",

    // Generate many Soax Proxy URLs
    "http://package-123456-country-${country(case=LOWER)}-sessionid-${rand(len=10,use=[lowerCaseChar,digit])}-sessionlength-10:ppp@proxy.soax.com:5000",

    // Generate many 711Proxy Proxy URLs
    "http://uuu-zone-custom-region-${country(case=UPPER)}-session-${rand(len=8,use=[digit])}-sessTime-10-sessAuto-1:ppp@global.711proxy.com:10000"
  ]
}

Important Notes​

  • Placeholders are expanded for each scraping execution.
  • Sticky sessions must remains stable for at least 15 minutes.
  • Don't hesitate to contact us if you need help.

Next Steps​

  • Review the scraper-specific documentation (Shopee, Lazada, etc.)
  • Review the Quick Start guide for a minimal integration example
  • Monitor usage and wallet balance in the Console