Skip to main content

Get Job Status

GET/api/v1/jobs/{job_id}/status

Overview​

Poll this endpoint to monitor the progress of your scraping job. Jobs move through several states from queued to completed or failed.

Path Parameters​

job_idstringrequired

The unique identifier of the job (UUID format)

Example: 550e8400-e29b-41d4-a716-446655440000

Query Parameters​

formatstring

Response format for AI agent integration (v2.60.0)

Options:

  • yaml - Token-efficient YAML format (40% savings)
  • md - Human-readable Markdown format (50% savings)

Default: JSON (no format parameter)

Response​

successboolean

Whether the request was successful

job_idstring

The job's unique identifier

typestring

Job type (spiderSite or spiderMaps)

statusstring

Current job status

Possible values:

  • queued - Waiting for worker
  • processing - Currently being processed
  • completed - Successfully finished
  • failed - Processing failed
  • cancelled - Cancelled by user
created_atstring

ISO 8601 timestamp when job was created

updated_atstring

ISO 8601 timestamp of last status update

worker_idstring

ID of the worker processing the job (if assigned)

Example Request​

curl https://spideriq.ai/api/v1/jobs/550e8400-e29b-41d4-a716-446655440000/status \
  -H "Authorization: Bearer <your_token>"

Example Responses​

{
  "success": true,
  "job_id": "550e8400-e29b-41d4-a716-446655440000",
  "type": "spiderSite",
  "status": "queued",
  "created_at": "2025-10-27T10:00:00Z",
  "updated_at": "2025-10-27T10:00:00Z",
  "worker_id": null
}

Polling Best Practices​

tip

Recommended polling strategy:

  • Poll every 2-5 seconds for jobs in queued or processing status
  • Stop polling when status is completed, failed, or cancelled
  • Implement exponential backoff for rate limit compliance

Python Polling Example​

import requests
import time

def wait_for_completion(job_id, auth_token, max_wait=300):
    """Poll job status until completion or timeout"""
    url = f"https://spideriq.ai/api/v1/jobs/{job_id}/status"
    headers = {"Authorization": f"Bearer {auth_token}"}

    start_time = time.time()
    while time.time() - start_time < max_wait:
        response = requests.get(url, headers=headers)
        data = response.json()

        status = data.get("status")
        print(f"Status: {status}")

        if status in ["completed", "failed", "cancelled"]:
            return data

        time.sleep(3)  # Poll every 3 seconds

    raise TimeoutError("Job did not complete within timeout period")

Status Flow​

Next Steps​

When status is completed:

When status is failed:

  • Check error message in results endpoint
  • Review job parameters and retry if needed