Submit SpiderLanding Job

POST/api/v1/jobs/spiderLanding/submit

Overview

SpiderLanding captures and analyzes landing pages from URLs (typically Facebook ad tracking links). It provides:

• Screenshots: Above-fold and full-page captures via Playwright
• HTML Bundles: Self-contained HTML with embedded assets via Monolith
• AI Extraction: Marketing content extraction via Claude (optional)
• Redirect Tracking: Full chain from tracking URL to final landing page

info

Perfect For:

• Competitor ad research and landing page analysis
• Marketing funnel documentation
• A/B test monitoring
• Archiving campaign landing pages

Best For

• Capturing landing pages from Facebook/Google ad tracking URLs
• Documenting competitor marketing funnels
• Archiving landing pages before they change
• Extracting marketing copy, CTAs, and testimonials
• Analyzing landing page design patterns

Request Body

payloadobjectrequired

Job configuration payload

properties

urlstringrequired

The URL to capture. Can be a tracking URL - all redirects will be followed.

Example: https://tracking.example.com/ad?id=123

ad_idstring

Optional ad identifier for correlation with your ad tracking system.

Example: fb_123456789

optionsobject

Capture configuration options.

options properties

capture_screenshotbooleandefault: true

Capture above-fold screenshot (viewport only).

capture_full_pagebooleandefault: true

Capture full-page screenshot (entire page height).

capture_html_bundlebooleandefault: true

Create self-contained HTML with embedded CSS, JS, and images.

note

Complex pages can produce large bundles (10-50MB).

extract_contentbooleandefault: true

Use Claude AI to extract marketing content (headlines, CTAs, testimonials).

Requires ANTHROPIC_API_KEY to be configured on the worker.

dismiss_popupsbooleandefault: true

Use AI to identify and dismiss cookie banners and popups.

scroll_for_lazy_loadbooleandefault: true

Scroll through the page to trigger lazy-loaded content before capture.

viewportobjectdefault: {"width": 1440, "height": 900}

Browser viewport dimensions.

viewport properties

widthintegerdefault: 1440

Viewport width in pixels.

heightintegerdefault: 900

Viewport height in pixels.

timeout_secondsintegerdefault: 60

Maximum time to wait for page load (network idle).

max_redirectsintegerdefault: 10

Maximum redirects to follow.

testbooleandefault: false

Route to test queue (spider-landing-jobs-test) for local testing.

Response

job_idstring

Unique job identifier (UUID).

typestring

Job type: spiderLanding

statusstring

Initial status: queued

created_atstring

ISO 8601 timestamp.

from_cacheboolean

Whether result was returned from deduplication cache.

messagestring

Status message (e.g., "Job submitted to TEST queue").

Result Schema

When the job completes, use GET /api/v1/jobs/{job_id}/results to retrieve:

{
  "success": true,
  "job_id": "uuid",
  "type": "spiderLanding",
  "status": "completed",
  "processing_time_seconds": 8.5,
  "worker_id": "spider-landing-main-1",
  "completed_at": "2026-01-19T12:00:05Z",
  "data": {
    "capture": {
      "initial_url": "https://tracking.example.com/ad?id=123",
      "final_url": "https://landing-page.com/offer",
      "redirect_chain": [
        "https://tracking.example.com/ad?id=123",
        "https://landing-page.com/redirect",
        "https://landing-page.com/offer"
      ],
      "screenshot_fold_url": "https://media.spideriq.ai/client-xxx/landings/job-id/fold.png",
      "screenshot_full_url": "https://media.spideriq.ai/client-xxx/landings/job-id/full.png",
      "html_bundle_url": "https://media.spideriq.ai/client-xxx/landings/job-id/page.html",
      "captured_at": "2026-01-19T12:00:05Z"
    },
    "extracted": {
      "meta": {
        "title": "Get 50% Off Today | Amazing Product",
        "description": "Limited time offer on our best-selling product",
        "og_image": "https://landing-page.com/og-image.jpg"
      },
      "content": {
        "headline": "Transform Your Business in 30 Days",
        "subheadline": "Join 10,000+ companies using our platform",
        "cta_primary": {
          "text": "Start Free Trial",
          "url": "https://landing-page.com/signup",
          "style": "button"
        },
        "value_propositions": [
          "Save 10+ hours per week",
          "Increase revenue by 30%",
          "No credit card required"
        ],
        "testimonials": [
          {
            "quote": "Best tool we've ever used",
            "author": "John Smith",
            "title": "CEO, TechCorp"
          }
        ],
        "trust_signals": [
          "10,000+ customers",
          "Featured in Forbes",
          "SOC 2 Certified"
        ]
      },
      "design": {
        "layout_type": "hero_features_testimonials_cta",
        "color_palette": ["#FF5733", "#1A1A1A", "#FFFFFF"],
        "font_families": ["Inter", "system-ui"],
        "has_video": false,
        "has_chat_widget": true,
        "form_fields": ["email", "name"]
      }
    },
    "errors": [],
    "duration_ms": 8500
  },
  "error_message": null
}

Response Fields

data.captureobject

Capture results

properties

initial_urlstring

The URL that was submitted.

final_urlstring

The final URL after all redirects.

redirect_chainarray

List of URLs in the redirect chain.

screenshot_fold_urlstring

Public URL for above-fold screenshot (PNG).

screenshot_full_urlstring

Public URL for full-page screenshot (PNG).

html_bundle_urlstring

Public URL for self-contained HTML bundle.

captured_atstring

ISO 8601 timestamp of capture.

data.extractedobject

AI-extracted marketing content (null if AI extraction disabled or failed)

properties

metaobject

Page metadata (title, description, og_image).

contentobject

Marketing content (headline, subheadline, CTA, value propositions, testimonials, trust signals).

designobject

Design analysis (layout type, colors, fonts, form fields).

data.errorsarray

List of non-fatal errors that occurred during capture.

data.duration_msinteger

Total processing time in milliseconds.

Examples

Basic Capture

curl -X POST "https://spideriq.ai/api/v1/jobs/spiderLanding/submit" \
  -H "Authorization: Bearer client_id:api_key:api_secret" \
  -H "Content-Type: application/json" \
  -d '{
    "payload": {
      "url": "https://example.com"
    }
  }'

Full Options

curl -X POST "https://spideriq.ai/api/v1/jobs/spiderLanding/submit" \
  -H "Authorization: Bearer client_id:api_key:api_secret" \
  -H "Content-Type: application/json" \
  -d '{
    "payload": {
      "url": "https://tracking.example.com/ad?id=123",
      "ad_id": "fb_123456789",
      "options": {
        "capture_screenshot": true,
        "capture_full_page": true,
        "capture_html_bundle": true,
        "extract_content": true,
        "dismiss_popups": true,
        "scroll_for_lazy_load": true,
        "viewport": {"width": 1920, "height": 1080},
        "timeout_seconds": 90,
        "max_redirects": 15
      }
    }
  }'

Screenshot Only (No AI)

curl -X POST "https://spideriq.ai/api/v1/jobs/spiderLanding/submit" \
  -H "Authorization: Bearer client_id:api_key:api_secret" \
  -H "Content-Type: application/json" \
  -d '{
    "payload": {
      "url": "https://competitor.com/landing",
      "options": {
        "capture_screenshot": true,
        "capture_full_page": true,
        "capture_html_bundle": false,
        "extract_content": false
      }
    }
  }'

Test Mode

curl -X POST "https://spideriq.ai/api/v1/jobs/spiderLanding/submit" \
  -H "Authorization: Bearer client_id:api_key:api_secret" \
  -H "Content-Type: application/json" \
  -d '{
    "payload": {
      "url": "https://example.com",
      "test": true
    }
  }'

Processing Time

Page Type	Expected Time
Simple static page	3-5 seconds
Marketing landing page	15-30 seconds
Complex SPA	30-60 seconds
Heavy page with timeout	60-90 seconds

note

Pages that never reach "network idle" state will timeout but still capture whatever has loaded.

Storage

Files are stored in your client's SpiderMedia bucket:

• Path: landings/{job_id}/
• Files: fold.png, full.png, page.html
• Public URL: https://media.spideriq.ai/{bucket_name}/landings/{job_id}/{filename}

Files are retained according to your SpiderMedia storage policy.

Error Handling

SpiderLanding uses graceful degradation - errors in one step don't fail the entire job:

Error	Behavior
Navigation timeout	Captures whatever loaded
Popup dismissal fails	Continues without dismissing
Screenshot fails	Other captures still attempted
HTML bundle fails	Falls back to simple HTML
AI extraction fails	Returns extracted: null

All errors are recorded in data.errors array.