IDAP (Internet Data Access Protocol)

What is IDAP?

IDAP is SpiderIQ's unified data access protocol. Think of it like IMAP for business data:

• SpiderIQ is the mail server — it produces and stores heavy data (leads, emails, media, profiles, verifications, website crawls)
• Your app / AI agent is the email client — it reads, flags, and syncs data without copying it

Instead of learning 14 different endpoint patterns for 14 different data types, IDAP gives you one fetch pattern, one sync pattern, one flag pattern that works across everything SpiderIQ produces.

Token Efficiency

All IDAP endpoints support ?format=yaml for 40–76% token savings when consumed by AI agents. Combine with ?fields= projection to fetch only what you need.

---

Data Ownership

Owner	Data
SpiderIQ	Business records, enrichment data, email bodies, media files, scraped content, verification traces
Your app	Task state, comments, workflow decisions, collaboration context
Bridge	idap:// references — lightweight pointers your app stores that resolve to SpiderIQ resources on demand

Your app never copies heavy data. It stores an idap:// reference and fetches live from SpiderIQ via IDAP.

---

The idap:// Reference Format

Every resource in SpiderIQ can be addressed with a standard URI:

idap://{resource_type}/{resource_id}

Examples:

idap://businesses/a1b2c3d4-e5f6-7890-abcd-ef1234567890
idap://domains/987fcdeb-51a2-3b4c-d5e6-f7890abcdef1
idap://contacts/c0ntact1-d2e3-f4a5-b6c7-d8e9f0a1b2c3
idap://emails/em41l-1d2e-3f4a-5b6c-7d8e9f0a1b2c
idap://phones/ph0ne-1d2e-3f4a-5b6c-7d8e9f0a1b2c
idap://linkedin_profiles/lnkd-1d2e-3f4a-5b6c-7d8e9f0a1b2c
idap://media/screenshot-456

---

Resource Types

Type	Source Worker	Description
businesses	SpiderMaps, SpiderMapsEnrich	Google Maps business data — name, address, rating, reviews, photos, popular times
domains	SpiderSite	Website crawl results — extracted content, company info, lead scoring, markdown compendiums
contacts	SpiderSite	People extracted from websites — name, title, email, phone, LinkedIn
emails	SpiderVerify	Email addresses with SMTP verification status and traces
phones	SpiderMaps, SpiderSite	Phone numbers extracted from Maps listings or websites
company_registry	SpiderCompanyData	Official company registration data from US/UK/EU registries
linkedin_profiles	SpiderPeople	LinkedIn profiles and AI research reports
media	SpiderLanding, SpiderMapsEnrich	Screenshots, photos, HTML bundles (binary — served via dedicated media proxy endpoint)

---

Authentication

IDAP uses standard SpiderIQ authentication:

Authorization: Bearer <client_id>:<api_key>:<api_secret>

All queries are automatically scoped to your tenant. No cross-tenant access is possible — a request for another tenant's resource returns 404 (not 403, to avoid leaking existence).

---

Response Formats

All GET endpoints accept a format query parameter:

Format	Content-Type	Best For
json (default)	application/json	Standard API consumption
yaml	text/yaml	AI agents (40–76% token savings)
md	text/markdown	Human-readable output

---

Flag System

Flags are the bidirectional state layer between your app and SpiderIQ. Your app writes flags to annotate resources (e.g., "this lead is qualified"), and SpiderIQ writes processing flags (e.g., "currently enriching").

Category	Flags	Set By
Lifecycle	new, in_progress, qualified, converted, closed	Your app
Disposition	interested, not_interested, no_response, opted_out	Your app
Quality	priority, low_quality, duplicate, stale	Either
Contact	contacted, do_not_contact, bounced	Either
Processing	enriching, verifying, crawling	SpiderIQ
Rejection	rejected	Your app

Flag Behavior

• rejected resources are excluded from list/sync responses by default
• do_not_contact prevents all outreach workers from touching the resource
• duplicate triggers FuzzIQ merge review
• Flag history is append-only for full audit trail

See Write Flags for the full API.

---

Endpoints

Method	Endpoint	Description
GET	/api/v1/idap/{type}/{id}	Fetch a single resource
GET	/api/v1/idap/{type}	List / sync resources with cursor pagination
POST	/api/v1/idap/{type}/{id}/flags	Write flags on a resource
POST	/api/v1/idap/batch	Batch fetch up to 100 resources
GET	/api/v1/idap/{type}/search	Full-text search
GET	/api/v1/idap/{type}/stats	Aggregate statistics
GET	/api/v1/idap/media/{media_id}	Stream media files

---

Quick Start

1. List your businesses:

curl "https://spideriq.ai/api/v1/idap/businesses?limit=5&format=yaml" \
  -H "Authorization: Bearer $TOKEN"

2. Fetch a single business with related emails and phones:

curl "https://spideriq.ai/api/v1/idap/businesses/abc123?include=emails,phones" \
  -H "Authorization: Bearer $TOKEN"

3. Flag a lead as qualified:

curl -X POST "https://spideriq.ai/api/v1/idap/businesses/abc123/flags" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "add": ["qualified", "priority"],
    "remove": ["new"],
    "flagged_by": "agent:my-agent",
    "reason": "High rating, verified email"
  }'