SpiderSearch Guide - SpiderIQ by Di-Atomic

Overview

SpiderSearch provides powerful full-text search capabilities across all your indexed job results. Built on OpenSearch, it enables instant search, faceted filtering, geo-radius queries, and autocomplete suggestions.

Full-Text Search

Search across business names, descriptions, categories, and more with relevance ranking.

Geo-Radius Queries

Find businesses within a specific distance from any location.

Faceted Filtering

Filter by category, rating, job type, and more with real-time counts.

Autocomplete

Type-ahead suggestions for building responsive search UIs.

How It Works

SpiderSearch uses a dual-write pattern to keep your search index synchronized with your data:

Auto-Indexing: When any SpiderMaps or SpiderSite job completes, results are automatically indexed to OpenSearch
Per-Client Isolation: Each client has their own index (leads_{client_id}) for complete data isolation
PostgreSQL as Source of Truth: Your data is always safely stored in PostgreSQL; OpenSearch provides fast search

OpenSearch indexing happens automatically on job completion. No additional API calls needed.

Search Endpoints

SpiderSearch provides specialized endpoints for different use cases:

Endpoint	Use Case	Key Features
`/search/businesses`	Local business directories	Geo-radius, ratings, categories
`/search/leads`	B2B lead search	ICP scores, email verification
`/search/all`	Universal search	Cross-source queries
`/search/autocomplete`	Type-ahead UI	Prefix matching, fuzzy search
`/search/aggregations`	Dashboards	Statistics, trends, counts

Basic Search

Search across your indexed data with a simple query:

curl -X GET "https://spideriq.ai/api/v1/search/businesses?query=coffee shop" \
  -H "Authorization: Bearer $CLIENT_TOKEN"

Response includes relevance-ranked results with facets:

{
  "success": true,
  "total": 156,
  "hits": [
    {
      "name": "Café Central",
      "category": "Coffee Shop",
      "rating": 4.7,
      "address": "12 Grand Rue, Luxembourg",
      "score": 8.5
    }
  ],
  "facets": {
    "categories": [
      {"name": "Coffee Shop", "count": 45},
      {"name": "Café", "count": 32}
    ]
  }
}

Geo-Radius Search

Find businesses within a specific distance from a location:

curl -X GET "https://spideriq.ai/api/v1/search/businesses?query=restaurant&lat=49.6117&lon=6.1319&radius_km=5" \
  -H "Authorization: Bearer $CLIENT_TOKEN"

The default radius is 50km. Adjust radius_km based on your use case: 5km for city-level, 50km for regional, 200km for country-wide searches.

Real-World Examples

VayaPin Restaurant Finder:

# Find Italian restaurants within 10km of Luxembourg City
curl -X GET "https://spideriq.ai/api/v1/search/businesses?\
query=italian&category=restaurant&lat=49.6117&lon=6.1319&radius_km=10&min_rating=4" \
  -H "Authorization: Bearer $CLIENT_TOKEN"

Gun Guard Retailer Locator:

# Find gun shops within 25km of a ZIP code location
curl -X GET "https://spideriq.ai/api/v1/search/businesses?\
category=gun%20shop&lat=38.8951&lon=-77.0364&radius_km=25" \
  -H "Authorization: Bearer $CLIENT_TOKEN"

Faceted Filtering

Use facets to build dynamic filter UIs. Search results include facet counts that update based on your filters:

# Get businesses with facet counts
curl -X GET "https://spideriq.ai/api/v1/search/businesses" \
  -H "Authorization: Bearer $CLIENT_TOKEN"

Response facets:

{
  "facets": {
    "categories": [
      {"name": "Restaurant", "count": 234},
      {"name": "Coffee Shop", "count": 156},
      {"name": "Bar", "count": 89}
    ],
    "rating_ranges": [
      {"name": "4-5 stars", "count": 312},
      {"name": "3-4 stars", "count": 145}
    ],
    "avg_rating": 4.2
  }
}

Building a Filter UI

// Fetch initial results with facets
const results = await searchBusinesses({ query: '' });

// Render category filters with counts
results.facets.categories.map(cat => (
  <FilterButton
    key={cat.name}
    label={`${cat.name} (${cat.count})`}
    onClick={() => applyFilter('category', cat.name)}
  />
));

// When user selects a filter, re-fetch with the filter applied
async function applyFilter(field, value) {
  const filtered = await searchBusinesses({
    query: currentQuery,
    [field]: value
  });
  // Facet counts update to reflect filtered results
  updateResults(filtered);
}

Lead Search with ICP Scoring

Search SpiderSite leads with ICP (Ideal Customer Profile) filtering:

# Find high-scoring SaaS leads with verified emails
curl -X GET "https://spideriq.ai/api/v1/search/leads?\
query=software&industry=SaaS&email_verified=true&min_icp_score=70" \
  -H "Authorization: Bearer $CLIENT_TOKEN"

ICP scores are generated by SpiderSite’s AI lead generation engine based on your icp_description parameter.

Autocomplete

Implement search-as-you-type with the autocomplete endpoint:

curl -X GET "https://spideriq.ai/api/v1/search/autocomplete?prefix=caf&size=5" \
  -H "Authorization: Bearer $CLIENT_TOKEN"

Response:

{
  "success": true,
  "suggestions": [
    {"text": "Café Central", "score": 9.5},
    {"text": "Café du Commerce", "score": 8.2},
    {"text": "Cafeteria Luxembourg", "score": 7.8}
  ]
}

React Implementation

import { useState, useEffect } from 'react';
import { debounce } from 'lodash';

function SearchBox({ authToken, onSearch }) {
  const [query, setQuery] = useState('');
  const [suggestions, setSuggestions] = useState([]);
  const [showSuggestions, setShowSuggestions] = useState(false);

  const fetchSuggestions = debounce(async (prefix) => {
    if (prefix.length < 2) {
      setSuggestions([]);
      return;
    }

    const res = await fetch(
      `https://spideriq.ai/api/v1/search/autocomplete?prefix=${prefix}`,
      { headers: { Authorization: `Bearer ${authToken}` } }
    );
    const data = await res.json();
    setSuggestions(data.suggestions || []);
    setShowSuggestions(true);
  }, 300);

  return (
    <div className="search-container">
      <input
        type="text"
        value={query}
        onChange={(e) => {
          setQuery(e.target.value);
          fetchSuggestions(e.target.value);
        }}
        onKeyDown={(e) => e.key === 'Enter' && onSearch(query)}
        placeholder="Search businesses..."
      />
      {showSuggestions && suggestions.length > 0 && (
        <ul className="suggestions">
          {suggestions.map((s, i) => (
            <li
              key={i}
              onClick={() => {
                setQuery(s.text);
                setShowSuggestions(false);
                onSearch(s.text);
              }}
            >
              {s.text}
            </li>
          ))}
        </ul>
      )}
    </div>
  );
}

Dashboard Aggregations

Get statistics for building dashboards:

curl -X GET "https://spideriq.ai/api/v1/search/aggregations" \
  -H "Authorization: Bearer $CLIENT_TOKEN"

Response includes counts, averages, and trends:

{
  "success": true,
  "total_documents": 32798,
  "by_job_type": [
    {"name": "spiderMaps", "count": 28500},
    {"name": "spiderSite", "count": 4298}
  ],
  "by_category": [
    {"name": "Restaurant", "count": 8234},
    {"name": "Coffee Shop", "count": 3421}
  ],
  "avg_rating": 4.2,
  "total_reviews": 456789,
  "verified_emails_count": 12456,
  "indexed_over_time": [
    {"date": "2026-01-20", "count": 2345},
    {"date": "2026-01-21", "count": 1876}
  ]
}

Advanced Search (POST)

For complex queries with many parameters, use the POST endpoint:

curl -X POST "https://spideriq.ai/api/v1/search" \
  -H "Authorization: Bearer $CLIENT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "italian restaurant",
    "filters": {
      "category": "Restaurant",
      "min_rating": 4.0,
      "job_type": "spiderMaps"
    },
    "geo": {
      "lat": 49.6117,
      "lon": 6.1319,
      "radius_km": 10
    },
    "page": 1,
    "per_page": 20,
    "sort_by": "rating"
  }'

Best Practices

Optimize Query Performance

Use specific queries rather than broad ones
Apply filters to narrow results before sorting
Use pagination (per_page=20) rather than fetching all results
Cache autocomplete results client-side for repeated prefixes

Build Responsive UIs

Debounce autocomplete requests (300ms recommended)
Show facet counts to help users filter effectively
Provide clear feedback when no results are found
Use geo-radius for location-aware apps

Handle Edge Cases

Check success field in responses
Handle empty result sets gracefully
Implement retry logic for temporary failures
Monitor error field in aggregations response

Integration Examples

Xano Backend

// In Xano function stack
const searchResults = await external_api_request({
  method: 'GET',
  url: 'https://spideriq.ai/api/v1/search/businesses',
  params: {
    query: input.search_query,
    lat: input.user_lat,
    lon: input.user_lon,
    radius_km: 25
  },
  headers: {
    Authorization: `Bearer ${env.SPIDERIQ_TOKEN}`
  }
});

return searchResults.hits;

React Native App

import { useState, useCallback } from 'react';

export function useSpiderSearch(authToken) {
  const [loading, setLoading] = useState(false);
  const [results, setResults] = useState([]);
  const [facets, setFacets] = useState(null);

  const search = useCallback(async (params) => {
    setLoading(true);
    try {
      const queryString = new URLSearchParams(params).toString();
      const res = await fetch(
        `https://spideriq.ai/api/v1/search/businesses?${queryString}`,
        { headers: { Authorization: `Bearer ${authToken}` } }
      );
      const data = await res.json();
      setResults(data.hits || []);
      setFacets(data.facets || null);
      return data;
    } finally {
      setLoading(false);
    }
  }, [authToken]);

  return { search, results, facets, loading };
}

Search Businesses

API Reference for business search

Search Leads

API Reference for lead search

SpiderMaps Guide

Learn about SpiderMaps data collection

FuzzIQ Deduplication

Prevent duplicate records

Guides

​Overview

Full-Text Search

Geo-Radius Queries

Faceted Filtering

Autocomplete

​How It Works

​Search Endpoints

​Basic Search

​Geo-Radius Search

​Real-World Examples

​Faceted Filtering

​Building a Filter UI

​Lead Search with ICP Scoring

​Autocomplete

​React Implementation

​Dashboard Aggregations

​Advanced Search (POST)

​Best Practices

​Integration Examples

​Xano Backend

​React Native App

​Related Resources

Search Businesses

Search Leads

SpiderMaps Guide

FuzzIQ Deduplication

Overview

How It Works

Search Endpoints

Basic Search

Geo-Radius Search

Real-World Examples

Faceted Filtering

Building a Filter UI

Lead Search with ICP Scoring

Autocomplete

React Implementation

Dashboard Aggregations

Advanced Search (POST)

Best Practices

Integration Examples

Xano Backend

React Native App

Related Resources