SSE Events

The /research and /research/upload endpoints return Server-Sent Events (SSE). Each event has a type and data field.

ConnectionCopy Anchor Link

curl -N -X POST https://api.krawl.sh/research \
  -H "Content-Type: application/json" \
  -H "X-API-Key: your-key" \
  -d '{"query": "AI agents", "mode": "deep"}'

Events arrive as:

event: session
data: {"session_id": "uuid"}

event: plan
data: {"status": {...}, "plan": [...]}

event: source
data: {"url": "...", "title": "...", "tool": "webSearch"}

event: report_chunk
data: {"text": "chunk of markdown..."}

event: result
data: {"markdown": "full report...", "sources": [...]}

event: done
data: {}

Keepalive pings are sent every 10 seconds.

Research streams include a sequential id on each event. Reconnect to GET /research/{session_id}/events with the last received Last-Event-ID to replay later buffered events. Replaying events does not restart research or repeat provider calls; durable execution recovery is separate.

Catalog at a glanceCopy Anchor Link

Every event Krawl emits, when it fires, and which renderer on the web app consumes it. The renderer column references components under apps/web/src/components/research/ — use it to map SSE → UI when wiring a new client.

Event TypesCopy Anchor Link

sessionCopy Anchor Link

Emitted first. Contains the session ID for steering and follow-up research.

{"session_id": "550e8400-e29b-41d4-a716-446655440000"}

planCopy Anchor Link

The research plan generated by the planning phase.

{
  "status": {"title": "Research plan ready, starting recursive deep research"},
  "plan": [
    {
      "title": "AI Agent Architectures",
      "todos": [
        "Research current AI agent frameworks",
        "Find recent papers on agent architectures"
      ]
    }
  ]
}

perspectivesCopy Anchor Link

STORM perspective discovery results. 3–5 perspectives for multi-angle research.

{
  "perspectives": [
    {
      "name": "AI Researcher",
      "description": "Focuses on technical architecture and capability benchmarks",
      "key_questions": ["What architectures dominate?", "How do agents handle tool use?"]
    }
  ]
}

thinkingCopy Anchor Link

Agent's internal reasoning between tool calls (flat/quick mode).

{
  "thought": "The search results cover frameworks but not benchmarks. I should search for agent evaluation methods.",
  "nextStep": "Searching for AI agent benchmarks"
}

queryCopy Anchor Link

A search query being executed.

{"query": "AI agent frameworks 2025", "tool": "webSearch"}

sourceCopy Anchor Link

A new source discovered during search.

{
  "url": "https://arxiv.org/abs/2401.00001",
  "title": "A Survey of AI Agents",
  "tool": "webSearch"
}

contentCopy Anchor Link

Content extracted from a source (flat mode).

{"url": "https://...", "content": "Extracted text...", "chars": 3000}

x_searchCopy Anchor Link

X/Twitter search results.

{
  "query": "AI agents",
  "text": "Summary of X discourse...",
  "tweets_count": 12,
  "citations": ["https://x.com/..."]
}

browse_pageCopy Anchor Link

Page content fetched via browsePage tool.

{"url": "https://...", "title": "...", "content_length": 5000}

report_chunkCopy Anchor Link

Streaming markdown chunk during synthesis. Arrives many times.

{"text": "## Introduction\n\nAI agents have evolved..."}

resultCopy Anchor Link

The final research result. Contains the complete report and all sources.

It includes a logical id, query, mode, created_at, measured duration_seconds, and persisted: false. This identity supports inline evidence-pack export, including ZDR runs. It does not imply that a saved report exists. Only enable saved-report downloads and sharing after the terminal usage.result_id acknowledges successful persistence.

{
  "markdown": "# Full Research Report\n\n...",
  "sources": [
    {"url": "...", "title": "...", "source_type": "exa", "content": "..."}
  ],
  "total_steps": 24,
  "total_learnings": 18,
  "model": "bedrock/us.anthropic.claude-opus-4-7",
  "session_id": "...",
  "learnings": [...],
  "structured": null
}

errorCopy Anchor Link

An error occurred during research.

{"message": "Research failed unexpectedly"}

doneCopy Anchor Link

Research stream is complete. No more events will be sent.

{}

node_startedCopy Anchor Link

A new node in the recursive research tree has started.

{
  "node_id": "...",
  "query": "AI agent frameworks",
  "depth": 3,
  "breadth": 4,
  "parent_id": null
}

node_completedCopy Anchor Link

A research tree node has completed.

{
  "node_id": "...",
  "learnings_count": 5,
  "sources_found": 8,
  "gaps_found": 2
}

learningCopy Anchor Link

A new learning extracted from search results.

{
  "content": "GPT-4 based agents can solve 78% of SWE-bench tasks",
  "source_url": "https://...",
  "confidence": 0.95
}

gap_detectedCopy Anchor Link

A knowledge gap identified during research.

{
  "question": "How do agents handle long-horizon planning?",
  "priority": 0.9,
  "suggested_tools": ["webSearch", "academicSearch"]
}

outline_updatedCopy Anchor Link

The dynamic report outline has been updated with new evidence.

{
  "sections": [
    {"title": "Agent Architectures", "evidence_count": 5},
    {"title": "Tool Use Patterns", "evidence_count": 3}
  ]
}

citation_verifiedCopy Anchor Link

A citation has been verified against the fetched source content.

{
  "claim": "GPT-4 achieves 78% on SWE-bench",
  "source_url": "https://...",
  "verified": true,
  "source_excerpt": "Our evaluation shows GPT-4..."
}

auditCopy Anchor Link

The full audit trail for the research session. When present, it precedes result so a disconnect immediately after the report preserves applied controls in the saved audit.

{
  "entries": [
    {
      "timestamp": "2025-01-15T10:30:00Z",
      "action": "search",
      "tool": "webSearch",
      "sources_found": 8
    }
  ]
}

steerCopy Anchor Link

The latest queued steering instruction was consumed at a quick agent turn, recursive query or synthesis/refinement boundary. It applies to that call and subsequent child queries/report scope; it cannot alter a provider request already in flight. Instructions superseded before the boundary are not emitted as applied events. The consumed directive also scopes structured output when output_schema is requested; the original research query remains available as context.

{"instruction": "Focus more on security aspects"}

chartCopy Anchor Link

Data visualization extracted from research findings.

{
  "chart_type": "bar",
  "title": "Market Cap Comparison",
  "data": {...}
}

structured_outputCopy Anchor Link

Structured data extracted per the provided output_schema.

{
  "fields": {
    "market_cap": {
      "value": "$400B",
      "confidence": 0.95,
      "citation": "https://...",
      "reasoning": "Multiple sources confirm..."
    }
  }
}

review_passCopy Anchor Link

Multi-pass synthesis review results.

{
  "pass_number": 1,
  "score": 0.72,
  "issues_count": 3
}

memory_recallCopy Anchor Link

Prior knowledge recalled from the cross-session knowledge base.

{
  "memories_count": 5,
  "memories": [
    {"content": "Ethereum merged to PoS in Sep 2022", "tags": ["ethereum"]}
  ]
}

documentCopy Anchor Link

Document upload processing status.

{
  "filename": "report.pdf",
  "chars_extracted": 15000,
  "status": "processed"
}

usageCopy Anchor Link

Terminal usage and cost telemetry, emitted near the end of a run. The meter snapshot is annotated with the pricing tier the request was billed at.

A retained run adds result_id only after saving its report successfully (null on a failed save). ZDR and playground runs do not acknowledge a saved result. Incremental snapshots with partial: true do not confirm persistence.

{
  "prompt_tokens": 48213,
  "completion_tokens": 9120,
  "total_tokens": 57333,
  "cost_usd": 0.412,
  "llm_calls": 24,
  "search_calls": 11,
  "source_count": 57,
  "cost_ceiling_usd": 1.5,
  "ceiling_hit": false,
  "by_model": {
    "bedrock/us.anthropic.claude-opus-4-7": {"calls": 1, "total_tokens": 18400, "cost_usd": 0.31}
  },
  "pricing": {
    "tier": "deep",
    "billed_as": "deep_krawl",
    "cost_ceiling_usd": 2.5,
    "x402_price_usd": 5.0
  }
}

changes_since_lastCopy Anchor Link

Auto-comparison with the previous result in the same session (emitted for follow-up research).

{
  "previous_result_id": "...",
  "has_changes": true,
  "new_findings": ["New finding 1"],
  "removed_claims": [],
  "changes": [{"metric": "price", "old_value": "$50k", "new_value": "$55k"}],
  "summary": "Since your last research: price increased..."
}

Event Order (Deep Mode)Copy Anchor Link

Typical event sequence for a deep research session:

1. session — session ID
2. memory_recall — prior knowledge (if any)
3. plan — research plan
4. perspectives — STORM perspectives (if query is complex enough)
5. node_started → query → source (repeated) → learning → node_completed (per tree node)
6. gap_detected → recurse (depth-1)
7. outline_updated
8. report_chunk (many) — streaming synthesis
9. review_pass — quality review
10. chart — data visualizations (if applicable)
11. structured_output — structured data (if output_schema provided)
12. citation_verified (multiple)
13. audit — full audit trail
14. result — final result
15. changes_since_last — comparison (if follow-up session)
16. done — stream complete

JavaScript Client ExampleCopy Anchor Link

const evtSource = new EventSource('/research', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'X-API-Key': 'your-key',
  },
  body: JSON.stringify({ query: 'AI agents', mode: 'deep' }),
});

// Note: EventSource only supports GET. For POST, use fetch with ReadableStream:
const response = await fetch('https://api.krawl.sh/research', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'X-API-Key': 'your-key',
  },
  body: JSON.stringify({ query: 'AI agents', mode: 'deep' }),
});

const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = '';

while (true) {
  const { done, value } = await reader.read();
  if (done) break;

  buffer += decoder.decode(value, { stream: true });
  const lines = buffer.split('\n');
  buffer = lines.pop() || '';

  for (const line of lines) {
    if (line.startsWith('event: ')) {
      const eventType = line.slice(7);
      // Next line is data
    }
    if (line.startsWith('data: ')) {
      const data = JSON.parse(line.slice(6));
      console.log(eventType, data);
    }
  }
}

Evidence time and accessCopy Anchor Link

Research sources carry optional retrieved_at (UTC), published_date, updated_date, date provenance, access_status, source_record_id and temporal_status. Missing fields on older reports mean unknown; never assign export time as retrieval time. Access states are inspected_text, partial_excerpt, provider_summary, metadata_only, fetch_failed, or unknown. These describe available evidence, not claim support or truth. A recent fetch does not make an old article current. Summary/metadata-only results do not establish that the original text was read.

Optional source_policy.as_of sets a UTC evidence cutoff; dates after it are quarantined in quarantined_sources, outside synthesis. Without it the cutoff is research start. This is not a historical web archive; undated content remains uncertain. See evidence methodology.

Evidence coverageCopy Anchor Link

The optional evidence_coverage result field gives descriptive counts of unique exact URLs discovered, explicitly fetched, retained, referenced in outgoing model input, and cited. Stages overlap: a retained search snippet may never have had its original page fetched. It also reports explicit failed/unavailable fetches, unknown fetch/access states, retained-host concentration, missing publication dates and their known range. A model-input reference may contain only metadata or a short excerpt; a citation is not a support verdict.

These counts do not establish representative sampling or correct selection bias. Hostnames do not prove independent origins; mirrors may share a publisher. Old reports without this field have unknown coverage. No extra LLM call is made; the measured instrumentation overhead is reported as measurement_overhead_ms, not a guarantee of zero added latency. See evidence methodology.