Building an AI-powered search with embeddings and Cloudflare Vectorize
Why edge-native vector search matters
Semantic search with embeddings has become table stakes for any application that needs users to find things by meaning rather than exact keywords. The traditional deployment for this capability is a standalone vector database (Pinecone, Weaviate, Qdrant, pgvector) sitting behind your API server — which introduces a cross-region network hop for every search query and a separate infrastructure component to secure, scale, and pay for.
Cloudflare Vectorize collapses that architecture. It runs as a binding inside a Cloudflare Worker, meaning your embedding query hits the same edge node that holds the index. The round-trip from user to vector result is measured in single-digit milliseconds from most of the world's population. No separate database to provision, no VPC peering to configure, no egress fees.
This article builds a complete semantic search system on Cloudflare's stack: Workers for the API, Vectorize for the index, R2 for document storage, and the Workers AI embedding model for turning text into vectors. The final system handles document ingestion, semantic query, hybrid re-ranking with keyword signals, and streaming result delivery.
Setting up the Cloudflare project
Start with a new Workers project using the c3 scaffolding tool and add the required bindings:
npm create cloudflare@latest semantic-search -- --type worker --lang ts
cd semantic-search
Create the Vectorize index. The dimensionality must match your embedding model — @cf/baai/bge-base-en-v1.5 produces 768-dimensional vectors:
npx wrangler vectorize create docs-index \
--dimensions 768 \
--metric cosine
Update wrangler.toml to wire up all the bindings:
name = "semantic-search"
main = "src/index.ts"
compatibility_date = "2025-01-01"
[[vectorize]]
binding = "VECTORIZE"
index_name = "docs-index"
[[r2_buckets]]
binding = "DOCS_BUCKET"
bucket_name = "semantic-search-docs"
[ai]
binding = "AI"
The TypeScript environment interface:
export interface Env {
VECTORIZE: VectorizeIndex;
DOCS_BUCKET: R2Bucket;
AI: Ai;
}
Ingesting documents into the index
Document ingestion has three steps: store the raw document in R2, split it into chunks, embed each chunk, and upsert into Vectorize. Each Vectorize vector can carry up to 10 KB of metadata, which you use to store the chunk text and document provenance so you can reconstruct search results without hitting R2 on every query.
interface DocumentChunk {
id: string;
documentId: string;
title: string;
text: string;
charStart: number;
charEnd: number;
}
function chunkText(
text: string,
chunkSize = 512,
overlap = 64
): Array<{ text: string; charStart: number; charEnd: number }> {
const chunks = [];
let start = 0;
while (start < text.length) {
const end = Math.min(start + chunkSize, text.length);
// Try to end at a sentence boundary
const boundary = text.lastIndexOf(". ", end);
const actualEnd = boundary > start + chunkSize / 2 ? boundary + 1 : end;
chunks.push({ text: text.slice(start, actualEnd), charStart: start, charEnd: actualEnd });
start = actualEnd - overlap;
}
return chunks;
}
async function ingestDocument(
env: Env,
documentId: string,
title: string,
content: string
): Promise<{ chunksIngested: number }> {
// Store raw document in R2
await env.DOCS_BUCKET.put(`documents/${documentId}.txt`, content, {
customMetadata: { title, documentId },
});
const rawChunks = chunkText(content, 512, 64);
const vectors: VectorizeVector[] = [];
for (let i = 0; i < rawChunks.length; i++) {
const chunk = rawChunks[i];
const chunkId = `${documentId}-${i}`;
// Generate embedding using Workers AI
const embeddingResult = await env.AI.run("@cf/baai/bge-base-en-v1.5", {
text: [chunk.text],
});
const embedding = embeddingResult.data[0];
vectors.push({
id: chunkId,
values: embedding,
metadata: {
documentId,
title,
text: chunk.text.slice(0, 4096), // Vectorize metadata limit
charStart: chunk.charStart,
charEnd: chunk.charEnd,
},
});
}
// Upsert in batches — Vectorize limit is 1000 vectors per batch
const batchSize = 100;
for (let i = 0; i < vectors.length; i += batchSize) {
await env.VECTORIZE.upsert(vectors.slice(i, i + batchSize));
}
return { chunksIngested: vectors.length };
}
Querying with semantic and keyword signals
Pure vector search returns results that are semantically similar to the query but may miss exact-match keywords. Hybrid search combines vector similarity scores with keyword presence to get the best of both worlds. Cloudflare Vectorize does not natively support keyword scoring, so you compute it client-side over the top-N retrieved results.
interface SearchResult {
id: string;
documentId: string;
title: string;
text: string;
vectorScore: number;
keywordScore: number;
hybridScore: number;
charStart: number;
charEnd: number;
}
function computeKeywordScore(text: string, query: string): number {
const queryTerms = query
.toLowerCase()
.split(/\s+/)
.filter((t) => t.length > 2);
const textLower = text.toLowerCase();
let matches = 0;
for (const term of queryTerms) {
if (textLower.includes(term)) matches++;
}
return queryTerms.length > 0 ? matches / queryTerms.length : 0;
}
async function search(
env: Env,
query: string,
topK = 10,
vectorWeight = 0.7
): Promise<SearchResult[]> {
// Embed the query
const embeddingResult = await env.AI.run("@cf/baai/bge-base-en-v1.5", {
text: [query],
});
const queryEmbedding = embeddingResult.data[0];
// Retrieve top-30 from Vectorize to give the re-ranking step room to work
const vectorResults = await env.VECTORIZE.query(queryEmbedding, {
topK: 30,
returnMetadata: true,
});
const keywordWeight = 1 - vectorWeight;
const scored: SearchResult[] = vectorResults.matches.map((match) => {
const meta = match.metadata as {
documentId: string;
title: string;
text: string;
charStart: number;
charEnd: number;
};
const vectorScore = match.score;
const keywordScore = computeKeywordScore(meta.text, query);
const hybridScore = vectorWeight * vectorScore + keywordWeight * keywordScore;
return {
id: match.id,
documentId: meta.documentId,
title: meta.title,
text: meta.text,
vectorScore,
keywordScore,
hybridScore,
charStart: meta.charStart,
charEnd: meta.charEnd,
};
});
return scored
.sort((a, b) => b.hybridScore - a.hybridScore)
.slice(0, topK);
}
The vectorWeight parameter gives you a runtime dial between pure semantic search (1.0) and pure keyword matching (0.0). For technical documentation where exact API names matter, 0.6 vector / 0.4 keyword tends to outperform pure semantic search. For natural language FAQs, 0.8/0.2 is a better starting point.
Building the Worker API
Wire the ingestion and search functions into a Hono router running inside the Worker:
import { Hono } from "hono";
import { cors } from "hono/cors";
const app = new Hono<{ Bindings: Env }>();
app.use("*", cors());
// POST /ingest — accepts { documentId, title, content }
app.post("/ingest", async (c) => {
const { documentId, title, content } = await c.req.json<{
documentId: string;
title: string;
content: string;
}>();
if (!documentId || !title || !content) {
return c.json({ error: "documentId, title, and content are required" }, 400);
}
if (content.length > 500_000) {
return c.json({ error: "Content exceeds 500 KB limit" }, 413);
}
const result = await ingestDocument(c.env, documentId, title, content);
return c.json({ success: true, ...result });
});
// GET /search?q=...&topK=10&vectorWeight=0.7
app.get("/search", async (c) => {
const query = c.req.query("q");
const topK = Math.min(parseInt(c.req.query("topK") ?? "10"), 50);
const vectorWeight = parseFloat(c.req.query("vectorWeight") ?? "0.7");
if (!query || query.trim().length < 2) {
return c.json({ error: "Query must be at least 2 characters" }, 400);
}
const results = await search(c.env, query, topK, vectorWeight);
return c.json({ results, count: results.length });
});
// DELETE /documents/:documentId — removes all vectors for a document
app.delete("/documents/:documentId", async (c) => {
const { documentId } = c.req.param();
// List and delete all chunk vectors for this document
// Vectorize doesn't have a filter-delete, so we track IDs ourselves
// In production, store chunk IDs in a D1 database during ingestion
await env.DOCS_BUCKET.delete(`documents/${documentId}.txt`);
return c.json({ success: true });
});
export default {
fetch: app.fetch,
};
Streaming search results for large corpora
For large document sets, you may want to stream search results back to the client rather than waiting for all N results to be scored. Workers support streaming responses via ReadableStream, which pairs well with EventSource on the frontend.
app.get("/search/stream", async (c) => {
const query = c.req.query("q");
if (!query) return c.json({ error: "Query required" }, 400);
const embeddingResult = await c.env.AI.run("@cf/baai/bge-base-en-v1.5", {
text: [query],
});
const queryEmbedding = embeddingResult.data[0];
const vectorResults = await c.env.VECTORIZE.query(queryEmbedding, {
topK: 20,
returnMetadata: true,
});
const encoder = new TextEncoder();
const stream = new ReadableStream({
async start(controller) {
for (const match of vectorResults.matches) {
const meta = match.metadata as {
documentId: string;
title: string;
text: string;
charStart: number;
charEnd: number;
};
const event = `data: ${JSON.stringify({
id: match.id,
score: match.score,
title: meta.title,
excerpt: meta.text.slice(0, 200),
})}\n\n`;
controller.enqueue(encoder.encode(event));
// Yield control so the runtime can flush
await new Promise((r) => setTimeout(r, 0));
}
controller.enqueue(encoder.encode("data: [DONE]\n\n"));
controller.close();
},
});
return new Response(stream, {
headers: {
"Content-Type": "text/event-stream",
"Cache-Control": "no-cache",
"Connection": "keep-alive",
},
});
});
Managing the index lifecycle
Vectorize indexes need maintenance over time. Documents get updated, deleted, or re-indexed with improved embeddings when you switch models. Keep a shadow D1 database table that maps documentId -> [chunkId, ...] so you can efficiently delete all vectors for a document without a full index scan:
-- In a D1 migration
CREATE TABLE document_chunks (
chunk_id TEXT PRIMARY KEY,
document_id TEXT NOT NULL,
ingested_at INTEGER NOT NULL
);
CREATE INDEX idx_document_chunks_doc_id ON document_chunks(document_id);
async function deleteDocument(
env: Env & { DB: D1Database },
documentId: string
): Promise<void> {
const { results } = await env.DB.prepare(
"SELECT chunk_id FROM document_chunks WHERE document_id = ?"
)
.bind(documentId)
.all<{ chunk_id: string }>();
const chunkIds = results.map((r) => r.chunk_id);
if (chunkIds.length > 0) {
// Delete in batches of 100
for (let i = 0; i < chunkIds.length; i += 100) {
await env.VECTORIZE.deleteByIds(chunkIds.slice(i, i + 100));
}
await env.DB.prepare(
"DELETE FROM document_chunks WHERE document_id = ?"
)
.bind(documentId)
.run();
}
await env.DOCS_BUCKET.delete(`documents/${documentId}.txt`);
}
Measuring search quality
Search quality is difficult to measure without ground truth data, but you can build a lightweight evaluation loop using a set of representative query-document pairs where you know the expected top result. Run these after every embedding model upgrade or re-indexing operation.
interface EvalPair {
query: string;
expectedDocumentId: string;
}
async function evaluateSearchQuality(
env: Env,
evalSet: EvalPair[]
): Promise<{ hitRate: number; mrr: number }> {
let hits = 0;
let reciprocalRankSum = 0;
for (const pair of evalSet) {
const results = await search(env, pair.query, 10);
const rank = results.findIndex(
(r) => r.documentId === pair.expectedDocumentId
);
if (rank !== -1) {
hits++;
reciprocalRankSum += 1 / (rank + 1);
}
}
return {
hitRate: hits / evalSet.length,
mrr: reciprocalRankSum / evalSet.length,
};
}
Hit rate (the fraction of queries where the expected document appeared in top-10) and MRR (mean reciprocal rank, which rewards finding the right document higher in the list) give you two complementary quality signals. For a typical documentation search system, a hit rate above 0.85 and MRR above 0.65 is a reasonable production threshold.
The combination of Workers AI for zero-ops embedding generation, Vectorize for globally distributed vector storage, and D1 for metadata tracking gives you a complete semantic search stack that deploys to 300+ Cloudflare edge locations with a single wrangler deploy command.