Semantic Search

AI-powered search using vector embeddings for better relevance

Semantic Search

Search tasks and docs by meaning, not just keywords. Uses native ONNX Runtime (via Go bindings) for local embedding inference — no external API calls, no sidecar process required.

In v0.20, the embedding pipeline was rewritten from a Bun sidecar to native ONNX Go bindings, making search faster and simpler to deploy.

Architecture

Tasks/Docs → Chunker → ONNX Runtime (Go) → Vector Index
Query → ONNX Runtime (Go) → Hybrid Search → Results

Quick Start

bash
# Enable during init
knowns init my-project
# → "Enable semantic search?" [y/n] → y
# → "Select model:" → gte-small (recommended)

# Or enable on existing project
knowns config set search.semantic.enabled true
knowns model download gte-small
knowns search --reindex

Model Management

ONNX models are stored at ~/.knowns/models/ (shared across projects). The Go binary loads models directly via ONNX Runtime — no Node.js or Python dependencies required.

ModelSizeSpeedBest For
gte-small~50MBFastMost projects (recommended)
all-MiniLM-L6-v2~45MBFastLarge codebases
gte-base~110MBMediumHigh accuracy
bge-small-en-v1.5~50MBFastEnglish text
bge-base-en-v1.5~110MBMediumEnglish, high quality
e5-small-v2~50MBFastGeneral use
bash
knowns model list              # List available models
knowns model download gte-small # Download model
knowns model set gte-small      # Set for project
knowns model status             # Check status
knowns model remove gte-small   # Remove model

Search Usage

bash
# Semantic search (hybrid mode)
knowns search "how to handle auth errors"

# Force keyword only
knowns search "auth error" --keyword

# Filter by type
knowns search "api design" --type doc
knowns search "login bug" --type task
knowns search "token rotation" --type memory

# Rebuild index
knowns search --reindex

# Check status
knowns search --status-check

Search Output

Results show similarity scores and match reasons:

#42 [in-progress] [high] (97%)
  Implement JWT authentication for API
  Matched by: semantic, keyword

DOC: guides/auth-patterns (92%)
  Section: ## JWT Best Practices
  Matched by: semantic

Configuration

In .knowns/config.json:

JSON
{
  "settings": {
    "semanticSearch": {
      "enabled": true,
      "model": "gte-small"
    }
  }
}

Config Options

KeyTypeDescription
enabledbooleanEnable semantic search
modelstringModel ID (e.g., gte-small)
huggingFaceIdstringCustom HuggingFace model
dimensionsnumberEmbedding dimensions

Indexing

Index auto-updates on create/update. Manual rebuild:

bash
knowns search --reindex

What Gets Indexed

  • Task title, description, acceptance criteria
  • Task implementation plan and notes
  • Document content (chunked by sections)
  • Imported docs

Memory Retrieval

Both project and global memory stores contribute to memory retrieval:

  • knowns init and knowns sync prepare semantic memory retrieval state
  • knowns search --type memory avoids letting unrelated doc/task chunks crowd out memory results
  • Runtime memory hooks can reuse semantic-backed memory candidates when available

Custom Models

Add custom HuggingFace models:

bash
# Add custom model
knowns model add Xenova/bge-large-en-v1.5 --dims 1024 --tokens 512

# Download and use
knowns model download bge-large-en-v1.5
knowns model set bge-large-en-v1.5
knowns search --reindex

MCP Integration

TypeScript
// Search with mode parameter (v0.20 consolidated)
search({
  action: "search",
  query: "authentication",
  mode: "hybrid", // "hybrid" | "semantic" | "keyword"
})

// Retrieve with citations
search({
  action: "retrieve",
  query: "authentication patterns",
})

Troubleshooting

IssueFix
Model not foundknowns model download gte-small
Index staleknowns search --reindex
Slow first searchNormal - model loads into memory
Search returns nothingCheck knowns search --status-check

CLI Commands Reference

bash
# Model management
knowns model                    # Show status
knowns model list               # List all models
knowns model download <id>      # Download model
knowns model set <id>           # Set for project
knowns model status             # Detailed status
knowns model add <hf-id>        # Add custom model
knowns model remove <id>        # Remove custom model

# Search
knowns search "<query>"         # Hybrid search
knowns search "<query>" --keyword    # Keyword only
knowns search --reindex         # Rebuild index
knowns search --status-check    # Check status