All insights

Structure plus reasoning beats flat similarity for complex domains

Across documents, code, and skills, the same pattern holds: structured knowledge navigated by reasoning outperforms flat indexes searched by similarity

Recurring pattern across PageIndex, Claude Code agentic search, and @arscontexta skill graphs · · 12 connections

The same architectural pattern keeps winning across completely different domains. For document retrieval, PageIndex builds tree structures and has LLMs reason through them — beating chunk-and-embed RAG. For code search, Claude Code abandoned RAG and vectors for Agentic search beats RAG for live codebases with grep/glob — the agent reasons about codebase structure, then looks directly. For agent skills, Skill graphs enable progressive disclosure for complex domains replaces monolithic SKILL.md files with interconnected knowledge that agents navigate by relevance.

The common thread: flat structures searched by similarity hit a ceiling. Complex domains need hierarchy (trees, graphs, directories) and reasoning (LLMs deciding where to look) rather than embedding distance (cosine similarity, nearest neighbors). This is why Files are the universal interface between humans and agents is so powerful — plain files naturally support hierarchy through directories and linking, giving both humans and agents navigable structure. It’s also why Context is the product, not the model — the structure you wrap around data determines retrieval quality more than the model powering the search. A comparative analysis of 10+ knowledge graph architectures confirms this is universal: Knowledge systems need dual-layer storage — narrative depth and structured queries can't share a format shows that every mature system discovers narrative depth and structured queries require different storage layers, and Navigation beats search for knowledge retrieval — let each data source keep its native query interface makes the explicit case that letting each data source keep its native query interface outperforms flattening everything into embeddings.