A search agent can crawl after a query arrives

When we first launched the product, we did not launch a search product first. We launched a search agent product first. Our search agent could go essentially crawl the web after a query arrived. And we’ve had products which sometimes take 10 minutes of research. That’s a lot of time to be able to crawl a lot of pages, if only you have enough of a map to know what to prioritize crawling, right? So you can make up for shortcomings. Like, index is oftentimes—you can think of it as a latency optimization. So if you give up on that dimension, if you’re competing with humans—that’s why our search agents were competing with the alternative, being outsourcing to humans to curate amazing data, right? So we said it seems like humans sitting on search engines are way easier to compete with than a search engine on day zero. So by building a product that was a search agent to do real work on top of web data, we were able to incrementally go build our index.