- Microsoft and Cloudflare want websites to behave like conversational AI apps
- Cloudflare’s AutoRAG indexes content automatically, promising seamless updates for website data
- NLWeb introduces structured MCP endpoints for AI agents seeking reliable access
Microsoft and Cloudflare have announced a new collaboration which seeks to make websites easier for both people and automated systems to query.
The initiative merges Microsoft’s “NLWeb” standard with Cloudflare’s “AutoRAG” infrastructure, providing a model for conversational search.
Instead of keyword-based navigation, the system allows natural language interaction, presenting direct answers rather than lists of links.
From search engines to answer engines
The goal of the new launch is to make any site function like an AI app, where human visitors and AI agents can ask questions and receive structured responses.
“Together, NLWeb and AutoRAG let publishers go beyond search boxes, making conversational interfaces for websites simple to create and deploy,” said R.V. Guha, creator of NLWeb, CVP, and Technical Fellow at Microsoft. “This integration will enable every website to easily become AI-ready for both people and trusted agents.”
Traditional search relies on keywords, leaving users to sift through multiple links to reach relevant information, but advocates of this new approach argue the model no longer matches expectations shaped by tools such as ChatGPT, Copilot, and Claude.
These systems provide immediate answers, and people increasingly expect websites to do the same.
However, while that framing is compelling, it assumes that all users prefer conversational responses over browsing.
The partnership also emphasizes the role of AI agents as a new class of “visitors.”
These automated systems typically scrape pages or follow keyword queries, but NLWeb introduces a structured “MCP” endpoint that gives them controlled access to site data.
This could reduce the inefficiency of scraping while allowing website owners to define the terms of access.
Yet questions remain about whether site operators will see tangible benefits or primarily additional technical burdens.
Cloudflare’s AutoRAG component handles crawling, indexing, and embedding site content into a managed vector database.
The system promises to keep information fresh with continuous updates and provide observability through Cloudflare’s AI Gateway.
While this sounds efficient, it effectively places indexing and access pipelines in the hands of a third-party infrastructure provider.
That raises concerns about cost, dependency, and whether site owners will cede too much control over how their data is handled.
By framing websites as first-class data sources for AI tools, Microsoft and Cloudflare are positioning themselves in direct competition with traditional search engines, particularly Google.
If AI writer models and LLM systems increasingly rely on structured access rather than scraping, search traffic patterns could change significantly.
To enable conversational search on a website through Cloudflare’s AutoRAG, users log into the Cloudflare Dashboard, create a new AutoRAG instance using the NLWeb Website quick deploy option, select the desired domain, and start indexing.
Once the process is complete, the site’s content becomes searchable through natural language queries, and owners can preview or embed the conversational interface to test how it will appear to visitors.
You might also like
link