RAG and AI search

Answering from a web page is a different use from training on it, and sites treat it differently.

A retrieval product fetches web content when a user asks a question and uses that content to write the answer.

Publishers see this use every day in their traffic, and many now address it directly. Some sites allow search indexing and refuse AI answers. Some allow answers and refuse training. The RightSignal Algorithm decides AI answer use as a purpose of its own, so the decision you receive is for the use you make.

Request accessTalk to us

Why answer use needs its own decision

Site signals increasingly separate the two uses. Content-Signal directives distinguish search, AI input and AI training. Crawler operators run separate crawlers for training, for search and for fetching on a user's request, and sites block them separately in robots.txt. Terms of use sometimes forbid automated summaries while leaving indexing open. A single yes or no for a site hides these distinctions. A decision per purpose keeps them.

Purpose

news.example.in

Our articles may not be used to train AI models or supplied to others as a dataset.
robots.txt
Allows your crawler
Reservation file
None found
Access barrier
None found
  • Search indexing

    Reading pages to build a search index that points people to the source.

    Terms and policies Read. Nothing against your purpose.

    Collect under the EU profile

  • AI training

    Using content to train or fine-tune an AI model.

    Terms and policies Refuses

    Do not collect under the EU profile

  • AI answer use

    Retrieving content when a question is asked and using it to generate the answer.

    Terms and policies Read. Nothing against your purpose.

    Hold for review under the EU profile

  • Text and data mining

    Automated analysis of content to find patterns, trends and other information.

    Terms and policies Read. Nothing against your purpose.

    Collect on conditions under the EU profile

  • Dataset building

    Assembling content into a dataset to sell or supply to others.

    Terms and policies Refuses

    Do not collect under the EU profile

Illustration. news.example.in is an invented site. The receipt for a real site shows the outcome for that site and the wording it rests on.

News content is treated with extra care

The law on AI answers built from news is unsettled in both places RightSignal covers. In India the interim ruling in ANI Media v OpenAI, which is under appeal, looked at answers produced by retrieval but did not finally decide the question. In the EU the Court of Justice is considering how the rights of press publishers apply to AI chatbots in Like Company v Google. While those questions are open, RightSignal holds AI answer use of news content for review by default, even where the site has not refused. You will see more holds on news sources than on others, and each hold shows you what a reviewer should read.

How RightSignal fits a retrieval product

Check the sources you index

Run your source list through Gate for AI answer use under your profile.

Respect the answer in your retriever

Leave out the sources that return a stop, and route the holds to a person.

Ask again on a schedule

A decision states when it stops being current. Your system asks for the site's answer again when it expires, and an unchanged answer costs nothing to confirm.

Keep the receipt with the index entry

When a publisher writes to you about an answer, you can show what its site said when you fetched the page.

Watch, which is coming soon, will recheck your sources on a schedule and tell you when one of them changes its rules.

Click the scene for the next step

Availability

In early access

AI answer use is one of the five purposes the RightSignal Algorithm decides. The ready-made policies in early access cover AI training and text and data mining. A policy for AI answer use is being added. Tell us in your access request that you need it, and we will tell you when you can run it.

What you gain

  • You learn which sources have refused answer use before a publisher tells you.
  • You hold dated evidence of each source's position, which gives a licensing conversation a factual starting point.
  • You can show customers that your product checks its sources for the use it makes of them.

Tell us which sources your answers depend on.