It reads what the site says, applies the RightSignal Algorithm under your legal profile and returns one clear outcome with the lines it relied on and a sealed receipt. You learn what to leave alone before you spend time and compute on it, and you keep a dated record for everything you take.
Know before you fetch.Know before you fetch.
01
What you tell Gate
Gate needs four things from you. You give it the address of a page. You name the crawler that will collect it, because a site may block one crawler and admit another. You name the purpose, because a site may allow search indexing and refuse AI training. You choose the legal profile, which is India, the EU or the India and EU Bridge.
The RightSignal Algorithm decides five purposes, each on its own.
Purpose
What it covers
Search indexing
Reading pages to build a search index that points people to the source.
AI training
Using content to train or fine-tune an AI model.
AI answer use
Retrieving content when a question is asked and using it to generate the answer.
Text and data mining
Automated analysis of content to find patterns, trends and other information.
Dataset building
Assembling content into a dataset to sell or supply to others.
A decision for one purpose is never a decision for another. This protects you from the common mistake of treating a site that welcomes search engines as a site that welcomes AI training.
Click the scene to try the next purpose
news.example.in
Our articles may not be used to train AI models or supplied to others as a dataset.
robots.txt
Allows your crawler
Reservation file
None found
Access barrier
None found
Search indexing
Reading pages to build a search index that points people to the source.
Terms and policies Read. Nothing against your purpose.
Collectunder the EU profile
AI training
Using content to train or fine-tune an AI model.
Terms and policies Refuses
Do not collectunder the EU profile
AI answer use
Retrieving content when a question is asked and using it to generate the answer.
Terms and policies Read. Nothing against your purpose.
Hold for reviewunder the EU profile
Text and data mining
Automated analysis of content to find patterns, trends and other information.
Terms and policies Read. Nothing against your purpose.
Collect on conditionsunder the EU profile
Dataset building
Assembling content into a dataset to sell or supply to others.
Terms and policies Refuses
Do not collectunder the EU profile
Illustration. news.example.in is an invented site. The receipt for a real site shows the outcome for that site and the wording it rests on.
In the hosted service you start from one of three ready-made policies. They cover AI training under each profile, and text and data mining under the EU profile. Policies for AI answer use, search indexing and dataset building are being added, so tell us which purpose you need when you request access.
Gate decides for a named crawler. The hosted service accepts the well-known crawler names in its registry, which include GPTBot, ClaudeBot, CCBot and Google-Extended. Registration of your own crawler's name is planned.
02
What Gate reads
Gate
Gate reads everything a site uses to say what it allows.
It reads robots.txt for the exact crawler you name, so a site that blocks one AI crawler and admits another is read correctly.
It reads text and data mining reservations in the site's reservation file, in response headers and in page tags.
It reads Content-Signal and Content-Usage directives, which state a site's preferences for search, AI answers and AI training.
It reads ai.txt, llms.txt and llms-full.txt, and it reads robots meta tags and headers on the page itself.
It reads the site's terms of use, legal notices, AI policies, copyright pages and licence pages.
It recognises open licences such as Creative Commons and the Government Open Data Licence India, together with the credit they ask for.
It records logins, paywalls, bot challenges and rate limits as barriers, and it never tries to pass them.
Gate reads policy pages in English, Hindi, French, German, Italian, Spanish and Dutch. A policy page in another language is held for a person to review. It is never treated as silence.
Gate reads a site's own terms and policy pages even where robots.txt closes them, because those pages are how a site states its rules. It records that it did so every time, and it reads nothing else against robots.txt.
03
What Gate tells you
Gate
Every answer is one of five outcomes, named with your profile.
Collect
Collect means RightSignal read the site's signals and found nothing that refuses or conditions your collection for that purpose.
Collect on conditions
Collect on conditions means collection may go ahead only while you meet the conditions listed. A condition can come from the site, such as an open licence that asks for credit. It can come from the profile, such as keeping copies only for as long as the mining needs them under the EU profile.
Hold for review
Hold for review means what the site says is unclear, conflicting or incomplete, and a person should look. Gate says what they should look at.
Information needed
Information needed means something the decision depends on could not be read or was not supplied. Gate names it.
Do not collect
Do not collect means Gate found a barrier, a robots.txt rule against your crawler or a clear refusal, or the profile otherwise stops this purpose.
Each decision also gives you the following.
It gives the reason in a sentence that a person outside the legal team can follow.
It quotes the exact lines the outcome rests on, with the address each came from and the time it was read.
It lists every evidence gap, which is a check Gate could not complete, such as a signal file that returned an error.
It names the version of the RightSignal Algorithm that decided it and the time at which the decision stops being current.
It says whether the AI reader and the policy-page assistant took part.
It carries a sealed receipt with its own receipt number.
You can set a policy to require full verification. Any evidence gap then holds the decision for review. Teams that prefer to wait for complete evidence choose this option.
Some conditions are standard for a profile. A named person in your organisation can accept them once for a policy, for up to twelve months. Each decision made while the acceptance stands names it, and the acceptance itself is sealed.
04
Where Gate fits in your pipeline
Before a crawl
Paste or upload your list of sites in the app. Gate shows the current answer for each site and lets you export the results. You remove the sites that return a stop and set aside the ones that need review. This is the cheapest moment to find a refusal, because you have not yet fetched or stored anything.
During a crawl
Your crawler calls the Gate API. For each site it can ask for the site bundle, which is one answer per site that covers the purposes in your policy and can be cached until it expires. Your pipeline then fetches, skips or flags, and stores the receipt number beside each item.
The receipt numbers in your dataset link every item to the decision that allowed it. When a customer or a publisher asks about a source, you look up the receipt and show the site's own words and the time they were read.
01/03
Click the scene to play this step again
05
Who Gate is for
Gate is for any team that collects web content by machine and expects to be asked about it later. Model builders use it to choose sources before a training run. Retrieval and AI search teams use it to check the sources behind their answers. Scraping and data services use it to give their own clients evidence with each delivery. Legal and compliance teams use the receipts to see that the organisation's policy was applied.
06
Test mode and live mode
Gate has two modes. In test mode you build and test your integration. Decisions carry test seals that are clearly marked, and the AI reader does not run. Test decisions are for integration only and are not evidence for real collection. In live mode decisions are sealed with RightSignal's service key, and the AI reader and the policy-page assistant can take part. Each decision says which mode made it.
07
What Gate does not do
Gate does not collect content for you. You run your own crawler and decide what to take. Gate does not pass a login, a paywall or a bot challenge, so it cannot tell you what a closed page says. Nothing in the hosted service lets a user turn a site's refusal into a collect. A decision speaks for one page, one crawler, one purpose, one profile and one moment, and it expires at the time it states. Sites change, a site can publish a signal wrongly and a reader can miss wording. Gate is designed to return a hold or a stop when it is unsure, so you will see holds on some sites where fuller facts might have allowed collection.