From a page to a sealed receipt
You ask. You name a page, the crawler that will collect it, your purpose and your legal profile.
RightSignal reads robots.txt. Its crawler identifies itself as RightSignal and fetches robots.txt first and alone. If robots.txt cannot be fetched because the server fails, RightSignal treats the site as closed.
RightSignal reads the site's signals. It reads the signal files, the page you asked about with its headers and tags, the home page and the site's terms and policy pages. It keeps to the site's own rate limits and stops at any barrier.
RightSignal records what it found. Every source is recorded with the exact time it was read. A page that could not be read is recorded as unreadable, with the reason. A failed read is never treated as silence and never as permission.
The RightSignal Algorithm decides. The signed rules are applied to the evidence for your purpose and profile, and they return one of five outcomes.
RightSignal seals the receipt. The decision, the evidence it rests on, the read times and the Algorithm version are sealed together. You receive the decision and the receipt number.
The signals RightSignal reads
| Kind of signal | Examples | Why it matters to you |
|---|---|---|
| Machine-readable signals | Robots.txt rules, text and data mining reservations in files, headers and tags, Content-Signal and Content-Usage directives, ai.txt, llms.txt and llms-full.txt, robots meta tags and headers, licence files. | These are the signals that regulators and courts in the EU look for first, and the ones a crawler is expected to follow. |
| Human-readable statements | Terms of use, legal notices, AI and content-use policies, copyright pages and licence pages, including Creative Commons and the Government Open Data Licence India. | This is where most sites state their conditions in full. A crawler that reads only robots.txt never sees them. |
| Access barriers | A login page, a paywall, a bot challenge, a block or a rate limit. | A barrier is the clearest way a site can withhold access. RightSignal records it and stops. |
Where AI helps, and where code decides
RightSignal uses AI models in two narrow places, and only in live mode. A policy-page assistant can look for a site's terms when they are hard to find, for example on a site whose links are drawn by scripts. An AI reader can help read the terms once they are found.
Neither one decides anything. The AI reader must quote the sentence it relies on, and RightSignal's own code checks that quotation against the page, word for word, before the reading can count. A reading that fails the check is discarded. The design lets an AI reading make a decision stricter and never more permissive. The signed rules of the RightSignal Algorithm reach every outcome, and a model never does.
www.example.in/terms-of-use
Youmaynotuseanycontentonthiswebsitetotrainartificialintelligencemodels.
Its quotation was checked against the page by code.
Every decision records whether the assistant and the reader took part. When they could not run, the decision is made without them and says so. Every model call is recorded, so a past decision can be replayed without asking a model again.
This design gives you the reach of a language model on long legal text and removes the risk that a model invents the text. What reaches your receipt is the site's own wording, checked by code.
The five outcomes
Collect
What it meansUnder the chosen profile, RightSignal read the site's signals and found nothing that refuses or conditions the proposed collection for that purpose.
GroupCollect
Collect on conditions
What it meansThe proposed collection may go ahead under the chosen profile only if, and for as long as, the stated conditions are met.
GroupCollect
Hold for review
What it meansWhat the site says is unclear, conflicting or incomplete in a way that needs a person's judgement.
GroupHold
Information needed
What it meansSomething the decision depends on could not be read or was not supplied.
GroupHold
Do not collect
What it meansRightSignal found an access barrier, a robots.txt disallow addressed to the crawler or a clear refusal, or the chosen profile otherwise stops this purpose.
GroupStop
Each purpose is decided on its own. An outcome is always named with its profile. No decision carries a score, a rating or a percentage, because a number would suggest a precision that the law does not have.
A Collect outcome is not a licence or permission from the site's owner, and it is not a statement about the law. It tells you what RightSignal read and found.
Conditions and evidence gaps
A condition is something you must do for a collection to stay within the outcome. A condition can come from the site, for example an open licence that asks you to credit the source. It can come from the profile, for example keeping copies only for as long as the mining needs them under the EU profile. The decision lists each condition in words.
An evidence gap is a check that RightSignal could not complete. A signal file may have returned an error, or a policy page may have been unreadable. Every gap is named in the decision. Where a gap touches something the decision depends on, the outcome is a hold. You always know what was not read, so you never mistake an incomplete reading for a clean one.
The RightSignal Algorithm
The RightSignal Algorithm is the signed and versioned body of rules under which RightSignal reads a site's signals and reaches a decision.
- Every version is signed. Each version has a number, such as 1.3.6. RightSignal signs each version, and the service will not run a version whose signature fails.
- Every decision names its version. A receipt verifies against the version that made it and keeps verifying after later versions are released.
- Versions change, and receipts do not. A later version may decide the same site differently. RightSignal never rewrites a receipt.
- The outer layer is public. RightSignal publishes the outcomes, the outcome groups, the purposes, the profiles, the kinds of signals read and how versions and receipts are signed.
- The full rules are disclosed under confidence. RightSignal discloses the rules of a specific version to a client's counsel, an auditor or a court under a non-disclosure agreement.
This arrangement gives you what a dispute requires. The reasoning behind any decision can be examined by the people entitled to examine it, and the version that applied on the day cannot be changed afterwards.
The principles RightSignal works to
- A hold is preferred to a wrong collect. Where what a site says is unclear, missing, unreadable or in conflict, the Algorithm is designed to return a hold or a stop. You will see holds on some sites where fuller facts might have allowed collection. That is intended.
- A refusal stays a refusal. Nothing in the hosted service lets a user turn a site's refusal into a collect.
- Outcomes are named and explained. RightSignal gives words and reasons and never a score.
- The same evidence gives the same answer. A recorded decision can be replayed under the same version and will give the same result.
- RightSignal says what it did. The one exception to robots.txt that RightSignal makes, which is reading a site's own policy pages, is disclosed and recorded every time.
What our testing shows
Measured results will be published here after the next measurement of the RightSignal Algorithm.
What RightSignal cannot yet do
RightSignal would rather tell you its limits than have you find them.
- It cannot read what a site has closed. RightSignal never passes a login, a paywall or a bot challenge.
- It reads seven languages. Policy pages are read in English, Hindi, French, German, Italian, Spanish and Dutch. A policy page in another language is held for review.
- It can miss wording. Terms shown only as an image, or loaded only after a visitor takes an action, may not be found. The AI reader can also miss a sentence. Where terms could not be read, the decision says so.
- It reads fewer wordings without the AI reader. In test mode, and in live mode when the AI reader cannot run, RightSignal's built-in reader recognises fewer ways of phrasing a refusal. The decision says when the AI reader did not take part. Use live mode for any decision you intend to keep as evidence.
- It returns many holds. A hold means a person must look, so RightSignal reduces review work and does not remove it.
- It decides about access and use only. RightSignal does not assess personal data, confidentiality or any other question a page may raise.
- It starts from ready-made policies. In early access, Gate offers three policies and a registry of well-known crawler names. Custom policies and your own crawler's name are planned.
- It cannot see the past without a record. For data collected before you used RightSignal, crawl-time evidence is limited to what an archive kept. The Trace page explains this limit.