What your clients are being asked
The companies that buy scraping are AI teams, data businesses and enterprises, and they face new questions. Their customers ask for data provenance. Their investors ask for it in diligence. If they serve the EU, the AI Act asks general-purpose model providers to describe the measures that third parties took when supplying data. In India, site owners cite Section 43 of the Information Technology Act in terms that a robots.txt check never reads. A client who cannot answer these questions will look for a supplier who helps them answer.
What changes when Gate is in your pipeline
| Without Gate | With Gate |
|---|---|
| You tell clients that you follow robots.txt. | You tell clients that each site's robots.txt, signal files, terms, licences and barriers were read before collection. |
| You tell clients that use of the data is their responsibility. | You give clients a receipt for each source, decided for their own purpose and profile, which they can keep and verify. |
| A site owner's complaint reaches you with no record on either side. | You can show the site owner what its own site said when you checked, and that you stopped where it said stop. |
| You compete on price and speed. | You compete on evidence as well, which is what enterprise buyers pay for. |
What you can say to your clients
Once Gate is in your pipeline, you can make statements like these and support each one.
- Every target site is checked before collection for the purpose you name, under the Indian or EU profile you choose.
- The check reads the site's terms and AI policies as well as its robots.txt.
- We do not collect from a site that returns Do not collect.
- You receive a sealed receipt for each source, and you can verify it without us.
- The checking service never passes a login, a paywall or a bot challenge.
These statements describe conduct and evidence. They do not say that a collection is permitted in law, and neither you nor RightSignal should say that. Responsibility for the use of data stays where your contract with your client places it. What changes is that both of you hold a record.
How it fits your service
- Collect fetch
- Collect on conditions fetch, and keep the conditions
- Hold for review flag for a person
- Do not collect skip
Your client names the target sites and the purpose.
Your pipeline asks Gate for each site's answer through the API.
You collect from the sites that return a collect, and you apply any conditions.
You skip the sites that return a stop and report the holds to your client with the quoted lines.
You deliver the data with the receipt numbers attached.
Gate works beside your crawler and does not replace it. Your crawler keeps its own identity and makes its own requests. Gate contains nothing that helps a crawler get past a site's controls, and RightSignal's acceptable use policy does not allow a client to disguise its crawler or rotate its identity.
What to know before you start
In early access
Gate decides for a named crawler. In early access, Gate accepts the well-known crawler names in its registry, and registration of your own crawler's name is planned. Gate offers no plugin for a particular scraping framework yet. You integrate through the API. The ready-made policies today cover AI training and, under the EU profile, text and data mining.