Parag Agarwal, the former CEO of Twitter, has unveiled Index, a platform from his artificial intelligence startup Parallel Web Systems, aimed at addressing one of the most contentious issues in AI. According to a report by Fortune, Agarwal claims the platform is designed to fairly compensate content creators when AI agents, rather than humans, consume their work. Agarwal's vision is built on the idea that AI agents will soon use the web far more than humans, reshaping both technology and business models.
The Challenge of AI Content Consumption
Unlike human readers who click through individual pages, AI agents can pull from dozens of sources simultaneously to complete tasks. This shift has raised concerns among publishers, who fear losing control over their content and revenue streams. The Index platform promises to offer transparency by showing publishers how AI agents use their material and introduces a new compensation model based on the Shapely value, a game theory concept that estimates each source's contribution to an AI agent's output. The more unique or valuable the source, the higher the payout.
Launch Partners for Index
Index is launching with a wide range of partners, including The Atlantic, Fortune, PR Newswire, and data providers like PitchBook, Enigma, RocketReach, and ZoomInfo. Independent creators such as Alex Heath's Sources, Packy McCormick's Not Boring, and Mario Gabriele's The Generalist are also onboard.
Industry Praise
Nicholas Thompson told Fortune, "AI agents are becoming the next major interface for accessing information, but the economics of the web have not caught up with that reality. Parallel is tackling this by creating a dynamic and scalable model for recognizing and compensating publishers."
A Different Approach from Licensing Deals
Agarwal argues that fixed-fee licensing deals like the ones signed by OpenAI with the Associated Press, Axel Springer, and News Corp will fail to work in the age of AI agents. Such deals risk leaving smaller publishers and startups out of the market. "If only a few large companies have access to premium content and no one else does, how will anyone compete?" Agarwal told Fortune. Index's model instead ties compensation directly to the value of the agent's work and the source's contribution.
Context of Legal Battles
The launch of Index comes at a time when legal battles are ongoing between AI companies and content owners. The New York Times sued OpenAI and Microsoft in 2023, alleging unauthorized use of millions of articles. Dow Jones and the New York Post have also filed suits against AI startup Perplexity. Authors have challenged AI firms over book usage, with Anthropic agreeing to a major settlement last year.
About the Author: TOI Tech Desk. The TOI Tech Desk is a dedicated team of journalists committed to delivering the latest and most relevant news from the world of technology to readers of The Times of India. The coverage spans gadget launches, reviews, trends, in-depth analysis, exclusive reports, and breaking stories that impact technology and the digital universe.
