The cost of publishing information has collapsed. Scientific journals, government agencies, multilateral institutions, corporations, non-government organizations, think tanks, universities, media organizations, consultants and individuals can now publish globally at negligible cost. Search engines made this information discoverable. Social media platforms accelerated distribution. Large language models have made it searchable, summarisable and conversational.

The result is not simply more information. Information produced under very different governance arrangements now appears side by side. A report from a multilateral development bank may appear alongside an anonymous website in a search result. An AI system may retrieve both as supporting material. The user receives information but often receives little context about the organizations that produced it.

For much of modern history, information quality was partly managed through institutions. Scientific findings passed through peer review. National statistics were released through statistical agencies. International agreements were published by treaty organizations. Newspapers maintained editorial processes. Credit rating agencies and, more recently, ESG rating providers developed methodologies to assess specific organizational attributes. None of these systems guaranteed accuracy. They did, however, provide identifiable processes, accountability mechanisms and reputational consequences.

Digital platforms changed how information is distributed. Search systems rank relevance, links and user behaviour. Social media systems rank engagement. Large language models identify statistical relationships across enormous collections of text. These approaches are highly effective at processing information at scale. They were not designed to assess the reliability of the institutions producing that information.

This becomes evident when AI systems move beyond retrieval and begin generating conclusions. Traditional search engines typically return sources. Generative systems increasingly return summaries, interpretations and answers. Research examining generative search systems has found that generated responses are not always fully supported by cited material. The issue is not confined to any particular model. AI systems inherit strengths and weaknesses from the information environments on which they depend.

Public debate has largely focused on misinformation, disinformation and fact checking. Fact checking addresses an important problem, but it operates at the level of individual claims. The volume of information now being produced makes comprehensive verification difficult. Policymakers, researchers, journalists, businesses and citizens routinely make decisions about which sources deserve attention long before individual claims can be investigated in detail.

A range of responses has emerged. Search providers continue refining quality signals. Academic publishing relies upon peer review. Rating agencies assess financial risk. ESG rating providers assess sustainability-related characteristics. Governments, development institutions and research organizations are increasingly deploying AI systems to improve access to information.

In 2024, Planetary.blue built an internal chatbot, EARPI.AI, to demonstrate how environmental information could be explored through a conversational interface. The platform illustrated how large collections of environmental and sustainability information could be made more accessible through AI-assisted search and dialogue. The United Nations Development Programme and other organizations have since experimented with AI-enabled access to policy papers, technical reports and development knowledge repositories. These initiatives make information easier to locate, interrogate and understand. They do not address a separate question that sits beneath them: how much confidence should users place in the organizations producing that information?

The answer is not straightforward because organizations operate under very different conditions. An international treaty organization, multilateral development bank, government agency, university, scientific publisher, publicly listed corporation, think tank, media organization, non-government organization and anonymous website may all publish on the same topic. Yet they face different standards of transparency, disclosure, review and accountability. They also face different levels of reputational risk if information they publish proves unreliable.

Reputation alone is not a measure of truth. Well-resourced organizations make mistakes. Small organizations sometimes produce exceptional work. Nevertheless, institutional reputation remains important because it reflects the consequences organizations face when information quality fails. Universities risk academic credibility. Corporations risk shareholder confidence. Governments face political scrutiny. Treaty organizations and multilateral institutions depend upon maintaining the confidence of member states. These incentives influence how information is produced, reviewed and maintained.

Financial markets confronted a related challenge decades ago. Investors could not independently investigate every borrower, transaction or security. Credit ratings emerged as a way of assessing characteristics associated with institutions and obligations rather than attempting to predict every future outcome. Information systems face a comparable problem. Users increasingly need context about the organizations behind information before they can make informed judgments about the information itself.

As AI systems become more capable, questions about source quality become more consequential. Search engines generally return lists of sources that users can inspect for themselves. Generative systems increasingly return synthesized answers. The quality of those answers remains tied to the quality of the information from which they are derived.

Search engines help users locate information. Fact checkers evaluate individual claims. AI systems help interpret and synthesize large bodies of knowledge. None of these functions directly address the reliability characteristics of the organizations producing information. As the volume of both human-generated and AI-generated content continues to expand, understanding the governance, transparency and reputational incentives of information sources may become a more important component of information governance.

About the Author

Steve Peters is Curator of the Biosphere Information Reliability Index (BIRI™), a subscription service that ranks the reliability characteristics of biosphere information sources. BIRI™ can be integrated into AI workflows through API and MCP server connections, including ChatGPT, Claude and private enterprise AI environments.

References

References are provided for reader convenience. BIRI™ rankings relate to the reliability characteristics of information sources and not to the accuracy of any specific document, publication or claim.

Google Search. How Search Works. https://www.google.com/search/howsearchworks/ - BIRI™ Source Ranking: 6

Google Search Central. Ranking Systems Guide. https://developers.google.com/search/docs/appearance/ranking-systems-guide - BIRI™ Source Ranking: 6

United Nations Development Programme. SDG AI Lab. https://www.undp.org/policy-centre/istanbul/sdg-ai-lab - BIRI™ Source Ranking: 9

United Nations Development Programme. Artificial Intelligence for Development Analytics (AIDA). https://aida.undp.org - BIRI™ Source Ranking: 9

United Nations Development Programme. Digital and Artificial Intelligence. https://www.undp.org/policy-centre/singapore/digital-and-artificial-intelligence - BIRI™ Source Ranking: 9

World Bank. World Development Report 2021: Data for Better Lives. https://www.worldbank.org/en/publication/wdr2021 - BIRI™ Source Ranking: 9

International Organization of Securities Commissions. ESG Ratings and Data Products Providers. https://www.iosco.org/library/pubdocs/pdf/IOSCOPD690.pdf - BIRI™ Source Ranking: 7

Organisation for Economic Co-operation and Development. OECD AI Principles. https://oecd.ai/en/ai-principles - BIRI™ Source Ranking: 8

United Nations Educational, Scientific and Cultural Organization. Guidance for Generative AI in Education and Research. https://unesdoc.unesco.org/ark:/48223/pf0000386693 - BIRI™ Source Ranking: 9

‍