Understanding and influencing decisions

Epistemic disclosure

A description of a Saxford Advisor's training data. The data was collected from 2020-2025, sampled from 18-75 years old, resident in Saxford, and funded by Saxford Council.

Description

Provide insights on the influences, and validity, of the information and data that cause an AI system to reach a particular decision.

This requires breaking down and presenting the information and data that an AI system uses to create its output.

In some contexts the epistemic disclosure will be more important, for example in public sector services that use AI systems. To achieve epistemic disclosure, the technical architecture will need to be designed in such a way that AI systems are intentionally composed as smaller machines, rather than one singular version of the truth. Examples of this kind of emerging architecture are called compositional chaining.

For example, when using a council advisory service, the user should be able to see more information about what data was used to train it, how it was collected and who funded its development.

IF thinks this pattern will become more important in the future, as technology and the content we consume become less trustworthy, or useful, with the increase of misinformation and AI generated content.

Advantages

  • Helps people understand how automated decisions were made.
  • Allows people to inspect and interrogate suggestions, decisions and the workings of an AI system.
  • Increased knowledge of the AI system supply chain will help the AI system provide manage internal risks and meet compliance requirements.

Limitations

  • Needs to be designed with user needs in mind, so that people do not get overwhelmed.
  • Very hard to achieve when using a singular Large Language Model.

Examples

  • YouTube →

    The health verification programme creates a badge under the name of a clinician or health professional identifying them as a genuine, licensed healthcare worker.

  • Google Flights’ Carbon emissions breakdown →

    Google Flights provides a breakdown of how it calculates its emissions data, at the point of use.

Related patterns