Towards metacognitive agents: integrating confidence in sequential decision-making

Confidence in natural cognition

A decision is a deliberative process leading to a choice.
Decision-makers need time to collect and process informative cues.
Decinion-making is often modeled as an accumulation-to-threshold process [Gold and Shadlen, 2007].
The balance between response time and accuracy (when available) is called the Speed/Accuracy Trade-off [Heitz, 2014].

For binary choices, a popular model is the Diffision Decision Model [Ratcliff and McKoon, 2008].

Illustration of the DDM model

Multi-alternative decisions are often modeled as a race between accumulators for each possible choice.

Illustration of a race model

Image credits: [Mamassian, 2016]

Uncertainty is inherent to all stages of neural computation [Fleming, 2024].
- It refers to probabilistic representations of information in the brain.
Confidence quantifies the degree of certainty associated with a decision.
- It refers to scalar values derived from those distributions [Meyniel et al., 2015].
More formally, confidence can be defined as the probability that a choice is correct given the evidence [Pouget et al., 2016].

In decisional focus models, confidence is directly indexed by the state of evidence at the time of choice.

Race model with BoE

Post-decisional focus models posit that evidence accumulation goes on after decision time to account for confidence.

2DSD model

Metacognition is the ability to monitor and regulate one’s cognitive processes [Flavell, 1979].
- Example: should I study more (or differently) for this exam?
As part of metacognitive monitoring, confidence judgments may inform the processes of cognitive control [Fleming and Lau, 2014].

Activity in the parietal cortex seems related to evidence accumulation during decision-making.
Separate and perhaps multiple brain areas are involved in confidence monitoring and reporting [Grimaldi et al., 2015]:
- Importance of the prefrontal cortex, more precisely the ventromedial prefrontal cortex (vmPFC), in the formation of confidence.
- Firing rates of many single neurons in the orbitofrontal cortex (OFC) of rats match confidence models [Kepecs et al., 2008].

Model combining:

a decision module based on an evidence accumulation model;
a metacognitive module in which confidence is used to tune the decision hyperparameters [Desender and Verguts, 2024].

Model was assessed on a classic perceptual task: Random Dot Motion discrimination.

Confident agent model

As expected, confidence is correlated with dot motion coherence, as is (oppositely) decision time.
Model is able to implement the SAT.

First model results

Any questions?

May 5, 2025 [Last modified: May 5, 2025]