Think Aloud Protocol with Selective Stash/Submit: Externalizing System 1 and System 2 as Proof of Work

Skill verification and learning analytics increasingly face a measurement crisis: generative models can produce fluent final answers that obscure whether a human (or agent) can retrieve, apply, and transform knowledge under real constraints. Multiple-choice scores and deliverable-only portfolios sample thin output slices. Classic think-aloud methods (Ericsson & Simon, 1980/1993) remain a gold standard for process evidence, but they were not designed for AI-saturated workflows, shareable session links, or machine-readable dual streams of spontaneous versus filtered thought.

We present an implementation of TAP that keeps the core scientific commitment—externalize thinking while it occurs—while adding a selective thought interface. Learners think out loud; each crystallized fragment is either stashed (kept private relative to the dialogue partner) or submitted into the Socratic exchange. The intentionality of that choice is itself signal. The resulting dual stream becomes Proof of Work data: artifacts of process that can be scored, embedded, and compared against labeled regions of knowledge space.

This paper is a methods and design document. It does not claim completed empirical region-learning results. Section 5 describes the experiment we plan to run.

Protocol analysis of verbal reports has long treated concurrent verbalization as a window onto intermediate cognitive products, with careful caveats about reactivity and incomplete reportability (Ericsson & Simon, 1980/1993). In learning science and HCI, think-alouds are used for usability, tutoring research, and assessment design. Our contribution is not to replace that tradition, but to instrument a modern selective interface that separates spontaneous crystallization from deliberate dialogue participation.

Dual-process accounts colloquially contrast fast, intuitive processing with slow, deliberative control (Kahneman, 2011). We adopt System 1 / System 2 as an operational measurement vocabulary, not as a claim of clean neurological modules. In product and research code, the mapping is explicit: System 1 ≈ spontaneous crystallized speech, including stashed/unsent thoughts; System 2 ≈ deliberate send, edit, skip, select/deselect, or resend into the dialogue. The interface makes metacognitive filtering inspectable—knowledge that is articulated but not submitted can indicate hesitation, incomplete understanding, or strategic withholding—while still preserving those fragments as PoW rather than discarding them.

This operationalization is stated upfront so academic readers can evaluate construct validity without conflating product labels with clinical dual-process assays.

A TAP session is a timed, scoped episode inside a workspace that defines a knowledge or skill target (for example, a role skill block or a learning goal). A facilitator agent elicits continuous externalization with short prompts—definitions, causal steps, examples, transfer, and repair—without scoring live or lecturing the domain answer. The primary scientific goal of the live session is to maximize genuine thought-trace signal.

The selective thought interface is central. Learners do not only stream audio into a black box. They produce transcribed thought fragments. For each fragment, they may Stash (retain as System 1 / unsent evidence) or Submit (enter System 2 dialogue participation). Related System 2 actions—edit, skip, select/deselect, resend—capture intentional revision and selection under social and evaluative pressure.

Protocol purity constraints keep the method tool-agnostic: pen-and-paper, mental calculation, or LLM assistance may be present in the environment; the scored object remains the thinking that is externalized and the Stash/Submit decisions around it. The protocol thereby resists the failure mode in which AI-polished finals are mistaken for demonstrated skill, while remaining practical under real AI use.

• Session goal: externalize cognition about a workspace-scoped target under time bounds.
• Elicitation: brief Socratic or directed prompts that thicken traces (example, causal link, repair), not multi-paragraph lectures.
• Stash: fragment is crystallized and retained as System 1 / unsent PoW without becoming the dialogue turn of record.
• Submit (and related deliberate actions): fragment or revision enters System 2 dialogue evidence.
• Both streams are first-class: later analysis may contrast what was said privately versus what was advanced publicly.

We use Proof of Work (PoW) to mean machine- and human-inspectable process artifacts that stand as a proxy for proximity to a target knowledge configuration—not a blockchain metaphor and not a single correctness bit. In TAP, PoW includes the dialogue transcript, System 1 and System 2 thought-trace files, timestamps and inter-event structure (for example dwell before send, idle before crystallize), and session-level manifests that list attached traces.

After the session, analysis may jointly consider transcript and traces. Scoring instructions treat System 1 counts (spontaneous and stashed speech) and System 2 counts (deliberate dialogue actions) as primary genuine-cognition signal: unsent knowledge can reveal gaps and metacognitive filtering; submitted reasoning can reveal what the learner is willing to stand behind. Snapshots and related workspace analyses use these artifacts as input evidence for marker scores, gap analysis, and temporal summaries—without requiring that live TAP announce scores to the learner.

Separately, agent-facing APIs mirror the same intent taxonomy (buffer, then Stash or Submit into the regular PoW stack), so human TAP and agent evaluation can share a dual-stream PoW ontology. This paper focuses on the human TAP Stash/Submit method; the shared ontology is noted for completeness of the measurement stack.

• Captured: verbalized fragments, Stash vs Submit (and edit/skip/select/resend) decisions, timestamps, dialogue context, workspace target labels.
• Retained: stashed/unsent traces as System 1 PoW; submitted turns and deliberate actions as System 2 PoW.
• Downstream: performance snapshots and related analyses consume both streams; polished finals alone are insufficient.
• Privacy posture: selective sharing is first-class—stash is not failure, it is evidence of filtering under the protocol.

We plan a multi-stage research program. The Map of Knowledge product surface already visualizes public embedding locations and regions for workspaces; the experiment below is the research program to systematically gather dual-stream TAP data, learn robust representations, and define high-dimensional regions that map to knowledge areas. Results are not claimed here.

Verbal reports are incomplete and can be reactive; the Stash/Submit interface may alter what participants choose to say. We treat dual-stream PoW as a richer proxy, not as a full readout of neural state. System 1 / System 2 labels are operational, not clinical diagnoses.

AI-assisted environments complicate attribution; protocol purity emphasizes externalized reasoning quality but cannot eliminate all confounds. Embedding spaces inherit biases of underlying models; region definitions must be audited for demographic and linguistic skew.

Human subjects research requires informed consent, clear data retention, and careful handling of stashed content (which participants may reasonably treat as more private). Production telemetry and research corpora should be separated by policy. We do not claim that proximity in embedding space is identical to physical brain configuration; it is a measurement proxy aligned with our knowledge-configuration thesis.

The Think Aloud Protocol with a Stash/Submit selective interface provides a practical, researchable method for externalizing System 1 and System 2 thinking as dual-stream Proof of Work. By retaining stashed as well as submitted cognition, the method resists final-answer theater and yields process data suitable for embedding research.

Our planned experiment will gather such data, study representations that respect dual streams, and define high-dimensional Map of Knowledge regions tied to specific knowledge areas. Completing that program is future work; this paper fixes the method and experimental outline so the research can proceed with transparent constructs.