Research
DATOM represents claims about how evidence should be represented and used in high-stakes environments. These claims are not self-evident; they must be tested against empirical results. This page documents our methodology, the questions we are currently testing, and the areas where our assumptions have failed or remain unproven.
Explicit Research Questions
- Q1: Under what conditions does the aggregation of replication outcomes reduce decision error compared to narrative synthesis?
- Q2: How should the presence of unresolved contradictory evidence materially affect quantified action thresholds?
- Q3: At what point of structural density does additional evidence cease to meaningfully reduce operational risk?
- Q4: Can a structured representation of "what was known when" successfully mitigate hindsight bias in institutional accountability audits?
Methodological Stance
We treat evidence as atomic. This means decomposing research into discrete units—Datoms—that capture population, measure, setting, and result. We do not aggregate via averaging; we represent structure via bipolar bonds (Supports/Contradicts).
- Negative Results: Failures to replicate are treated as high-resolution data that narrow the boundaries of a claim, rather than as noise.
- Disagreement: We preserve contradictory findings within the knowledge graph. Consensus is not a goal; legibility is.
- Provisional Confidence: Confidence scores are time-dependent and update as the graph evolves. They reflect structural stability, not absolute truth.
Active Research Threads
Thread: Quantified Confidence as Structural Density
Impact: Determining if decision-makers can trust a score derived purely from evidence geometry.
Evidence to date: Initial trials using the Remote Work Datomer show that the SCS-64 score correctly identified volatility in coordination-heavy tasks.
Falsification: This assessment would change if a claim with high structural density consistently resulted in unforeseen operational failure.
Thread: Next-Best Evidence (NBE) Optimization
Impact: Identifying which specific experiment would resolve the most uncertainty for the lowest cost.
Evidence to date: Proof-of-concept modeling indicates that targeting "structural gaps" in the graph provides higher information gain than repeating established measures.
Falsification: This would change if experimental results in NBE-targeted areas failed to update the global confidence score.
Known Limitations & Failures
- Assumption Fragility: Early versions of the extraction logic relied too heavily on LLM-based summarization, which smoothed over critical methodological nuances. This approach was discarded in favor of human-in-the-loop atomic verification.
- Inconclusive Evidence: Our current framework struggle to represent qualitative evidence with the same precision as quantitative measures. This is an unresolved gap in the DATOM standard.
- Scalability Limits: The manual effort required to build a "Canonical Proof Object" remains high. We have not yet proven that this can be done at the scale of an entire scientific field.
Research vs. Product
DATOM is a work in progress. While the representation of evidence as atomic units is sufficiently stable for operational use in diligence workflows, the automated confidence updating algorithms remain exploratory. Limitations are documented in our technical appendices to prevent over-trust by users.
Relationship to Prior Work
Our work builds upon the Open Science Framework (OSF) regarding transparency and the Causal Mapping literature for representing relationships. However, the use of structural density as a proxy for decision readiness remains largely unresolved in the existing literature.
Engagement
We invite technical partners to replicate our extraction methods, challenge our confidence scoring logic, or contribute evidence to existing Datomers. Falsification is the primary mechanism of improvement for this system.
Uncertainty is unavoidable. Decisions under uncertainty are hard. DATOM exists to make that difficulty explicit rather than hide it.