MoreRSS

site iconLessWrongModify

An online forum and community dedicated to improving human reasoning and decision-making.
Please copy the RSS to your reader, or quickly subscribe to:

Inoreader Feedly Follow Feedbin Local Reader

Rss preview of Blog of LessWrong

AI Mental Health Chatbots for Low-Resource Settings: A Prioritization Framework

2025-12-02 01:41:49

Published on December 1, 2025 5:41 PM GMT

Summary: We're building an AI-powered mental health chatbot targeting populations with severe mental healthcare shortages. This post presents our framework for prioritizing which conditions and regions to focus on first, synthesizing data on global mental health workforce gaps, existing digital resources across 15+ diagnostic categories, and AI intervention suitability. A key consideration is “breaking the cycle of trauma and tyranny” – addressing conditions that contribute to insecure attachment and power-seeking behavior that perpetuate conflict and authoritarianism.

Note: This is the summary of our preliminary findings including personal observations and inferences. We consider this level of certainty sufficient for current purposes in this early exploratory phase. We’ve written this article with the assistance of Claude and Gemini. We seek further advice and suggestions for the refinement or reframing of the project’s scope.

Introduction

The supply of mental health workers per 100,000 population ranges from 67 in high-income countries to 1 in low-income countries. In all settings, though, there are people whose mental health problems are not addressed for lack of affordable and accessible care.

Recent advances in large language models (LLMs) present an opportunity to partially address this gap through scalable, low-cost interventions. Our team is developing an AI mental health chatbot and hopes to make it useful for populations with the least access to traditional mental healthcare.

However, mental health is vast: various diagnostic manuals contain hundreds of diagnoses each, which overlap in complex ways, and mental health needs vary dramatically across cultural contexts. We cannot effectively serve everyone simultaneously. This post outlines our systematic approach to prioritization and solicits feedback on our reasoning and potential blind spots.

Our Context and Constraints

Team composition: Multilingual team with fluency in English, German, Hindi, Tamil, Estonian, Finnish, and Mandarin.

Unique advantage: Team lead has direct connections within communities struggling with Cluster B personality disorders (ASPD, BPD, HPD, NPD) and familiarity with mentalization-based treatment (MBT), potentially enabling culturally competent outreach to highly stigmatized populations typically underserved by existing resources. Our team also includes licensed psychologists and published psychology researchers.

Long-term motivation: Interest in “breaking the cycle of trauma and tyranny” – addressing the intergenerational transmission of trauma, insecure attachment, and personality pathology that contributes to authoritarian leadership and societal instability. This framework also suggests that healing trauma and fostering secure attachment in this generation can reduce power-seeking pathology and conflict risk in the next.

Current stage: Pre-launch prioritization phase. We’re determining which conditions and populations to serve first, rather than attempting a one-size-fits-all approach.

Methodology: Systematic Resource Mapping

Before prioritizing, we conducted a comprehensive landscape analysis across 15+ major diagnostic categories, examining:

  1. Existing self-help resources (workbooks, apps, online communities) for each specific disorder
  2. Evidence-based interventions and their amenability to AI delivery
  3. Global mental health workforce distribution using WHO data
  4. Technology adoption patterns and infrastructure constraints
  5. Cultural considerations affecting mental health help-seeking
  6. Intergenerational impact on attachment security and power-seeking behavior

Our analysis covered:

  1. Mood disorders (depression, bipolar I, bipolar II, cyclothymic disorder, dysthymia/persistent depressive disorder, disruptive mood dysregulation disorder, premenstrual dysphoric disorder)
  2. Anxiety disorders (generalized anxiety disorder/GAD, panic disorder, agoraphobia, social anxiety disorder/social phobia, specific phobias, separation anxiety disorder, selective mutism)
    1. Trauma and stressor-related disorders (PTSD, complex PTSD, acute stress disorder, adjustment disorders, reactive attachment disorder, disinhibited social engagement disorder)
    2. Obsessive-compulsive and related disorders (OCD, body dysmorphic disorder, hoarding disorder, trichotillomania/hair-pulling disorder, excoriation/skin-picking disorder)
  3. Personality disorders (Cluster A: paranoid, schizoid, schizotypal; Cluster B: antisocial/ASPD, borderline/BPD, histrionic/HPD, narcissistic/NPD; Cluster C: avoidant, dependent, obsessive-compulsive)
  4. Psychotic disorders (schizophrenia, schizoaffective disorder, schizophreniform disorder, brief psychotic disorder, delusional disorder, psychotic depression, substance-induced psychotic disorder)
  5. Neurodevelopmental disorders (ADHD, autism spectrum disorder/ASD, intellectual disabilities, communication disorders including speech sound disorder and childhood-onset fluency disorder/stuttering, specific learning disorders including dyslexia, dyscalculia, and dysgraphia, motor disorders including developmental coordination disorder/dyspraxia, tic disorders including Tourette syndrome)
  6. Substance use disorders (alcohol use disorder, opioid use disorder, cannabis use disorder, stimulant use disorder including cocaine and amphetamines, sedative/hypnotic/anxiolytic use disorder, tobacco use disorder, hallucinogen use disorder, inhalant use disorder, gambling disorder)
  7. Feeding and eating disorders (anorexia nervosa, bulimia nervosa, binge eating disorder, avoidant/restrictive food intake disorder/ARFID, pica, rumination disorder)
  8. Sleep-wake disorders (insomnia disorder, hypersomnolence disorder, narcolepsy, obstructive sleep apnea, central sleep apnea, sleep-related hypoventilation, circadian rhythm sleep-wake disorders, non-rapid eye movement sleep arousal disorders including sleepwalking and sleep terrors, nightmare disorder, rapid eye movement sleep behavior disorder, restless legs syndrome)
  9. Somatic symptom and related disorders (somatic symptom disorder, illness anxiety disorder/hypochondriasis, conversion disorder/functional neurological symptom disorder, factitious disorder, psychological factors affecting other medical conditions)
  10. Dissociative disorders (dissociative identity disorder/DID, dissociative amnesia, depersonalization/derealization disorder, other specified dissociative disorder/OSDD)
  11. Sexual disorders
    1. Sexual dysfunctions (erectile disorder, female sexual interest/arousal disorder, male hypoactive sexual desire disorder, female orgasmic disorder, delayed ejaculation, premature/early ejaculation, genito-pelvic pain/penetration disorder)
    2. Paraphilic disorders (voyeuristic disorder, exhibitionistic disorder, frotteuristic disorder, sexual masochism disorder, sexual sadism disorder, pedophilic disorder, fetishistic disorder, transvestic disorder)
  12. Disruptive, impulse-control, and conduct disorders (oppositional defiant disorder, intermittent explosive disorder, conduct disorder, antisocial personality disorder, pyromania, kleptomania)

For each category, we assessed resource availability (extensive/moderate/limited/very limited), identified gaps, and analyzed cultural/technological adoption patterns.

This categorization is one possible one among many. The complexity and ontological uncertainty of mental health as a field (at least in terms of nosology and diagnosis) is reflected in the abundance of various frameworks, such as the National Institute of Mental Health's Research Domain Criteria, research by the Hierarchical Taxonomy of Psychopathology, the Diagnostic and Statistical Manual, and the Psychodynamic Diagnostic Manual and related frameworks.

Key Finding: Dramatic Workforce Disparities

Using the latest WHO Mental Health Report data, we identified severe disparities in mental health workforce availability:

Global averages by World Bank income group (specialized mental health workers per 100,000 population):

  1. High-Income Countries (HIC): 67.2
  2. Upper-Middle-Income Countries (UMIC): 19.3
  3. Lower-Middle-Income Countries (LMIC): 2.4
  4. Low-Income Countries (LIC): 1.1

By WHO region:

  1. EUR (Europe): 80.4 per 100k
  2. AMR (Americas): 22.2 per 100k
  3. WPR (Western Pacific): 14.1 per 100k
  4. EMR (Eastern Mediterranean): 4.7 per 100k
  5. SEAR (South-East Asia): 4.0 per 100k
  6. AFR (Africa): 2.2 per 100k

This represents a 60-fold difference between highest and lowest resourced regions. In practical terms: a person with depression in Norway has access to ~80 mental health workers per 100,000 people, while someone in Uganda has access to ~0.1 – an 800-fold difference.

The Trauma-Tyranny Cycle: A Developmental Perspective on Long-Term Impact

Beyond immediate suffering, untreated mental health conditions – particularly trauma-related disorders and resulting attachment pathology – contribute to a self-perpetuating cycle that shapes political stability and conflict risk across generations.

The Cycle Model

The cycle operates as follows:

  1. Wars, societal collapse, and adverse childhood experiences → cause widespread trauma and chronic stress
  2. Trauma and parental mental health problems → disrupt healthy attachment formation in children
  3. Insecure attachment and unprocessed trauma → increase the susceptibility to (and rate of) power-seeking dictators
  4. Authoritarian leadership and poor institutional decision-making → increases risk of wars and societal collapse, perpetuating the cycle

This framework suggests that mental health interventions – particularly those addressing trauma, attachment, and personality pathology – have downstream effects on political stability, institutional quality, and conflict risk that compound across generations.

Evidence Base

Research supporting elements of this cycle:

  • Trauma transmission: Parental trauma predicts insecure attachment in children; war trauma affects parenting practices across generations
  • Attachment and leadership: Studies link insecure attachment patterns to authoritarian followership and preference for "strong man" leaders
  • Personality pathology and power: Cluster B traits (particularly NPD and ASPD) overrepresented in positions of political power
  • Developmental origins: Most personality disorders rooted in childhood trauma, neglect, and attachment disruption
  • Malleability: Personality pathology treatable; attachment patterns can shift; trauma can heal – suggesting interventions can break the cycle

Why This Matters for Prioritization

This framework suggests we should weight conditions not only by immediate burden but by their role in perpetuating intergenerational cycles of suffering and instability:

High long-term impact conditions:

  • PTSD and complex trauma: Direct cycle driver; prevents secure parenting
  • Personality disorders (especially Cluster B): Direct link to power-seeking and authoritarian tendencies
  • Attachment-disrupting conditions: Depression, anxiety, substance use in parents affect children's attachment security
  • Childhood conduct problems: Early intervention prevents crystallization into ASPD

High-risk populations:

  • Conflict-affected regions: Active cycle perpetuation; highest intervention value
  • Parents and prospective parents: Breaking intergenerational transmission
  • Adolescents and young adults: Critical window before personality patterns rigidify and before they become parents

Intervention modalities with cycle-breaking potential:

  • Trauma healing: Reduces transmission to next generation
  • Parenting support: Directly improves children's attachment security
  • Personality disorder treatment: Reduces power-seeking behavior; improves parenting
  • Resilience building: Strengthens population-level resistance to authoritarian messaging

This lens makes conditions like PTSD, personality disorders, and perinatal mental health higher priority despite some challenges, because successfully treating one generation protects the next.

Prioritization Framework

We developed a multi-tier framework weighing 20+ criteria across seven domains:

Tier 1: Core Feasibility

Safety & Risk Profile

  • Can we deliver interventions without significant risk of harm?
  • Do we have robust crisis protocols for high-risk situations?
  • Can we reliably identify and escalate emergencies?

Key insight: This criterion should filter out conditions before other considerations. Active psychosis, acute suicidality, severe eating disorders in crisis, and mania present risks that outweigh potential benefits of unsupervised AI intervention.

Language Capacity

  • Does our team have native/fluent speakers for seeking feedback, noticing and responding to problems?
  • Can we avoid mere translation in favor of genuine cultural competence?

Technology Access & Literacy

  • Smartphone penetration in target regions
  • Data costs relative to local income
  • Digital literacy rates
  • Internet infrastructure reliability

Equity & Justice

  • Prioritizing most underserved over most profitable
  • Ensuring accessibility for lowest-income users

Cultural Sensitivity

  • Avoiding imposition of Western psychiatric models on non-Western contexts
  • Incorporating local healing traditions
  • Collaborating with local communities and professionals

Transparency & Limitations

  • Clear communication about AI capabilities and limitations
  • Avoiding dependency creation
  • Providing pathways to human care

Tier 2: Impact Potential

Mental Health Workforce Gap

  • Where is the treatment gap largest?
  • Where will AI provide the highest marginal benefit?

Disease Burden & Prevalence

  • DALYs (Disability-Adjusted Life Years) lost
  • Absolute number of people affected
  • Regional variation in prevalence

Stigma & Barriers to Traditional Care

  • Where does stigma prevent help-seeking?
  • Where do cultural/gender restrictions limit access to human therapists?
  • Where might anonymous AI access lower barriers?

Attachment Security Impact

  • Does treating this condition improve parenting capacity?
  • Will treatment reduce transmission of insecure attachment to children?
  • Does the condition directly disrupt attachment formation?

High impact: Perinatal depression/anxiety, PTSD, substance use, personality disorders (all affect parenting)

Moderate impact: Depression, anxiety in parents; childhood trauma-related conditions

Power-Seeking & Authoritarianism Risk

  • Does the condition involve patterns associated with malevolent leadership? (NPD, ASPD, sadism)
  • Does healing reduce power-seeking behavior or improve use of power?
  • Does treatment reduce susceptibility to authoritarian messaging?

High impact: Cluster B personality disorders, especially NPD/ASPD combinations; trauma creating "might makes right" worldviews

Moderate impact: Any condition improving emotional regulation and reducing reactivity to threats

Conflict & Instability Risk

  • Is the condition prevalent in conflict zones, perpetuating cycles?
  • Does the condition directly increase interpersonal violence risk?
  • Does healing improve institutional decision-making quality?

High impact: PTSD in conflict zones, ASPD, substance use disorders, impulse control disorders

Moderate impact: Conditions affecting judgment and emotional regulation

Critical Developmental Windows

  • Can we intervene before personality patterns rigidify? (adolescence/early adulthood)
  • Can we intervene before individuals become parents?
  • Can we heal parents before patterns transmit to children?

High impact: Adolescent/young adult populations; perinatal interventions; parenting support

Population-Level Resilience

  • Does healing this condition make populations more resistant to manipulation?
  • Does treatment promote secure attachment at scale?
  • Does intervention build what Antonovsky calls "sense of coherence" (comprehensibility, manageability, meaningfulness)?

High impact: Trauma healing, attachment-focused interventions, mental health literacy programs

Tier 3: AI Suitability

Amenability to Structured Interventions

AI is most effective for conditions with structured, manualized treatments:

  • Excellent fit: CBT for depression/anxiety, CBT-I for insomnia, exposure protocols, behavioral activation, psychoeducation
  • Moderate fit: Motivational interviewing, DBT skills training, habit tracking, mentalization practice
  • Poor fit: Complex trauma requiring relational depth, severe personality disorders needing nuanced therapeutic tensions, conditions requiring physical examination

Self-Help Amenability

  • Does evidence support self-directed interventions?
  • Can people improve without immediate professional involvement?

Data & Training Resources

  • Quality of LLM training data for condition
  • Availability of evidence-based treatment manuals
  • Ability to validate AI responses against gold standards

Tier 4: Market Gap Analysis

Existing Digital Solutions

  • Where are markets oversaturated vs. underserved?
  • Where do existing solutions fail to serve LMICs?

Our finding: Dramatic inequality mirrors workforce gaps. Most mental health apps target English-speaking HIC markets. Very few quality apps exist in Hindi (500M+ speakers), Bengali (230M+ speakers), or Tamil (80M+ speakers). African markets almost entirely neglected except South Africa.

Cultural Adaptation Needs

  • Where do Western psychiatric models fail to translate?
  • Where is somatic expression of distress more common?
  • Collectivist vs. individualist therapy frameworks

Existing Workbook/Professional Resource Availability

  • Can we adapt existing evidence-based resources?
  • Do gaps indicate lack of proven interventions or just lack of accessibility?

Tier 5: Strategic Considerations

Scalability Potential

  • Size of potential user base
  • Growth trajectory of condition awareness/diagnosis
  • Platform effects and community features

Regulatory & Liability Landscape

  • Regulatory requirements vary by region and intervention type
  • Risk increases with diagnostic/treatment claims vs. psychoeducation/support

Monetization Potential

  • Willingness to pay varies by region and condition
  • Venture capital funding opportunities
  • Grant funding opportunities (WHO, NGOs, government programs)
  • Freemium viability for impact at scale

Partnership Opportunities

  • NGO/WHO initiatives in target regions
  • Research institutions for validation studies
  • Local healthcare systems for integration
  • Telehealth providers for triage/adjunct services

Measurement & Validation

  • Can we measure impact using validated scales?
  • Feasibility of clinical validation studies
  • User engagement and retention metrics

Condition Prioritization: Rankings and Rationale

Using this framework, we ranked conditions by overall suitability. Note that the assessment of fuzzy regional factors of suitability is heavily informed by AI.

Tier 1: Highest Priority

PTSD (Prioritize Conflict-Affected Regions)

  • Burden: High in conflict-affected regions (Afghanistan, Sudan, South Sudan, Syria, Yemen, DRC, Myanmar, Pakistan border regions, Northeast Nigeria)
  • Gap: Extreme shortage of trauma-trained therapists
  • AI Fit: Good – PE and CPT components are structured
  • Safety: Moderate risk – requires robust crisis protocols
  • Existing Resources: Very few culturally appropriate apps for conflict-affected LMICs
  • Stigma: Extremely high in many cultures; AI may lower barriers
  • Cultural: Trauma narratives culturally specific; requires careful adaptation
  • Cycle Impact: ⭐⭐⭐⭐⭐ – PTSD is the primary cycle driver. Traumatized parents have difficulty providing secure attachment; PTSD directly transmits across generations via parenting practices and epigenetics; conflict-zone trauma creates conditions for the next generation of authoritarian leaders; healing trauma breaks the cycle at its source.

Personality Disorders – Strategic Focus on Cluster B (NPD, ASPD, BPD, HPD)

Our team's unique positioning: Given team lead's connections in NPD/ASPD/HPD communities and MBT training, we have potential advantages in serving this highly stigmatized population.

  • Burden: ~10% of population; severe functional impairment
  • Gap: MASSIVE stigma prevents help-seeking; very few specialists even in HICs
  • AI Fit: MODERATE-DIFFICULT – MBT requires nuanced mentalizing that challenges AI, BUT psychoeducation and skill-building components could help
  • Safety: Moderate-High risk depending on disorder (ASPD risk assessment, BPD self-harm)
  • Existing Resources: Very limited for Cluster B; most resources focus on "surviving" people with NPD/BPD rather than helping them
  • Cultural: Cluster B presentations culturally mediated; requires deep cultural knowledge
  • Cycle Impact: ⭐⭐⭐⭐⭐ – This is the other primary cycle driver. Cluster B disorders (especially NPD and ASPD) are directly associated with power-seeking behavior, authoritarian leadership, and malevolent use of power. These conditions arise from childhood trauma and transmit intergenerationally through disrupted attachment. Healing personality disorders directly reduces the pool from which malevolent leaders emerge. BPD, while less associated with power-seeking, severely disrupts parenting and attachment.

The compassionate case: As I've argued elsewhere, people with NPD and ASPD are not “evil” – they are using brilliant childhood adaptations to survive impossible situations. These adaptations become maladaptive in adulthood but can heal with appropriate support, typically in just a few years of therapy. Many individuals with these conditions desperately want help but cannot access it due to stigma, cost, and scarcity of trained therapists.

The strategic case: The overlap between Cluster B traits and positions of power means that even small improvements in this population have outsized effects on institutional quality, conflict risk, and the next generation's wellbeing. If we can help even a fraction of people with these conditions, the downstream effects on politics, violence, and intergenerational trauma transmission could be substantial.

Possible approach: Focus on psychoeducation, mentalization skills practice, emotion regulation – NOT replacement for intensive therapy but potentially helpful adjunct for people unable/unwilling to access traditional care due to stigma. Clear about AI limitations. Strong safety protocols for violence risk. Initial target: adults with NPD/ASPD seeking help (not those court-mandated or uninterested in change).

Conduct Disorder / Childhood Trauma Interventions

  • Burden: Common in high-adversity environments
  • Gap: Very few child mental health services in LMICs
  • AI Fit: Moderate – parenting interventions structured; child-facing interventions more challenging
  • Safety: Moderate – requires careful age-appropriate design
  • Challenges: Would need separate child-focused platform; consent/privacy issues
  • Cycle Impact: ⭐⭐⭐⭐⭐ – Early intervention prevents personality disorder crystallization. Conduct disorder is precursor to ASPD; childhood trauma is the root cause of most personality pathology. Intervening in childhood/adolescence is the most effective cycle-breaking point, before patterns rigidify. Biggest challenge: reaching children requires a different platform approach.

Perinatal Mental Health (Depression, Anxiety)

  • Burden: Massive need in your regions (maternal mortality links)
  • Gap: Low resources for perinatal mental health
  • AI Fit: Good for psychoeducation, CBT components
  • Safety: Moderate-High risk (infanticide, severe postpartum psychosis require emergency response)
  • Opportunity: WHO priority area; partnership potential
  • Cycle Impact: ⭐⭐⭐⭐⭐ – This is a peak intervention point for attachment security. Perinatal mental health directly affects infant attachment formation; this is the most critical developmental window; treating mothers prevents transmission to the next generation at the source.

Tier 2: High Priority

Depression (Mild–Moderate)

  • Burden: Leading cause of disability globally; ~280M people affected
  • Safety: Low risk if severe/suicidal cases properly filtered and escalated
  • AI Fit: Excellent – CBT and behavioral activation are highly structured
  • Evidence: Strong self-help efficacy data
  • Workforce Gap: Massive gap in LIC/LMIC (treatment gap >80%)
  • Existing Resources: Many apps exist BUT dramatic language gap (almost nothing quality in Hindi/Tamil/Bengali for LMIC contexts)
  • Measurement: PHQ-9 validated globally
  • Cultural: Depression presents across cultures but may manifest somatically – requires adaptation
  • Cycle Impact: ⭐⭐⭐ – Parental depression significantly disrupts attachment security; reduces parenting capacity; transmits intergenerationally

Anxiety Disorders (GAD, Social Anxiety, Panic)

  • Burden: ~300M affected globally; highly disabling
  • Safety: Low risk
  • AI Fit: Excellent – CBT protocols, exposure hierarchies, grounding techniques all structured
  • Evidence: Strong self-help efficacy
  • Gap: Similar to depression – huge LMIC gap, language barriers
  • Measurement: GAD-7, SPIN validated globally
  • Cultural: Anxiety universal but expression varies; requires cultural adaptation
  • Cycle Impact: ⭐⭐⭐ – Anxious parenting affects children's attachment security; hypervigilance transmits intergenerationally; anxiety increases susceptibility to threat-based authoritarian messaging

Substance Use Disorders (Harm Reduction Focus)

  • Burden: Major cause of DALYs in many LMICs
  • Gap: Extreme stigma prevents help-seeking; very few services
  • AI Fit: Good for motivational interviewing, harm reduction education, tracking
  • Safety: Moderate – requires crisis protocols for overdose risk, withdrawal
  • Cultural: Highly stigmatized; AI anonymity major advantage
  • Challenges: Cultural/religious sensitivities (alcohol in Muslim countries, substance use stigma in conservative societies)
  • Cycle Impact: ⭐⭐⭐⭐ – Parental substance use severely disrupts attachment; increases violence and neglect; intergenerational transmission common; substance use associated with impulsive violence and poor institutional decision-making

Tier 3: Medium to Low Priority

Insomnia (Primary & Comorbid)

  • Burden: ~30% of adults affected; impacts physical and mental health
  • Safety: Zero acute risk
  • AI Fit: PERFECT – CBT-I is highly manualized and structured
  • Evidence: CBT-I self-help proven effective (comparable to therapist-delivered)
  • Gap: Very few quality apps in target languages despite universal problem
  • Measurement: Sleep diary, ISI scale
  • Cultural: Low stigma = higher engagement; universal relevance
  • Unique advantage: "Gateway" condition – treating insomnia often improves comorbid depression/anxiety
  • Cycle Impact: ⭐⭐ – Better sleep improves emotional regulation and parenting quality; indirect effects on attachment security

OCD

  • Burden: Highly disabling; ~2–3% prevalence
  • AI Fit: Excellent – ERP is highly structured
  • Gap: Very few ERP-trained therapists even in HICs
  • Safety: Low risk, beyond the danger of reinforcing compulsions
  • Existing Resources: Few quality apps in any language
  • Cultural: Presentations vary (religious scrupulosity, contamination fears vary culturally)
  • Cycle Impact: ⭐ – Minimal direct effect on attachment or power-seeking, though severe OCD can impair parenting

ADHD (Adults & Adolescents)

  • Burden: Growing awareness in LMICs; severe underdiagnosis
  • AI Fit: Excellent for skills training (time management, organization, emotional regulation)
  • Safety: Zero acute risk
  • Gap: Massive – most LMICs have near-zero ADHD services for adults
  • Existing Resources: MANY productivity apps BUT few culturally adapted for India/Africa; mostly assume HIC work contexts
  • Cultural: ADHD increasingly recognized cross-culturally but stigma varies
  • Cycle Impact: ⭐⭐ – Untreated ADHD in parents complicates parenting; emotion dysregulation affects children; but not directly linked to power-seeking or authoritarianism

Somatic Symptom Disorders

  • Burden: Very common in target regions (somatic expression of distress culturally normative in many Asian/African contexts)
  • Gap: Almost NO existing digital resources
  • Cultural Fit: Highly relevant – Western psychology often fails to address
  • Challenges: Requires medical rule-outs (liability risk); validation complex
  • Opportunity: Major gap to fill with culturally appropriate approaches
  • Cycle Impact: ⭐⭐ – Chronic pain/illness affects parenting capacity; but not directly linked to attachment disruption or power-seeking

Bipolar Disorder

  • High safety risk (mania, suicidality)
  • Medication essential (beyond AI scope)
  • Complex case management needs
  • Cycle Impact: ⭐⭐ – Untreated bipolar disrupts parenting, but with medication most people stable

Eating Disorders

  • High medical risk requiring monitoring
  • Lower prevalence in initial target regions (though rising)
  • Complex interventions
  • Cycle Impact: ⭐ – Minimal direct cycle effects except in severe cases affecting parenting

Psychotic Disorders

  • HIGH safety risk
  • Medication usually essential
  • Anosognosia limits engagement
  • BUT: Family psychoeducation could be valuable supportive intervention
  • Cycle Impact: ⭐ – Most people with schizophrenia are not violent or power-seeking; primary impact is on individual/family suffering

Geographic Prioritization: Country Rankings

Using mental health workforce data (per 100,000 population), World Bank income classifications, language accessibility, technology infrastructure, and conflict/trauma exposure, but ignoring strategic, marketing, or funding considerations. Fuzzy regional, cultural, and historical impressions again draw heavily on AI.

Tier 1: Highest Priority Markets 🎯

India

  • Mental health workers: ~0.3–0.6 per 100k (vs. 67.2 in HICs)
  • Population: 1.43 billion
  • Languages: Hindi (550M speakers), Tamil (80M speakers), English (widespread)
  • Income: LMIC (but wide internal variation)
  • Technology: Rapidly growing smartphone penetration; good mobile infrastructure in urban/suburban areas
  • Mental Health Burden: High rates of depression, anxiety, suicide
  • Conflict/Trauma: Kashmir conflict; communal violence; high rates of adverse childhood experiences
  • Cycle Status: ⭐⭐⭐ – Significant trauma exposure; growing but incomplete mental health infrastructure; critical window to intervene before patterns rigidify
  • Rationale: Largest addressable market with our language capabilities; enormous gap; growing mental health awareness
  • Challenges: Digital divide (rural vs. urban); data costs; diverse cultural contexts

Pakistan

  • Mental health workers: ~0.2–0.5 per 100k
  • Population: 231 million
  • Languages: English (official), Urdu (mutually intelligible with Hindi)
  • Income: LMIC
  • Technology: Growing smartphone adoption; less infrastructure than India
  • Mental Health Burden: High; extreme stigma particularly around women's mental health
  • Conflict/Trauma: Afghan border terrorism; internal sectarian violence; TTP attacks; drone strike trauma; significant PTSD burden
  • Cycle Status: ⭐⭐⭐⭐ – Active conflict perpetuating trauma cycles; very low mental health capacity; strong stigma preventing help-seeking
  • Rationale: Second-largest Urdu/Hindi-speaking population; severe gap; AI anonymity crucial given stigma; trauma healing critical for conflict de-escalation
  • Challenges: Political instability; conservative cultural norms; lower female digital access

Afghanistan

  • Mental health workers: ~0.02–0.05 per 100k (among world's lowest)
  • Population: 41 million
  • Languages: English (limited), but potential Dari/Pashto development
  • Income: LIC
  • Technology: Growing mobile penetration despite infrastructure challenges
  • Mental Health Burden: Extreme – decades of war
  • Conflict/Trauma: 40+ years continuous conflict; Taliban rule trauma; highest trauma burden globally
  • Cycle Status: ⭐⭐⭐⭐⭐ – Active cycle perpetuation at crisis levels. Entire generations traumatized; minimal mental health infrastructure; current authoritarianism driven by trauma cycles. Highest need but also highest access barriers.
  • Rationale: Most acute trauma burden; greatest potential cycle-breaking impact
  • Challenges: Security situation; female access restrictions; language barrier (would need Dari/Pashto); political complications

Nigeria

  • Mental health workers: ~0.2–0.3 per 100k
  • Population: 220 million
  • Languages: English (official)
  • Income: LMIC
  • Technology: Variable – good in urban areas, limited in rural
  • Mental Health Burden: High; stigma extreme
  • Conflict/Trauma: Boko Haram in northeast (mass trauma, kidnappings); farmer-herder violence; Niger Delta conflict; significant PTSD burden
  • Cycle Status: ⭐⭐⭐⭐ – Active conflict zones; trauma perpetuating instability; religious extremism linked to trauma cycles
  • Rationale: Largest African market; English-speaking; enormous gap; trauma healing critical in conflict zones
  • Challenges: Infrastructure variability; cultural diversity (250+ ethnic groups); data costs; religious considerations

South Sudan

  • Mental health workers: <0.05 per 100k
  • Population: 11 million
  • Languages: English (official)
  • Income: LIC
  • Technology: Very limited but growing mobile access
  • Mental Health Burden: Extreme – ongoing conflict
  • Conflict/Trauma: Continuous war since independence; mass displacement; extreme violence exposure; one of world's highest trauma burdens
  • Cycle Status: ⭐⭐⭐⭐⭐ – Acute cycle perpetuation; virtually no mental health services; urgent intervention needed
  • Rationale: Desperate need; English-speaking; potential for enormous impact
  • Challenges: Infrastructure extremely limited; ongoing conflict; very low literacy

Democratic Republic of Congo

  • Mental health workers: ~0.05 per 100k
  • Population: 99 million
  • Languages: French (official), some English
  • Income: LIC
  • Technology: Growing mobile penetration despite poor infrastructure
  • Mental Health Burden: Extreme – decades of conflict
  • Conflict/Trauma: 25+ years of war; mass rape as weapon; child soldiers; extreme violence; ongoing Eastern Congo conflict
  • Cycle Status: ⭐⭐⭐⭐⭐ – Severe trauma perpetuating instability; virtually no services
  • Rationale: Massive trauma burden; enormous need
  • Challenges: Language barrier (would need French); infrastructure; ongoing violence; complexity

Myanmar

  • Mental health workers: ~0.1 per 100k
  • Population: 54 million
  • Languages: English (some), Mandarin (some)
  • Income: LMIC
  • Technology: Previously growing, now complicated by military coup
  • Mental Health Burden: High and worsening
  • Conflict/Trauma: Military coup trauma; Rohingya genocide; ethnic conflicts; civil war
  • Cycle Status: ⭐⭐⭐⭐⭐ – Active authoritarian violence; trauma-driven conflict cycles; dramatic example of cycle in action
  • Rationale: Clear case of trauma-tyranny cycle; potential intervention point
  • Challenges: Political situation; military restrictions; language barriers; safety concerns

Kenya

  • Mental health workers: ~0.5 per 100k
  • Population: 54 million
  • Languages: English, Swahili
  • Income: LMIC
  • Technology: Relatively advanced mobile infrastructure (M-Pesa model)
  • Mental Health Burden: Moderate rates; growing awareness
  • Conflict/Trauma: Post-election violence (2007–08); Al-Shabaab attacks; inter-ethnic tensions
  • Cycle Status: ⭐⭐⭐ – Historical trauma; relatively stable now but at risk; preventive intervention valuable
  • Rationale: Best African tech infrastructure; English-speaking; relatively strong civil society; good test case for preventive approach
  • Challenges: Would need Swahili for broader reach

Bangladesh

  • Mental health workers: ~0.1–0.2 per 100k
  • Population: 170 million
  • Languages: Bengali/English
  • Income: LMIC
  • Technology: Rapidly improving mobile infrastructure
  • Mental Health Burden: High rates of depression, anxiety
  • Conflict/Trauma: Liberation war trauma (1971); Rohingya refugee crisis; natural disasters; high rates of interpersonal violence
  • Cycle Status: ⭐⭐⭐ – Historical trauma; refugee crisis stress; refugee population particularly high-need
  • Rationale: Large Bengali-speaking population; severe gap; growing digital access; Rohingya camps could be specific intervention target
  • Challenges: Would require Bengali language development (related to Hindi but distinct)

Yemen

  • Mental health workers: ~0.02–0.05 per 100k
  • Population: 33 million
  • Languages: Arabic (no team capacity currently)
  • Income: LIC
  • Technology: Infrastructure severely damaged by war
  • Mental Health Burden: Extreme – humanitarian catastrophe
  • Conflict/Trauma: Ongoing civil war; Saudi bombing; famine; cholera; complete societal breakdown
  • Cycle Status: ⭐⭐⭐⭐⭐ – Worst humanitarian crisis globally; entire population traumatized; desperately needs intervention
  • Rationale: Extreme need; enormous potential impact if accessible
  • Challenges: Language barrier (would need Arabic); infrastructure destroyed; ongoing war; access extremely limited

Tier 2: Secondary Priority Markets

Syria (ongoing conflict, Arabic language barrier but extreme need)

Ethiopia (123M, recent Tigray conflict, English educational language)

Sudan (46M, ongoing conflict, English secondary)

Tanzania (65M, LIC, English/Swahili)

Uganda (47M, LIC, English, LRA conflict legacy)

Nepal (30M, LMIC, English, Hindi understood, Maoist conflict legacy)

Open Questions and Request for Feedback

We welcome any feedback, and are particularly interested in:

  1. Prioritization blind spots. What important criteria are we missing? What are we overweighting or underweighting?
  2. Funding and partnerships. Can we safely bootstrap in the US with VC funding and expand to other countries later?


Discuss

Which planet is closest to the Earth, and why is it Mercury?

2025-12-02 01:16:13

Published on December 1, 2025 5:16 PM GMT

Which planet is closest to Earth, on average? I used to think it was Venus, followed by Mars. But this paper claims it is instead Mercury.

At first this seemed to make no sense. Just picture the orbits of the planets: they're a bunch of concentric circles (approximately). Venus' orbit completely encloses Mercury's. Every point of it is closer to the Earth's orbit than Mercury's orbit is. And indeed, that's how you get that Venus is the closest planet to Earth, followed by Mars, and then Mercury: take the difference between the radius of their orbits.

I don't think you get to call your image "Sized to Scale" when Jupiter occupies half the distance between it and Mars, but okay.

But that doesn't actually get you the average distance. If two planets happen to be lined up (at the same point in their orbit) then yes, the distance between them is the difference between their orbital radii. But if one of them is on the opposite side of the Sun as the other (at the opposite point in their orbit), then the distance between them is the sum of their radii, and Mercury is the closest planet to Earth!

So, to figure out what planet is closest to the Earth on average, you have to actually do the math.

Actually Doing the Math

Let's calculate the average distance between the Earth (circular orbit of radius ) and another planet (circular orbit of radius ). We'll suppose each planet has an independent uniform probability of being at each point in its orbit, but because of symmetry, we can take our coordinate system to have Earth in the x axis, and only worry about the varying position of the other planet relative to it.

The distance between the planets, given by the cosine rule,  is . To find the average, we have to integrate this over  between  and , and divide by . That looks like a really unpleasant thing to integrate, but luckily our source paper tells us the answer, which turns out to be:

where  is the "elliptic integral of the second kind". After looking up the formula for this elliptic integral on Wikipedia, I was able to wrangle our expression for the distance into the paper's formula:

Skippable Math

We start with the average distance by the cosine rule:

And we want to get to an expression involving the elliptic integral of the second kind, which is:

To turn that cosine into a sine squared, we substitute  and use the identity , getting:

Now we have the sine squared, but the sign in front of it is positive (two minuses that cancel out), and we want it to be negative. So we substitute again, by , and use the fact that :

Rearranging a bit, we get:

After all the substitutions, the integral is between  and . However, since  is an even function, the parts of the integral from  to  and from  to  have the same value, so we can rewrite as:

Dividing and multiplying by , we get:

Where the integral finally looks like our elliptic integral, with  equal to . Replacing it in the expression, it becomes:

Exactly the expression from the original paper!

According to the paper, this average distance strictly increases with the radius .[1] So, the lowest average distance is to the planet with the smallest orbit, i.e. Mercury. Problem solved!

But Why Though

...problem not really solved. While this does prove that Mercury is the closest planet to Earth, it doesn't actually help explain why. Is there a simple reason we can actually understand that explains why planets with smaller radii are closer to us?

Yes. Consider, instead of a random point on the inner planet's orbit, two points A and B, at the same angle above the vertical:

Between these two points, the average horizontal distance to the Earth is just the Earth's orbital radius, , and so doesn't depend on the other orbit's radius; and the vertical distance is the same for both points, . So increasing the radius  doesn't change the average horizontal distance at all, and increases the average vertical distance; of course this means the average total distance is increased!

Of course, you may notice this was not completely valid reasoning, since horizontal distances and vertical distances don't add, they combine through Pythagoras' theorem. To turn this verbal insight into an actual proof, we need to write down the formula for the average between the two points A and B's distances, take the derivative with respect to , and see if it is positive. As it turns out, it is:

Skippable Math 2

The angle of the point B is , so its cosine is . The sum of the distances is then:

(The average is just half of this, and the factor of  doesn't make a difference as to whether it's increasing with , so we're discarding it.)

The derivative of this expression with respect to  is:

Which, replacing the expressions for  and , is:

Since  and  are positive, this expression is greater than zero if and only if its denominator is positive, so we want:

Now, if in the expression for  we replace  with , its value becomes lesser or equal (with equality only when ):

And we can do the same for :

( is always positive because we're taking  between  and , so we don't need the absolute value here.)

Substituting in our equation, we get:

The term in the right side of this inequality is certainly greater than or equal to zero: if  is less than zero, it's the sum of a term and its negative, which is 0; if it's more than zero, it's the sum of two positive terms, which is more than zero.

Since the inequality is strict when  is not 0, our initial expression for the derivative may be zero when  is zero, but is positive otherwise. This means that the sum of the distances  grows with the radius , and so does their average.

The average distance over the entire circle is equivalent to an integral over averages like this (divided by ), with  varying from  to , so it also grows with the radius.

So the intuitive explanation does turn into a viable proof that Mercury really is, on average, the closest planet to Earth and to every other planet.

  1. ^

    Actually, it doesn't even say that, it just says "the distance between two orbiting bodies is at a minimum when the inner orbit is at a minimum".



Discuss

How middle powers may prevent the development of artificial superintelligence

2025-12-02 00:48:58

Published on December 1, 2025 4:48 PM GMT

In this paper, we make recommendations for how middle powers may band together through a binding international agreement and achieve the goal of preventing the development of ASI, without assuming initial cooperation by superpowers.

You can read the paper here: asi-prevention.com

In our previous work Modelling the Geopolitics of AI, we pointed out that middle powers face a precarious predicament in a race to ASI. Lacking the means to seriously compete in the race or unilaterally influence superpowers to halt development, they may need to resort to a strategy we dub “Vassal’s Wager”: allying themselves with a superpower and hoping that their sovereignty is respected after the superpower attains a DSA.

Of course, this requires superpowers to avert the extinction risks posed by powerful AI systems, something over which middle powers have little or no control over. Thus, we argue that it is in the interest of most middle powers to collectively deter and prevent the development of ASI by any actor, including superpowers.

In this paper, we design an international agreement that could enable middle powers to form a coalition capable of achieving this goal. The agreement we propose is complementary to a “verification framework” that can prevent the development of ASI if it achieves widespread adoption, such as articles IV to IX of MIRI’s latest proposal.

Our proposal tries to answer the following question: how may a coalition of actors pressure others to join such a verification framework, without assuming widespread initial participation?

Key Mechanisms

Trade restrictions. The agreement imposes comprehensive export controls on AI-relevant hardware and software, and import restrictions on AI services from non-members, with precedents ranging from the Chemical Weapons Convention and the Nuclear Non-Proliferation Treaty.

Reactive deterrence. Escalating penalties—from strengthened export controls to targeted sanctions, broad embargoes, and ultimately full economic isolation—are triggered as actors pursue more and more dangerous AI R&D outside of the verification framework.

Preemptive self-defense rights. The coalition recognizes that egregiously dangerous AI R&D constitutes an imminent threat tantamount to an armed attack, permitting members to claim self-defense rights in extreme cases.

Escalation in unison. The agreement would establish AI R&D redlines as well as countermeasures tied to each breach. These are meant to ensure that deterrence measures are triggered in a predictable manner, in unison by all participants of the agreement. This makes it clear to actors outside of the agreement which thresholds are not to be crossed, while ensuring that any retaliation by actors receiving penalties are distributed among all members of the coalition.

Though these measures represent significant departures from established customs, they are justified by AI’s unique characteristics. Unlike nuclear weapons, which permit a stable equilibrium through mutually assured destruction (MAD), AI R&D may lead to winner-take-all outcomes. Any actor who automates all the key bottlenecks in Automated AI R&D secures an unassailable advantage in AI capabilities: its lead over other actors can only grow over time, eventually culminating in a decisive strategic advantage.

Path to Adoption

We recommend that the agreement activates once signatories represent at least 20% of the world’s GDP and at least 20% of the world’s population. This threshold is high enough to exert meaningful pressure on superpowers; at the same time, it is reachable without assuming that any superpower champions the initiative in its early stages.

This threshold enables middle powers to build common knowledge of their willingness to participate in the arrangement without immediately antagonizing actors in violation of the redlines, and without paying outsized costs at a stage when the coalition commands insufficient leverage.

As the coalition grows, network effects may accelerate adoption. Trade restrictions make membership increasingly attractive while non-membership becomes increasingly costly.

Eventually, the equilibrium between competing superpowers may flip from racing to cooperation: each superpower could severely undermine the others by joining the coalition, leaving the final holdouts facing utter economic and strategic isolation from the rest of the world. If this is achieved early enough, all other relevant actors are likely to follow suit and join the verification framework.

Urgency

The agreement's effectiveness depends critically on timing. Earlier adoption may be achieved through diplomatic and economic pressure alone. As AI R&D is automated, superpowers may grow confident they can achieve decisive strategic advantage through it. If so, more extreme measures will likely become necessary.

Once superpowers believe ASI is within reach and are willing to absorb staggering temporary costs in exchange for a chance at total victory, even comprehensive economic isolation may prove insufficient and more extreme measures may be necessary to dissuade them.

The stakes—encompassing potential human extinction, permanent global dominance by a single actor, or devastating major power war—justify treating this challenge with urgency historically reserved for nuclear proliferation. We must recognize that AI R&D may demand even more comprehensive international coordination than humanity has previously achieved.



Discuss

Becoming a Chinese Room

2025-12-02 00:34:30

Published on December 1, 2025 4:34 PM GMT

[My novel, Red Heart, is on sale for $4 this week. Daniel Kokotaijlo liked it a lot, and the Senior White House Policy Advisor on AI is currently reading it.]

“Formal symbol manipulations by themselves … have only a syntax but no semantics. Such intentionality as computers appear to have is solely in the minds of those who program them and those who use them, those who send in the input and those who interpret the output.”
John Searle, originator of the “Chinese room” thought experiment

A colleague of mine, shortly before Red Heart was published, remarked to me that if I managed to write a compelling novel set in China, told from Chinese perspectives — without spending time in the country, having grown up in a Chinese-culture context, or knowing any Chinese language — it would be an important bit of evidence about the potency of abstract reasoning and book-learning. This, in turn, may be relevant to how powerful and explosive we should expect AI systems to be.

There are many, such as the “AI as Normal Technology” folks, who believe that AI will be importantly bottlenecked on lack of experience interacting with the real world and all its complex systems. “Yes, it’s possible to read about an unfamiliar domain, but in the absence of embodied, hands-on knowledge, the words will be meaningless symbols shuffled around according to mere statistical patterns,” they claim.[1] ChatGPT has never been to China, just as it hasn’t really “been” to any country. All it can do is read.[2] Can any mind, no matter how fast or deep, build a deep and potent understanding of the world from abstract descriptions?

I’m not an LLM, and there may be important differences, but let’s start with the evidence. Did I succeed?

“It greatly surprised and impressed me to learn that Max had not once traveled to China prior to the completion of this novel. The scene-setting portions of every chapter taking place in China reveals an intimate familiarity with the cultures, habits, and tastes of the country in which I was raised, all displayed without the common pitfall that is the tendency to exoticize. I’d have thought the novel written by someone who had lived in China for years.”
— Alexis Wu, Chinese historical linguist and translator

“I now believe that you have a coauthor that was raised in China - the Chinese details are quite incredible, and if you don’t have a Chinese coauthor or editor that’s really impressive for someone who hasn’t been to China.”
Red Heart is a strikingly authentic portrayal of AI in modern China—both visionary and grounded in cultural truth.”
— Zhang San,[3] Senior AI Executive

How did I do it? And what might this suggest about whether understanding can be built from text alone?

Writing About China

I definitely got some things wrong, when writing the book.

Shortly before the book came out, concerned that it might be my only chance to safely visit the mainland,[4] I visited Shenzhen (and Hong Kong) as a tourist. Most of Red Heart takes place in Guizhou, not Guangdong, where Shenzhen is, but Guizhou is still pretty close, and similar in some ways — most particularly the humidity. The entire novel only has a single offhand reference to humidity, despite involving a protagonist that regularly goes in and out of carefully air-conditioned spaces! Southern China is incredibly humid (at least compared to California), and to my inner-perfectionist it stands as a glaring flaw. Augh!

Most issues that I know about are like the humidity — details which are absent, rather than outright falsehoods. I wish I had done a better job depicting fashion trends and beauty standards. I wish I’d emphasized how odd it is for the street-food vendor to only take cash. That sort of thing.

I’m sure there are a bunch of places where I made explicit errors, too. One of the most important parts of my process was getting a half-dozen Chinese people to read early drafts of my novel and asking them to look for mistakes. There were a bunch,[5] and it was extremely common for one Chinese reader to catch things that another reader didn’t, which implies that there are still more errors that I haven’t yet heard about because the right kind of Chinese reader hasn’t left a review yet. (If this is you, please speak up, either in the comments here or on Amazon or Goodreads! I love finding out when I’m wrong — it’s the first step to being right.) One of my biggest take-aways from learning about China is that it’s an incredibly large and diverse country (in many ways more than the USA[6]), and that means that no single person can do a comprehensive check for authenticity.

a group of buildings that are next to each other

But also, I think I got most things right, or at least as much as any novel can. Well before sending the book to any Chinese people, I was reading a lot about the country as part of my work as an AI researcher. China is a technological powerhouse, and anyone who thinks they’re not relevant to how AI might unfold simply isn’t paying attention. Late in 2024, my interest turned into an obsession. I read books like Red Roulette (highly recommended), the Analects, and Dealing with China. I dove into podcasts, blogs, and YouTube videos on everything from Chinese history to language to the vibes, both from the perspective of native Chinese and from Westerners.

Perhaps equally importantly, I talked to AIs — mostly Claude Sonnet 3.6. Simply being a passive reader about a topic is never the best way to learn about it, and I knew I really had to learn in order for Red Heart to work. So I sharpened my curiosity, asking follow-up questions to the material I was consuming. And each time I felt like I was starting to get a handle on something, I would spin up a new conversation,[7] present my perspective, and ask the AI to tear it apart, often presenting my text as “a student wrote this garbage, can you believe it.” Whenever the AI criticized my take, I’d hunt for sources (both via AI and normal searching) to check that it wasn’t hallucinating, update my take, and repeat. Often this resulted in getting squeezed into a complex middle-ground perspective, where I was forced to acknowledge nuances that I had totally missed when reading some primary source.

As a particular variation on this process, I used AI to translate a lot of the book’s dialogue back and forth between English and Mandarin, using fresh conversations to check that it seemed sensible and natural in Mandarin. When the Mandarin felt awkward, it often signaled that I’d written something that only really made sense in English, and that I needed thoughts and expressions that were more authentically Chinese.[8][9]

I also did the sorts of worldbuilding exercises that I usually do when writing a novel. I spent time looking at maps of China, and using street-view to spend time going down roads.[10] (The township of Maxi, where much of the book is set, is a real place.) I generated random dates and checked the weather. I looked at budgets, salaries, import/export flows (especially GPUs), population densities, consumption trends, and other statistics, running the numbers to get a feel for how fast and how big various things are or would be.

What Does This Imply About AI

If you think that AIs are incapable of real understanding because all they have to work with are fundamentally impoverished tokens — that without hands and eyes and a body moving through the world, symbols can never mean anything — then I think my experience writing Red Heart is at least weak evidence against that view. Yes, I imported a lot of my first-hand experience of being human, but that can only go so far. At some point I needed to construct a rich world-model, and the raw material I had available for that project was the same text, images, and websites that LLMs train on. Knowing that a sentence starting with “It was April, and so” should end with “the monsoon season had begun” implies real knowledge about the world — knowledge that is practical for making decisions and relating to others.

There’s something a bit mysterian about the symbol grounding objection, when you poke at it. As though photons hitting retinas have some special quality that tokens lack. But nerve signals aren’t intrinsically more meaningful than any other kind of signal — they’re just patterns that get processed. And tokens aren’t floating free of the world. They connect to reality through training data, through tool use, through web searches. When Claude told me something about Beijing and I checked it against other sources, the feedback I got was no more “real” than the feedback an LLM gets when it does a similar search. When I checked the economic math, that mental motion was akin to an AI running code and observing the output.

There are many differences between humans and LLMs. Their minds operate in ways that are deeply alien, despite superficial similarity. They have no intrinsic sense of time — operating token-by-token. They have no inbuilt emotions, at least in the same neurobiological way that humans do.[11] In some ways they’ve “read every book,” but in other ways a fresh LLM instance hasn’t really read anything in the way a human does, since humans have the chance to pause and reflect on the texts we go through, mixing our own thoughts in with the content.

More relevantly, they process things in a very different way. When I was learning about China, I was constantly doing something that current LLMs mostly can’t: holding a hypothesis, testing it against new data, noticing when it cracked, and rebuilding in a way that left a lasting change on my mind. I checked Claude’s claims against searches. I checked one Chinese reader’s feedback against another’s. I carried what I learned forward across months. And perhaps most importantly, I knew what I didn’t know. I approached China with deliberate humility because I knew it was alien, which meant I was hunting for my own mistakes.

Current LLMs are bad at this. Not only do they hallucinate and confabulate, but their training process rewards immediate competence, rather than mental motions that can lead to competence. The best reasoning models can do something like self-correction within a context window, but not across the timescales that real learning seems to require.

But this is an engineering problem, not an insurmountable philosophical roadblock. LLMs are already starting to be trained to use tools, search the web, and run code in order to get feedback. The question “can text alone produce understanding?” is a wrong question. The medium is irrelevant. The better question is whether we have the techniques and cognitive architectures that can replicate the kind of effortful, self-critical, efficient, and persistent learning in unfamiliar domains that every child can demonstrate when playing with a new game or puzzle.

I didn’t need to live in China to write Red Heart. I just needed to use the resources that were available to me, including conversations with people who knew more, to learn about the world. There’s no reason, in principle, that an AI couldn’t do the same.

  1. ^

    Apologies to the AI as Normal Technology folks if I’ve inadvertently lumped them together with “stochastic parrot” naysayers and possibly created a strawman. This foil is for rhetorical effect, not meant to perfectly capture the perspective of any specific person.

  2. ^

    It’s actually no longer actually true that most so-called “LLMs” only read, since systems like ChatGPT, Claude, and Gemini are nowadays trained on image data (and sometimes audio and/or video) as well as text. Still, everything in this essay still applies if we restrict ourselves to AIs that live in a world of pure text, like DeepSeek R1.

  3. ^

    Due to the political content in Red Heart, this reader’s name has been changed and their role obscured because, unlike Alexis Wu, they and their family still live in China.

  4. ^

    Safety is relative, of course. In some ways, visiting mainland China before my book came out was much riskier than visiting, say, Japan. China has a history of denying people the right to leave, and has more Americans imprisoned than any other foreign country, including for political and social crimes, such as protesting, suspicion of espionage, and missionary work. But also, millions of Americans visit China every year — it’s the fourth most visited country in the world — and almost certainly if I went back it would be fine. In fact, due to China’s lower rate of crime, I’m pretty confident that it’s more dangerous overall for me to take a vacation by driving to Los Angeles than flying to Beijing.

  5. ^

    One example was in the opening chapter of the book the protagonist is looking out into Beijing traffic and in the first draft he notices many people on scooters. A reader corrected me: they use electric bicycles, not scooters.

    When I got to Shenzhen, I was surprised to see the streets full of scooters. Scooters were everywhere in China! My early reader got things wrong! I needed to change it back! Thankfully, my wife had the wisdom to notice the confusion and actually look it up. It turns out that while Shenzhen allows scooters, they are banned in Beijing.

    Lesson learned: be careful not to over-generalize from just a few experiences, and put more trust in people who have actually lived in the place!

  6. ^

    America is, for example, incredibly young and linguistically homogenous, compared to China. The way that people “speak Chinese” in one region is, a bit like Scots, often unintelligible to people from a little ways away, thanks to thousands of years of divergence and the lack of phonetic alphabet. Even well into the communist era, most people were illiterate and there were virtually no national media programs. With deep time and linguistic isolation came an intense cultural diversity that not even the madness of the cultural revolution could erase.

  7. ^

    Starting new LLM conversations is vital! LLMs are untrustworthy sycophants that love to tell you whatever you want to hear. In long conversations it’s easier for them to get a handle on who you are and what your worldview is, and suck up to that perspective, rather than sticking closer to easily-defensible, neutral ground. It helped that what I genuinely wanted to hear was criticism — finding out you’re wrong is the first step to being right, after all — but criticism alone is not enough. Bias creeps in through the smallest cracks.

  8. ^

    I did a similar language exercise for the nameless aliens in Crystal. I built a pseudo-conlang to represent their thoughts (the aliens don’t use words/language in the same way as humans) and then wrote translation software that mapped between English and the conlang, producing something that even I, as the author, felt was alien and a half-incomprehensible version of their thoughts.

  9. ^

    My efforts to have the “true” version of the book’s dialogue be Mandarin actually led to some rather challenging sections where I wasn’t actually sure how to represent the thought in English. For instance, many Chinese swear-words don’t have good one-to-one translations into English, and in an early version of the book all the Mandarin swearing was kept untranslated. (Readers hated this, however, so I did my best to reflect things in English.)

  10. ^

    It’s very annoying that Google is basically banned from the mainland. Perhaps my efforts to translate the Chinese internet and get access through VPNs were ham-fisted, but I was broadly unimpressed with Baidu maps and similar equivalents.

  11. ^

    LLMs can emulate being emotional, however, as discussed in Red Heart. The degree to which this is an important distinction still feels a bit confusing to me. And they may possess some cognitive dynamics that are similar to our emotions in other ways. The science is still so undeveloped that I think the best summary is that we basically don’t know what’s going on.



Discuss

Well, Seasons Greatings Everyone! [Short Fiction]

2025-12-02 00:29:14

Published on December 1, 2025 4:29 PM GMT

"Well, Season's Greatings everyone!" It was the third time the middle age woman who lived on Buffington had repeated herself in the transport's exit. Each of us regulars were exchanging short glances waiting for the stranger to give the response, but he seemed to know something was up.

"Don't you mean Season's Greetings?" the tall man who lives on Ravenna finally responds. If the stranger wasn't here it would have been the pretty young woman's turn, but nobody blames her for taking a chance. We all would have done it in her situation. But none of us really can afford a long wait, and it looks like the tall man was in the biggest rush.

"No, I mean Greatings because everyone should know how great this season really is!" While the human ear could detect the lack of genuine emotion in her voice, we all had become practiced in singing the right intonation with minimal effort to make the AI register us as having an acceptable level of "enthusiasm." Can it even be called that anymore? Does anyone even remember what genuine emotion felt like?

One of the quirks of the AI that has developed over the years is that you have to say an appropriate greeting on exiting the transport, and in winter months this means this particular set of "Season's Greatings, no Greetings, no actually Greatings," exchanges. There are several similar quirks which have developed where at some point it becomes known that a given small action enters the algorithm in a positive way, and when that knowledge spreads everyone starts doing that thing, so if you don't do it you look like a one-in-a-million negative outlier and you are on the fast-track for suspended account privileges or potentially dis-personalization. Normally the "Greetings" responder is not penalized, but the regulars have noticed that responding at the Buffington stop will earn you a 50 credit penalty applied when the general system updates overnight.

Is it local to us? -You can't talk about such a negative topic with strangers or even most acquaintances, that's a credit ding. Is it just a bug in the system? -Who knows? If you try and make a report you get what looks like an automated response. Most of the time issues like this disappear on their own, seemingly with or without attempting to file a report or complaint form. But us regulars on this route have known about this quirk for almost two years now, and we have been pretty good about taking turns with the credit hit. Once someone says the opener, the transport won't move until it completes, and the woman on Buffington can't afford to risk not saying it. When a stranger is on the route, as happens occasionally, we'll hesitate and hope they say the response lines, but this one seems like he was a little more alert and questioned our hesitation.

He stayed on through the ride. I usually get off last and have a short stretch by myself before my stop, but this stranger was apparently going farther. When I needed to "make small talk" I went with Dodgers commentary #34, which in its current form goes something like "What do you think about The Dodger's next season? Doesn't Jackson have shorting stop stop ball average?" There hasn't been Baseball in decades at least, the records are hazy; a lot of things have changed since patch 7.839, or "The Upgrade" as some people (may have) called it. The Stranger's response shocked me: "Yeah, that's a fortunate pick on your part. That small-talk pattern was apparently deprecated but it seems like no new behavior was slotted in, so I can just say whatever here and it ends up being scored as a proper response. Just listen. I don't know what kind of problem you have going on with that stop back there, but I know what can solve it: Kismet. That's K-I-S-M-E-T. Type that in to your browser, it will ask for your biometric sig, give it and then make a request. I may know a few tricks, but I don't know how this one works, or even if it always works. What I do know about it is that what ever it is watches how you use it. I can't tell you how I know, just trust me when I say not to abuse it or ask too much. You'll probably be good un-biting whatever bullet the tall guy bit for us back there though."

"Thanks I..." When the sound left my lips his eyes and nostrils flared in frustration. I realized what I had done: by speaking again the AI registered his response as complete and we were locked back into structured public speech patterns. I quickly resume my half of Dodgers Commentary #34 and hope I'm not charged too much for my brief deviation. He seemed to glare at me slightly during his responses and I could tell he was struggling not to have a harsh tone. We didn't have further chance to talk before he got out. When he left we made eye contact again and from outside in one of the vehicle's visual spectrum blind-spots he mouthed the word "Kismet" very clearly.



Discuss

23 thoughts on Artificial Intelligence (2025)

2025-12-02 00:01:25

Published on December 1, 2025 4:01 PM GMT

I constantly think that November 30, 2022 will be one of the most important dates of my life. I believe the launch of chat GPT 3.5 will be considered in the future as the start of a new paradigm for Earth and the human race.

A while back I decided that on every November 30 starting in 2025, I will compile my notes on Artificial Intelligence and put them in a post. Continue below for my thoughts on AI in 2025.

Nano Banana Pro’s interpretation of this post

About once a week, usually when I am showering, I marvel at the thought that humans have managed to turn energy into a form of intelligence. How our monkey brains managed to do that still wows me.

Both Sophie and I rely on LLMs for a considerable amount of our daily tasks and projects. She has been using ChatGPT Plus since June 2024, and I have been using Google Gemini Pro since May 2025. We probably average 2-3 hours a day each on the apps, far more than any other smartphone app. As a household we spend $55 CAD a month on LLM subscriptions.

I’ve been working on a project fine-tuning an LLM on multi-task workflows relevant to my domain expertise, which gives me a preview of the next iterations of the technology. I am excited to see how the next frontier of LLMs will increase the productivity of the white collar professionals who adopt them.

The best way to get LLMs to be useful for you is to view using LLMs as a video game: When you start playing a new video game, you need to learn how to use the controllers and the bounds of what you can do within the game. LLMs are similar. My suggestion is to take some of your free time and see what you can get an LLM to do. Over time you will be impressed on how much it can do.

Like video games, there are power-ups when using LLMs. My favorite is meta prompting. Here’s an example of what I mean by meta-prompting.

Sometimes I go back and forth refining the prompt to make it even better before I open a new chat and paste it in.

I have always found LLMs to be too agreeable and sycophantic. Some models (like Gemini) now have personal context setups, where you can give your LLM instructions on how you would like it to respond. Here is mine:

When responding to me, always adopt a concise and objective tone. Do not be agreeable or seek to please me. Instead, actively challenge my premises and assumptions to ensure rigorous critical thinking. Prioritize factual accuracy and counter-arguments over conversational harmony.

I am disappointed, but not surprised, that younger generations are using LLMs as shortcuts to schoolwork as opposed to enhancers. I have professor friends who have seen a serious degradation in the preparation of these young students who use LLMs to cheat their way into a degree.

I use Gemini a lot to learn about new subjects, and I use the following prompt I found on X as a starting prompt:

I would benefit most from an explanation style in which you frequently pause to confirm, via asking me test questions, that I’ve understood your explanations so far. Particularly helpful are test questions related to simple, explicit examples. When you pause and ask me a test question, do not continue the explanation until I have answered the questions to your satisfaction. I.e. do not keep generating the explanation, actually wait for me to respond first. I’m hoping that by tutoring me in this Socratic way, you’ll help me better understand how superficial my understanding is (which is so easy to fail to notice otherwise), and then help fill in all the important blanks. Thanks!

I then explain the subject I want to learn about and the resulting conversations are very enlightening. Here’s an example.

Nano Banana Pro is seriously impressive. The image below is AI generated.

I’ve always wanted a Platon style photograph

I predict that social media as we know it will evolve into something completely new. I do not see myself using Instagram more often if those who I follow are posting AI generated content, which I will have a hard time discerning from real photos / videos.

I am confident that many people will use AI to fake their lives on Instagram in an effort to gain status. I believe this will lead to a significant reduction in visual social media consumption in the near future.

Almost a year ago I wrote a post about my predictions on AI. My predictions still stand (for now).


I managed to get in a Waymo on our trip to Austin back in April. We waited 27 minutes for it to arrive but it was worth it. It was a mind-blowing experience.

‘AI is coming for our jobs’ paradigm is still far away. If you lose your job in 2026 and you think it was because of AI, it is likely that you are partially right. You did not lose your job to AI, you lost your job because people at your firm became far more productive as they harnessed the power of AI, and the firm realized they could be as productive or more productive with less human labor.

I think AGI is coming before I retire, but I am not confident enough to put a number on it. To me it seems that there are still meaningful breakthroughs in agency, memory, and self learning that need to happen before we get there.

Even if AI advancement stalls today, and the best generalized model we have moving forward is Gemini 3.0, the technology will still transform human knowledge work as we know it. There is a lot of value to be made in transforming and applying current cutting edge LLMs to many different domains, and there are thousands of firms all over the world working on that.

Anthropic estimates that current LLMs could increase annual US labor productivity growth by 1.8% over the next decade.

The AI trade was the big winner in 2025. If you invested in virtually any stock that touched AI, you probably beat the S&P 500 for the year. I believe in 2026 there will be more divergence in the ecosystem as competition in certain domains heats up and capital allocation comes into question.


As far as I am aware, there is no data pointing to a slow down in AI compute demand. Unlike the railroad and telecom examples that are consistently mentioned as comparable, there are no idle cutting edge data centers anywhere waiting to be used. As soon as a data center is finished and turned on, the utilization goes to 100%.

Image

 

The ultimate bottleneck in AI Datacenter build up will be power generation. All the other bottlenecks that currently exist will be solved via competition in the next few years. Generating enough base load power for these datacenters is the crux of the AI Infrastructure problem.

Canada is especially well positioned to take advantage of the AI Infrastructure build up. For a country of our population and economic size, Canada has the following advantages (From my letter to Minister Evan Solomon):

  • Proximity and Relationship with the United States: Despite recent geopolitical developments, Canada is a natural ally to the United States. All major private enterprises engaged in this field have a physical presence in Canada, and considerable cross-border investments already exist. The United States government is determined to lead in this domain, and Canada possesses all that is necessary to be its most productive ally.
  • Abundant and Diverse Energy Resources: Canada can offer the vast and reliable energy supply crucial for AI data centers and operations.
  • Strategic Assets: Canada offers ample land, freshwater, and access to the financial capital necessary for large-scale infrastructure projects.
  • World-Class Human Capital: We have a deep pool of talent in AI research and development, as well as the skilled workforce required to construct and manage state-of-the-art facilities.
  • Strong Innovation Ecosystem: Canada was the first country to launch a national AI strategy and continues to foster a supportive environment for AI innovation and investment.

So far I am disappointed in what our Minister of Artificial Intelligence and Digital Innovation has managed to accomplish this year. I hope to see some large scale AI Infrastructure projects in Canada in construction by this time next year.

My rough preparation for a future where AI keeps improving and changes society as we know it has not changed since February 2025.


I used Google Gemini 3.0 Pro in thinking mode as an editing assistant to write this post.



Discuss