Neural Networks and Brain Scaling

Seed idea. Sparked by Children of Time (1.2) — Portia’s ~60k neurons vs a human’s ~100B.

The hook

The book frames intelligence as a function of scale. That’s essentially a scaling law. Worth exploring how capability tracks “brain size” (neurons / parameters).

Threads to grow

• Miniature scaling-law demo: capability vs neuron/param count.
• Where do diminishing returns kick in? Where do new abilities emerge?
• Biological vs artificial neuron counts — what the comparison does and doesn’t capture.
• Link to Genetic Algorithms and Benevolent Viruses via neuroevolution.

Side Project / Prompt Seed

If I hand this file to Claude: let’s build something. Start by digging up relevant research, then prototype a proof of concept in code.

Premise: capability as a function of “brain size.” Build a miniature scaling-law experiment.

What to build (POC): train a series of small MLPs at increasing width/depth on one fixed task (MNIST or a synthetic function). Sweep param/neuron count — e.g. start near Portia’s ~60k and scale up. Plot capability (accuracy/loss) vs size. Look for the diminishing-returns curve and any sharp “emergent” jumps.

Research to pull first:

• Neural scaling laws (Kaplan et al. 2020; Hoffmann/Chinchilla 2022).
• Emergent abilities of large models — and the “are they real or a metric artifact?” debate.
• Biological neuron counts across species vs. behavioral complexity.

Stretch: vary architecture (not just size) at fixed param budget — show capability isn’t only raw count. Bridges to Emergent and Collective Intelligence.