A
Alpha
novels_all_20260225152704_30v8activenovels15.90M params1m 33s elapsed · ~16h 9m remaining
8L / 384D / 8H · helios · bpe · adamw· Created Feb 25, 2026 3:27 PM
Step 90 / 50,0000.2%
6.7429
Loss?
6.7429
Best Loss?
-9.3% from start
-
Val Loss?
7.86e-5
Learning Rate?
7,063
Throughput?
tok/s (avg)
1,165
Speed?
ms/iter (avg)
0.734
Grad Norm?
avg: 0.989
655.4K
Tokens
processed
195ms
Forward
17% of step
922ms
Backward
79% of step
12ms
GPU Sync
1% of step
730
GPU Ops
per step
2.2%
MFU
model FLOPS util
4.7x
Bwd/Fwd
ratio
Loss Curve ?
Symbio semantics: this chart stitches many fresh candidate evaluations onto one global step axis. Loss resets near switches are expected because candidates are re-initialized. Compare local candidate shapes and the global frontier, not a single continuous model trajectory.
Search semantics
Validation / selection
Run diagnostics
Search Trajectory + Frontier
Candidate-local train/val loss segments on a shared step axis, with switch events and global frontier overlays.
Search-aware view
Architecture
Layers?8
Embedding?384
Heads?8
Vocab?2,000
Context?512
Dropout?0
Parameters?15.90M
Training Config
Total iters?50,000
Batch size?16
Max LR?0.0003
Optimizer?adamw
Backend?helios
Tokenizer?bpe
Seed?42
Weight decay?0.1
Grad clip?5
Eval interval?500
GPU & VRAM
Learning Rate
Grad Norm
Step Time Breakdown
Step Time Breakdown
Clip Telemetry
SymbiogenesisSWIGLU
1.74
Wt Entropy
bits
20.0
Eff. Rank
6.7603
Free Energy
3.904
Pop Entropy
nats
0.0739
Complexity
0.0552
Fitness
76
CUSUM Alerts
of 80 steps
12
Batch Size
adaptive
CUSUM Change-Point Monitor
Weight Entropy
Effective Rank
Free Energy
Fitness Score
Population Entropy
Adaptive Batch Size
Phase Change / Gelation
Current
Regime Shift
Stability
0%
Phase Changes
2
Regime Shifts
1
Major training disruption. Multiple monitors triggered simultaneously — gradient behavior, clipping, and/or throughput all changed. This can mean the model has found a new loss landscape, or training is becoming unstable. Check learning rate and gradient norms.
Phase Timeline
Step 11Step 61
Loss Oscillation (Harmonic Analysis)
Evolutionary Search
Generations
1
Candidates
1
Activations
1
Best Loss
6.7429
Total Steps
80
#CandidateActivationGenLossFitnessStepsMutation
1G-Alphagelu06.74290.055280origin
Generation Summary
G01c6.7429
Architecture Diversity
Convergence vs Diversity (Tug-of-War)
Current Mode
Converging
Diversity Pressure
80%
Convergence Momentum
100%
Convergence Progress
100%
Phase Portrait: Diversity Pressure vs Convergence Momentum
Low diversity / high momentum = lock-in convergence
High diversity / high momentum = productive exploration
Low diversity / low momentum = stalled collapse
High diversity / low momentum = diversity stalling convergence
Tug-of-War Trace (Time Domain)
Positive tension means recent frontier improvement is outpacing diversity pressure (search is converging). Negative tension means exploration pressure is dominating recent convergence momentum (search is broadening or getting “stumped”).
Strongest Convergence
step 18
tension 0.200
Strongest Diversity Push
step 11
tension -0.800
Best Frontier
6.7429
progress 100%
Evolutionary Lineage Tree
Lineage Tree
100%
Activation Flow (Sankey)
Activation Switch Log
StepFromToGenPrev StepsBest LossFinal LossFitnessTree
11-→gelu0----
Search Candidates
#NameActivationGenParentStepsBest LossBest ValAvg LossFitnessAvg tok/sAlerts
1G-Alphagelu0-806.7429-7.07320.05527,07276
Activation Distribution
gelu
80 (100%)
Oscillation & Heat Capacity
Activation Evolution Radial
Symbio Config
{
  "cusumSensitivity": 4,
  "cusumBaselineWindow": 5,
  "metricsInterval": 10,
  "trackWeightEntropy": true,
  "trackEffectiveRank": true,
  "trackFreeEnergy": true,
  "trackMIProfiles": true,
  "trackPopulationMetrics": true,
  "freeEnergyBeta": 0.01,
  "miNumBins": 30,
  "adaptiveBatch": true,
  "batchMin": 8,
  "batchMax": 64,
  "batchStep": 4,
  "calmStepsBeforeRestore": 200,
  "fitnessAlpha": 1,
  "complexityMode": "entropy",
  "diversityBonus": 0.08,
  "diversityDecay": "cosine",
  "searchMode": "ffn-activation-search",
  "activationPool": [
    "gelu",
    "silu",
    "relu",
    "swiglu",
    "universal",
    "kan_spline"
  ],
  "searchStrategy": "evolutionary",
  "populationSize": 6,
  "generations": 16,
  "selectionStrategy": "topk",
  "tournamentK": 3,
  "mutationRate": 0.6,
  "stepsPerCandidate": 500,
  "rankBy": "valLoss",
  "perfWeight": 0,
  "stabilityWeight": 0,
  "writeReport": true,
  "writeCandidates": true,
  "writeSummary": true
}
Checkpoints (0) ?
No checkpoints saved
{
  "vocabSize": 2000,
  "blockSize": 512,
  "nLayer": 8,
  "nEmbd": 384,
  "nHead": 8,
  "dropout": 0,
  "ffnActivation": "swiglu",
  "ffnDim": 1024
}
{
  "iters": 50000,
  "batchSize": 16,
  "lr": 0.0003,
  "lrMin": 0,
  "warmupIters": 500,
  "beta1": 0.9,
  "beta2": 0.95,
  "eps": 1e-8,
  "weightDecay": 0.1,
  "gradClip": 5,
  "evalInterval": 500,
  "evalIters": 10,
  "seed": 42,
  "backend": "helios",
  "tokenizer": "bpe",
  "optimizer": "adamw",
  "logLevel": "info",
  "trace": false,
  "gradAccumSteps": 1,
  "sampleInterval": 100,
  "spikeThreshold": 10,
  "syncEvery": 1,
  "gcEvery": 0,
  "packed": false,
  "symbio": true,
  "symbioConfig": {
    "cusumSensitivity": 4,
    "cusumBaselineWindow": 5,
    "metricsInterval": 10,
    "trackWeightEntropy": true,
    "trackEffectiveRank": true,
    "trackFreeEnergy": true,
    "trackMIProfiles": true,
    "trackPopulationMetrics": true,
    "freeEnergyBeta": 0.01,
    "miNumBins": 30,
    "adaptiveBatch": true,
    "batchMin": 8,
    "batchMax": 64,
    "batchStep": 4,
    "calmStepsBeforeRestore": 200,
    "fitnessAlpha": 1,
    "complexityMode": "entropy",
    "diversityBonus": 0.08,
    "diversityDecay": "cosine",
    "searchMode": "ffn-activation-search",
    "activationPool": [
      "gelu",
      "silu",
      "relu",
      "swiglu",
      "universal",
      "kan_spline"
    ],
    "searchStrategy": "evolutionary",
    "populationSize": 6,
    "generations": 16,
    "selectionStrategy": "topk",
    "tournamentK": 3,
    "mutationRate": 0.6,
    "stepsPerCandidate": 500,
    "rankBy": "valLoss",
    "perfWeight": 0,
    "stabilityWeight": 0,
    "writeReport": true,
    "writeCandidates": true,
    "writeSummary": true
  }
}