Alpha

super_chat_20260225143238_sa7hactivechat17.44M params25m 39s elapsed · ~215h 41m remaining

8L / 384D / 8H · helios · bpe-4k · adamw· Created Feb 25, 2026 2:33 PM

Step 208 / 50,0000.4%

7.9278

Loss?

7.9004

Best Loss?

-5.3% from start

Val Loss?

1.44e-5

Learning Rate?

394

Throughput?

tok/s (avg)

15,595

Speed?

ms/iter (avg)

1.280

Grad Norm?

avg: 1.958

1.14M

Tokens

processed

7289ms

Forward

47% of step

8241ms

Backward

53% of step

9ms

GPU Sync

0% of step

887

GPU Ops

per step

0.1%

MFU

model FLOPS util

1.1x

Bwd/Fwd

ratio

Loss Curve ?

Symbio semantics: this chart stitches many fresh candidate evaluations onto one global step axis. Loss resets near switches are expected because candidates are re-initialized. Compare local candidate shapes and the global frontier, not a single continuous model trajectory.

Search semantics

Validation / selection

Run diagnostics

Search Trajectory + Frontier

Candidate-local train/val loss segments on a shared step axis, with switch events and global frontier overlays.

Search-aware view

Architecture

Layers?8

Embedding?384

Heads?8

Vocab?4,000

Context?512

Dropout?0.1

Parameters?17.44M

Training Config

Total iters?50,000

Batch size?12

Max LR?0.00005

Optimizer?adamw

Backend?helios

Tokenizer?bpe-4k

Seed?42

Weight decay?0.1

Grad clip?1

Eval interval?1000

GPU & VRAM

Learning Rate

Grad Norm

Step Time Breakdown

Clip Telemetry

SymbiogenesisSWIGLU

1.78

Wt Entropy

bits

20.0

Eff. Rank

7.9952

Free Energy

3.903

Pop Entropy

nats

0.0747

Complexity

0.0367

Fitness

155

CUSUM Alerts

of 186 steps

Batch Size

adaptive

CUSUM Change-Point Monitor

Weight Entropy

Effective Rank

Free Energy

Fitness Score

Population Entropy

Adaptive Batch Size

Phase Change / Gelation

Current

Transitioning

Stability

Phase Changes

Regime Shifts

Training dynamics are shifting. The model may be entering a new loss basin or the learning rate is hitting a critical threshold. This often happens before a breakthrough or a plateau.

Phase Timeline

Step 1Step 166

Loss Oscillation (Harmonic Analysis)

Evolutionary Search

Generations

Candidates

Activations

Best Loss

7.9004

Total Steps

186

#	Candidate	Activation	Gen	Loss	Fitness	Steps	Mutation
1	gen0_universal_5	universal	0	7.9004	0.0340	24	origin
2	gen0_kan_spline_6	kan_spline	0	7.9111	0.0369	16	origin
3	gen1_universal_7	universal	1	7.9278	0.0367	26	origin
4	gen0_relu_3	relu	0	7.9298	0.0369	30	origin
5	gen0_swiglu_4	swiglu	0	8.0675	0.0343	30	origin
6	gen0_gelu_1	gelu	0	8.1062	0.0341	30	origin
7	gen0_silu_2	silu	0	8.1295	0.0349	30	origin

Generation Summary

G06c7.9004

G11c7.9278

Fitness Progression

Architecture Diversity

Convergence vs Diversity (Tug-of-War)

Current Mode

Exploration Dominant

Diversity Pressure

59%

Convergence Momentum

Convergence Progress

100%

Phase Portrait: Diversity Pressure vs Convergence Momentum

Low diversity / high momentum = lock-in convergence

High diversity / high momentum = productive exploration

Low diversity / low momentum = stalled collapse

High diversity / low momentum = diversity stalling convergence

Tug-of-War Trace (Time Domain)

Positive tension means recent frontier improvement is outpacing diversity pressure (search is converging). Negative tension means exploration pressure is dominating recent convergence momentum (search is broadening or getting “stumped”).

Strongest Convergence

step 9

tension 0.200

Strongest Diversity Push

step 69

tension -0.920

Best Frontier

7.9004

progress 100%

Evolutionary Lineage Tree

Lineage Tree

100%

Activation Flow (Sankey)

Activation Switch Log

Step	From		To	Gen	Prev Steps	Best Loss	Final Loss	Fitness
1	-	→	gelu	0	-	-	-	-
31	gelu	→	silu	0	30	8.1062	8.1110	0.0341
61	silu	→	relu	0	30	8.1295	8.1295	0.0349
91	relu	→	swiglu	0	30	7.9298	7.9318	0.0369
121	swiglu	→	universal	0	30	8.0675	8.0675	0.0343
152	universal	→	kan_spline	0	24	7.9004	7.9149	0.0340
181	kan_spline	→	universal	1	16	7.9111	7.9111	0.0369

Search Candidates

#	Name	Activation	Gen	Parent	Steps	Best Loss	Best Val	Avg Loss	Fitness	Avg tok/s	Alerts
1	gen0_universal_5	universal	0	-	24	7.9004	-	8.1068	0.0340	394	24
2	gen0_kan_spline_6	kan_spline	0	-	16	7.9111	-	8.1146	0.0369	180	16
3	gen1_universal_7	universal	1	-	26	7.9278	-	8.1159	0.0367	394	26
4	gen0_relu_3	relu	0	-	30	7.9298	-	8.0959	0.0369	7,230	30
5	gen0_swiglu_4	swiglu	0	-	30	8.0675	-	8.2667	0.0343	2,234	19
6	gen0_gelu_1	gelu	0	-	30	8.1062	-	8.2366	0.0341	7,313	22
7	gen0_silu_2	silu	0	-	30	8.1295	-	8.2695	0.0349	2,329	18

Activation Distribution

universal

50 (27%)

gelu

30 (16%)

silu

30 (16%)

relu

30 (16%)

swiglu

30 (16%)

kan_spline

16 (9%)

Oscillation & Heat Capacity

Activation Evolution Radial

Symbio Config

{
  "cusumSensitivity": 4,
  "cusumBaselineWindow": 5,
  "metricsInterval": 10,
  "trackWeightEntropy": true,
  "trackEffectiveRank": true,
  "trackFreeEnergy": true,
  "trackMIProfiles": true,
  "trackPopulationMetrics": true,
  "freeEnergyBeta": 0.01,
  "miNumBins": 30,
  "adaptiveBatch": true,
  "batchMin": 8,
  "batchMax": 64,
  "batchStep": 4,
  "calmStepsBeforeRestore": 200,
  "fitnessAlpha": 1,
  "complexityMode": "entropy",
  "diversityBonus": 0.05,
  "diversityDecay": "cosine",
  "searchMode": "ffn-activation-search",
  "activationPool": [
    "gelu",
    "silu",
    "relu",
    "swiglu",
    "universal",
    "kan_spline"
  ],
  "searchStrategy": "evolutionary",
  "populationSize": 6,
  "generations": 12,
  "selectionStrategy": "topk",
  "tournamentK": 3,
  "mutationRate": 0.5,
  "stepsPerCandidate": 30,
  "rankBy": "valLoss",
  "perfWeight": 0,
  "stabilityWeight": 0,
  "writeReport": true,
  "writeCandidates": true,
  "writeSummary": true
}

Checkpoints (0) ?

No checkpoints saved

{
  "vocabSize": 4000,
  "blockSize": 512,
  "nLayer": 8,
  "nEmbd": 384,
  "nHead": 8,
  "dropout": 0.1,
  "ffnActivation": "swiglu",
  "ffnDim": 1024
}

{
  "iters": 50000,
  "batchSize": 12,
  "lr": 0.00005,
  "lrMin": 0.000005,
  "warmupIters": 1000,
  "beta1": 0.9,
  "beta2": 0.95,
  "eps": 0.000001,
  "weightDecay": 0.1,
  "gradClip": 1,
  "evalInterval": 1000,
  "evalIters": 10,
  "seed": 42,
  "backend": "helios",
  "tokenizer": "bpe-4k",
  "optimizer": "adamw",
  "logLevel": "info",
  "trace": false,
  "gradAccumSteps": 1,
  "sampleInterval": 500,
  "spikeThreshold": 10,
  "syncEvery": 1,
  "gcEvery": 0,
  "packed": false,
  "symbio": true,
  "symbioConfig": {
    "cusumSensitivity": 4,
    "cusumBaselineWindow": 5,
    "metricsInterval": 10,
    "trackWeightEntropy": true,
    "trackEffectiveRank": true,
    "trackFreeEnergy": true,
    "trackMIProfiles": true,
    "trackPopulationMetrics": true,
    "freeEnergyBeta": 0.01,
    "miNumBins": 30,
    "adaptiveBatch": true,
    "batchMin": 8,
    "batchMax": 64,
    "batchStep": 4,
    "calmStepsBeforeRestore": 200,
    "fitnessAlpha": 1,
    "complexityMode": "entropy",
    "diversityBonus": 0.05,
    "diversityDecay": "cosine",
    "searchMode": "ffn-activation-search",
    "activationPool": [
      "gelu",
      "silu",
      "relu",
      "swiglu",
      "universal",
      "kan_spline"
    ],
    "searchStrategy": "evolutionary",
    "populationSize": 6,
    "generations": 12,
    "selectionStrategy": "topk",
    "tournamentK": 3,
    "mutationRate": 0.5,
    "stepsPerCandidate": 30,
    "rankBy": "valLoss",
    "perfWeight": 0,
    "stabilityWeight": 0,
    "writeReport": true,
    "writeCandidates": true,
    "writeSummary": true
  }
}