concordance_v2_20260227051643_7hv1completedconcordance306.73M params1m 13s elapsed · Updated 45d ago
21L / 1024D / 16H · helios · bpe-64k · adamw· Created Feb 27, 2026 5:16 AM
Step 5 / 5100.0%
9.4053
Loss?
9.4053
Best Loss?
-7.0% from start
-
Val Loss?
6.95e-5
Learning Rate?
139
Throughput?
tok/s (avg)
14,791
Speed?
ms/iter (avg)
4.614
Grad Norm?
avg: 6.916
10.2K
Tokens
processed
1625ms
Forward
11% of step
12370ms
Backward
84% of step
78ms
GPU Sync
1% of step
9,203
GPU Ops
per step
0.2%
MFU
model FLOPS util
7.6x
Bwd/Fwd
ratio
Loss Curve ? click any chart to add markers
?
?
?
?
Architecture
Layers?21
Embedding?1024
Heads?16
Vocab?19,777
Context?512
Dropout?0
Parameters?306.73M
Training Config
Total iters?5
Batch size?1
Max LR?0.0003
Optimizer?adamw
Backend?helios
Tokenizer?bpe-64k
Seed?42
Weight decay?0.1
Grad clip?1
Eval interval?100
Throughput (tok/s)
Step Time (ms/iter)
GPU & VRAM
Perplexity
Train/Val Gap
No validation data
Learning Rate
Grad Norm
Smoothed Loss (EMA)
Loss Velocity
Insufficient data
Gradient Clipping
GPU Operations
Step Time Breakdown
Forward
Backward
Grad Norm
Optimizer
GPU Sync
Data
Timing Phase Lines
Backward / Forward Ratio
Transformer Layer Analysis
Gradient Norm Heatmap
Per-Layer Gradient Evolution
Checkpoints (0) ?
No checkpoints saved
Sample Generations (3)
#CheckpointPrompt (preview)Generated
1-The 45d ago
Prompt
The
Output
The Wiki gardenIllinois hazilia"]={"3/0147Illinois -e, 4697"},["certificprov說o 057sprov說sakgelsreward449ia"]={"3/0147r449gelus"]={"1/019765arozniar辦(ll. dia"]={"3/0147eggC057ssm095saksak449ia"]={"3/0147øhotel room -D w:an虞ingenious locgel
2-When in the Course of human events45d ago
Prompt
When in the Course of human events
Output
When in the Course of human eventsshareggeggConviction_Politpow.d sakkibunti貞057for his Illinois ing 4697"},["certific辦anticip4697"},["certificnsIllinois ing 449Drem3"},["comp.quialterate "},["circumlustr4697"},["certific4697"},["certificILProtective Devop449-Illinois locCaim4697"},["certificnot be 4697"},["certificProtective Devsing Illinois Iguaçu ,
3-It was a dark and stormy45d ago
Prompt
It was a dark and stormy
Output
It was a dark and stormy ia"]={"3/0147us"]={"1/019765辦k辦002ia"]={"3/0147for his -ince s, and arozniafor his 譜ksen (saksak 辞rewardeggs lュquialterate atus"]={"0/0169originreward arily alIllinois ocals貞貞Protective Devatastro"]={"1/0204697"},["certificC449gave ollateral consequence fürchtet.adeo"]={"2/012990
Model Config (JSON)
{
"vocabSize": 19777,
"blockSize": 512,
"nLayer": 21,
"nEmbd": 1024,
"nHead": 16,
"dropout": 0,
"ffnActivation": "swiglu"
}Training Config (JSON)
{
"iters": 5,
"batchSize": 1,
"lr": 0.0003,
"lrMin": 0,
"warmupIters": 0,
"beta1": 0.9,
"beta2": 0.95,
"eps": 1e-8,
"weightDecay": 0.1,
"gradClip": 1,
"evalInterval": 100,
"evalIters": 10,
"seed": 42,
"backend": "helios",
"tokenizer": "bpe-64k",
"optimizer": "adamw",
"logLevel": "info",
"trace": false,
"gradAccumSteps": 4,
"sampleInterval": 100,
"spikeThreshold": 0,
"syncEvery": 1,
"gcEvery": 1,
"packed": false,
"symbio": false,
"symbioConfig": null
}