⚠️ Disclaimer — this is Phase 1 of a multi-phase project. The codebase represents the foundational scaffold only. All infrastructure (self-training, research agent, self-modification loop, self-critic) is built but untested under real autonomous conditions. The 340M model has not been trained.sum_mod3is stuck at 32.7% on the 4.92M baseline. Do not take the current results as representative of the system's eventual capability — they are a working proof-of-concept for the architecture, not a finished system.
Standard models are trained for one purpose. To handle a new task, you either fine-tune (expensive, can forget old tasks) or maintain separate models (wasteful). The core idea here is different: make a single model that reconfigures its own weights based on what it's being asked to do.
A hypernetwork is a neural network that outputs the weights of another network. Rather than having fixed parameters W, the model has a function H(t) that generates task-specific weights:
W_task = H(task_description)
y = f(x, W_task)
This is powerful because the same base network can solve fundamentally different tasks — not by learning a shared representation, but by generating different weight configurations per task. The hypernetwork learns how to reconfigure, not what to store.
Generating full weight matrices from scratch is expensive and hard to train. Instead, the hypernetwork generates deltas (ΔW) that modify a frozen base model:
W_final = W_base + α · ΔW
The base model knows how to process sequences (via self-attention and FFN). The hypernetwork only needs to learn how to nudge those weights toward the right computation for each task. This keeps the model stable — the base stays general — while the deltas provide specialization.
The scalar α (learned per layer) controls how strongly to apply each delta. This is the Scope Predictor: it decides how task-specific a given layer needs to be.
Naive delta injection tends to either overfit (deltas too large) or have no effect (deltas too small). The TaskAdapter bridges this with a low-rank bottleneck:
ΔW = U · V^T where U ∈ R^{H×r}, V ∈ R^{H×r}, r << H
The hypernetwork generates U and V (rank-r matrices), and their product produces the delta. This constrains the delta to a low-dimensional subspace — enough to meaningfully modify behavior without destabilizing the base. It's the same principle as LoRA, but generated dynamically at inference time, conditioned on the task description rather than on a static adapter.
Training the hypernet doesn't end when loss converges. The system monitors for proxy shortcuts: the hypernet learns a correlation that works in training but doesn't implement the actual task. For example, sum_mod3 learned to correlate with digit magnitude (r=-0.43) instead of actual modulo-3 arithmetic.
When the SelfCritic detects a proxy, it:
- Quarantines the proxy variant
- Triggers the TaskGenerator to propose a harder variant of the same task
- Routes the variant through MetaAdapterTrainer for fast fine-tuning
- Merges the result back into the main training loop
The loop is autonomous: human sets strategy, the agent executes tactics, metrics drive routing decisions.
| Standard fine-tuning | This approach | |
|---|---|---|
| Per-task weights | Separate model or adapter per task | Generated on-the-fly from task description |
| catastrophic forgetting | Risk when fine-tuning base | Base is frozen; deltas are additive |
| Zero-shot new tasks | Not possible | Hypernet reads a new task description → generates new deltas |
| Self-correction | Manual review | Autonomous SelfCritic loop |
| Architecture growth | Full retrain for new tasks | Register new task type, expand embedding |
# Interactive REPL
python3 main.py
# Single prediction
python3 main.py --predict 42 fib_check
# Self-correction loop (retry until correct)
python3 main.py --retry 1234 digit_sum
# Full benchmark on all 10 tasks
python3 main.py --benchmark
# Train the model
python3 main.py --train --epochs 50 --batch-size 64flowchart LR
A["task description\n\"sum of digits\""] --> B["HyperNetworkV3i\nLSTM"]
B --> C["weight deltas\nΔW per layer"]
C --> D["Base Transformer\nfrozen"]
A --> E["digit tokens\n1 2 3 4"]
E --> D
D --> F["PredictionHead\nper task"]
F --> G["class prediction"]
flowchart TD
subgraph INPUT
T["task description"]:::box
D["digit input"]:::box
K["task ID (0-9)"]:::box
end
subgraph HYPERNET["HyperNetworkV3i"]
ENC["TaskEncoder\nchar-level LSTM"]:::box
EMB["TaskEmbed\n128D → 256D"]:::box
LSTM["4-layer LSTM\nper-layer deltas"]:::box
end
subgraph ADAPTER["TaskAdapter (rank r)"]
DOWN["H → r down-proj"]:::box
MID["SiLU + Dropout"]:::box
UP["r → H up-proj"]:::box
G["gate g ∈ [0,1]"]:::box
end
subgraph BASE["Base Transformer"]
L1["LayerNorm + RoPE"]:::box
ATT["Multi-head\nSelf-Attention"]:::box
FFN["FFN"]:::box
end
T --> ENC --> EMB --> LSTM
LSTM -->|ΔW_attn_q| ATT
LSTM -->|ΔW_attn_k| ATT
LSTM -->|ΔW_ffn| FFN
LSTM -->|g| G
K --> EMB
D --> L1
ATT -. "delta\ninjection" .-> ADAPTER
FFN -. "delta\ninjection" .-> ADAPTER
ADAPTER -->|g · ΔW| ATT
ADAPTER -->|g · ΔW| FFN
style HYPERNET fill:#1a1a2e,stroke:#7b68ee,color:#fff
style ADAPTER fill:#1a2a1a,stroke:#3cb371,color:#fff
style BASE fill:#1a1a2e,stroke:#4a9eff,color:#fff
style INPUT fill:#2a1a2e,stroke:#ff7b54,color:#fff
classDef box fill:#2a2a4a,stroke:#aaa,color:#fff,rx:4
flowchart LR
E["Training Epoch"] --> P["PerformanceMonitor"]
P --> DIFF["DifficultyEstimator"]
DIFF --> CURR["CurriculumManager"]
subgraph LOOP["SelfCritic loop"]
SC["SelfCritic.judge()\nproxy detection"]:::critic
TG["TaskGenerator\nnew variant"]:::gen
MA["MetaAdapterTrainer\nfast fine-tune"]:::train
end
CURR -. "difficulty\nthreshold" .-> SC
P -. "accuracy\nmetrics" .-> SC
SC -->|quarantine proxy| MA
SC -->|new harder variant| TG
TG -->|variant spec| MA
MA -->|merged weights| P
style LOOP fill:#0a1a0a,stroke:#3cb371,color:#fff
classDef critic fill:#3a1a0a,stroke:#ff7b54,color:#fff
classDef gen fill:#1a2a3a,stroke:#4a9eff,color:#fff
classDef train fill:#2a1a3a,stroke:#9b59b6,color:#fff
| Component | Config | Params |
|---|---|---|
| Base Transformer | 8 layers, 256 hidden, 4 heads, FFN=512 | ~1M |
| HyperNetworkV3i | 4-layer LSTM, 256D task embedding | ~0.4M |
| TaskAdapter | bottleneck rank 8, 3 injection pts/layer | ~2.5M |
| PredictionHeads | 10 tasks × per-class heads | ~0.5M |
| Total (small) | 4.92M |
Full model (planned): ~340M — same arch at scale.
# On-the-fly generation (no pre-processed data needed)
cd training
python3 train_v6.py --epochs 50 --batch-size 64 --lr 3e-4
# With pre-generated Arrow data (10-50x faster loading)
python3 train_v6.py --epochs 50 --batch-size 64 --arrow-dir ../data/arrow
# Generate the Arrow data first
cd ..
python3 vectorize_dataset.py --split allpython3 vectorize_dataset.py # both train + val
python3 vectorize_dataset.py --split train # train only
python3 vectorize_dataset.py --dry-run # show what would be generatedEdit TrainConfig in training/train_v6.py (line ~580):
| Parameter | Default | Description |
|---|---|---|
epochs |
50 | Training epochs |
batch_size |
64 | Batch size |
lr |
3e-4 | Learning rate |
weight_decay |
1e-2 | AdamW weight decay |
num_train |
12000 | Training samples |
num_val |
1200 | Validation samples |
hidden_size |
256 | Transformer hidden dim |
num_layers |
8 | Transformer layers |
hnet_dim |
256 | Hypernetwork LSTM size |
adapter_rank |
8 | TaskAdapter bottleneck rank |
arrow_dir |
None | Use Arrow loader if set |
Training writes to experiments_v6/training_history.json each epoch:
python3 -c "
import json
with open('experiments_v6/training_history.json') as f:
h = json.load(f)
e = h[-1]
print(f'Epoch {e[\"epoch\"]}: val={e[\"val_acc\"]:.1%} loss={e[\"val_loss\"]:.3f}')
print('Task accuracies:', json.dumps(e['task_acc'], indent=2))
"28 epochs on RTX 3050 (~1.4h wall time):
| Task | Accuracy | Status |
|---|---|---|
| ascending | 100.0% | solved |
| digit_sum | 96.4% | solved |
| reverse | 100.0% | solved |
| fib_check | 100.0% | solved |
| last_digit | 100.0% | solved |
| mode_digit | 86.4% | near-solved |
| count_odd | 96.4% | solved |
| all_unique | 85.5% | near-solved |
| min_digit | 100.0% | solved |
| sum_mod3 | 32.7% | stuck — proxy shortcut, not modulo |
Overall: 57.2% → 89.7% val accuracy (+32.6pp over 28 epochs)
The hypernet learned a proxy that correlates with digit_sum (r=-0.43) but doesn't implement actual modulo-3. Every variant detected as a proxy by SelfCritic is correctly flagged — the infrastructure works. The issue is that with the current 4.92M model, there's insufficient capacity to learn the actual modulo operation, and the proxy provides a local minimum.
Fix path: Scale to the 340M model, which has enough capacity to learn sum_mod3 properly. The proxy-detection infrastructure (self_training/self_critic.py) will catch it if it recurs.
flowchart RL
subgraph ROOT["meta_transformer/"]
subgraph CORE["core"]
MAIN["main.py"]:::file
RA["research_agent.py"]:::file
VD["vectorize_dataset.py"]:::file
DL["data_loader_arrow.py"]:::file
INIT["__init__.py"]:::file
end
subgraph MODELS["models/"]
MTV["meta_transformer_v3.py"]:::file
end
subgraph TRAIN["training/"]
TV["train_v6.py"]:::file
end
subgraph ST["self_training/"]
MST["meta_self_trainer.py"]:::critic
SC["self_critic.py"]:::critic
CM["curriculum_manager.py"]:::gen
TG["task_generator.py"]:::gen
DE["difficulty_estimator.py"]:::gen
PM["performance_monitor.py"]:::gen
MA["meta_adapter_trainer.py"]:::train
RS["research_state.py"]:::gen
end
subgraph INF["inference/"]
SML["self_modification_loop_v3.py"]:::file
end
subgraph DATA["data/arrow/"]
TR["train/"]:::folder
VL["val/"]:::folder
SI["zd21_SciInstruct/"]:::folder
end
subgraph EXP["experiments_v6/"]
BM["best_model.pt"]:::bin
CK["checkpoint_latest.pt"]:::bin
TH["training_history.json"]:::file
TL["training_log.txt"]:::file
end
end
MAIN --> MTV
MAIN --> TV
RA --> ST
TV --> DL
VD --> DL
ST --> MTV
style CORE fill:#0f1a2e,stroke:#4a9eff,color:#fff
style MODELS fill:#1a0f2e,stroke:#9b59b6,color:#fff
style TRAIN fill:#0f1a0f,stroke:#3cb371,color:#fff
style ST fill:#1f1a0a,stroke:#f39c12,color:#fff
style INF fill:#1a1a1a,stroke:#aaa,color:#fff
style DATA fill:#0f0f1a,stroke:#aaa,color:#fff
style EXP fill:#0f0f0f,stroke:#aaa,color:#fff
classDef file fill:#1a2a4a,stroke:#4a9eff,color:#fff,rx:3
classDef folder fill:#1a2a1a,stroke:#3cb371,color:#fff,rx:3
classDef bin fill:#2a1a1a,stroke:#e74c3c,color:#fff,rx:3
classDef critic fill:#3a1a0a,stroke:#ff7b54,color:#fff,rx:3
classDef gen fill:#0a1a2a,stroke:#4a9eff,color:#fff,rx:3
classDef train fill:#1a0a1a,stroke:#9b59b6,color:#fff,rx:3
Legend: self_training/ modules are colored by role — 🟠 critic, 🔵 generator, 🟣 trainer, 🟢 core.
The architecture supports a closed-loop self-improvement cycle:
flowchart TD
subgraph TRAIN["Training Loop"]
EPO["Training Epoch"] --> PM["PerformanceMonitor"]
PM --> DE["DifficultyEstimator"]
DE --> CM["CurriculumManager"]
CM --> SC["SelfCritic.judge()"]
PM -. "accuracy\nmetrics" .-> SC
end
subgraph CRITIC["Proxy Detection"]
SC -. "quarantine\nproxy" .-> Q["Quarantined"]
SC -. "new harder\nvariant" .-> TG["TaskGenerator"]
SC -. "verdict" .-> MS["MetaSelfTrainer"]
end
subgraph ADAPTER["Adaptation"]
TG --> MA["MetaAdapterTrainer\nfast fine-tune"]
MA --> MW["Merged\nWeights"]
MW --> PM
end
MS -->|"continue / stop\n/ quarantine"| EPO
style TRAIN fill:#0d1f0d,stroke:#3cb371,color:#fff
style CRITIC fill:#1f0d0d,stroke:#ff7b54,color:#fff
style ADAPTER fill:#0d0d1f,stroke:#4a9eff,color:#fff
Run the self-training harness:
python3 research_agent.py --mode agentOr drive it programmatically via self_training/meta_self_trainer.py.
This project builds on several lines of prior research:
| Paper | arXiv | Key idea |
|---|---|---|
| HyperNetworks (Ha et al., 2016) | 1609.09106 | Dynamic weight generation via hypernetwork — the foundational concept |
| Attention is All You Need (Vaswani et al., 2017) | 1706.03762 | Transformer architecture with multi-head self-attention and positional encoding |
| FiLM: Feature-wise Linear Modulation (Perez et al., 2017) | 1709.07871 | Conditioning via feature-wise affine transformation — similar to TaskAdapter gating |
| ReZero (Bachlechner et al., 2020) | 2002.05566 | Gated residual connections for fast convergence and stability |
| LoRA: Low-Rank Adaptation (Hu et al., 2021) | 2106.09685 | Low-rank weight decomposition for efficient fine-tuning |
| AdapterHub (Rücklé et al., 2020) | 2007.07779 | Adapter-based transfer learning for NLP |
| Meta-Learning with Hypernetworks (Wichrowska et al., 2017) | 1703.04561 | Learned optimizers as hypernetworks |
| Modular Meta-Learning (Alet et al., 2018) | 1806.10140 | Compositional task-specific modules |
| Task-Agnostic vs Task-Specific (Jospe et al., 2020) | 2009.01758 | Analysis of hypernetwork approaches to MAML |
- Train the 340M model — the architecture is ready, just needs GPU time (~6-12h on RTX 3050)
- Fix sum_mod3 at scale — 340M should have enough capacity to learn modulo-3 without proxy
- Run full self-training loop — current self-training modules are built but haven't been exercised on real training data
- Verify proxy detection — SelfCritic infrastructure exists but needs real difficulty-scaled variants to validate
- GSM8K evaluation —
datasets/gsm8k/data exists; evaluate transfer learning
The core hypothesis: a hypernetwork that reads task descriptions and generates weight deltas can make a single model adapt to multiple tasks without fine-tuning. The architecture went through several iterations:
- v1/v2: Frozen base + hypernet → causality-inert deltas (deltas too small to change logits meaningfully)
- v3/v4: Joint base+hypernet training → marginal improvement
- v5: TaskAdapter bottleneck injection → significant improvement
- v6 (current): HyperNetworkV3i + TaskAdapter + PredictionHeads → 89.7% val accuracy
The stuck sum_mod3 task at the 4.92M scale is the primary remaining challenge. The self-training infrastructure was built specifically to detect and route around such proxy shortcuts.