Tate Hertel

Research

Rigorous evaluation,
honest reporting.

Independent replications, ablation studies, and empirical evaluations. I publish negative results too — knowing what doesn't work is as valuable as knowing what does.

Conditional Memory for Small Language Models cover

April 2026

Conditional Memory for Small Language Models

An Empirical Evaluation of N-Gram Hash Lookup Tables

We investigate whether N-gram hash lookup tables (Engram modules) can improve small language models. Across four experiments spanning three model sizes, Engram modules reduce perplexity by 22-30% — yet produce zero measurable improvement on downstream benchmarks.

Machine LearningNLPConditional MemoryNegative Result
Read paper