A collection of projects from classes and late-night coding sessions. Some are polished, some are experiments, all are valuable learning experiences.
Designed and trained a 44M parameter Mixture-of-Experts model from scratch on TinyStories Dataset, achieving a perplexity of 3.0. Implemented Noisy Top-2 Gating to dynamically route tokens, reducing active parameters by 75% while maintaining model quality.
Engineered a fused Self-Attention CUDA kernel using SRAM tiling and thread-level parallelism, achieving a 3x speedup over naive implementations. Implemented Online Softmax to eliminate O(N²) memory complexity by computing attention without full matrix materialization.
Architected a budgeting and wealth tracking platform integrating SimpleFin via OAuth to aggregate real data across institutions. Enforced end-to-end type safety using tRPC and TypeScript, with secure session management via BetterAuth.