M&E Engineer / Systems Architect

Daniel Lew

Fixing systems in real life and building systems online.

Infrastructure Design Principles

Three pillars

Noise Reduction

Signal extraction from high-volume, low-signal data streams โ€” RSS feeds, chat logs, knowledge graphs. Threshold tuning and relevance scoring strip out the churn before anything expensive runs, so downstream systems only ever see what matters.

Cost-Friendly

Per-purpose model routing, batch processing, and tiered validation keep expensive compute to a minimum. Each task is matched to the cheapest tool that can do the job well, and validation escalates only when the payoff justifies it.

Efficiency

A cascading LLM architecture tries the cheapest model first and escalates only when confidence is low. Desirepath algorithms let concepts form organically from real usage, so structure emerges where it is actually needed instead of being imposed up front.

Key Algorithms

How the systems think

Cascading LLM Architecture

A task enters at the cheapest capable model and only escalates when confidence is low โ€” most work never touches an expensive model.

Task in
incoming request
DeepSeek-V3
cheap model โ€” first attempt
Confidence gate
high โ†’ done ยท low โ†’ escalate
GPT-4 / Claude Opus
expensive โ€” only when needed
Up to ~55ร— cost savings vs. routing every task to a frontier model. Measured on typical mixed workloads; see the case study for methodology.

Desirepath Algorithm

Concepts form organically the way footpaths do โ€” from repeated use, not top-down design.

Visitation frequency
how often a node is revisited
Hub detection
nodes many paths pass through
Novelty scoring
reward for new connections
High-traffic routes between ideas harden into named concepts over time.