# Dibyaprakash Pradhan — complete site content Source: https://diby.app Generated from the same content that renders the HTML pages, so this file and the site cannot disagree. Dibyaprakash Pradhan builds AI foundation architectures and models at AntEngage — designed from first principles rather than derived from transformers. Also builds pcbeditor.com at RoboDIB, an AI circuit design tool that turns plain English into verified schematics. Published work on mixture-of-experts routing, checkpoint compression and speech emotion recognition. Bengaluru, India. ## Identity and contact Name: Dibyaprakash Pradhan Also written: Dibya Prakash Pradhan Role: Founder & CTO at AntEngage (https://antengage.com/) — AI foundation architectures and models Also: Founder at RoboDIB (https://robodib.com) — pcbeditor.com Location: Bengaluru, India Email: dibyaprakash@robodib.com Areas of expertise: Foundation model architecture, Novel neural architectures, Model efficiency, AI systems research, Mixture of Experts, Expert routing, Model compression, LLM inference, Speech emotion recognition, Real-time voice systems, Electronic design automation, Distributed systems Profiles: https://www.linkedin.com/in/dibyaprakash-pradhan/, https://github.com/dibyapp, https://twitter.com/dibyapp, https://www.youtube.com/@dibyaprakash-pradhan, https://peerlist.io/dibyaprakash, https://www.f6s.com/member/dibyaprakash-pradhan, https://www.crunchbase.com/person/dibyaprakash-pradhan, https://www.producthunt.com/@dibyapp, https://instagram.com/dibyapp, https://www.quora.com/profile/Dibyaprakash-Pradhan ## Experience ### Chief Technology Officer (CTO) — AntEngage Duration: Apr 2024 - Present Location: Bengaluru, Karnataka, India · On-site Technical direction for a conversational-AI platform that holds real calls: voice agent architecture, the streaming path underneath it, and the cloud footprint it runs on. Also where the mixture-of-experts work started, from needing inference to fit the hardware we had. - Own technical strategy and architecture across the product surface. - Lead the engineering team and the design partnership around it. - Build and operate the cloud infrastructure the platform runs on. - Ship the AI features that decide who to reach, when, and what to say. Technologies: React, Node.js, AWS, Docker, Kubernetes, TensorFlow, MongoDB ### Senior Software Engineer - III — GeekyAnts Duration: Mar 2024 - Nov 2024 Location: Bengaluru, Karnataka, India · On-site Led delivery on client-facing applications and moved monolithic services onto a microservice footing without a rewrite. - Led development of multiple client-facing web applications. - Implemented microservices architecture to improve system scalability. - Collaborated with cross-functional teams to deliver projects on time. Technologies: Spring Boot, JavaScript, TypeScript, React, Node.js, AWS, GraphQL ### Senior Software Engineer - I — GeekyAnts Duration: Mar 2023 - Mar 2024 Location: Bengaluru, Karnataka, India · On-site Owned enterprise Spring Boot services end to end. Mostly performance work — the kind where the win is a page that loads 40% faster and nobody notices, which is the point. - Developed and maintained enterprise-level Spring boot applications. - Optimized application performance and reduced loading times by 40%. - Mentored junior developers and conducted code reviews. Technologies: JavaScript, React, Redux, Node.js, Express, MongoDB, Spring Boot ### Software Engineer - III — GeekyAnts Duration: Mar 2022 - Mar 2023 Location: Bengaluru, Karnataka, India Built product front-ends in React against Spring Boot services, working directly with design in short cycles. - Built robust front-end interfaces using React and modern JavaScript. - Collaborated with UX/UI designers to implement pixel-perfect designs. - Participated in agile development processes and sprint planning. Technologies: JavaScript, Spring Boot, React, HTML/CSS, Git, RESTful APIs ### Client Success Associate — Avanze Duration: Jan 2021 - Mar 2022 Location: Bangalore Urban, Karnataka, India · On-site Sat between clients and engineering, turning business requirements into specifications developers could build against. The most useful detour of my career: most failed software fails at that translation layer, not in the code. - Managed client relationships and ensured successful project delivery. - Gathered requirements and translated them into technical specifications. - Coordinated between development teams and clients to resolve issues. Technologies: Project Management, Java, PHP, Client Relations, Technical Documentation ### Freelance Developer — Pyloom Innovations Duration: Jan 2015 - Present Location: Remote A long-running independent practice. Custom web and mobile builds for startups and established businesses, owned from the first conversation through deployment. - Developed custom web and mobile applications for various clients. - Created innovative solutions for startups and established businesses. - Managed end-to-end project lifecycle from concept to deployment. Technologies: Full Stack Development, Mobile Apps, UI/UX Design, Consulting ## Open source projects ### MoE-Watcher-Modifier URL: https://diby.app/open-source/moe-watcher-modifier Repository: https://github.com/dibyapp/MoE-Watcher-Modifier Language: Python | Stars: 5 | Forks: 2 MoE-Watcher-Modifier is an open-source, model-agnostic toolkit that analyses which experts a Mixture-of-Experts model actually uses, ranks them by importance, and rewrites the checkpoint with fewer experts — shrinking the model without touching training or inference code. Most Mixture-of-Experts checkpoints ship far more experts than any single workload needs. MoE-Watcher-Modifier measures which experts your traffic actually routes to, ranks them, and writes out a smaller checkpoint containing only the ones that earn their place. It works on any safetensors MoE checkpoint and requires no changes to your training or serving code. Who it is for: - Teams serving a large MoE model on hardware that can barely hold it - Anyone running Qwen3-Next, Mixtral, DeepSeek, Phi-3.5-MoE or OLMoE locally - Researchers studying expert utilisation and routing collapse - Engineers who want real routing data from production traffic, not synthetic benchmarks Key capabilities: - Model-agnostic — works with any safetensors MoE checkpoint, schema auto-detected from config.json - Router-only mode profiles a checkpoint in minutes on CPU, loading only the router tensors - Full-model mode captures true routing decisions from your own prompts - Daemon mode is a transparent proxy that accumulates routing stats from live traffic with no added response latency - Rewrites checkpoints on CPU — memory use is bounded by the largest shard, not the whole model - Emits a pruning manifest so every produced checkpoint is traceable back to its source Requirements: - torch >= 2.0.0 - safetensors >= 0.4.0 - rich >= 13.0.0 (optional — nicer terminal output) - transformers >= 4.40.0 (optional — full-model mode only) Quickstart: Install ``` pip install torch safetensors rich pip install transformers # only for full-model mode ``` 1. Inspect the checkpoint — Auto-detects the schema and prints the layer and expert layout. ``` python3 moe_monitor.py inspect --model-dir /path/to/checkpoint ``` 2. Collect expert usage stats — Router-only mode needs no GPU and finishes in minutes — the recommended first pass. ``` python3 moe_monitor.py router-only \ --model-dir /path/to/checkpoint \ --output ./stats/report.json \ --keep-experts 128 \ --new-topk 4 \ --samples 1024 \ --strategy per-layer ``` 3. Dry-run the rewrite — Validates the plan against the checkpoint without writing anything. ``` python3 moe_prune.py dry-run \ --model-dir /path/to/checkpoint \ --plan ./stats/report-plan.json ``` 4. Write the pruned checkpoint ``` python3 moe_prune.py prune-checkpoint \ --model-dir /path/to/checkpoint \ --plan ./stats/report-plan.json \ --output-dir ./pruned-128experts \ --copy-support-files ``` Supported model families The schema is detected automatically from the checkpoint's config.json. Pass --schema to override it when a model reports something unexpected. Schema | Models qwen3_next | Qwen3-Coder-Next, Qwen3-Next qwen_moe | Qwen1.5-MoE, Qwen2-MoE mixtral | Mixtral-8x7B, Mixtral-8x22B deepseek | DeepSeek-V2, DeepSeek-V3, DeepSeek-R1 phi3_moe | Phi-3.5-MoE olmoe | OLMoE Three ways to collect routing statistics Router-only mode loads just the router weight tensors — a few megabytes — and probes each router with random unit-norm hidden states. No GPU, no full model load, minutes even on very large checkpoints. It captures structural routing preferences, which is enough for a first pruning pass. Full-model mode loads the whole model and installs forward hooks on every MoE gate, then runs your real prompts through it. This captures true routing decisions on your actual workload, and needs enough RAM or VRAM to hold the model. Daemon mode is the one worth reaching for in production. It runs as a transparent HTTP proxy in front of any OpenAI-compatible model server — ollama, vLLM, llama.cpp, LM Studio, text-generation-webui — and accumulates routing statistics from live user traffic in a background thread. Requests are forwarded untouched, so nothing is added to the response path. How the daemon works Your application points at the daemon instead of the backend. For each request the daemon extracts the prompt text, tokenises it with the checkpoint's own tokenizer, looks up token embeddings from embed_tokens.weight, forwards each token's hidden state through the router weights, and records the selected experts per layer. Every N requests it writes a report and prints a ready-to-run prune command. Because it only ever loads the router weights and the embedding table, the daemon runs comfortably alongside the model server on the same box — Linux, macOS or Windows, AMD, NVIDIA or CPU-only. What the rewriter actually does The pruner drops expert tensors absent from the keep list, renames retained experts to compact IDs from 0 to target-1, slices each router weight matrix to match the retained expert IDs, updates config.json with the new expert count and top-k, and writes a pruning_manifest.json for traceability. With --copy-support-files it also carries across the tokenizer, generation config and other auxiliary files, so the output directory is a complete, loadable checkpoint rather than a bag of tensors. No GPU is required and peak memory is bounded by the largest shard. The iterative workflow The strongest results come from iterating. Run router-only on the original checkpoint for an initial structural plan, prune, then load the now-smaller checkpoint — which may finally fit in memory — and run full-model mode on it for real routing statistics. Re-rank with a different keep count, prune again, and finish with a finetune or distillation pass on domain data to recover quality. Each round replaces guesswork with a stronger routing signal than the round before it. Frequently asked questions: Q: What is MoE-Watcher-Modifier? A: It is an open-source Python toolkit for analysing and pruning Mixture-of-Experts models. It measures which experts a checkpoint actually routes to, ranks them by importance, and rewrites the checkpoint with only the experts worth keeping — reducing model size without any change to training or inference code. Q: Which MoE models does it support? A: Any safetensors MoE checkpoint. Schemas ship for Qwen3-Next and Qwen3-Coder-Next, Qwen1.5-MoE and Qwen2-MoE, Mixtral-8x7B and 8x22B, DeepSeek-V2, V3 and R1, Phi-3.5-MoE, and OLMoE. The schema is auto-detected from config.json and can be overridden with --schema. Q: Do I need a GPU to prune a MoE model with it? A: No. Router-only analysis and the checkpoint rewrite both run on CPU. Peak memory during the rewrite is bounded by the largest shard rather than the size of the whole model. A GPU is only needed for full-model mode, which loads the model to capture real routing decisions. Q: How do I get expert usage statistics from production traffic? A: Run the daemon. It is a transparent HTTP proxy that sits in front of any OpenAI-compatible model server — ollama, vLLM, llama.cpp, LM Studio — and accumulates routing statistics from live requests in a background thread, adding no latency to responses. After N requests it prints a ranked expert table and the exact prune command to run. Q: Does pruning experts hurt model quality? A: Removing experts is lossy, which is why the toolkit ranks experts on your own routing data rather than pruning blindly, and reports the expert coverage fraction so you can see how much routing mass the keep list captures. The recommended workflow finishes with a finetune or distillation pass on domain data to recover quality. Q: Is it safe to run against my only copy of a checkpoint? A: The pruner never modifies the source checkpoint — it writes a new directory. There is also a dry-run subcommand that validates the plan against the checkpoint structure without writing any files. --- ### Multitenancy in Spring Boot URL: https://diby.app/open-source/spring-boot-multitenancy Repository: https://github.com/dibyapp/mt-base-service Language: Java | Stars: 1 | Forks: 7 mt-base-service is an open-source Spring Boot reference implementation of multitenancy, covering tenant registration, per-tenant datasource resolution and Flyway-managed schema migration. Every B2B product eventually needs to keep one customer's data away from another's, and multitenancy done casually is how that becomes a data leak. This is the version I would want to start from: a Spring Boot service with tenant organisation entities, datasource records per tenant, JPA repositories and mappers, and Flyway migrations — the structural decisions made once, properly. Who it is for: - Teams starting a new B2B or SaaS product on Spring Boot - Engineers who have inherited a single-tenant service that now needs isolation - Anyone weighing schema-per-tenant against database-per-tenant Key capabilities: - Tenant organisation entities with DTOs and mappers, kept separate from request handling - Datasource records per tenant, so isolation is a data concern rather than scattered conditionals - Flyway migrations checked into the repository for reproducible tenant schema setup - A base REST controller other services can extend rather than copy - The most-forked of my repositories — teams use it as a starting point, not a demo Requirements: - Java 17+ - Maven (wrapper included) - A relational database Quickstart: Clone and build ``` git clone https://github.com/dibyapp/mt-base-service.git cd mt-base-service ./mvnw clean install ``` Configure and run — Point src/main/resources/application.yml at your database, then start the service. ``` ./mvnw spring-boot:run ``` How it is structured The service separates the tenant model from the request layer. Organisation entities and their DTOs are mapped explicitly rather than exposed straight out of JPA, datasource records describe where each tenant's data lives, and repositories keep the lookup in one place. A base REST controller gives downstream services a common starting point, so tenant handling is inherited rather than reimplemented per endpoint. Why multitenancy is worth getting right early Retrofitting isolation into a service that assumed one customer is among the more expensive migrations in enterprise software, because the assumption is rarely in one place. It is in queries, in caches, in background jobs, and in every endpoint written before anyone thought about it. Deciding tenant resolution and data isolation up front costs a few days. Deciding it after the second customer signs costs a quarter. Frequently asked questions: Q: How do I implement multitenancy in Spring Boot? A: Resolve the tenant on each request, isolate data per tenant rather than filtering in queries, and manage tenant schema with versioned migrations. This repository implements that shape: organisation entities with mappers, datasource records per tenant, JPA repositories, and Flyway migrations checked into source control. Q: Is it schema-per-tenant or database-per-tenant? A: The service models tenant datasources as first-class records, so where a tenant's data physically lives is a configuration decision rather than something hardcoded into the application logic. Q: Can I use it as a starting point for a commercial product? A: It was written to be forked — which is what most people do with it. Clone it, point the configuration at your database, and replace the organisation model with yours. --- ### Location from IP Address URL: https://diby.app/open-source/location-from-ip-address Repository: https://github.com/dibyapp/location-from-ip-address-service Language: Java | Stars: 0 | Forks: 0 location-from-ip-address-service is an open-source Spring Boot service that resolves an IP address to a geographic location through a GeoIP lookup and renders the result on a map. A self-hosted alternative to paying a third-party API per request for something as routine as turning an IP address into a country and city. The service exposes a lookup endpoint, models the resolved location as a first-class type, and includes a map view so you can see the answer rather than read coordinates. Who it is for: - Teams doing analytics or fraud checks who would rather not send every user IP to a vendor - Anyone localising content by region without a per-request bill - Java developers who want a working GeoIP example rather than a snippet Key capabilities: - Lookup endpoint returning structured location data for an IP address - A dedicated map view rendering the resolved location visually - Server location modelled as its own type instead of a loose map of strings - Self-hosted, so user IP addresses never leave your infrastructure Requirements: - Java 17+ - Maven (wrapper included) Quickstart: Clone and run ``` git clone https://github.com/dibyapp/location-from-ip-address-service.git cd location-from-ip-address-service ./mvnw spring-boot:run ``` Why self-host a GeoIP lookup IP geolocation through a commercial API is easy to adopt and awkward to live with. It puts a per-request cost on a lookup that should be nearly free, adds a network hop to a hot path, and quietly turns every user's IP address into data you are sending to someone else. A local GeoIP database answers the same question with none of those properties. This service is the thin layer around it that most projects end up writing anyway. Frequently asked questions: Q: How do I get a location from an IP address in Java? A: Query a local GeoIP database rather than a remote API. This Spring Boot service does exactly that: it takes an IP address, resolves it against a GeoIP lookup, returns structured location data, and can render the result on a map. Q: Does it send user IP addresses to a third party? A: No. The lookup runs against a local GeoIP database inside your own infrastructure, which is the main reason to run it instead of calling a hosted geolocation API. --- ### RSS GitHub Notifier for Slack URL: https://diby.app/open-source/rss-github-notifier-for-slack Repository: https://github.com/dibyapp/rss-notifier-for-slack Language: Java | Stars: 0 | Forks: 0 rss-notifier-for-slack is a small open-source Java utility that watches GitHub activity over RSS and posts updates into Slack channels. Release and repository notifications delivered by email are notifications nobody reads. This is a deliberately small Java program that polls a GitHub RSS feed and pushes what it finds into Slack, where the team already is. No framework, no service to operate — one class doing one job. Who it is for: - Teams who want repository activity in Slack without wiring up a full integration - Anyone monitoring a repository they do not own or control - Developers who want a readable example of RSS polling plus Slack webhooks in Java Key capabilities: - Single-class Java program — readable end to end in one sitting - Polls any GitHub RSS feed, including repositories you have no admin rights on - Posts into Slack channels via webhook - No framework and no runtime service to maintain Requirements: - Java 17+ - A Slack incoming webhook URL Why RSS rather than webhooks GitHub webhooks are the better mechanism when you administer the repository. RSS wins when you do not — you can follow releases and activity on any public repository without asking anyone for access, which is exactly the case where a notification is most useful. Frequently asked questions: Q: How do I send GitHub notifications to Slack? A: If you administer the repository, a webhook integration is the direct route. If you do not, poll the repository's RSS feed and post updates to a Slack incoming webhook — which is what this small Java utility does. Q: Can it watch repositories I do not own? A: Yes. It reads public RSS feeds, so it works on any public repository without needing admin rights or an installed app. ## Research ### Random-probe expert rankings degrade generation at 3% pruning URL: https://diby.app/research/random-probe-rankings-degrade-generation Published: 2026-04-21 — Negative result, published here Keywords: expert pruning quality, MoE degradation, random probe routing statistics, OLMoE, negative result, routing collapse repetition Router-only profiling is cheap enough to be nearly free, which makes it tempting to prune on its ranking alone. This experiment tests how far that ranking can be trusted by making the smallest meaningful cut — two experts per layer, top-k unchanged — and comparing generations against the unmodified checkpoint. The cut is not free. The failure mode is informative about why, and points at what the ranking would have to be built from instead. Motivation Router-only profiling loads a few megabytes of router weights and probes them with random unit-norm hidden states. It runs on CPU in under a second for a 16-layer model. Because it is so cheap, the obvious temptation is to prune directly on its ranking. The question this experiment asks is how much that ranking is worth. Rather than testing an aggressive prune — where degradation would be unsurprising and uninformative — it tests the smallest cut that is still a cut. Setup OLMoE-1B-7B, 16 layers, 64 experts per layer, top-k 8. Router-only statistics over 1,024 samples produced 131,072 router selections, with 100% expert coverage — every expert was selected at least once. The plan drops the two lowest-ranked experts in each layer and leaves top-k at 8. The intervention is deliberately minimal: a 3.1% reduction in expert count with the routing width unchanged. Property | Original | Pruned Experts per layer | 64 | 62 Top-k | 8 | 8 Total tensors | 3,219 | 3,123 Checkpoint size | 13.0 GB | 13.44 GB Reduction | — | 3.1% of experts Both checkpoints were then run greedily — do_sample=False, 30 new tokens — over the same five prompts, on the same hardware, in the same session. Result Greedy generation, original versus 62-expert prune. Four of five prompts survive; one fails outright. Prompt | Outcome The capital of France is | Identical output def fibonacci(n): | Different branch structure, still valid Mixture of Experts models work by | Different wording, still correct The largest planet in the solar system is | Correct, slightly repetitive To make a cup of tea, you need to | Collapses into a repetition loop The tea prompt is the informative one. The original drifts in the ordinary way a 1B model drifts — boiling water, a kettle, a stove. The pruned model emits "The water is boiling." and then emits it again, and again, for the remainder of the budget. Two experts out of sixty-four, with routing width unchanged, is enough to turn a coherent continuation into a loop. Why random probes fail here The planner drops the two experts scoring lowest against random hidden states. Random hidden states are not the hidden states your prompts produce. An expert that looks marginal under isotropic noise may sit directly on the path of a common real input pattern — and the routing distribution is close enough to uniform that the ranking has very little margin to be wrong with. This connects to the earlier measurement on Qwen3-Coder-Next, where the routing Gini was 0.1006 and the largest deviation from uniform anywhere in the model was 1.42×. When experts are nearly equally used, the difference between rank 62 and rank 64 is close to noise, and a ranking derived from the wrong input distribution will order them close to arbitrarily. Flat routing makes pruning a priced trade and simultaneously makes the price hard to estimate from structure alone. What follows from this Router-only rankings are a structural baseline, not a production keep-list. Use them to decide whether a model is worth pruning, not which experts to remove. Coverage is not sufficient as a safety metric. This run reported 100% coverage and still degraded — an expert being selected at least once says nothing about how much depends on it. The keep-list should be derived from the workload. Collecting real routing statistics from live traffic is what the daemon proxy exists for; the two dropped experts would then genuinely be the least-used ones on that workload. Any prune deserves a side-by-side generation check before it is trusted, and the check should include prompts long enough to expose looping. The broader point is that cheap measurement and good measurement are different things, and the gap between them is exactly where a compression pipeline quietly loses quality. Publishing the failure is more useful than publishing the compression ratio. Reproduction python3 moe_monitor.py router-only \ --model-dir allenai/OLMoE-1B-7B-0924 \ --output ./stats/olmoe-report-62.json \ --keep-experts 62 \ --new-topk 8 \ --samples 1024 \ --strategy per-layer python3 moe_prune.py prune-checkpoint \ --model-dir allenai/OLMoE-1B-7B-0924 \ --plan ./stats/olmoe-report-62-plan.json \ --output-dir ./OLMoE-1B-7B-pruned-62 \ --copy-support-files The report and plan JSON for both this run and the 64 → 32 run are committed in the repository under stats/, together with the full session log. --- ### Pruning a Mixture-of-Experts checkpoint without a GPU URL: https://diby.app/research/pruning-moe-checkpoints-without-a-gpu Published: 2026-04-21 — Systems result, published here Keywords: MoE checkpoint pruning, safetensors rewriting, vLLM fused_moe, CPU-only model compression, expert pruning without GPU, OLMoE pruning This began as a failed attempt to fit Qwen3-Coder-Next-FP8 — 80 GB, 512 experts, 48 layers — onto a single NVIDIA L4 with 22 GB of VRAM. Every route through vLLM hit the same wall, and the wall turned out to be structural rather than configurational. The resolution was to stop treating pruning as an inference-time problem and treat it as a file-format problem instead. This note documents the failure, the design that came out of it, and the measured rewrites. The wall The original goal was narrow: run Qwen3-Coder-Next-FP8 — 80 GB, 512 experts across 48 layers — on a single NVIDIA L4 with 22 GB of VRAM. Pruning experts is the obvious lever, and the obvious place to pull it is the serving framework. That does not work. In vLLM 0.19.0, fused_moe/layer.py allocates all 512 expert tensors on the GPU inside __init__, before any CPU offload path engages. The allocation happens during construction, so there is no flag to set and no hook to intercept — the model has to fit before you are given the opportunity to make it smaller. You cannot use the inference stack to shrink a model that the inference stack cannot load. The constraint is structural, not configurational. The interesting question is the general one this failure implies: can a tool prune any MoE checkpoint, for any runtime, on any hardware — so that nobody has to repeat this? Design: treat it as a file-format problem The rewriter never calls from_pretrained. It opens the safetensors shards directly and processes them one at a time, copying through the tensors named in the keep-list and dropping the rest. Four properties follow immediately. No GPU required at any point in the rewrite. Peak memory is bounded by the largest single shard, not by the size of the model — an 80 B checkpoint rewrites on a laptop. No dependence on framework versions, because no framework is loaded. The source checkpoint is never modified; the rewrite writes a new directory. The rewriter drops expert tensors absent from the keep-list, renames retained experts to compact IDs from 0 to target−1, slices each router weight matrix to match, updates config.json with the new expert count and top-k, and writes a pruning manifest recording the source and the plan that produced the output. Measured rewrites Validated end to end on OLMoE-1B-7B — 16 layers, 64 experts per layer, top-k 8, three shards, 13.0 GB. Hardware was an NVIDIA L4 box, but the rewrite itself ran on CPU. Two rewrites of the same source checkpoint. The 62-expert output is larger than its source: safetensors metadata overhead per shard exceeds the two dropped experts. Rewrite | Tensors | Size | Elapsed Source (64 experts, top-k 8) | 3,219 | 13.0 GB | — 64 → 32, top-k 8 → 4 | 1,683 | 7.39 GB | 7.6 s 64 → 62, top-k unchanged | 3,123 | 13.44 GB | 16.1 s A dry-run subcommand validates a plan against the checkpoint structure without writing anything; on this model it completed in 0.1 s. That matters more than it sounds, because a pruning plan that disagrees with the checkpoint layout should fail in a tenth of a second rather than after a 16-second write. Two bugs the end-to-end test surfaced Both were only findable by actually loading a pruned checkpoint, which is an argument for end-to-end tests over unit tests when the contract you depend on belongs to somebody else. Schema drift between documentation and checkpoint The OLMoE schema expected router tensors at model.layers.{n}.mlp.router.weight, following the model card and the transformers source. The published checkpoint uses model.layers.{n}.mlp.gate.weight. Inspection reported router_keys_found: 0 against expert_keys_found: 3072 — a shape that unambiguously indicates a naming mismatch rather than a missing component, which is why the inspect subcommand reports both counts separately. Missing format metadata The pruned shards loaded fine as safetensors and crashed the transformers loader with an invalid-metadata error. The rewriter had been writing its own provenance keys into the shard metadata and, in doing so, dropped the format: pt entry that transformers requires. metadata = { "format": "pt", # required by the transformers loader "moe_pruner": "true", "source_shard": str(src.root / shard_name), } Writing custom metadata is safe. Replacing the metadata dict is not. What this does and does not establish It establishes that the mechanical part of expert pruning — deciding what to keep, rewriting the checkpoint, producing something a standard loader accepts — is cheap, portable and independent of the serving stack. Seven seconds on CPU for a 13 GB model. It establishes nothing about quality. The 64 → 32 output loads and generates, and what it generates is incoherent: prompts collapse into token repetition. Removing half the experts without any recovery training breaks routing, exactly as expected. Making the mechanics free does not make the decision free, and the decision is the hard part. That question — how much quality a given prune actually costs, and whether random-probe rankings are good enough to choose by — is taken up separately. --- ### Expert routing in Qwen3-Coder-Next is close to uniform URL: https://diby.app/research/expert-routing-qwen3 Published: 2026-04-21 — Experimental result, published here Keywords: Mixture of Experts, expert routing, routing collapse, Qwen3-Coder-Next, model compression, checkpoint pruning, expert coverage Mixture-of-Experts checkpoints are widely assumed to collapse onto a small set of favoured experts, which would make pruning a matter of removing dead weight. Profiling every MoE layer of Qwen3-Coder-Next does not support that assumption: routing is only mildly concentrated, cold experts are rare, and their distribution across layers is uneven. This changes what pruning buys and how it should be measured. Setup All 48 MoE layers of Qwen3-Coder-Next (FP8) were profiled in router-only mode: the router weight tensors are loaded from the safetensors checkpoint and probed with random unit-norm hidden states, without materialising the experts themselves. 1,024 samples produced 491,520 router selections across 512 experts per layer. Configuration and cost of both profiling runs. Both executed on CPU. Model | Layers | Experts | Top-k | Samples | Coverage | Elapsed Qwen3-Coder-Next FP8 | 48 | 512 → 128 | 10 → 4 | 1,024 | 99.34% | 4.98 s OLMoE-1B-7B | 16 | 64 → 32 | 8 → 4 | 512 | 100% | 0.38 s Result: routing is only mildly concentrated The routing distribution has a Gini coefficient of 0.1006, where 0 is perfectly uniform. The busiest expert in the model was selected 1,453 times; the quietest 488. A factor of three between extremes across 512 experts is not the winner-take-all behaviour the collapse narrative predicts. Expert selection counts, 48 layers × 512 experts. The absence of strong vertical banding is the finding: utilisation is spread rather than concentrated. Routing mass by expert percentile, against what uniform routing would give. The largest deviation anywhere is 1.42×. Bucket | Observed | Uniform | Ratio Top 1% | 1.42% | 1% | 1.42× Top 5% | 7.02% | 5% | 1.40× Top 10% | 13.22% | 10% | 1.32× Top 20% | 25.04% | 20% | 1.25× Top 50% | 57.12% | 50% | 1.14× Cold experts are rare but unevenly distributed Mean cold experts per layer: 3.396 of 512, roughly 0.66%. But 45 of 48 layers contained at least one cold expert, and a single layer contained 27. The scarcity is real; its distribution is not uniform across depth. This asymmetry is the argument for per-layer keep-lists over one global ranking. A model-wide opinion about which experts matter will be wrong for the layers that disagree with it, and here most layers disagree slightly. Per-layer keep-list construction for the 512 → 128 plan, with top-k reduced from 10 to 4. Interpretation Because routing is close to uniform, pruning is not the removal of dead capacity — it is a priced trade. Cutting 512 experts to 128 retained 99.34% of observed routing mass, which means 0.66% of it is now unserved and those tokens route elsewhere. Coverage is the number that makes the trade legible, and it should be reported alongside any compression ratio. Random probes measure structural router preference, not workload. A domain-concentrated workload will produce a different distribution — which is what full-model and daemon modes exist to capture. Reproduction pip install torch safetensors rich python3 moe_monitor.py router-only \ --model-dir Qwen/Qwen3-Coder-Next-FP8 \ --output ./stats/report-512.json \ --keep-experts 128 \ --new-topk 4 \ --samples 1024 \ --strategy per-layer The report JSON backing every number above is committed in the repository under stats/, alongside the plan file and the generated figures. --- ### Cognitive Analysis of Emotion from Voice Using Deep Learning URL: https://diby.app/research/cognitive-analysis-emotion-voice Published: 2024-01-01 — Artificial Intelligence Research Keywords: Speech Emotion Recognition, Cognitive Psychology, Deep Learning, wav2vec, HuBERT, Whisper, Explainable AI, Affective Computing, Prosody, Valence and arousal, Cross-corpus generalisation, Speaker adaptation Voice carries emotion before words do. This paper connects two literatures that rarely constrain each other — cognitive accounts of human emotion perception, and transformer-based speech emotion recognition — and proposes a system architecture in which model outputs are tied back to the vocal cues a human listener would have cited. It covers the psychological grounding, the acoustic correlates those models imply, the datasets the field relies on and what they systematically omit, the model families that hold up under noise, why environmental context changes what a classifier should be allowed to conclude, how such a system should be evaluated, and what follows for deployment in settings where being wrong quietly is worse than being uncertain loudly. Introduction Emotion recognition from speech is essential for making voice-enabled AI more human-like. Humans naturally understand emotions like happiness, sadness, anger or fear from vocal tone alone. This work examines how cognitive psychology and deep learning methods together can help machines infer emotional state from voice signals — and, more importantly, explain the inference. Psychological background Ekman's theory: emotions such as anger, happiness, sadness, fear, surprise and disgust are treated as universal and recognisable through voice. Russell's dimensional model: emotion described along two axes — valence (positive to negative) and arousal (high to low energy). Scherer's component process model: emotion identified through multiple vocal cues including pitch, loudness and voice quality. People mainly use pitch, loudness, speech rate and voice quality to identify emotion. High pitch and loud voice often indicate anger or excitement; low pitch and soft voice suggest sadness or calm. These are the cues an explanation should be able to name. The three accounts are not rivals so much as different resolutions of the same phenomenon, and each one implies a different output contract for a system built on it. Ekman's categories imply a classifier. Russell's plane implies a regressor over two continuous axes. Scherer's model implies something closer to a structured description — a set of cue observations from which a label is derived. The third is the hardest to build and the only one that can explain itself, which is the argument this paper makes for it. Acoustic correlates Between the psychology and the model sits a layer that is often skipped: what, physically, is being measured. The cues listeners rely on are not abstract. They are fundamental frequency and its contour, intensity, timing, and the spectral characteristics of the glottal source. Acoustic correlates of the cues listeners report using. A system that claims to explain itself should be able to point at one of these. Cue | Acoustic correlate | Typical direction Pitch | F0 mean, range and contour slope | Raised in anger, fear, joy; lowered in sadness Loudness | Intensity, dynamic range | Raised in anger and excitement Rate | Speaking rate, pause frequency and length | Faster in fear and joy; slower in sadness Voice quality | Jitter, shimmer, harmonics-to-noise ratio | Breathy in fear; tense or creaky in anger Articulation | Formant precision, vowel space | Reduced in sadness and fatigue Two properties of this table matter for system design. First, no cue is diagnostic on its own — raised F0 with high intensity and tense phonation reads as anger, while raised F0 with breathy phonation and irregular pauses reads as fear. Emotion is carried by the configuration, not by any single dimension. Second, every correlate here also varies with speaker identity, language, and recording conditions, which is where most of the field's generalisation failures originate. Datasets The corpora the field relies on. All four are valuable; none of them sound like a phone call from a moving vehicle. Corpus | Language | Character IEMOCAP | American English | Dyadic dialogue, multiple emotions RAVDESS | English | Acted speech, professional actors EmoDB | German | Strong, clearly separated expressions CREMA-D | English | Diverse speakers, crowdsourced labels Three omissions run through all of them. They are predominantly acted rather than elicited, which produces expressions that are clearer and more prototypical than spontaneous speech. They are recorded in quiet conditions with close microphones. And they are dominated by a small number of languages and speaker populations. A model trained on this material learns to recognise performed emotion in studio conditions, and the field's headline accuracies are quoted on that task. The consequence shows up as a cross-corpus generalisation gap: models that score well within a corpus lose a substantial margin when evaluated on a different one, even for the same nominal label set. That gap, rather than within-corpus accuracy, is the number that predicts field behaviour. Deep learning approaches CNN and LSTM models capture local and temporal emotion cues from spectrograms. Transformer speech encoders — wav2vec, HuBERT, Whisper — learn deeper representations from large unlabelled corpora and remain effective under noise where earlier architectures degrade sharply. The reason self-supervised encoders help here is specific rather than general. Emotional speech corpora are small — thousands of utterances where a speech recognition corpus has thousands of hours — so a model trained from scratch on them overfits to speaker identity and recording channel. Pre-training on large unlabelled speech gives representations that already separate phonetic content from speaker and channel characteristics, which is precisely the factorisation an emotion head needs. Fine-tuning then has to learn far less. CNN over spectrograms: strong on local spectral texture, weak on long-range prosodic contour. LSTM and temporal convolution: model contour and timing, but degrade with channel mismatch. Self-supervised transformers (wav2vec 2.0, HuBERT): robust representations, the current default backbone, and the layer at which emotion is most decodable is generally not the final one. Whisper encoders: trained on weakly supervised multilingual audio, so they tolerate real-world noise and accent variation better than corpus-trained alternatives. One practical detail is worth stating because it is easy to get wrong: emotional information in these encoders is not concentrated in the top layer. Middle layers frequently carry more of it, since upper layers are optimised toward the pre-training objective's linguistic targets. Probing layer by layer before choosing where to attach the head is cheap and usually pays. Cognitive interpretability These models converge on cues similar to those humans use. Attention mechanisms and explainable-AI techniques can surface why a model classified a segment as angry or sad by pointing at the part of the signal that drove the decision. This is what turns a score into something an operator can accept or override. A classifier that outputs a label and a confidence, with no account of the cue behind it, cannot be audited — and therefore cannot be deployed anywhere the answer matters. Environmental context Recognition accuracy varies with acoustic background. Detecting anger or excitement is materially harder in heavy traffic, where loudness and pitch cues overlap with ambient noise. A model trained on studio-clean acted speech will report that overlap as emotion with full confidence. Context-aware processing is therefore not a refinement — without it, the system is confidently wrong in exactly the environments real calls originate from. Proposed architecture Audio preprocessing that treats the acoustic environment as an explicit input rather than an assumed constant. Feature extraction combining the cues humans use — pitch, loudness, rate — with learned representations, rather than replacing one with the other. A transformer speech encoder such as wav2vec or Whisper, fine-tuned on emotional data. Contextual adaptation that adjusts inference according to background conditions. Output pairing each label with the cue that produced it, so the reasoning can be disputed and not just the score. Adaptive learning that personalises to individual speakers over time; one voice's baseline is not another's. Evaluation Weighted accuracy on a held-out split of a single corpus is the field's default metric and it is close to useless for predicting deployment behaviour. Emotional corpora are class-imbalanced, so weighted accuracy rewards predicting the majority class; and a within-corpus split shares speakers, channel and elicitation protocol between train and test, so it measures memorisation of the recording setup as much as recognition of emotion. Report unweighted average recall alongside accuracy, so minority classes cannot be ignored for free. Evaluate cross-corpus, training on one dataset and testing on another. The drop is the honest number. Enforce speaker-independent splits. A speaker appearing in both train and test inflates every metric reported. Test under additive noise and channel simulation at realistic signal-to-noise ratios, not only on clean audio. Report calibration, not just correctness — a system whose confidence means nothing cannot support an escalation threshold. Compare against human inter-annotator agreement on the same material. Where annotators disagree, the ceiling is not 100%, and a model exceeding that agreement is fitting annotator idiosyncrasy. The last point deserves emphasis because it changes what a good result looks like. Emotion labels are perceptual judgements with genuine disagreement between raters. Treating a majority-vote label as ground truth discards that disagreement, and a model trained to reproduce it is being asked to be more certain than the humans were. Deployment considerations Moving from a benchmark to a live system introduces constraints the literature rarely addresses. A voice agent running this inference has a latency budget measured in tens of milliseconds, shared with transcription and response generation. Affect estimates must be produced incrementally over a stream rather than over a complete utterance, which means the system commits to a reading before the evidence is complete and must be able to revise it. There are also consequences that are not technical. Inferring emotional state is a stronger claim about a person than transcribing what they said, and it is more likely to be wrong in ways that correlate with speaker characteristics — accent, age, vocal health, cultural norms of expression. A system that routes calls differently based on inferred anger will route some people differently for reasons that have nothing to do with their emotional state. Prefer influencing routing and escalation over recording a permanent judgement about a person. Expose the cue behind every inference so a human can overrule the reasoning rather than only the outcome. Set thresholds per deployment, using calibration data from that acoustic environment. Audit outcomes by speaker group; correlated error is the failure mode that matters and it is invisible in aggregate accuracy. Treat low confidence as a first-class output. Abstention is a valid answer and usually the correct one under mismatch. Limitations and open questions This work is a synthesis and an architectural proposal, not an empirical study. It does not report benchmark results for the proposed system, and the architecture's central claim — that tying outputs to named acoustic cues improves operator trust without materially costing accuracy — is untested here and is the obvious next step. Whether cue-grounded explanations survive contact with real operators, or are ignored the way most confidence scores are. How much of the cross-corpus gap is closed by context-aware preprocessing versus by better pre-training. Whether speaker adaptation over a call improves accuracy enough to justify holding per-speaker state, with the privacy cost that implies. How categorical and dimensional outputs should be reconciled when a downstream system needs one decision. Conclusion Integrating cognitive insight with deep learning yields voice emotion systems that perform well and remain interpretable to the people operating them. Accounting for environmental context improves both accuracy and usability. The result is applicable to customer service, healthcare and personal assistants — domains where being wrong quietly is worse than being uncertain loudly. The argument throughout has been that accountability is an architectural property rather than a reporting one. A system that cannot name the cue behind its inference cannot be corrected, cannot be audited, and should not be given authority over how a person is treated — regardless of what it scores on a held-out split. ## Notes ### What a voice says before the words do URL: https://diby.app/notes/what-voice-says-before-words-do Published: 2026-06-02 | 6 min | Topics: Speech, Emotion recognition, Voice systems A speech system can transcribe perfectly and still handle a call badly, because the words were fine and the caller was furious. Notes on why affect belongs in the pipeline, and why a confidence score is not an explanation. Every voice product eventually meets the same failure. The transcript is perfect. The intent classification is correct. The response is well-formed and entirely wrong, because the caller was three minutes into an escalating problem and the system answered as though they had just said hello. Words carry what someone is asking for. Prosody carries how much trouble you are in. Humans read the second channel constantly and mostly without noticing — pitch, loudness, rate, voice quality — and it arrives earlier than the sentence does. Two traditions that rarely talk There is a cognitive literature describing how people hear emotion: Ekman's categorical account, Russell's valence-arousal plane, Scherer's component process model with its multiple vocal cues. And there is a deep learning literature that classifies emotion from spectrograms with CNNs and LSTMs, and more recently with transformer speech encoders — wav2vec, HuBERT, Whisper — that hold up far better in noise. The two bodies of work rarely constrain each other, and that is the gap my paper is about. The interesting question is not whether a transformer can beat a benchmark. It is whether the model can be made accountable to the cues a person would have cited. Why accountability is a shipping requirement A classifier that outputs "angry, 0.91" and nothing else cannot be deployed into a clinic or a contact centre. Not because the number is wrong, but because there is no way to audit it, no way to argue with it, and no way to explain to the person it was wrong about what happened. Attention maps and post-hoc explanation methods that point back at the actual signal — this pitch rise, that loudness spike — turn a score into something a human operator can accept or override. That is the difference between a research result and a feature. The context problem the benchmarks hide Most speech emotion datasets are acted and studio-clean. IEMOCAP, RAVDESS, EmoDB, CREMA-D are all valuable and none of them sound like a phone call from a moving car. This matters more than a few points of accuracy. A raised voice on a noisy street is not the same signal as a raised voice in a quiet office — loudness and pitch cues overlap with ambient noise, and a model trained on clean audio will report the overlap as emotion with complete confidence. Context-aware processing is not a refinement here; without it the system is confidently wrong in exactly the environments real calls come from. The full paper covers the psychological grounding, dataset landscape and proposed architecture in more detail. Where this lands in a real pipeline Preprocess with the acoustic environment as an explicit input, not an assumed constant. Combine the cues humans use — pitch, loudness, rate — with learned representations, rather than replacing one with the other. Fine-tune a transformer speech encoder on emotional data rather than training from scratch. Emit a label with the cue that produced it, so an operator can disagree with the reasoning and not just the score. Adapt to individual speakers over time; the baseline for one voice is not the baseline for another. None of this is exotic. It is mostly a matter of refusing to treat affect as a bolt-on classifier and instead giving it the same standing in the pipeline as transcription — with the same expectation that it can be inspected when it gets something wrong. --- ### Profiling a MoE model without loading it URL: https://diby.app/notes/profiling-moe-models-without-loading-them Published: 2026-05-14 | 5 min | Topics: Mixture of Experts, Inference, Tooling A 48-layer, 512-expert checkpoint profiled in 4.98 seconds on CPU. The trick is that expert routing is decided by a few megabytes of router weights, not by the hundreds of gigabytes behind them. The obvious way to find out which experts a Mixture-of-Experts model uses is to run the model. Load the checkpoint, install hooks on every gate, push prompts through, record what the router picks. It works, it is accurate, and it requires enough memory to hold a model you may be trying to shrink precisely because it does not fit. There is a cheaper path, and it comes from a structural observation: the routing decision does not depend on the experts. It depends on the router — one weight matrix per MoE layer, mapping a hidden state to a score per expert. Everything else in the checkpoint is what happens after the decision. Reading only the routers Safetensors checkpoints are indexed, so you can pull individual tensors without materialising the rest of the file. Load only the router weight matrices — a few megabytes across the whole model — probe each one with random unit-norm hidden states, and record which experts come out on top. Router-only profiling runs, on CPU. Model | Layers | Experts | Samples | Elapsed Qwen3-Coder-Next (FP8) | 48 | 512 | 1,024 | 4.98 s OLMoE-1B-7B | 16 | 64 | 512 | 0.38 s Five seconds for a 48-layer sweep changes what the tool is for. At that cost, profiling stops being an experiment you schedule and becomes something you run while deciding whether to bother. What random probes can and cannot tell you The honest limitation: random hidden states are not your tokens. What this measures is the router's structural preference — the shape of the decision boundary in isolation — rather than the distribution your workload induces. Good for: a first pass, structural comparison between layers, spotting cold experts, deciding whether a model is worth pruning at all. Not sufficient for: a production keep-list on a domain-specific workload. Complementary to: full-model mode with your real prompts, and the traffic daemon. Closing the loop with live traffic The strongest signal is the one your users generate. The daemon runs as a transparent HTTP proxy in front of any OpenAI-compatible server — ollama, vLLM, llama.cpp, LM Studio — and accumulates routing statistics from real requests in a background thread. It works using the same observation. The daemon loads the router weights and the embedding table, extracts prompt text from each forwarded request, tokenises it with the checkpoint's own tokenizer, and pushes the token embeddings through the routers itself. The backend serves the request untouched; nothing is added to the response path. python3 moe_monitor.py daemon \ --model-dir /path/to/checkpoint \ --backend http://localhost:11434 \ --listen-port 8080 \ --keep-experts 128 \ --new-topk 4 \ --report-every 50 The iterative version of this is the one I would actually recommend. Profile cheaply, prune once, and the resulting checkpoint may finally fit in memory — at which point you can afford full-model mode on it and get a far better ranking for the second pass. --- ### Expert routing is flatter than you think URL: https://diby.app/notes/expert-routing-is-flatter-than-you-think Published: 2026-04-21 | 7 min | Topics: Mixture of Experts, Routing, Model compression Profiling all 48 layers of Qwen3-Coder-Next gives a routing Gini of 0.1006. Almost nothing is dead, almost nothing dominates — which changes what pruning a Mixture-of-Experts checkpoint can and cannot buy you. The folk model of a Mixture-of-Experts layer is that it degenerates. You train 512 experts, a few dozen win, the rest sit cold, and pruning is a matter of deleting the dead weight. It is a satisfying story and it makes compression sound easy. It did not survive contact with the measurements. Profiling all 48 MoE layers of Qwen3-Coder-Next with router-only probes produced a routing Gini coefficient of 0.1006 — much closer to uniform than to collapsed. The busiest expert in the model was selected 1,453 times across 491,520 router selections; the quietest was selected 488 times. A factor of three between the extremes of 512 experts is not a winner-take-all distribution. Expert selection counts across 48 layers × 512 experts, router-only mode, 1,024 probe samples. The absence of strong banding is the result: utilisation is broadly spread rather than concentrated in a few columns. What the concentration curve says The clearest way to read the distribution is to ask what share of routing mass the busiest experts take, and compare it against what uniform routing would give them. Routing mass by expert percentile, Qwen3-Coder-Next, 48 layers, 1,024 samples. The third column is the ratio to uniform. Bucket | Observed share | Uniform | Ratio Top 1% | 1.42% | 1% | 1.42× Top 5% | 7.02% | 5% | 1.40× Top 10% | 13.22% | 10% | 1.32× Top 20% | 25.04% | 20% | 1.25× Top 50% | 57.12% | 50% | 1.14× Every bucket is above uniform, so there is real preference in the router — but the largest ratio anywhere in the model is 1.42×. Compare that against the mental model where the top 10% of experts carry most of the forward pass. That model would put the top-10% share somewhere north of 60%. The measured value is 13.22%. Gini 0.1006. For reference, a perfectly uniform distribution is 0 and a fully collapsed one approaches 1. Cold experts are rare and unevenly placed If experts are not concentrated, are any of them actually unused? Barely. Across 48 layers the mean number of cold experts per layer was 3.396 out of 512 — about 0.66%. But the distribution of that scarcity is itself uneven: 45 of the 48 layers had at least one cold expert, and one layer had 27. That asymmetry is the practical argument for per-layer pruning over a single global ranking. A global keep-list applies one model-wide opinion about which experts matter to a layer that may disagree. The per-layer strategy ranks within each layer and keeps that layer's own winners. So what does pruning actually buy? Cutting 512 experts down to 128 — a 4× reduction — retained 99.34% of observed routing mass. That number is better than the flat distribution would suggest, and the reason is that top-k routing is being reduced alongside the expert count, from k=10 to k=4. You are not only removing experts; you are also asking the router to commit harder to the ones that remain. The honest framing is this: because routing is close to uniform, pruning is not free salvage of dead capacity. It is a real trade, and coverage is the number that tells you how much you traded. A run that reports 99.34% coverage has left 0.66% of observed routing mass unserved, and those tokens now route somewhere else. Flat routing means pruning is a decision about acceptable loss, not a free lunch. Coverage is how you price it. Reproducing this Router-only mode loads just the router weight tensors — a few megabytes — and probes each router with random unit-norm hidden states. The full 48-layer sweep took 4.98 seconds on CPU. No GPU, no model load. python3 moe_monitor.py router-only \ --model-dir Qwen/Qwen3-Coder-Next-FP8 \ --output ./stats/report-512.json \ --keep-experts 128 \ --new-topk 4 \ --samples 1024 \ --strategy per-layer The caveat worth stating plainly: random probes measure the router's structural preferences, not what your traffic does. A workload concentrated on one domain will produce a very different picture. That is what full-model mode and the traffic daemon are for — and the gap between the two is itself worth measuring. ## Writing ### Smart Triage: The Cure for Spam Calls and Lost Patients Published: 2025-10-28 | Category: Healthcare | 4 min read URL: https://blog.antengage.com/smart-triage-for-clinics Spam, wrong leads and wasted time — why your front desk needs something better. Opening: Your phone rings every 40 seconds. Half of them are real patients, the rest are insurance chasers, lab partners… ### The Patient Engagement Maze: Quality vs Quantity Published: 2025-09-29 | Category: Healthcare | 3 min read URL: https://blog.antengage.com/the-patient-engagement-maze-quality-vs-quantity Every clinic tries the same fixes. Each one opens another leak. Play the maze — each move looks like progress, and each one has a cost. Opening: Boss walks in: “Leads are flat. Push more calls.”… ### The Follow-Up Leak After the Visit — "India clinic reality check" Published: 2025-08-18 | Category: Healthcare | 5 min read URL: https://blog.antengage.com/missed-follow-ups-clinic-india Addressing follow-up challenges in Indian clinics: identifying the leaks and implementing solutions for better patient outcomes. Opening: Imagine a patient leaves with a lab or imaging order, suggested by the doctor, but never comes back… ## Products ### AntEngage URL: https://antengage.com Site: https://antengage.com Role: Founder & CTO | Category: AI foundation models The company I lead as Founder & CTO, and where the foundation model work happens. We are building model architectures from first principles rather than deriving them from the transformer lineage — the thesis being that the efficiency and capability ceiling of the current generation is an artefact of its architecture, not of what is possible. AntEngage also ships conversational AI for engagement, which is where much of my voice and real-time systems work comes from. ### PCBEditor URL: https://diby.app/products/pcbeditor Site: https://pcbeditor.com Role: Founder at RoboDIB | Category: Electronic design automation An AI-native circuit design tool. Describe a circuit in plain English and it generates the schematic, the circuit itself, component footprints, the netlist and the routed traces — then runs AI design-rule checking, handles copper pour, and renders the layer stack and the finished board in 3D. Most tools bolt a model onto one step of the flow; here every stage is generated rather than drawn. Edit by chat, export to KiCad or Gerber when it is ready to fabricate. Runs in the browser with nothing to install, and it is free. Built at RoboDIB. Capabilities: - AI schematic and circuit generation from a plain-English description - AI-generated component footprints - AI netlist extraction and trace routing - AI design rule checking (DRC) - AI copper pour - 3D layer stack-up and 3D board visualisation - Chat-based editing of an existing design - Export to KiCad and Gerber - Browser-based, nothing to install, free ### RoboDIB URL: https://robodib.com Site: https://robodib.com Role: Founder | Category: Robotics & hardware My hardware startup — where the robotics work lives and where PCBEditor was built. It is the physical counterpart to the model work at AntEngage, and the reason 'AI and robotics' shows up in how I describe what I am building. ### Fonix AI URL: https://fonix.ai Site: https://fonix.ai Role: Builder | Category: Voice AI A voice and messaging layer for AI agents — placing and receiving calls, handling WhatsApp and email, and exposing the whole thing programmatically so an agent can run a conversation end to end rather than just draft one. ### AIVoice Live URL: https://aivoice.live Site: https://aivoice.live Role: Builder | Category: Voice AI Focused on the streaming, low-latency end of conversational systems — where architecture decisions are felt immediately by whoever is on the line. ## Talks and tutorials ### What is gRPC — HTTP vs gRPC for your next API URL: https://diby.app/videos/grpc-vs-http-for-your-next-api Watch: https://www.youtube.com/watch?v=WGhdVpvvlnU Published: 2024-04-04 | Channel: GeekyAnts | Topics: APIs, gRPC, Distributed systems HTTP is universal, human-readable and understood by every tool in the stack. gRPC is faster on the wire, strongly typed at the boundary, and awkward everywhere a browser is involved. This conversation works through which properties actually decide the choice for a new API. What the comparison is actually about HTTP with JSON is the default for good reasons. It is platform-independent, inspectable with tools everyone already has, tolerant of schema drift, and supported by every client that has ever existed. Those properties are worth more than they look, and they are the reason most APIs should stay exactly where they are. gRPC trades several of them away. It uses HTTP/2 as a transport and Protocol Buffers as a wire format, which makes serialisation compact and fast and gives both sides a schema they must agree on. What you lose is readability on the wire, casual debugging, and — without a proxy layer — direct browser support. Where the trade pays Service-to-service traffic inside your own infrastructure is where gRPC is at its strongest. The calls are frequent, the payloads are repetitive, both ends are yours, and the schema is an asset rather than an obstacle. Streaming is first-class rather than bolted on, which matters for anything that pushes rather than polls. The typing is often underrated as a benefit. A generated client that will not compile against a changed message is a whole class of integration bug caught before deployment rather than during it. Where it does not Public APIs consumed by third parties, anything a browser calls directly, and low-traffic endpoints where serialisation was never the bottleneck. In those cases the operational cost — proxies, tooling, the fact that nobody can debug it with curl — outweighs a performance win you will not measure. The honest summary is that this is a decision about coupling and traffic shape, not about which protocol is better. Key points: - HTTP wins on ubiquity, inspectability and tolerance of change. - gRPC wins on serialisation cost, first-class streaming and an enforced schema. - Internal service-to-service traffic is where the trade usually pays. - Public and browser-facing APIs are where it usually does not. --- ### Diving deep into GraalVM — native images with Spring Boot URL: https://diby.app/videos/graalvm-native-images-spring-boot Watch: https://www.youtube.com/watch?v=EbKIUNYdPPY Published: 2023-09-19 | Channel: GeekyAnts | Topics: JVM, GraalVM, Spring Boot GraalVM compiles a JVM application ahead of time into a native executable. Startup drops from seconds to milliseconds and memory footprint falls sharply, which changes what the JVM is viable for. The cost is a closed-world assumption that breaks the reflective tricks much of the Java ecosystem is built on. What native image changes A conventional JVM application starts by loading classes, verifying bytecode, interpreting, and eventually letting the JIT compile the hot paths. That warm-up is invisible in a long-running server and fatal in anything short-lived. GraalVM's native-image does the compilation ahead of time and produces a standalone binary with no JVM to start. Startup moves from seconds to milliseconds and resident memory drops substantially, because there is no class loading, no bytecode verification and no JIT infrastructure to carry. The closed-world assumption The ahead-of-time compiler must know at build time everything the program might reach. That is the whole basis of the optimisation, and it is also the constraint that makes the migration non-trivial: reflection, dynamic proxies, runtime classpath scanning and dynamic class loading are exactly what a lot of Java frameworks — Spring included — have historically relied on. The practical answer is configuration and framework support. Spring's ahead-of-time processing generates the reflection and proxy hints the native compiler needs, which moves most of the work from the application author to the framework. What remains is a longer build and a real testing obligation: the native binary is a different artefact from the JAR and can fail in ways the JAR does not. When it is worth it Serverless functions, CLI tools, short-lived jobs, and anything scaling to zero — cases where startup dominates the lifetime. Also memory-constrained deployments where footprint per instance sets the bill. Long-running services with steady traffic are the weakest case. The JIT eventually produces better peak throughput than the ahead-of-time compiler, so trading it away for a startup time nobody experiences is a poor bargain. Key points: - Native image trades JIT peak throughput for millisecond startup and a smaller footprint. - The closed-world assumption is what makes it fast and what makes migration work. - Framework AOT processing handles most reflection hints for you. - Best for serverless, CLIs and scale-to-zero; weakest for steady long-running services. --- ### Resolving a location from an IP address in Java URL: https://diby.app/videos/ip-address-geolocation-in-java Watch: https://www.youtube.com/watch?v=LyijGBs1vKQ Published: 2020-05-27 | Channel: Programmatic DIB | Topics: Java, GeoIP, Backend Turning an IP address into a country, region and city is a routine requirement for analytics, fraud checks and localisation. Doing it against a local GeoIP database rather than a hosted API keeps the lookup fast, free per request, and keeps user IP addresses inside your own infrastructure. Why the lookup is a database problem IP geolocation is a range lookup. Address blocks are allocated to organisations and regions, and a GeoIP database is essentially a sorted mapping from address ranges to location records. Resolving an address means finding the block that contains it. Because it is a local lookup rather than a network call, it is fast enough to run inline on a request without thinking about it — which is not true of an API round-trip. Why not just call a hosted API Hosted geolocation APIs are easy to adopt and awkward to live with. They put a per-request cost on something that should be nearly free, add a network hop to a hot path, introduce a third-party dependency into your availability story, and quietly turn every user's IP address into data you are sending elsewhere. That last point is the one that tends to matter most in review. An IP address is personal data in several jurisdictions, and shipping it to a vendor on every request is a decision worth making deliberately rather than by default. Accuracy expectations Country-level resolution is generally reliable. City-level is approximate and degrades badly for mobile networks, corporate egress and anyone behind a VPN. Any product decision built on top of it should be tolerant of being wrong — geolocation is a hint, not an identity. Key points: - A GeoIP lookup is a local range query, fast enough to run inline. - Self-hosting avoids per-request cost, a network hop and a data-sharing decision. - Country-level accuracy is good; city-level is approximate. --- ### Camera text recognition on Android — building a Google Lens-style OCR app URL: https://diby.app/videos/android-camera-text-recognition-ocr Watch: https://www.youtube.com/watch?v=G7wmpZE-qow Published: 2020-06-05 | Channel: Programmatic DIB | Topics: Android, OCR, Computer vision Reading text off a live camera feed is a pipeline problem rather than a single API call: acquire frames, hand them to a recognition engine, and reconcile a stream of noisy per-frame results into something stable enough to show a user. The pipeline Three stages. Camera frames arrive continuously; a detector runs recognition on a frame; the result is drawn back over the preview. The interesting engineering is in the gaps between those stages rather than in any one of them. Recognition is slower than the frame rate, so every frame cannot be processed. The usual approach is to process the most recent frame whenever the detector is free and drop the rest, which keeps latency bounded instead of building a backlog. Why on-device matters here On-device recognition means no network round trip per frame, which is what makes live overlay feel immediate rather than laggy. It also means the camera feed never leaves the phone — a meaningful property for an application pointed at documents. Where accuracy comes from Most real-world OCR failures are acquisition failures rather than model failures: motion blur, poor lighting, extreme angle, or text too small in frame. Guiding the user toward a better frame does more for perceived accuracy than any post-processing. Temporal aggregation helps too. A single frame is noisy; agreement across several consecutive frames is a much stronger signal, and it stops the overlay flickering between readings. Key points: - Process the latest frame and drop the rest — never queue frames behind a slow detector. - On-device recognition buys both latency and privacy. - Most OCR errors are acquisition problems; guide the user to a better frame. - Aggregating across frames stabilises a noisy per-frame result. --- ### Which language should you use for Android development? URL: https://diby.app/videos/best-language-for-android-development Watch: https://www.youtube.com/watch?v=SLAv8sKCSqA Published: 2020-05-25 | Channel: Programmatic DIB | Topics: Android, Kotlin, Java Android can be written in Java, Kotlin, C++ through the NDK, or any of several cross-platform frameworks. The languages differ less in capability than in how much ceremony they demand and how well they match the platform's own direction. The realistic options Java is the platform's original language and still runs an enormous amount of shipped code. Kotlin is the language Google now leads with — it interoperates completely with Java, removes a great deal of boilerplate, and handles nullability in the type system rather than at runtime. C++ through the NDK is for the narrow set of cases where you need it: existing native libraries, or workloads where the managed runtime is genuinely the bottleneck. Cross-platform frameworks trade some platform fidelity for sharing a codebase with iOS. How to actually choose For a new native Android application the answer is Kotlin, and the reason is ecosystem direction rather than language aesthetics — documentation, samples and library APIs increasingly assume it. For an existing Java codebase there is no need to rewrite. The interop is good enough that new code can be Kotlin while old code stays as it is, which is how most migrations actually happen. Cross-platform is a product decision rather than a technical one. It is the right call when the shared surface is large and platform-specific behaviour is minimal, and the wrong call when the application lives or dies on feeling native. Key points: - Kotlin for new native Android work — the ecosystem has moved. - No need to rewrite Java; interop lets both coexist. - NDK only where a native library or a real performance constraint requires it. - Cross-platform is a product trade-off, not a language comparison. --- ### Python CGI programming — the architecture, and a first program URL: https://diby.app/videos/python-cgi-programming-architecture Watch: https://www.youtube.com/watch?v=-OA5Qh8_7E8 Published: 2020-05-31 | Channel: Programmatic DIB | Topics: Python, Web fundamentals, CGI CGI is the oldest way to make a web server run a program, and understanding it explains why every framework since has been built the way it is. One process per request is a beautifully simple model with exactly one fatal property. The model A request arrives. The server starts a new process, hands it the request through environment variables and standard input, and reads the response off standard output. The process exits. Nothing is shared and nothing survives. That isolation is the model's great virtue. There is no shared state to corrupt, a crash affects exactly one request, and the program is trivially easy to reason about because it starts clean every time. Why nothing works this way any more Process creation is expensive, and CGI pays that cost on every single request. Under load the server spends more time forking than it does answering. Everything that followed — FastCGI, mod_python, WSGI, ASGI and the application servers built on them — exists to keep the interpreter alive across requests and amortise the startup. The trade is that state now persists between requests, which is where a large share of modern web bugs come from. CGI did not have that problem because it did not have that opportunity. Why it is still worth understanding The request/response contract CGI established — environment for metadata, stdin for the body, stdout for the response — is essentially still the interface WSGI and ASGI present, with the process boundary removed. Learning it makes the abstractions above it legible rather than magical. Key points: - CGI runs one fresh process per request: perfect isolation, terrible throughput. - Everything since exists to amortise interpreter startup across requests. - Persistent state is the cost of that optimisation, and the source of new bug classes. - The request/response contract it defined still shapes WSGI and ASGI. ## Shorts ### Why vLLM? URL: https://diby.app/videos/why-vllm Watch: https://www.youtube.com/shorts/TndMkkajWpg Published: 2026-03-15 | Topics: LLM inference, Serving, vLLM Serving a language model well is mostly a memory management problem. vLLM's contribution is treating the KV cache like virtual memory — paging it — which raises how many requests you can hold concurrently far more than a faster kernel would. The bottleneck is memory, not compute Every in-flight request holds a key-value cache proportional to its context length. Naive serving allocates that cache contiguously for the maximum possible length, so most of the reserved memory is never used — and the wasted space, not the GPU's arithmetic throughput, is what caps concurrency. vLLM's PagedAttention allocates the cache in fixed-size blocks that need not be contiguous, the same trick operating systems use for virtual memory. Fragmentation collapses, more requests fit at once, and throughput rises without touching the model. Why this matters beyond throughput Blocks that are not tied to a single sequence can be shared. Requests with a common prefix — a system prompt, a shared document — can reference the same cached blocks instead of each holding a copy, which is a large saving in exactly the workloads production systems actually run. Key points: - LLM serving is capped by KV cache memory before it is capped by compute. - PagedAttention pages the cache in blocks, removing fragmentation. - Shared prefixes can share blocks rather than duplicating them. --- ### This is why quantization matters URL: https://diby.app/videos/why-quantization-matters Watch: https://www.youtube.com/shorts/MnGwUyYud84 Published: 2026-03-18 | Topics: Quantization, LLM inference, Model efficiency Quantization stores weights at lower precision — 8-bit or 4-bit instead of 16. The obvious win is that the model fits. The less obvious and often larger win is that inference is memory-bandwidth bound, so moving fewer bytes per token makes it faster too. Two wins, not one The first is capacity: halving precision roughly halves the weight memory, which decides whether a model fits on the hardware you have at all. The second is speed, and it surprises people. Generating a token requires reading essentially every weight. That makes decoding memory-bandwidth bound rather than compute bound, so halving the bytes read per token can nearly halve the time per token — even though the arithmetic is unchanged. What it costs Precision loss is real but unevenly distributed. Most weights tolerate aggressive quantization; a small number of outlier channels do not, and naive uniform quantization on those is where quality collapses. Modern schemes handle outliers separately, which is most of why 4-bit is usable at all. The honest framing is the same as for pruning: this is a priced trade, and the price should be measured on your workload rather than assumed from a benchmark. Key points: - Lower precision buys capacity and speed, because decoding is bandwidth bound. - Quality loss concentrates in a few outlier channels, not uniformly. - Measure the cost on your own workload rather than trusting a benchmark number. --- ### Training an AI model is slow — back-propagation explained URL: https://diby.app/videos/why-training-is-slow-backpropagation Watch: https://www.youtube.com/shorts/FTk-6y6GJX0 Published: 2026-03-20 | Topics: Training, Back-propagation, Deep learning Training is slow because every step runs the network forwards, then runs a second pass backwards to compute how every parameter should change, and the backward pass needs the forward pass's intermediate values kept in memory the whole time. What the backward pass costs Back-propagation is the chain rule applied efficiently: compute the loss, then walk backwards through the network working out how much each parameter contributed to it. The backward pass costs roughly twice the forward pass, so a training step is around three times the work of inference on the same batch. That factor is the cheap part of the explanation. The expensive part is memory. Why memory is the real constraint The backward pass needs the intermediate activations the forward pass produced. They must be held from the moment they are computed until the gradient reaches them, which means activation memory scales with both batch size and depth — and it frequently, not the parameters, is what decides the largest batch you can run. Gradient checkpointing is the standard answer: discard most activations and recompute them during the backward pass, trading extra compute for a large reduction in memory. Choosing to do more arithmetic in order to use less memory is a recurring theme in this field. Key points: - A training step is roughly three times the work of inference on the same batch. - Activation memory, not parameter count, often caps batch size. - Gradient checkpointing trades recomputation for memory. --- ### Gradient descent explained URL: https://diby.app/videos/gradient-descent-explained Watch: https://www.youtube.com/shorts/1pGHhVhr79g Published: 2026-03-23 | Topics: Optimisation, Gradient descent, Deep learning Gradient descent is the whole of learning reduced to one instruction: measure which direction increases the error, and take a small step the other way. Everything else in an optimiser is a refinement of how large that step should be. The idea The gradient points in the direction of steepest increase of the loss. Step the opposite way and the loss goes down. Repeat a few million times and the parameters end up somewhere useful. The step size — the learning rate — is the parameter that decides whether this works. Too large and the updates overshoot and diverge; too small and training is correct but takes forever. Most practical difficulty in training is some version of this tension. Why it is stochastic in practice Computing the gradient over the entire dataset for a single step is prohibitive, so it is estimated from a mini-batch instead. The estimate is noisy, and the noise turns out to be useful — it helps the optimiser escape poor regions rather than settling into the first flat spot it finds. Modern optimisers build on this by keeping running statistics of past gradients to scale each parameter's step individually, which is why they converge in far fewer steps than plain gradient descent. Key points: - Step against the gradient; repeat. That is the entire learning rule. - The learning rate is the parameter that decides whether training works at all. - Mini-batch noise is a feature, not just a compromise. ## Consulting services Available for hire. Contact: dibyaprakash@robodib.com or WhatsApp +918763871481. ### General Consulting Open-ended advisory for founders and engineering leaders: what to build, what to buy, what to stop doing. Useful when the decision is expensive and reversible only at cost. ### Spring Boot Training Hands-on training for teams working in Java and Spring Boot — service design, testing, performance, and the patterns that keep an enterprise codebase workable three years in. Delivered from years of building and reviewing exactly these systems. ### Software Architecture Designing systems that hold up: service boundaries, data flow, streaming and messaging, and the cloud footprint underneath. Particularly relevant for AI and voice products where latency is a product feature, not an implementation detail. ### Software Strategy Turning a product ambition into a sequenced technical plan — what gets built in which order, what the team needs to exist first, and where the real risk sits. ### Team Building Strategy How to hire, structure and grow an engineering org that ships. Drawn from leading teams as CTO and from mentoring engineers through senior levels. ### Architecture Review A structured read of an existing system with findings you can act on: bottlenecks, coupling, failure modes, and the two or three changes that would matter most. ### Security Review An assessment of how a system handles authentication, tenancy, data isolation and its exposed surface — with prioritised remediation rather than an undifferentiated list. ### Frequently asked questions Q: What kind of work do you take on? A: Consulting and advisory across software architecture, AI and voice systems, technology strategy, team building, and architecture or security reviews. Engagements range from a single review to ongoing advisory alongside a team. Q: What are you strongest at? A: Conversational and voice AI systems, streaming architectures, and enterprise Java and Spring Boot. That combination — real-time AI on top of systems that have to stay up — is where most of my work sits. Q: Do you work with teams outside India? A: Yes. I'm based in Bengaluru and have run remote engagements since 2015 through Pyloom Innovations. Email or WhatsApp is the fastest way to sort out time zones. Q: How do I start a conversation? A: Email dibyaprakash@robodib.com or send a WhatsApp message. Mention which service you're interested in and enough context about the system or team, and I'll come back with whether I'm the right fit.