Feb 2026 — Present
Senior Consultant
Straive · Hyderabad · Data Science & AI
- Designed and built Jetstream, an AI-powered experimentation platform where A/B tests are planned, monitored, and decided in one place — a production CUPED variance-reduction engine, chi-square Sample Ratio Mismatch checks that can block a bad ship decision, and a daily AI briefing across the whole experiment portfolio. Built solo end to end: FastAPI + React, ~10,000 lines, 64 unit tests on the statistics engines, containerized and deployed as a feedback beta.
- Rebuilt the conversion covariate behind app experimentation, benchmarked eight variance-reduction estimators, and designed two original refinements — mechanism-matched metrics and a "Zero-to-Hero" noise-reduction technique that cut standard error by a median 9.1% across the 36 of 64 experiments where it applied, roughly 3× the ceiling of covariate tuning alone — taking conclusive experiments from 13 → 24 of 64 at full test coverage.
- Built the retrieval layer behind New For You, a personalized homepage carousel — clustering each shopper's history into interest centroids and retrieving against an 888K-product embedding space. Lifted ranking AUC to 0.88 from 0.49 against the user-embedding baseline, took shoppers scoring above chance from 47% → 94%, and nearly doubled catalog coverage to 57%.
- Shipped an executive out-of-stock dashboard quantifying $74.7M in annual recoverable revenue across the client's North America e-commerce, engineering a two-segment methodology that combines behavioral session data with inventory time-series modeling to deliver CFO-ready numbers.
- Built an end-to-end item-item similarity engine for 221K+ products, blending 1,024-dimensional semantic embeddings with structured attribute scoring to surface the ten most relevant items per product across a catalog driving $6.8B in sales.
- Ran persona-level demographic analysis across 2M+ customers, segmenting five behavioral personas to surface $234M in revenue patterns and inform merchandising strategy.
Read more on the technical approachShow less
- Jetstream — design discipline built into the product. Setup is a stepped flow ending in an "Experiment Contract" that records the pre-specified primary analysis method, locked at launch. The decision engine only uses the CUPED-adjusted result when CUPED was pre-specified beforehand and the covariate is eligible, and states which basis it used on screen — the guardrail that stops post-hoc method shopping. Standard and CUPED results are reconciled so no two screens can disagree: the point estimate never moves, only the standard error shrinks, so CUPED changes your confidence in a result and never its direction.
- Jetstream — AI that can't invent things. The daily monitoring pass scans the portfolio and diffs against the previous briefing, but the scan itself is deterministic: the model narrates it and cannot add or drop items, so briefings are reproducible and auditable. Generated insights must cite the evidence they came from — uncited insights are dropped rather than displayed. Anomaly detection runs statistically on daily per-variant series before the model is asked to explain a cause.
- New For You — diagnosis and honest evaluation. I diagnosed why the baseline failed rather than just beating it: user–product cosine similarity sat near 0.2 against roughly 0.9 for interest–product, meaning the user embedding wasn't living in the same space as the product embeddings. I then validated against what shoppers actually did next across 77,439 shoppers and 384,313 next-actions, which showed 75% of next-actions were re-engagement with already-seen products that the product excludes by design — so the novelty filter, not the model, caps the achievable recall. Recall@300 of 12.2% sits close to the ~16% structural ceiling, and I designed the evaluation with a historical cutoff to rule out leakage.
- Variance reduction — the finding that mattered more than the gain. I added 56 app-native behavioral features to the existing 64 and cross-fit predictions by experiment so no visitor's score came from a model that had seen their own test, avoiding the leakage that would bias CUPED. That covariate work hit a hard ceiling — correlation with conversion topped out near 0.24 and ROC-AUC near 0.68 across nine model families — so I proved the ceiling rather than keep chasing it, and moved the leverage elsewhere. Mechanism-matched metrics (judging each test on the metric its change actually targets) took detections from 13 → 20 of 64; "Zero-to-Hero," which imposes a known-zero effect on visitors who couldn't have been affected instead of estimating it, added a median 9.1% standard-error cut and brought the final count to 24. A looser 80% confidence bar would have shown 28, but roughly 7 of those are expected false positives, so I rejected it. The headline conclusion I delivered was that precision was never the bottleneck: the median non-significant experiment sat ~17× of traffic away from significance, so effect size and traffic were the real constraint — and I recommended bigger treatments and more sensitive primary metrics over more statistics.
Across all 64 evaluated experiments at 100% coverage; production variance-reduction methods covered under 50%. Zero-to-Hero applied to the 36 experiments with an eligible pre-exposure signal.