About · Staff AI Engineer
Ankit Jha
I build AI systems end to end: agents, retrieval and the inference stack underneath them. 7 years of shipping ML and LLM products at Amazon, Meesho, Paytm, Innovaccer and PhysicsWallah, and a founder’s view of running one in production at MarketLane AI.
- Years shipping ML and LLM systems
- 7+
- Products under review moderation
- 2.5M
- Production AI systems shipped
- 10+
- Upstream PRs across 8 repos
- 14
01 / Story
How I got here
I started at Amazon in 2019, building machine learning and NLP models for the product catalog in three marketplaces. That is where I learned what scale does to a model: a 2.5M-product catalog in several languages has a long tail that no demo set captures.
At Meesho most of my work was deep learning: models for trust and safety and for search, trained and served on Spark and Databricks. At Paytm I built machine learning and deep learning systems across travel and payments, from train waitlist prediction and flight pricing to card fraud detection, and a multimodal LLM tool for merchant KYC.
Today I am a Staff AI Engineer at PhysicsWallah. Before that I led text-to-SQL work at Innovaccer. In parallel I build MarketLane AI, a market-intelligence platform for traders. Running a product alone meant owning everything around the model too: queues and workers, two clouds, tracing, alerting and the cloud bill.
That pulled me down the stack. I now contribute to vLLM, llm-d and NVIDIA Dynamo on request routing, scheduling and inference correctness. I like hard problems that need both research rigour and solid engineering, and I like fixing them at the root.
The whole stack, not one layer
Agents, retrieval, model serving and the infrastructure under them. I can debug a bad answer down to the prompt, the retriever, the router or the KV cache, and fix it where it actually breaks.
Measure before shipping
Every change is judged on accuracy, cost and latency together, against a bar set before the experiment runs. On MarketLane Research that meant two techniques shipped and eleven were rejected with a reason.
Built for real scale
Catalogs of millions of products, payment flows with millions of transactions, and multi-marketplace data in several languages. Correctness has to hold on the long tail, not the demo.
Own it in production
Queues, workers, tracing, alerting, cost ledgers and on-call. Running my own startup taught me that a model is a small part of an AI product, and operating it is most of the work.
02 / Experience
Where I’ve built
2026 – Present
PhysicsWallah (PW)Staff AI Engineer
Building production AI systems at one of India’s largest education platforms.
- Staff-level ownership of AI engineering: system design, model and serving choices, and evaluation that decides what ships.
- LLMs
- Agentic AI
- RAG
- Python
2025 – 2026
InnovaccerLead AI Engineer
Led text-to-SQL for a healthcare data platform: plain-language questions answered with correct, safe SQL over large and complex schemas.
- Owned the text-to-SQL problem end to end: schema and context retrieval, query generation, validation before execution, and evaluation of correctness on real questions.
- Treated accuracy as a measured property, with test sets and regression checks, so model and prompt changes are shipped on evidence.
- LLMs
- Text-to-SQL
- LangGraph
- RAG
- Langfuse
- Python
- SQL
2025 – Present
MarketLane AIFounder · in parallel
Building the context layer for the stock market: an AI-driven market-intelligence platform for technical analysts and swing traders.
- Built the AI stock-selection service and rebuilt it three times, from one cached LLM call to a verified multi-stage pipeline. The production version hits 90% on top setups for about $4.5 per 100-symbol scan. Case study →
- Tried and measured several agentic designs (single agent, analyst plus critic, multi-agent ranking) and shipped the one that balanced accuracy, cost and latency, not the most complex one.
- Built an AI research channel that answers any market question with cited evidence from SEC filings, XBRL facts and market data. Case study →
- Built trading journals that turn a trader’s own history into AI insights on patterns, mistakes and discipline.
- Owned all of the engineering and ops: queue-driven workers on AWS (SQS, Lambda), Vertex AI and Cloud Run on GCP, PostgreSQL with pgvector, OpenTelemetry and Langfuse tracing, CloudWatch alerting, SLOs, and a spend ledger that ties every model call to a stage and a dollar figure.
- Claude
- Gemini multimodal
- FastAPI
- PostgreSQL + pgvector
- AWS
- GCP
- Langfuse
- OpenTelemetry
- Phoenix
2023 – 2025
PaytmSenior Data Scientist
Machine learning, deep learning and LLM systems for travel and payments, on data with millions of transactions and users.
- Train waitlist confirmation: predicted the probability that a waitlisted ticket confirms, shown to users at booking time.
- Dynamic flight pricing: models that adjust fares to demand and availability.
- Credit and debit card fraud detection on high-volume, real-time transaction streams.
- Multimodal LLM tool for shopkeeper KYC: reads documents and storefront images to verify merchants and cut manual review.
- PyTorch
- Multimodal LLMs
- PySpark
- Airflow
- Triton
- Kubernetes
- AWS
2022 – 2023
MeeshoData Scientist II
Deep learning for trust, safety and search on one of India’s largest e-commerce marketplaces.
- Built the compliance platform that uses deep learning models to find illegal and prohibited listings across the catalog and block new ones before they go live.
- Developed a transformer-based search retrieval service that improved how well products match what shoppers search for.
- PySpark
- Databricks
- TensorFlow
- PyTorch
- Transformers
- Milvus
2019 – 2022
AmazonData Scientist · Research Analyst
Machine learning and NLP for the product catalog in the US, Italy and Spain marketplaces.
- Engineered a transformer-based review moderation system over a catalog of 2.5M products. It separates on-topic from off-topic reviews per product so the off-topic ones can be removed.
- Standardized product sizes for 1M products on Amazon Italy and Spain, resolving cases where the description, attributes and images disagree on the size.
- SageMaker
- PyTorch
- Hugging Face
- Python
- SQL
- AWS
03 / Open source
Working upstream
I contribute to the open-source LLM serving stack, on the problems that only show up under real traffic: request routing and flow control, prefill/decode disaggregation, session affinity, prefix-cache management and memory-safety bugs. 8 of my 14 pull requests across 8 repositories are merged upstream so far. Each one has a write-up with the root cause and the test that reproduces it.
- vLLMInference engine, prefix cache, router
- llm-dKubernetes-native routing and scheduling
- NVIDIA DynamoDistributed serving, NIXL transfer
04 / Toolkit
What I work with
AI and GenAI
- LLMs
- Agentic AI
- RAG
- Text-to-SQL
- NLP
- Multimodal AI
- PyTorch
- TensorFlow
- Keras
Agents and LLMOps
- LangGraph
- Langfuse
- Arize Phoenix
- OpenTelemetry
- Evaluation design
Inference and serving
- vLLM
- llm-d
- NVIDIA Dynamo
- Triton
- Truefoundry
Infra and ops
- AWS
- GCP
- Kubernetes
- Docker
- Airflow
- Grafana
- CloudWatch
- CI/CD
Data
- PySpark
- Databricks
- SQL
- PostgreSQL + pgvector
- Milvus
- Distributed ML
Languages
- Python
- C++
- Go
- Rust
- Java
05 / Education
Education
2015 – 2019
International Institute of Information Technology, Bhubaneswar
Bachelor of Technology in Computer Science and Engineering
Get in touch
Hiring for a senior or staff AI engineer, or building something hard?
I’m happy to talk with teams working on agents, retrieval, or LLM inference at scale, and with founders who need someone who can take an AI product from prototype to production. Open to full-time roles and to consulting engagements.