Research, tools, and experiments from our engineering team.
Labs is where we publish the work that doesn't fit a case study - open-source tools, AI benchmarks, datasets, and technical research from the StartxLabs engineering team.
Open experiments
Tools, datasets, and research we've made publicly available.
RAG Evaluation Harness
A lightweight Python harness for evaluating retrieval-augmented generation pipelines across faithfulness, relevance, and groundedness - with RAGAS integration.
Latency Benchmarks: LLM Inference Providers
We benchmarked 8 hosted LLM inference providers across P50/P95 latency, token throughput, and cost-per-1k tokens for production workloads.
Prompt Drift Detector
A CLI tool that tracks semantic drift in LLM outputs over time - useful for catching prompt sensitivity issues before they reach production users.
Clinical NER Benchmark
Annotated dataset of 12,000 de-identified clinical notes with named entity labels for conditions, medications, procedures, and dosages. Released under CC BY 4.0.
Agentic Loop Failure Modes
A taxonomy of failure modes observed in production agentic AI systems - from tool hallucination to infinite retry loops - with mitigation patterns.
Vector Store Migration Toolkit
Utilities for migrating embeddings between Pinecone, Weaviate, Qdrant, and pgvector without downtime - including dimension-mismatch handling.
How we think about Labs
Build in the open
We share real tools, real benchmarks, and real failure modes - not sanitised marketing content.
Practitioner-first
Everything here comes from engineering teams shipping AI to production, not from conference talks.
No hype
We only publish things we've tested ourselves. If a technique doesn't hold up, we say so.