Synth

What do they actually do

Synth ships a serverless post‑training platform that helps teams improve multi‑step AI agents by running controlled experiments (prompt edits or model weight updates), scoring them with custom judges, and returning concrete outputs to merge back into production (updated prompts, suggested code changes, or fine‑tuned models). It’s exposed as serverless APIs with an SDK/CLI and real‑time dashboards; teams can wrap existing agent code behind simple HTTP routes so they don’t need to re‑architect their app to use it (docs). Two built‑in prompt optimizers—GEPA and MIPRO—run variants and compare results across benchmarks; the platform also supports supervised fine‑tuning and reinforcement learning with cost/budget controls and multi‑provider model support (OpenAI, Gemini, Groq, etc.) (algorithms, benchmarks, serverless post-training APIs).

Today, early customers are teams building vertical, multi‑step agents. Public posts mention closed‑beta usage at the scale of “tens of thousands” of agent trajectories and show measurable gains on benchmarks (e.g., Banking77 87% → 100% with GEPA; YC cites a 33% relative improvement on a SWE‑Bench agent) (YC profile, LinkedIn, benchmarks). The company maintains examples and a Python client intended for CI/production workflows (GitHub). Looking ahead, their roadmap emphasizes automating more of the “research engineer” loop (hosted judges, multi‑stage/workflow optimization, broader model/topology support, and cost/UX improvements) (changelog, workflows overview).

Who are their target customer(s)

• Early-stage startups building multi-step AI assistants: They need more reliable behavior without long manual experiment cycles. Failures are hard to reproduce, and they lack cheap tooling to run many variants and see which changes actually help.
• Research engineers / ML engineers responsible for agent performance: They spend weeks on ad‑hoc experiments and custom evaluators, with high provider costs and no automated loop to run, score, compare, and safely roll back changes.
• Product teams with customer-facing automation (support bots, code assistants, document workflows): When agents fail, it drives churn or support load. It’s hard to tell if the issue is the prompt, model, or orchestration, and fixes risk breaking other flows without repeatable evaluation.
• MLOps / platform teams integrating agents into CI/CD: They lack observable, production‑safe experiments that enforce cost limits and produce reviewable artifacts (updated prompts/checkpoints). Deploying improvements is manual and risky.
• Small AI teams with limited labeled data or annotation budgets: They can’t afford big labeling or training runs. Trial‑and‑error is costly without automation to test many prompt variants or small model updates with objective scoring.

How would they acquire their first 10, 50, and 100 customers

• First 10: Founder/engineer‑led pilots with hands‑on instrumentation of one failing flow, running optimizations, and delivering a measurable improvement the team can merge. Source pilots via YC/early‑startup networks and targeted outreach, using a one‑page playbook and GitHub example to keep friction low.
• First 50: Productize self‑serve with templates, CI/CD examples, and reproducible notebooks; run workshops showing “before/after” on common failures. Nurture a community channel and convert active users into paid pilots with clear success criteria.
• First 100: Ship one‑click integrations (CI templates, GitHub Actions) and list in model/agent marketplaces; add a light product‑led sales motion to close mid‑sized pilots with SLAs. Use published benchmarks and 3+ case studies to drive inbound, and a referral/credits program to accelerate growth.

What is the rough total addressable market

Top-down context:

Industry reports estimate the AI agents market reaching roughly $50–53B by 2030, with MLOps around $16.6B, indicating a large parent market for agent software and tooling (MarketsandMarkets, Grand View Research).

Bottom-up calculation:

Illustratively, if 50,000 teams operate multi‑step agents by 2030 and spend $50k–$200k per year on agent optimization, evaluation, and post‑training, that implies a $2.5B–$10B serviceable market for hosted experiment/evaluation tooling.

Assumptions:

• Tens of thousands of teams will operate multi‑step agents by 2030 (e.g., ~50k).
• Per‑team annual spend on hosted agent optimization/evaluation averages $50k–$200k (includes evaluation jobs, judges, and post‑training).
• A meaningful share of teams prefer managed platforms over in‑house scripts for reliability, scale, and CI/CD integration.

Who are some of their notable competitors

• LangSmith (LangChain Labs): Observability and evaluation for chains/agents (traces, evals, prompt/model comparisons). Overlaps on evaluation but doesn’t advertise Synth’s serverless post‑training optimizers like GEPA/MIPRO.
• OpenAI Evals: Open‑source framework to build test suites and graders for LLMs; strong on judges/benchmarks but it’s tooling, not a hosted platform that runs large post‑training jobs end‑to‑end.
• PromptLayer: Prompt registry and testing workbench with versioning and A/B tests. Focuses on prompt management/logging rather than running server‑side optimizers or producing fine‑tuned checkpoints.
• promptfoo: Open‑source CLI/library for automated prompt/agent testing, CI integration, and red teaming. Closer on testing, but not a hosted system that mutates prompts or fine‑tunes models server‑side.
• Weights & Biases: General ML experiment tracking and evaluation (incl. LLM features) with artifacts/versioning. Broad ML infra rather than a specialist agent optimizer with built‑in evolutionary/instruction strategies.