Model Eval Studio
选 AI 模型像开盲盒——参数表看不懂、第三方跑分不可信、自己测又没时间。Model Eval Studio 让你截图就识别模型、自动跑多维度对比，给出一份能拍板的选型报告。系统支持自定义评测集（多模态、推理、代码、长上下文都能配），自动产出 leaderboard + 成本曲线 + 失败案例分析。适合 AI 应用团队的架构师、CTO、模型采购决策者，把过去 2 周的选型会压成一个下午的可视化对比。
Teams often choose models from screenshots, third-party claims, or scattered traces without a repeatable way to compare quality, cost, and latency.
I framed evaluation as a workspace: identify the artifact, extract comparable signals, and produce reports that can be diffed across models and runs.
The project shows how model selection can move from vibes to an auditable comparison flow with clearer cost-per-quality tradeoffs.
- 01Designed the evaluation workflow and comparison report shape.
- 02Built the interactive Next.js UI and result surfaces.
- 03Centered the product around practical model choice rather than abstract leaderboard scores.
Model evaluation is engineering infrastructure, not a benchmark spreadsheet — upload a screenshot, let vision models identify the source, and surface hard metrics (latency, token cost, output quality) in a comparison report you can act on.
Three-stage pipeline. (1) Vision models identify the model in the uploaded artifact. (2) Hard metrics are extracted from the trace or the artifact itself. (3) A structured report is generated and persisted for diffing across runs.
- 01Vision-first identification — supports screenshots, charts, terminal output, and rendered UI without prior labeling.
- 02Comparison diffing — runs are versioned; you can see how the same model changed across deploys.
- 03Cost-per-quality scoring — surfaces the Pareto frontier, not just the leaderboard.