Provider Benchmark

Provider Benchmark posts from the AIHubMix team.

  • Pareto DeepSeek V4.1 Flash review: only 5 hits in 18 hot requests, answers indistinguishable from official

    Pareto DeepSeek V4.1 Flash review: only 5 hits in 18 hot requests, answers indistinguishable from official

    In the 2026-11-03/04 APB Full run, Pareto's four scores are Q 44.58, F 93.42, P 57.69 and R 100, with an overall of 68.72. The 8k hot requests hit 2 of 9 and 32k hit 3 of 9. On 2026-11-03/04, DeepSeek V4.1 Flash 1017 was tested on six providers through AIHubMix with the APB Full suite, using the same questions, parameters and request scripts for each. This APB Profile covers Pareto only: what each section measured and how it came out. The numbers below are scores, not a ranking or a provider

    11 min readProvider Benchmark
  • Fireworks DeepSeek V4.1 Flash review: stop requests return empty body, 8k cache hot requests hit 3 of 9

    Fireworks DeepSeek V4.1 Flash review: stop requests return empty body, 8k cache hot requests hit 3 of 9

    In the 2026-11-03/04 APB Full run, Fireworks' Q is 41.42, and the 95% interval of its gap to the official channel includes 0. F is 71.78: requests with a stop sequence returned an empty body, the APB v2 parameter-contract check failed, and reported input tokens do not match the native count. K is 60.68. On 2026-11-03/04, DeepSeek V4.1 Flash 1017 was tested on six providers through AIHubMix with the APB Full suite, using the same questions, parameters and request scripts for each. This APB Prof

    13 min readProvider Benchmark
  • DeepSeek official channel DeepSeek V4.1 Flash review: comparison baseline, 15 invalid requests at single concurrency

    DeepSeek official channel DeepSeek V4.1 Flash review: comparison baseline, 15 invalid requests at single concurrency

    In the 2026-11-03/04 APB Full run, the DeepSeek official channel is the answer-quality baseline. Its four scores are Q 42.45, F 94.00, P 86.14 and R 100, with an overall of 73.76, only 0.37 from Alibaba Cloud: not distinguishable. Fifteen single-concurrency speed requests in window 1 were invalid. On 2026-11-03/04, DeepSeek V4.1 Flash 1017 was tested on six providers through AIHubMix with the APB Full suite, using the same questions, parameters and request scripts for each. This APB Profile co

    12 min readProvider Benchmark
  • Alibaba Cloud DeepSeek V4.1 Flash review: 1.34s first body token, all 18 hot requests hit cache, answers indistinguishable from official

    Alibaba Cloud DeepSeek V4.1 Flash review: 1.34s first body token, all 18 hot requests hit cache, answers indistinguishable from official

    In the 2026-11-03/04 APB Full run, Alibaba Cloud's four scores are Q 41.75, F 89.78, P 88.58 and R 100, with an overall of 73.38. That is only 0.37 from the DeepSeek official channel's overall, and the 95% interval of the Q gap includes 0: the two cannot be told apart. On 2026-11-03/04, DeepSeek V4.1 Flash 1017 was tested on six providers through AIHubMix with the APB Full suite, using the same questions, parameters and request scripts for each. This APB Profile covers Alibaba Cloud only: what

    12 min readProvider Benchmark
  • Baidu DeepSeek V4.1 Flash review: tool calling on par with official, frequent no-available-channel pulls down answer quality and speed

    Baidu DeepSeek V4.1 Flash review: tool calling on par with official, frequent no-available-channel pulls down answer quality and speed

    In the 2026-11-03/04 APB Full run, Baidu's BFCL accuracy of 86.40% is on par with the official channel's 86.13%. But 975 HLE and AA-LCR question-instances got no complete answer, and 50 of 90 ten-way speed requests returned no_available_channel. Q is 38.00 and P is 37.77. On 2026-11-03/04, DeepSeek V4.1 Flash 1017 was tested on six providers through AIHubMix with the APB Full suite, using the same questions, parameters and request scripts for each. This APB Profile covers Baidu only: what each

    12 min readProvider Benchmark
  • ByteDance DeepSeek V4.1 Flash review: answers indistinguishable from official, input counts don't match

    ByteDance DeepSeek V4.1 Flash review: answers indistinguishable from official, input counts don't match

    In the 2026-11-03/04 APB Full run, ByteDance's Q is 42.05, and the 95% interval of its gap to the DeepSeek official channel includes 0. F is 74.98: the APB v2 parameter-contract check failed, and reported input tokens do not match the native count. On 2026-11-03/04, DeepSeek V4.1 Flash 1017 was tested on six providers through AIHubMix with the APB Full suite, using the same questions, parameters and request scripts for each. This APB Profile covers ByteDance only: what each section measured an

    13 min readProvider Benchmark