Qwen4-Max

(qwen.ai)

150 points | by ilreb 1 hour ago

8 comments

  • maplet 21 minutes ago
    It scores a 43 on Artificial Analysis with an average cost of $3.27 [1]. For reference, GPT-6.1 Sol (medium) gets a 46 with an average of $0.36.

    [1]: https://artificialanalysis.ai/models/qwen4-max

    • esperent 16 minutes ago
      The table on that page shows gpt 6.1 sol (max) at $1.27 per task and it's the only model cheaper than Qwen 4 max.

      By comparison opus 5.5 (max) is $10.01.

      If they are not testing Chinese models with less than max thinking, then we should only compare them to other models also at max thinking.

    • montroser 2 minutes ago
      Yeah, it's in the same ballpark as Kimi 3.1. Both of which are obviously open weight, so that's at least a big differentiator from Sol. Let's hope against hope that the much smaller flash model might beat deepseek v4.1 flash and run for as cheap.
  • seizethecheese 1 hour ago
    I hope they are able to serve 4 faster. Qwen 3.8 is a great model but I’ve regretted including it in the battle system for opencode [1] we have been building because it scores so low.

    And it’s a shame because the only reason it scores so low is that it’s slooooow. (Like 30 tokens per second vs opus near 80.) If you exclude voting, Qwen 3.8 does well.

    (We allow early voting contrary to Arena.ai because I think it makes sense for agents, where it doesn’t for chat. This means our data takes speed into account.)

    Also, it would be ideal if they served this with third parties. Part of the slowness with 3.8 is just not a lot of providers. Their open source version is much faster, but we couldn’t include it because it doesn’t handle images (another own goal by Alibaba).

    [1]: https://agentduel.com/leaderboard

  • dan_thinks 1 hour ago
    This is exciting. The continued focus on efficiency alongside quality is encouraging - it makes me hopeful to see what smaller models will become capable of.

    Looking forward to the "Flash" and 27B versions.

  • nelaggy 10 minutes ago
    is the open weights lag widening? these open models don't seem to be improving as quickly as ant/oai
    • taneq 7 minutes ago
      Might just be the commercial offerings scaling up parameters so much? Like for like the open models don’t seem to be lagging.
  • m00dy 33 minutes ago
    I bet most of us waiting for 27b model and want to see how good it will be.
    • kylehotchkiss 20 minutes ago
      M5U locked and loaded.

      I wish there were more ~60b dense options like ripple too!

    • McGlockenshire 30 minutes ago
      My 3060 yearns for the next generation MoE!
  • anana_ 45 minutes ago
    Artificial Analysis score doesn't look too hot. Doesn't even surpass Kimi 3.1
    • rbehrends 3 minutes ago
      While I doubt that Qwen 4 Max can seriously compete with frontier models, the AA index has become all but useless, due to the introduction of new benchmarks in the aggregate which are of questionable value as a measure of "intelligence" [1]. AA has seemingly also abolished updating the Coding Agents Index, which was an okay benchmark for coding ability, and older models are completely absent from the v5 version of the full index.

      [1] Especially Terminal-Bench-Science 0.1 stands out. Example problems:

      - https://hub.harborframework.com/tasks/terminal-bench-science...

      - https://hub.harborframework.com/tasks/terminal-bench-science...

      Probably very relevant to people working in these fields, but it is honestly not surprising if a lab with limited compute will not train for these technologies, and it says pretty much nothing about more common use cases. This is where you really need a breakdown of capabilities and not an increasingly meaningless aggregate. I also suspect that this benchmark measures knowledge retrieval more than raw reasoning ability.

    • anon373839 41 minutes ago
      AA has become pretty meaningless. They just keep adjusting the recipe to get a preferred ranking result, with no supporting evidence for their decisions.
      • the_duke 5 minutes ago
        To be fair, a lot of the popular benchmarks have become pretty meaningless.

        Models often get great scores despite being a lot worse than the frontier in practice.

      • gr_norm 18 minutes ago
        Yeah, I don't know why people bother with it anymore. It no longer reflects what you're likely to see in real-world usage, at least IME.
      • stratts 17 minutes ago
        One thing I find confusing about AA is that they show GPT 6.1 dominating the lower end of the Pareto line, with eg, GPT 6.1 Sol at medium supposedly costing far less per task than DeepSeek V4.1 Flash at high.

        Yet every time I compare the two in something like Pi, Sol ends up being 5-10x the price for a simple query, something like 7-10c vs. less than 0.5c for DeepSeek. What's the deal?

        • blackoil 11 minutes ago
          Same for me coding tasks taking $20-50 are easily done by deepseek in $5-10
    • selectodude 42 minutes ago
      Anthropic finally figured out how to block them from distilling Opus.
      • m00dy 24 minutes ago
        how so ?
    • AAisthetiobeofA 22 minutes ago
      > Artificial Analysis

      Ah yes, the reverse-benchmaxers of OpenAI models.

  • nixosbestos 14 minutes ago
    Really would love if these publications have a huge, bright red bolded section "How to use this right now". Is it on openrouter? Is Qwen hosting it already? Maybe I'm just ignorant?

    EDIT: oh, I guess this isn't open weights, so presumably I'll have to get it through Qwen or whoever they might partner with (Bedrock?).

  • bbor 1 hour ago
    [flagged]
    • aiqq 48 minutes ago
      [dead]
    • potwinkle 27 minutes ago
      Another Qwen doesn't seem all that scary, a few nation-state actors have likely exfiltrated weights of much stronger non-open models and abliterated them for use in large-scale collection on targets foreign and domestic. But god forbid someone mods a video game or jailbreaks their phone without applying for Claude Mythos or Daybreak, right?
    • zamadatix 30 minutes ago
      If Anthropic truly believes we are on the verge of catastrophe but nobody is listening because they've been crying wolf for years about it while they get rich then they should truly do something. By "do something" I don't mean try again to convince the world to slow down as they IPO their bleeding edge research and solution (which is creating the doubt you mention) but just say "hey, we're so sure we're months away from worldwide disaster that we're shutting down our AI operations completely in hopes we can all put some attention to this before it's too late - will you follow and work together to come up with ways to prevent this problem from happening".

      After all, unless their goal to say "I told you so but I wanted to make money more than I wanted you to believe me so we could stop a billion people from dying" then what good is the current approach doing them?

    • pella 51 minutes ago
      > Ah yes, an open-weights

      This is not an open-weights modell. You have to wait for the smaller models

      "Smaller models in the family are coming to open source, built on the same lineage and trained with the same generation of techniques, tuned for what small models do best."

      • hadlock 21 minutes ago
        Pretty excited about Qwen 4 Flash. We're running qwen 3.5 35b-a3b for low effort, rapid CRM enrichment, but Q4F will be great for more complex tasks.
        • electroglyph 9 minutes ago
          you can use flash next right now and it's already a massive improvement over that old 35b