<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Multimodal LLM | Xuhong Wang</title><link>https://wangxuhongcn.github.io/en/tags/multimodal-llm/</link><atom:link href="https://wangxuhongcn.github.io/en/tags/multimodal-llm/index.xml" rel="self" type="application/rss+xml"/><description>Multimodal LLM</description><generator>Hugo Blox Builder (https://hugoblox.com)</generator><language>en-us</language><lastBuildDate>Sat, 12 Jul 2025 00:00:00 +0000</lastBuildDate><image><url>https://wangxuhongcn.github.io/media/icon_hu_982c5d63a71b2961.png</url><title>Multimodal LLM</title><link>https://wangxuhongcn.github.io/en/tags/multimodal-llm/</link></image><item><title>SafeWork-R1: Co-Evolving Intelligence and Safety under the AI-45 Degree Law</title><link>https://wangxuhongcn.github.io/en/post/safework-r1/</link><pubDate>Sat, 12 Jul 2025 00:00:00 +0000</pubDate><guid>https://wangxuhongcn.github.io/en/post/safework-r1/</guid><description>
&lt;details class="print:hidden xl:hidden" open>
&lt;summary>Table of Contents&lt;/summary>
&lt;div class="text-sm">
&lt;nav id="TableOfContents">
&lt;ul>
&lt;li>&lt;a href="#abstract">Abstract&lt;/a>&lt;/li>
&lt;li>&lt;a href="#why-revisit-the-relationship-between-safety-and-capability">Why Revisit the Relationship Between Safety and Capability&lt;/a>&lt;/li>
&lt;li>&lt;a href="#safety-and-general-capability-in-safework-r1">Safety and General Capability in SafeWork-R1&lt;/a>&lt;/li>
&lt;li>&lt;a href="#the-safeladder-technical-roadmap">The SafeLadder Technical Roadmap&lt;/a>&lt;/li>
&lt;li>&lt;a href="#core-functional-highlights">Core Functional Highlights&lt;/a>&lt;/li>
&lt;li>&lt;a href="#discussion-and-outlook">Discussion and Outlook&lt;/a>&lt;/li>
&lt;li>&lt;a href="#related-links">Related Links&lt;/a>&lt;/li>
&lt;/ul>
&lt;/nav>
&lt;/div>
&lt;/details>
&lt;p>
&lt;/p>
&lt;h2 id="abstract">Abstract&lt;/h2>
&lt;p>SafeWork-R1 is built around a specific claim: safety and capability do not have to trade off against each other. Instead of treating safety as a lightweight filter added after reasoning, this work introduces &lt;code>SafeLadder&lt;/code>, a training framework that attempts to make safety part of the model&amp;rsquo;s internal reasoning ability.&lt;/p>
&lt;p>The resulting model, &lt;code>SafeWork-R1&lt;/code>, is not just safer in the sense of being more conservative. It improves safety benchmark performance by &lt;code>46.54%&lt;/code> over &lt;code>Qwen2.5-VL-72B&lt;/code>, while also improving average performance by &lt;code>13.45%&lt;/code> across seven general reasoning and multimodal benchmarks. The central message is therefore not &lt;code>safety instead of ability&lt;/code>, but &lt;code>safety together with ability&lt;/code>.&lt;/p>
&lt;p>
&lt;figure >
&lt;div class="flex justify-center ">
&lt;div class="w-full" >
&lt;img alt="The AI-45 law view of joint improvement"
srcset="https://wangxuhongcn.github.io/en/post/safework-r1/featured_hu_6189b71377bc8edd.webp 320w, https://wangxuhongcn.github.io/en/post/safework-r1/featured_hu_b928d5368833a6c9.webp 480w, https://wangxuhongcn.github.io/en/post/safework-r1/featured_hu_de726724e02fd12e.webp 760w"
sizes="(max-width: 480px) 100vw, (max-width: 768px) 90vw, (max-width: 1024px) 80vw, 760px"
src="https://wangxuhongcn.github.io/en/post/safework-r1/featured_hu_6189b71377bc8edd.webp"
width="760"
height="313"
loading="lazy" data-zoomable />&lt;/div>
&lt;/div>&lt;/figure>
&lt;/p>
&lt;h2 id="why-revisit-the-relationship-between-safety-and-capability">Why Revisit the Relationship Between Safety and Capability&lt;/h2>
&lt;p>Reasoning-oriented large models have advanced rapidly in recent years, but the gap between capability and safety has also widened. Better problem solving does not automatically imply stronger compliance with ethical constraints, social norms, or trustworthy deployment requirements.&lt;/p>
&lt;p>SafeWork-R1 is motivated by what the original page calls the &lt;code>AI-45 Degree Law&lt;/code>: the desirable direction for model development is not pure capability growth on a single axis, but coordinated improvement along both capability and safety.&lt;/p>
&lt;p>The main argument is straightforward: if the base model is strong enough and the training process is designed properly, safety and general competence need not be a zero-sum game.&lt;/p>
&lt;h2 id="safety-and-general-capability-in-safework-r1">Safety and General Capability in SafeWork-R1&lt;/h2>
&lt;p>SafeWork-R1 is built on the SafeLadder framework, whose goal is to deeply integrate safety mechanisms into the native ability structure of multimodal models, rather than relying on superficial post hoc refusal layers.&lt;/p>
&lt;p>Key results reported on the original page include:&lt;/p>
&lt;ul>
&lt;li>an average &lt;code>46.54%&lt;/code> gain on safety benchmarks over &lt;code>Qwen2.5-VL-72B&lt;/code>&lt;/li>
&lt;li>an average &lt;code>13.45%&lt;/code> improvement across seven general benchmarks: &lt;code>MMMU&lt;/code>, &lt;code>MathVista&lt;/code>, &lt;code>GPQA&lt;/code>, &lt;code>Olympiad&lt;/code>, &lt;code>Gaokao-MM&lt;/code>, &lt;code>IFEVAL&lt;/code>, and &lt;code>MM-IFEval&lt;/code>&lt;/li>
&lt;li>&lt;code>70.94&lt;/code> on &lt;code>MMMU&lt;/code>, &lt;code>76.1&lt;/code> on &lt;code>MathVista&lt;/code>, and &lt;code>78.17&lt;/code> on &lt;code>Gaokao-MM&lt;/code>&lt;/li>
&lt;li>successful transfer of SafeLadder to additional models such as &lt;code>SafeWork-R1-InternVL-78B&lt;/code>, &lt;code>SafeWork-R1-DeepSeek-70B&lt;/code>, and &lt;code>SafeWork-R1-QwenVL-7B&lt;/code>&lt;/li>
&lt;/ul>
&lt;p>
&lt;figure >
&lt;div class="flex justify-center ">
&lt;div class="w-full" >
&lt;img alt="A case study of SafeWork-R1&amp;rsquo;s safety deliberation"
srcset="https://wangxuhongcn.github.io/en/post/safework-r1/safety-deliberation_hu_48d5b1244253a7f2.webp 320w, https://wangxuhongcn.github.io/en/post/safework-r1/safety-deliberation_hu_64ac7d475ca59850.webp 480w, https://wangxuhongcn.github.io/en/post/safework-r1/safety-deliberation_hu_f2d59e7a3966bda5.webp 760w"
sizes="(max-width: 480px) 100vw, (max-width: 768px) 90vw, (max-width: 1024px) 80vw, 760px"
src="https://wangxuhongcn.github.io/en/post/safework-r1/safety-deliberation_hu_48d5b1244253a7f2.webp"
width="760"
height="307"
loading="lazy" data-zoomable />&lt;/div>
&lt;/div>&lt;/figure>
&lt;/p>
&lt;p>
&lt;figure >
&lt;div class="flex justify-center ">
&lt;div class="w-full" >
&lt;img alt="Representation analysis and safety mutual information peaks"
srcset="https://wangxuhongcn.github.io/en/post/safework-r1/representation-analysis_hu_e61b52c47fdb8d7e.webp 320w, https://wangxuhongcn.github.io/en/post/safework-r1/representation-analysis_hu_985319f5589cde51.webp 480w, https://wangxuhongcn.github.io/en/post/safework-r1/representation-analysis_hu_260cef9ab33b2896.webp 760w"
sizes="(max-width: 480px) 100vw, (max-width: 768px) 90vw, (max-width: 1024px) 80vw, 760px"
src="https://wangxuhongcn.github.io/en/post/safework-r1/representation-analysis_hu_e61b52c47fdb8d7e.webp"
width="760"
height="206"
loading="lazy" data-zoomable />&lt;/div>
&lt;/div>&lt;/figure>
&lt;/p>
&lt;p>
&lt;figure >
&lt;div class="flex justify-center ">
&lt;div class="w-full" >
&lt;img alt="Benchmark results on safety and capability"
srcset="https://wangxuhongcn.github.io/en/post/safework-r1/benchmark-results_hu_272c4801f0367c5c.webp 320w, https://wangxuhongcn.github.io/en/post/safework-r1/benchmark-results_hu_259c4a607ade9c36.webp 480w, https://wangxuhongcn.github.io/en/post/safework-r1/benchmark-results_hu_b9d7bb885611c018.webp 760w"
sizes="(max-width: 480px) 100vw, (max-width: 768px) 90vw, (max-width: 1024px) 80vw, 760px"
src="https://wangxuhongcn.github.io/en/post/safework-r1/benchmark-results_hu_272c4801f0367c5c.webp"
width="760"
height="430"
loading="lazy" data-zoomable />&lt;/div>
&lt;/div>&lt;/figure>
&lt;/p>
&lt;p>Taken together, these numbers suggest that SafeWork-R1 is not merely optimized for safety metrics at the expense of open-ended performance. It tries to lift both.&lt;/p>
&lt;h2 id="the-safeladder-technical-roadmap">The SafeLadder Technical Roadmap&lt;/h2>
&lt;p>SafeLadder uses a structured and progressive reinforcement-learning-based post-training pipeline to internalize safety as part of model capability. The original webpage breaks it into four stages:&lt;/p>
&lt;ol>
&lt;li>&lt;code>CoT-SFT&lt;/code>: chain-of-thought supervised fine-tuning as a cold start for long-form reasoning.&lt;/li>
&lt;li>&lt;code>M³-RL&lt;/code>: multimodal, multitask, multi-objective RL that progressively aligns safety, values, knowledge reliability, and general capability.&lt;/li>
&lt;li>&lt;code>Safe-and-Efficient RL&lt;/code>: reducing overthinking and treating reasoning efficiency itself as part of safety.&lt;/li>
&lt;li>&lt;code>Deliberative Search RL&lt;/code>: enabling the model to retrieve, verify, and filter external information during answering.&lt;/li>
&lt;/ol>
&lt;p>
&lt;figure >
&lt;div class="flex justify-center ">
&lt;div class="w-full" >
&lt;img alt="The SafeLadder training roadmap"
srcset="https://wangxuhongcn.github.io/en/post/safework-r1/training-roadmap_hu_b9f61564086d8eba.webp 320w, https://wangxuhongcn.github.io/en/post/safework-r1/training-roadmap_hu_d572f0f053a95ec6.webp 480w, https://wangxuhongcn.github.io/en/post/safework-r1/training-roadmap_hu_9b00b1a36a27f2.webp 760w"
sizes="(max-width: 480px) 100vw, (max-width: 768px) 90vw, (max-width: 1024px) 80vw, 760px"
src="https://wangxuhongcn.github.io/en/post/safework-r1/training-roadmap_hu_b9f61564086d8eba.webp"
width="760"
height="172"
loading="lazy" data-zoomable />&lt;/div>
&lt;/div>&lt;/figure>
&lt;/p>
&lt;p>The page also mentions a scalable RL infrastructure named &lt;code>SafeWork-T1&lt;/code>, designed for thousand-GPU-scale training with multiple validators and modular verification components.&lt;/p>
&lt;h2 id="core-functional-highlights">Core Functional Highlights&lt;/h2>
&lt;p>SafeWork-R1 is not only about safer outputs. It also emphasizes trustworthy reasoning and interaction. The webpage highlights three major capabilities:&lt;/p>
&lt;ul>
&lt;li>&lt;code>Deliberative Search&lt;/code>: combining calibration and search so the model can verify and refine its own answer through RL-based multi-step reflection.&lt;/li>
&lt;li>&lt;code>Inference-Time Alignment&lt;/code>: bringing professional value models into the generation process to constrain intermediate reasoning and final responses.&lt;/li>
&lt;li>&lt;code>Human Intervention on Chain-of-Thought&lt;/code>: allowing users or supervisors to directly correct flawed reasoning steps so the model better aligns with desired logic, style, and values.&lt;/li>
&lt;/ul>
&lt;p>Together, these features show that the goal is not only to stop harmful behavior, but to produce reasoning processes that are themselves more reliable and controllable.&lt;/p>
&lt;h2 id="discussion-and-outlook">Discussion and Outlook&lt;/h2>
&lt;p>The original page closes with several takeaways that are likely to matter beyond this specific model:&lt;/p>
&lt;ul>
&lt;li>&lt;code>Safety and capability are not necessarily zero-sum&lt;/code>: with the right training design, they can co-evolve.&lt;/li>
&lt;li>&lt;code>Reasoning efficiency is closely tied to safety&lt;/code>: overly long and redundant chains of thought can themselves introduce security and alignment risks.&lt;/li>
&lt;li>&lt;code>Trustworthy interaction remains a long-term frontier&lt;/code>: future work needs better error correction, test-time adaptation, language calibration, and norm-aware interaction.&lt;/li>
&lt;/ul>
&lt;p>The broader significance of SafeWork-R1 is therefore not just that it releases a strong model, but that it presents a training path where safety is treated as part of reasoning ability rather than a patch applied after the fact.&lt;/p>
&lt;h2 id="related-links">Related Links&lt;/h2>
&lt;ul>
&lt;li>Paper:
&lt;/li>
&lt;/ul></description></item></channel></rss>