Autonomous closed-loop evolution

The core bottleneck for Agent capability is training data. Traditional pipelines rely on heavy human labeling—costly, slow, and hard to inject multi-path exploration. U2 Flash builds a sandbox and a data-synthesis loop so the model learns multi-strategy attempts and stable convergence:

• 

  Dual-track data generation

  For a batch of tasks, multiple strong teacher models produce correct trajectories while a weak model produces incorrect ones, forming paired right/wrong training samples.

• 

  Supervised analysis & key-action marking

  A supervisor model with harness compares paired trajectories, produces analysis reports and flowcharts, and marks decisive actions.

• 

  Prompted CoT generation

  From key-action marks, generate prompts and chain-of-thought to focus the weak model on its weak spots.

• 

  Weak-model re-sampling

  The weak model re-samples and produces correct trajectories guided by prompted CoT.

• 

  Cleaning & rewrite

  Clean and rewrite prompts and reasoning traces to improve data quality.

• 

  Fine-tune & reinforce (SFT + async Agent RL + OPD)

  SFT on cleaned data, then async Agent RL in a real sandbox, then multi-teacher OPD to distill specialist skills back into one model; the enhanced model becomes the next-round weak model, closing the loop.

With this loop we built nearly 100k high-quality SWE tasks across mainstream languages. The model proposes tasks, summarizes, and improves the pipeline itself—supervisors intervene only at key steps—so it strengthens its own weak spots without heavy labeling, and with harness optimization internalizes the capability.

Self-strengthening in real environments

Traditional RLHF depends on large preference labels and lagged rewards. U2 Flash uses Agentic RL so the model asynchronously collects trajectories in a sandbox and self-play optimizes policies:

• 

  Async sampling

  Multiple rollout workers run tasks and collect trajectories (code execution, tool calls, environment interaction) without blocking policy updates, boosting throughput.

• 

  GRPO policy optimization

  Following DeepSeekMath’s GRPO (Group Relative Policy Optimization), relative advantages are estimated within a trajectory group—no separate Critic—cutting memory and cost while diverse in-group paths encourage multi-solution exploration.

• 

  Sparse reward + long-horizon credit

  Agent rewards are sparse; key-action marks and trajectory-level advantages back-propagate success/failure to decisive actions, addressing long-horizon credit assignment.

• 

  Supervisor only at key steps

  Aligned with the closed loop—the model explores freely while the supervisor intervenes only at critical forks, balancing exploration and quality.

Multi-teacher on-policy distillation

After SFT + Agentic RL, U2 Flash introduces OPD (On-Policy Distillation) to merge specialist skills into one model without loss:

• 

  Domain experts

  Train an expert per domain (math / code / Agent / instruction following) with SFT plus domain GRPO to form an expert pool.

• 

  Multi-teacher merge

  Distill N experts into one student that minimizes reverse KL to each teacher on its own on-policy trajectories.

  L_OPD(θ)=∑_{i=1..N}w_ic · D_KL(π_θ || π_{E_i})
• 

  Engineering

  Load teacher weights on demand, cache hidden states only, and order samples by teacher—multi-teacher distillation under limited compute.

U2 Flash deeply participates in these training loops—not only a model being trained, but a participant in the training process itself.