Extract Local skills
An external analyzer compares an eligible successful trajectory with a failed one, turning recent experience into shared procedural guidance.
ALFWORLD SUCCESS98.57%+2.86 pp over SEED-API
WEBSHOP SUCCESS92.97%+5.47 pp over SEED-API
BACKBONEQwen3-4BRL from the base checkpoint · no SFT
THE IDEABetter models.
Better models.
Better skills.
Learning together.
Experience can become part of a model’s parameters, or live outside the model as reusable textual skills. But as the model learns, the skills it needs also change.
SkillDuet connects these two forms of learning. Local skills extracted from recent trajectories guide on-policy distillation. Model response feedback then selects which skills to consolidate into persistent Global guidance.
The updated model and its evolving skills shape the next round of experience—closing the loop between internalization and consolidation.
THE LEARNING FILMExperience in.
Experience in.
Two kinds of learning out.
Watch experience become a skill, a learning signal, and lasting guidance.
Six connected chapters. One continuous learning loop.
01 / EXPERIENCE
On-policy trajectories
8 rollouts per task
τ₁ successτ₂ failureτ₃ held-outτ₄ held-outτ₅ held-outτ₆ held-outτ₇ held-outτ₈ held-out
02 / EXTRACT
Local skill
Analyze a success–failure pair.
Qwen3.8-27B analyzer03 / INTERNALIZE
Model parameters θ
Local-guided OPD + RL
All eligible Local skills04 / SELECT
Strength |Δ|
Response shifts on held-out trajectories
05 / CONSOLIDATE
Global skills G
Top 50% · every 10 steps
Parameter learningLocal skillPersistent skill evolutionSwipe to explore →
01
Act with the current model and Global.
The policy collects eight trajectories per task using the current Global guidance. Environment feedback identifies successful and failed interactions, supplying fresh experience for both learning paths.
(policy θ, Global G) → 8 on-policy trajectoriesTwo paths, different selection rules. All eligible Local skills can supervise OPD. Only eligible, scored candidates selected by Strength enter Global consolidation. Groups without an eligible success–failure pair use per-trajectory Local skills for OPD only.
SKILLS IN ACTIONSkills are instructions.
Skills are instructions.
Watch them become actions.
From a missed prerequisite to a reusable procedure. Explore three examples of how experience becomes guidance, with instructions highlighted token by token.
TASK
Clean a butter knife, then place it on the dining table.
THREE RECORDED LOCAL OMISSIONS
✓
Fit typeL0006 · required attribute omitted
Check field✓
Round shapeL0021 · required attribute omitted
Check field✓
YouthL0034 · required attribute omitted
Check field↳ One shared procedure: verify every required field.
RECORDED UPDATE / STEPS 11–20
37selected Locals▤ ▤ ▤
→14first-layer summaries▤ ▤
→8second-layer summaries▤ ▤
→5updated Global rules▤
+ 5 existing Global rules G4 preserved verbatim
Identify the object and its required final state.
FROM EXPERIENCE TO SKILL
OBSERVED FAILURE
Without Global, the recorded Clean example reaches the 30-action limit without completing the task.
LOCAL LESSON · PARAPHRASED
Complete the state-changing prerequisite before placing the object at its destination.
GUIDED EXECUTION
With learned Global, the agent follows clean-before-place and completes the recorded task in 7 actions.
Appendix F · ButterKnife–DiningTable. The action sequence shown is illustrative, not the original 7-action trace.
instruction → action01 / 04 · LOOP
>identifythebutterknife
Animated procedures and skill wording are explanatory reconstructions. Outcomes, source IDs, and synthesis counts explicitly attributed to the paper are reported evidence.
METHODLearn continuously.
Learn continuously.
Consolidate selectively.
The model supplies two kinds of response feedback: token-level supervision for learning, and a trajectory-level signal for deciding what to retain.
Internalize with OPD
The policy scores the same sampled response with and without Local guidance. Confidence-gated token supervision complements the RL objective.
Consolidate into Global
Rank eligible candidates by cached Strength, keep the top half, and synthesize them with existing Global guidance every ten steps.
THE SELECTION SIGNAL
Keep what changes the response.
Strength measures the magnitude of a skill’s effect on held-out response log probabilities. Scores are cached when each skill is generated; scoring requires no extra environment interaction.
Strength(L) = 1|HL| ∑r ∈ HL |Δr(L)|Magnitude is a ranking signal, not proof of benefit.
300training steps
8trajectories per task
10steps per consolidation
50%Local retention ratio
WHY SKILLS MUST EVOLVEUseful now.
Useful now.
Useful later?
Fixed guidance helps early in training, but its benefit can fade as the policy learns. On ALFWorld, the same skills shift from a large positive gain to a negative mean gain.
+26.98ppSteps 1–50
+9.64ppSteps 51–100
−2.00ppSteps 101–150
Fixed-skill GRPO probe; mean guided-minus-unguided rollout success. These are motivation experiments, separate from the main benchmark results.
RESULTSStronger agents.
Stronger agents.
Across two environments.
Qwen3-4B on household tasks in ALFWorld and product-search tasks in WebShop. The matched SEED-API baseline uses the same initial checkpoint and external analyzer.
SUCCESS RATE (%) ↑
Household task completion
98.57%
↗ +2.86 pp vs. SEED-API
Main-table results. SkillDuet and SEED-API: step-300 checkpoints, 140 evaluation trajectories each. See Appendix E for the other baselines’ checkpoint and evaluation details.
View the full reproduced-results table
| Method | ALFWorld ALL ↑ | WebShop Score ↑ | WebShop Success ↑ |
|---|---|---|---|
| Vanilla | 37.38 | 13.85 | 0.00 |
| GRPO | 86.43 | 92.96 | 89.84 |
| SDAR | 87.86 | 89.83 | 86.72 |
| SEED | 96.09 | 86.23 | 75.78 |
| SEED-API | 95.71 | 91.31 | 87.50 |
| SkillDuet | 98.57 | 95.18 | 92.97 |
A CLOSER LOOK
What makes the duet work?
Explore selection, retention, and the relationship between a model and its skills.
SELECTION CRITERION
The magnitude of the response shift matters.
In the historical ALFWorld candidate pool, Strength achieves 97.9% peak validation success and 88.26% normalized AUC, outperforming reward-weighted Utility, success–failure Contrast, and Random selection.
This earlier candidate-pool study is distinct from the main framework’s independent ten-step windows.
FROM EXPERIENCE TO GUIDANCESkills that carry
Skills that carry
experience forward.
Consolidation combines task-specific lessons into reusable procedures, while retaining the conditions under which those procedures apply.
CITATION
Read on arXiv Build on SkillDuet.
If you find this work useful, please consider citing our paper.
BibTeX
@article{ji2026skillduet,
title = {SkillDuet: Co-Evolving Models and Skills through Response Feedback},
author = {Ji, Xingguang and Dong, Ziqiang and Zhang, Yubo and Leng, Xi
and Yang, Yang and Jiang, Guanjun},
journal = {arXiv preprint arXiv:2611.12109},
year = {2026},
url = {https://arxiv.org/abs/2611.12109}
}