WorkForge – In-Depth Review of the Smart Agent Training Environment Construction Framework Jointly Launched by Tencent and Fudan University

1. What is WorkForge

WorkForge is a framework for constructing training environments for intelligent agents, jointly developed by Tencent HunYuan and Fudan University. It aims to address the core bottleneck of scarce high-quality long-range task data in current intelligent agent training. Starting from expert workflow blueprints, the framework enables agents to automatically retrieve real-world resources to set up workspaces. Based on factual anchors within these workspaces, it generates task instructions and dual verification standards, allowing agents to produce traceable trajectory data during execution. To date, WorkForge has built 16.7K verifiable environments across 40 professional domains, capable of boosting the GDPval score of a 35B model by 28 points, bringing it closer to the performance of state-of-the-art models. It provides a scalable and verifiable data production paradigm for training intelligent agents' capabilities in workplace tasks.

Technical Positioning and Domain: WorkForge lies at the intersection of intelligent agent training data engineering and evaluation verification. It focuses on addressing the supply of training data for long-range task execution capabilities of large language models in real-world workplace environments. Its unique positioning lies in integrating "environment construction – task generation – result verification" into an automated pipeline, making training data production no longer reliant on costly manual annotation.

Development Background: This framework was jointly developed by the HunYuan large model team at Tencent and the School of Computer Science and Technology at Fudan University. Both parties have deep expertise in foundational model training, intelligent agent systems, and data engineering. The development motivation stems from the common industry challenge of balancing "environment authenticity" and "task verifiability" – while LLM-synthesized environments are low-cost, they lack the complexity of real-world operations, whereas manually constructed environments are costly and difficult to scale.

Core Value: WorkForge addresses three levels of practical issues in intelligent agent training: first, it replaces LLM-synthesized data with real-world resources, giving the training environment authentic directory structures, file dependencies, and operational friction; second, it ensures that task instructions, reference solutions, and scoring criteria are all derived from observable facts within the workspace, eliminating hallucinatory evaluation requirements; third, it reduces the cost of environment construction from weeks of manual labor to batch generation through an automated pipeline, enabling the large-scale production of 16.7K environments.

Technical Features: The core technical advantages of WorkForge lie in two major innovations: the "fact-anchored task-verification co-design" and the "procedural + semantic dual-track verification mechanism." The former ensures that task instructions, reference solutions, and scoring criteria are all derived from observable facts within the workspace. The latter captures different dimensions of delivery quality through complementary verification signals, resulting in a significantly improved hybrid screening effect (ALE 22.11) compared to using either signal alone (15.62/14.89).

2. Key Features

• Scenario Blueprint Construction: This module is responsible for distilling expert knowledge into reusable workflow templates, clearly defining deliverable types, required resources, and key decision points. The blueprint serves as the top-level constraint for task generation, ensuring structural consistency and professionalism across different instances, allowing tasks within the same domain to have comparable difficulty gradients.

• Real-World Workspace Synthesis: The agent iteratively executes a search—read—download—preview loop to collect real code repositories, datasets, and web resources from the internet, and constructs a workspace according to the original directory structure. Comparative experiments show that the number of files in a real workspace is 5.4 times that of an LLM-synthesized environment, the number of file types is 2.4 times greater, and the volume is 13 times larger. Evidence is distributed across multi-level directory trees, forcing the agent to explore before making judgments.

• Environment Profiling and Fact Anchor Extraction: The system automatically analyzes the role positioning of each file in the workspace, cross-file dependencies, and records verifiable "fact anchors." This profiling result forms the factual basis for all subsequent task generation and verification operations, ensuring that each task instruction can be matched with specific evidence within the workspace.

• Task Generation: By combining the scenario blueprint with the environment profile, the system automatically generates task instructions rooted in the actual content of the workspace, along with hidden reference solutions. Key facts in the reference solutions are rechecked against the workspace to ensure task feasibility and answer accuracy, eliminating hallucinatory evaluation requirements from the source.

• Dual Verification Mechanism: Composed of a programmatic validator and an LLM semantic judge, operating in parallel. The programmatic validator checks numerical correctness, whether the target file is generated, and whether the workspace state meets expectations, based on an instruction function. The semantic validator, an LLM judge, scores open-ended deliverables item by item according to evaluation criteria and normalizes the scores. Both tracks of scores are retained separately, allowing flexible configuration of filtering strategies.

• Trajectory Collection and Scoring: The agent autonomously performs tasks under the condition of only having visibility to the workspace and task instructions. The entire process of operations (averaging 31.8 steps and 76,000 tokens) is saved along with the final score as training data. This long-range interaction trajectory fully covers the complete workflow of exploration, evidence collection, error correction, and delivery, providing high-quality material for model post-training.

• Scalable Environment Generation: Based on a unified pipeline, 16.7K verifiable work environments can be generated with one click, covering 40 professional fields and 60 file types. This significantly reduces the engineering cost of manually building environments, making large-scale agent training data production feasible.

• Agent Post-Training: Using the generated long-range trajectory data to fine-tune the model significantly enhances its ability to perform workplace tasks. Experiments show that the A3B model's GDPval score improves by 28.1 points, outperforming all specialized agent models with less than 100B parameters; the 27B model even surpasses the 397B model with 15 times more parameters.

3. How to Use

WorkForge is currently positioned as a research framework, and the paper does not yet publicly disclose specific code repositories or data set download channels. The following usage process is based on the research paradigm described in the paper:

1. Environment Preparation: Researchers need to prepare a server environment with sufficient computational resources, and it is recommended to use NVIDIA A100/H100 level GPUs for model inference and training. On the software side, Python 3.9+, PyTorch 2.0+, and the HuggingFace Transformers library must be installed, along with proper network access permissions to support the agent retrieving real resources from the internet.

2. Define Scenario Blueprint: Researchers must structure the expert knowledge of the target domain, defining deliverable types (such as reports, tables, PPTs), required resource categories (code repositories, data sets, web resources), and key decision points. The quality of the blueprint directly affects the relevance of subsequent task generation; it is recommended to start with a single vertical domain and gradually expand.

3. Configure the Retrieval Agent: Set up the retrieval agent's search strategy, download filtering rules, and preview validation logic. This agent is responsible for collecting resources from GitHub, public data set platforms, and web pages, and for setting up the workspace according to the original directory structure. It is important to configure reasonable download whitelists and security filtering mechanisms to avoid introducing malicious code or non-compliant content.

4. Build Execution Environment and Generate Profile: Run the build pipeline to generate the workspace, then trigger the profiling module to analyze the role, dependencies, and factual anchors of each file. This phase outputs an environment profile report, and researchers can sample-check the profile quality to ensure the accuracy of factual anchor extraction.

5. Generate Tasks and Validation Criteria: Automatically generate task instructions, reference solutions, programmatic checks, and semantic scoring guidelines based on the blueprint and profile. It is recommended to manually sample-check the generated results to confirm that the task difficulty is appropriate and the validation criteria are executable. Adjust blueprint parameters and regenerate as needed.

6. Collect Trajectories and Perform Post-training: Let the target agent execute the tasks in a controlled environment, collecting complete operation trajectories and scores. Then use the trajectory data to perform supervised fine-tuning or reinforcement learning on the model, evaluating the improvement in benchmark metrics such as GDPval, APEX, and ALE.

4. Pros and Cons Analysis

5. Comparative Analysis with Similar Tools

Selection Recommendations:
For agent training data generation scenarios, WorkForge stands out due to its real-world workspace and long-range trajectory data, making it the more optimal choice currently. It is especially suitable for R&D teams that need to develop models' abilities to explore, investigate, and deliver in cluttered file environments. Its scalability with 16.7K environments allows enterprises to obtain high-quality training data at a lower cost.

For agent capability evaluation scenarios, AgentBench and τ-bench provide more mature standardized evaluation protocols, ideal for cross-team benchmarking. ClawGym is suitable for research teams that need to quickly validate different environment construction strategies within a unified framework, but its synthetic environments significantly lag behind WorkForge in terms of realism.

6. Editor's Summary

WorkForge demonstrates clear technological innovation in the field of intelligent agent training data engineering. Its core contribution lies in integrating "environment construction—task generation—result verification" into a scalable automated pipeline, and addressing the long-standing issue of traceable evaluation in the industry through a factual anchoring mechanism. Experimental data shows that the A3B model's GDPval improved by 28.1 points, and the 27B model outperformed the 397B model, providing strong evidence for the hypothesis that "data quality determines the upper limit of model capabilities."

From a practical value perspective, WorkForge's contribution to the industrialization of intelligent agents manifests in three aspects: first, it reduces the cost of environment construction from weeks of manual effort to pipeline-based batch generation, enabling data production at industrial-scale efficiency; second, the operational friction and long-term interaction requirements introduced by real workspaces make the training data more aligned with actual workplace scenarios, thereby increasing the success rate of model migration to real-world tasks; third, the dual-track verification mechanism combining procedural and semantic validation provides a reusable methodology for evaluating intelligent agent capabilities.

In terms of target users, this framework is primarily aimed at three categories: enterprise AI laboratories requiring large-scale training data, academic teams researching intelligent agent data engineering, and research institutions needing to build specialized agent evaluation environments in bulk. For individual developers, the current non-open-source status of the paper means they will have to wait for an official release.

In terms of future development potential, WorkForge's architectural design offers good scalability—the scenario blueprint mechanism allows for rapid migration to new domains, and the dual-track verification mechanism can be adapted to more task types. If the team proceeds as planned with open-sourcing the code and dataset, and continues to expand domain coverage, WorkForge has the potential to become a significant option for intelligent agent training data infrastructure. Worth noting is that balancing the compliance of resource collection with the efficiency of environment construction will be a continuous challenge during its large-scale deployment.

7. Application Scenarios

• Agent Training Data Generation: Enterprises or research institutions can use WorkForge to scale the generation of high-quality long-range trajectory data for specific vertical domains, which can be used for post-training their own work agents. Compared to the traditional method of manually building environments, this solution can reduce data production costs by several orders of magnitude while ensuring the authenticity and verifiability of the data.

• Digital Workforce Training in the Workplace: For 40 specialized fields including office work, data analysis, finance, and education, WorkForge can train "digital employees" capable of delivering real deliverables such as reports, spreadsheets, and PPTs. These models have full capabilities to explore file systems, cross-verify multi-source information, and iteratively refine deliverables when performing actual work tasks.

• Agent Capability Evaluation: Its fact-anchored dual-track verification mechanism can serve as an evaluation framework to test an agent's ability to explore, gather evidence, and deliver results in a cluttered file environment. Evaluation organizations can build standardized test sets based on WorkForge to quantitatively assess an agent's long-range task execution capabilities.

• Digitalization of Domain Workflows: Enterprises can document internal expert experience as scenario blueprints and use WorkForge to quickly and in bulk generate different instances of similar tasks for employee training or capability assessments. In this scenario, the blueprint mechanism ensures the professionalism of the tasks, while fact anchors guarantee the objectivity and traceability of the evaluation criteria.

• Foundation for Vertical Agent Development: In professional scenarios requiring cross-verification of multiple source files, such as financial quantification, code review, and compliance checks, WorkForge can be used as an environmental infrastructure to support the iterative development of upper-layer agent products. R&D teams can continuously generate new tasks within a unified environment system, driving the continuous evolution of model capabilities.

8. FAQ

Q: Is WorkForge currently open source? How can I obtain the code and data?
A: According to the paper, WorkForge is currently a research framework, and the paper does not mention any public channels for open-source code or datasets. It is recommended to follow the official announcements from Tencent Hun Yuan and the relevant team at Fudan University for updates on future open-source plans.

Q: What is the fundamental difference between WorkForge and the approach of directly synthesizing training environments with LLMs?
A: LLM-synthesized environments typically generate simple, centralized workspaces. In contrast, WorkForge acquires workspaces by retrieving and preserving the original directory structure from the real world, achieving 5.4 times more files, 2.4 times more file types, and 13 times more volume compared to synthetic environments. Evidence in real environments is distributed across multi-level directory trees, forcing agents to first explore and then make judgments, resulting in authentic operational friction.

Q: How does WorkForge's dual verification mechanism work specifically?
A: The dual verification mechanism consists of two tracks: a procedural validator and an LLM semantic reviewer. The procedural validator checks deterministic requirements based on instruction functions, such as whether numbers are correct, if target files are generated, and if the workspace state meets expectations. The semantic validator, composed of an LLM reviewer, scores open-ended deliverables item-by-item according to evaluation criteria and normalizes the scores. Both tracks' scores are retained separately, and the combined filtering effect (ALE 22.11) is significantly better than using either signal alone (15.62/14.89).

Q: Does the training effectiveness of WorkForge vary with model scale?
A: Experiments show that WorkForge achieves significant improvements across different model scales. The A3B model's GDPval increases by 28.1 points from the baseline to 73.6, outperforming all specialized models with less than 100B parameters. The 27B model even surpasses the 397B model with 15 times more parameters after training. This indicates that high-quality long-range trajectory data provides particularly notable gains for smaller models.

Q: What professional domains does WorkForge support for task generation?
A: WorkForge has established 16.7K verifiable environments covering 40 professional domains and 60 file types. Specific domains include office work, data analysis, finance, education, etc. Enterprises can also convert internal expert knowledge into scenario blueprints to quickly expand into new vertical domains.

Q: What hardware configuration is required to use WorkForge?
A: Due to the involvement of large-scale environment construction, model inference, and training, it is recommended to use an NVIDIA A100/H100-level GPU cluster. The specific configuration depends on the task scale: the environment construction phase mainly consumes CPU and network resources, while the trajectory collection and post-training phases require high GPU computational power.

Q: Can the verification standards in WorkForge produce hallucinations or errors?
A: WorkForge effectively reduces the risk of hallucinations through a factual anchoring mechanism. All task instructions, reference solutions, and scoring criteria are derived from observable facts in the workspace. Key facts in the reference solutions are rechecked in the workspace to ensure that the verification standards align with the actual content of the workspace, eliminating evaluation requirements without factual basis.

9. Project Links

• arXiv Technical Paper: https://arxiv.org/pdf/2610.04906 (Official technical report, containing complete methodology and experimental data)