APUS-OpenJev-v1: In-Depth Review of APUS AI Lab's Open-Source Edge-Side Decision-Making Large Model

1. What is APUS-OpenJev-v1

APUS-OpenJev-v1 is an open-source edge-side System 1 decision-making large model series developed by APUS AI Lab. It is positioned similarly to the closed-source Jev model from TypeSafe AI, offering three weight configurations: 4B, 9B, and 35B-A3B. The model can quickly select the next action in tasks such as browser automation, form filling, and workflow routing. APUS-OpenJev-v1 introduces an innovative dynamic depth computation mechanism with effort="low/high": in low mode, it runs only 16 layers combined with candidate-aware projection, achieving fast speed and low cost; in high mode, it uses full depth for higher accuracy. Through the native language decision-making interface, the model takes the task goal, context, and candidate operations as natural language inputs, binding compact Tokens to candidate actions, significantly reducing inference overhead while maintaining decision quality.

Technical Positioning and Domain: Belongs to the intersection of natural language processing and intelligent decision-making, focusing on System 1 rapid decision-making scenarios. Unlike traditional large models that emphasize generative tasks, this model highlights the ability to make immediate action choices in scenarios such as browser automation and business rule routing. It has a clear technical positioning for edge-side deployment and low-latency inference, filling the gap in the open-source community for structured decision-making large models.

Development Background: Developed and open-sourced by APUS AI Lab, which has deep expertise in the field of artificial intelligence applications. The team observed that existing large models face issues such as unstable output formats and high inference costs when handling structured decision-making tasks. Therefore, they drew inspiration from the design of TypeSafe AI's closed-source Jev model to develop this fully open-source decision-making model, aiming to promote the adoption of decision intelligence in edge-side scenarios.

Core Value: Addresses three major pain points of traditional large models in decision-making scenarios: first, the output format hallucination problem, by constraining outputs within a candidate set through the native language decision-making interface; second, the issue of high inference costs, by enabling on-demand resource allocation via the effort="low/high" mechanism; third, data privacy concerns, by supporting fully localized deployment, ensuring that data never leaves the enterprise environment.

Technical Features: Utilizes cross-depth joint post-training and self-distillation techniques to maintain reliable judgment power even with half the computational load in shallow exits; candidate-aware output projection reduces the output matrix computation from the full vocabulary to K candidates, significantly lowering computational load and memory access costs; supports multiple model configurations to meet diverse needs, ranging from lightweight edge deployment to high-performance production environments.

2. Key Features

• Candidate Action Decision-Making: The model outputs the next business action based on the task objective, page context, and candidate operation descriptions. By binding compact Tokens such as A, B, C to candidate actions, the model only needs to calculate the probability of valid candidate Tokens to make a decision, avoiding format errors and invalid outputs that may occur in open-ended generation with traditional large models.

• Browser Automation: The model can identify DOM elements such as buttons, input fields, and navigation components on web pages, assisting the Agent in performing actions like clicking, scrolling, and navigating. This feature is based on semantic understanding of DOM structure, converting browser operations into structured action sequences. It is suitable for scenarios such as web testing, data collection, and automated workflow execution.

• Business Rule Routing: Applicable to business decision-making scenarios such as customer service ticket routing, workflow approvals, and form submissions. The model can understand business rules and contextual information, automatically selecting the correct processing path and reducing manual intervention. For example, in a customer service system, it can automatically determine the most appropriate department or response strategy based on the user's question.

• Dynamic Computation Control: Flexibly switch between high throughput and high quality using the effort="low/high" parameter. In low mode, only 16 layers of Transformer are executed, combined with candidate-aware output projection, making it suitable for high-frequency routine actions such as scrolling or clicking "Next." In high mode, the full 32-layer inference is executed, suitable for high-risk operations such as payments and permission confirmations, achieving optimal allocation of computational resources.

• Form Text Generation: In addition to decision-making in multiple-choice formats, the model can also generate open-ended content such as search keywords and input text. Through the generate_text() interface, the model can produce specific text inputs for scenarios requiring search terms or form content, expanding its application scope beyond structured decisions.

• Local Millisecond-Level Deployment: Supports deployment on single-GPU cards and vLLM service platforms, reducing cloud API latency and invocation costs. When deployed locally with a specified vLLM path, the P50 latency for the 9B version is as low as 25.58ms, and for the 4B version it is only 18ms, meeting the strict latency requirements of real-time decision-making scenarios.

• Multi-Model Specification Options: Offers three versions: 4B, 9B, and 35B-A3B, each tailored to different needs—lightweight edge devices, general production use, and high-performance requirements. The 4B model is suitable for resource-constrained edge devices, the 9B model balances accuracy and performance, and the 35B-A3B model employs a MoE architecture, providing stronger model capabilities for complex decision-making tasks.

3. How to Use

1. Prepare the Environment: Install PyTorch, Transformers, and the huggingface_hub library with CUDA support, and ensure that your local machine or server has sufficient GPU memory. It is recommended to use an NVIDIA GPU with at least 8GB of memory for smooth operation of the 4B model, and at least 16GB of memory for the 9B model.

2. Download the Model: Execute hf download apus-ailab/APUS-OpenJev-v1 --include "9B-3000/**" --local-dir ./APUS-OpenJev-v1 to download the required model directory. Users can choose to download the corresponding directories for the 4B, 9B, or 35B-A3B models based on their needs, avoiding unnecessary storage costs.

3. Enter the Model Directory: Switch to the downloaded model subdirectory, and you can directly use the built-in openjet_runtime inference component. This runtime component encapsulates core logic such as model loading, inference, and output mapping, simplifying the deployment process.

4. Load the Model: Use OpenJet.from_pretrained(".", device="cuda:0") to load the model onto the specified GPU. Here, you can specify different GPU device numbers, supporting flexible configuration in multi-GPU environments.

5. Construct a Decision Request: Input the task objective, current context, and candidate actions by providing the instructions, state, and candidates fields. instructions describe the task instructions, state provides the current environment status, and candidates list all possible candidate actions.

6. Perform Fast Decision-Making: Call model.decide(request_data, effort="low") to obtain high-frequency routine actions with low computational cost. This mode is suitable for decision scenarios sensitive to latency and with low risk, such as page scrolling or clicking "Next" buttons.

7. Perform Deep Decision-Making: Call model.decide(request_data, effort="high") to obtain complete inference results suitable for critical actions such as payments or submissions. This mode leverages the full network depth, providing higher decision accuracy, and is appropriate for high-risk operation scenarios.

8. Generate Workflow Text: For scenarios requiring the generation of search terms or form content, call model.generate_text() to produce specific text. This interface supports open-ended text generation, addressing the limitations of pure decision-making interfaces.

9. Deploy Production Services: Run python deployment/serve_vllm.py --port 8000 to launch an OpenAI-compatible interface using vLLM. This interface supports standard RESTful API calls, making it easy to integrate into existing business systems and enabling scalable deployment.

4. Pros and Cons Analysis

5. Comparative Analysis with Similar Tools

Selection Recommendations: For enterprises sensitive to data privacy and cost, APUS-OpenJev-v1 is an ideal choice. Its local deployment capability ensures that data remains within the enterprise environment, and the effort="low/high" mechanism effectively controls inference costs. For teams needing rapid deployment with limited technical resources, Jev's commercial API provides a convenient integration method, but data leakage and ongoing cost issues must be carefully considered.

For technically strong teams seeking deep customization of decision logic, the open-source nature of APUS-OpenJev-v1 offers the greatest flexibility. As a similar open-source alternative, SemIf is suitable for scenarios requiring extremely high model transparency. Traditional general-purpose large models are better suited for comprehensive scenarios requiring both decision-making and generation tasks, but they require additional investment in prompt engineering and output constraints development.

6. Editor's Summary

The release of APUS-OpenJev-v1 marks a significant turning point in the evolution of on-device decision-making large models, transitioning them from closed-source commercial products to open-source communities. From a technological innovation perspective, the model's pioneering effort="low/high" dynamic computation depth mechanism holds substantial engineering value. It breaks the traditional paradigm of large models, which operate under a "single inference, fixed cost" model, allowing Agent systems to dynamically allocate computational resources based on the importance and certainty of actions. The joint post-training across depths and self-distillation techniques ensure that the shallow layer outputs maintain reliable judgment capabilities even with half the computational load, making this "tiered computation" design philosophy worthy of adoption across the industry. The candidate-aware output projection technology further optimizes inference efficiency from an engineering standpoint, reducing the output matrix computation from the full vocabulary to just K candidates, thereby providing technical support for on-device deployment.

In terms of practical value, the model achieves an 85% accuracy rate on the official 80-question frozen benchmark with its 9B version, surpassing the 82.5% accuracy of the closed-source commercial product Jev. This demonstrates the competitiveness of open-source models in specific tasks. The P50 latency of 25.58ms meets the stringent requirements of real-time decision-making scenarios, while its capability for localized deployment addresses the rigid data security needs of industries such as finance and government affairs. The model offers three specifications: 4B, 9B, and 35B-A3B, covering the complete spectrum of requirements from lightweight on-device deployment to high-performance production, thereby lowering the adoption threshold for businesses of different scales.

This model is primarily targeted at developers requiring rapid decision-making capabilities, AI Agent application builders, enterprise automation process designers, and technical decision-makers focused on data privacy and cost control. For scenarios such as browser automation, intelligent customer service, and workflow approvals, APUS-OpenJev-v1 provides ready-to-use solutions. As the open-source community continues to contribute and the ecosystem matures, such on-device decision-making models are expected to be deployed in more vertical domains, driving the evolution of AI from "content generation" to "decision-making."

7. Application Scenarios

• Browser Automation: The model quickly determines the next action, such as clicking, scrolling, or navigating, based on webpage DOM, button, and input field information. In scenarios like web testing, data collection, and RPA process automation, the model can understand page structure and make precise operational decisions, significantly improving the execution efficiency and stability of automated workflows.

• Enterprise Process Routing: In approval processes, ticket handling, or form transitions, the model automatically selects processing paths such as submission, rejection, or handover based on business rules. The model understands complex business rules and contextual information, reducing the need for manual judgment and intervention, thereby enhancing the efficiency and consistency of process handling. It is suitable for enterprise-level applications such as OA systems and ERP systems.

• Customer Service Ticket Assignment: By combining user questions, historical information, and candidate departments, the model can real-time determine the most appropriate customer service team or action. The model quickly analyzes user intent and question type, enabling intelligent ticket routing, reducing response time, and improving customer satisfaction. It is applicable to customer service centers, technical support, and similar scenarios.

• Finance and Compliance Review: For high-risk operations, use effort="high" for in-depth judgment to identify abnormal transactions, permission risks, or non-compliant data flows. In high-risk scenarios, the model ensures decision accuracy by leveraging full network depth, while also supporting complete on-premise deployment to meet the strict data security and compliance requirements of the financial industry.

• Edge-side Intelligent Assistant: Deploy lightweight models on local devices or private servers to provide low-latency, privacy-protecting decision-making capabilities for high-frequency Agent tasks. The 4B model can run on resource-constrained devices, offering real-time intelligent decision support for mobile applications, smart hardware, and other scenarios, while ensuring that user data remains on the device.

8. FAQ

Q: What is the difference between APUS-OpenJev-v1 and general large language models?
A: APUS-OpenJev-v1 focuses on System 1 rapid decision-making tasks, using a language-native decision interface that constrains outputs within predefined candidate sets, avoiding format hallucinations. General large language models excel at open-ended text generation and require additional prompting engineering and output constraints in structured decision-making scenarios, typically with higher inference costs.

Q: How to choose between effort="low" and effort="high" modes?
A: The effort="low" mode runs only 16 layers of Transformer, suitable for high-frequency routine actions such as scrolling and clicking "Next," with low computational cost and fast response. The effort="high" mode performs full 32-layer inference, ideal for high-risk operations like payment confirmation and permission approval, offering higher accuracy. It is recommended to dynamically select based on the importance and risk level of the action.

Q: What deployment methods does the model support?
A: The model supports local deployment on a single GPU card and service-oriented deployment using vLLM. For local deployment, you can run the openjet_runtime inference component. For service deployment, you can execute python deployment/serve_vllm.py --port 8000 to launch an OpenAI-compatible interface, facilitating integration into existing business systems.

Q: How to choose between the 4B, 9B, and 35B-A3B versions?
A: The 4B version is suitable for resource-constrained edge devices and lightweight on-device scenarios. The 9B version balances accuracy and performance, making it appropriate for general production scenarios. The 35B-A3B version employs a MoE architecture, providing stronger model capabilities for complex decision-making tasks, and is recommended for scenarios with extremely high accuracy requirements.

Q: Can the model handle Chinese business scenarios?
A: The model's training data and benchmark tests are primarily in English. However, due to the design of the language-native decision interface, it is theoretically possible to adapt the model to Chinese scenarios through natural language descriptions. It is recommended to first conduct small-scale testing in Chinese business scenarios, evaluate the model's performance, and then proceed with large-scale deployment.

Q: What hardware configuration is required to deploy the model?
A: It is recommended to use an NVIDIA GPU that supports CUDA. The 4B model requires at least 8GB of GPU memory, the 9B model is recommended to have 16GB or more of GPU memory, and the 35B-A3B model requires a higher configuration. Specific memory requirements also depend on the inference batch size and sequence length.

9. Project Links

• HuggingFace Model Library: https://huggingface.co/apus-ailab/APUS-OpenJev-v1