Evaluation of SenseNova Loop-9B-Preview: A Cyclic Multimodal Architecture from SenseTime Challenging 27B-Level Complex Tasks with 9B Parameters

1. What is SenseNova Loop-9B-Preview

SenseNova Loop-9B-Preview is a 9B-parameter-level multimodal preview model open-sourced by SenseTime. Its core innovation lies in the use of the Looped Transformer cyclic architecture, which retains the first and last four layers of the original decoder and replaces the intermediate layers with a parameter-shared cyclic computation module. By expanding the effective computational depth through multiple iterations, the model achieves an execution depth of 56 layers with approximately 9B parameters. The model introduces the mHC (Manifold-Constrained Hyper-Connections) mechanism to enhance the stability of feature interactions. In official evaluations, it outperforms Qwen3.5-27B in office and agent tasks, delivering large-model-level capabilities in complex tasks with fewer parameters. The model weights are publicly available on GitHub and Hugging Face.

Technical Positioning and Domain: Belongs to the field of multimodal large language models, positioned as a cyclic multimodal model designed for long context and agent workflows. It supports text and image inputs, with a training context length of up to 262,144 tokens, and is capable of handling code generation, office content creation, and complex agent task execution.

Development Background: Developed by SenseTime, leveraging its deep expertise in computer vision and AI large models. Previously, SenseTime had launched the "RiRi Xin" series of large model systems. SenseNova Loop-9B-Preview represents an important open-source practice in exploring efficient parameter utilization and inference cost control. It is among the first open-source models in China to adopt a 9B-level cyclic architecture.

Core Value: Addresses the industry challenge of limited capabilities in small-parameter models for complex reasoning tasks. By using parameter-shared cyclic computation, it approaches the performance of 27B-level dense models in office and agent tasks while keeping inference costs under control, providing a high-performance model solution for resource-constrained scenarios. At the same time, its open-source nature promotes community research and industrial applications.

Technical Features: Employs the Looped Transformer cyclic architecture and dual-stream mHC feature interaction mechanism, achieving a simulated computational depth of 56 layers with approximately 9B parameters; includes the MTP inference acceleration mechanism, supports multimodal input and 256K ultra-long context, and demonstrates outstanding performance in multi-step agent evaluations such as AutomationBench.

2. Key Features

• Code Generation: Achieves state-of-the-art performance in code-writing tasks, supporting full development scenarios. The model can complete code completion, script writing, and end-to-end development of small projects, providing developers with efficient coding assistance. It demonstrates consistent performance in terms of code quality and logical correctness.

• Office Content Generation: Integrated with the Xiaohuanxiang client, capable of generating office deliverables such as PPT designs and long documents. The model can automatically generate structured presentation content, data analysis reports, and animated demonstration pages based on user needs, covering high-frequency document creation and reporting scenarios in daily office work, significantly improving office efficiency.

• Agent Tasks: Performs exceptionally well in multi-step Agent evaluations such as AutomationBench, capable of completing complex, long-chain tasks involving narratives and interactions. The model has the ability to perform multi-step tool calling, web operations, and task planning, making it suitable for building Agents focused on automated office work and process execution. It can autonomously decompose complex tasks and execute them step by step.

• Long Context Processing: Trained to support a maximum context length of 262,144 tokens, enabling the model to process large-volume documents such as entire contracts, financial reports, and paper collections in one go. With its ultra-long context window, the model can generate structured summaries, perform in-depth Q&A, and extract information from long documents, making it suitable for document understanding and long-chain task scenarios.

• Multimodal Input Understanding: Supports dual-modal input of text and images, allowing the model to understand image content and generate corresponding text output. This capability is applicable to rich media content generation scenarios such as travel topics with mixed text and images, marketing pages, and narrative websites, expanding the model's application boundaries in visual understanding and cross-modal reasoning.

• Inference Acceleration Mechanism: Built-in MTP (Multi-Token Prediction) acceleration mechanism is now available, significantly improving inference throughput efficiency. Speculative decoding solutions such as Dspark and Dflash2 are currently under development, and in the future, they will further optimize inference speed and reduce deployment and operational costs in production environments.

• Thinking Mode Switch: Supports the enable_thinking parameter to control the mode. When deep reasoning is required, the thinking mode can be activated, allowing the model to perform internal reasoning and verification before generating the final response. This feature effectively enhances the quality and accuracy of responses in complex tasks such as mathematical reasoning and logical analysis.

3. Application Scenarios

• Long Document Understanding and Analysis: Leveraging a 256K ultra-long context window, the model can process large-volume documents such as contracts, financial reports, and thesis collections in one go, generating structured summaries and supporting in-depth Q&A. Professionals in fields like legal, finance, and research can directly upload long documents to quickly obtain key information extraction and point analysis, significantly reducing manual reading time.

• Agent Workflow: Excels in scenarios involving multi-step tool calling, web operations, and task planning, making it ideal for building Agent systems for automated office work and process execution. Enterprises can integrate the model into their internal systems to achieve automated data collection, information organization, and process approval. The model can independently decompose complex tasks and execute them step by step, reducing the cost of manual intervention.

• Office Content Generation: After integrating with the Xiao Huanhong client, the model can directly generate PPT designs, data analysis reports, and animated presentation pages. Office users can input a topic or data to receive complete, visually appealing presentation drafts and documents, covering high-frequency office delivery scenarios such as daily reports and project proposals.

• Code Development Assistance: Code generation capabilities are at a leading level among peers, capable of handling code completion, script writing, and end-to-end development of small projects. When developers are writing complex functions, debugging errors, or implementing specific features, they can obtain code recommendations and implementation plans through conversational interaction, improving development efficiency and code quality.

• Multimodal Content Creation: Supports text and image inputs, suitable for generating rich media content such as travel features, marketing pages, and narrative websites with mixed text and images. Content creators can upload reference images and describe their requirements, and the model will generate corresponding text and page structure by integrating visual information, enriching the forms of creative expression.

4. How to Use

1. Environment Setup: Install the PyTorch framework that matches your CUDA version, then execute pip install "transformers==5.18.0" accelerate safetensors to install the necessary dependencies. It is recommended to use a Linux operating system with an NVIDIA GPU (recommended with 16GB or more VRAM) to achieve better inference performance.

2. Obtain the Model: Download the sensenova/SenseNova-Loop-9B-Preview weights from Hugging Face to a local directory. Ensure that the directory contains the configuration file, tokenizer file, chat template, and custom model code. The model uses a custom architecture, so it is essential to retain all model implementation files from the repository.

3. Load the Model: In Python code, call AutoModelForCausalLM.from_pretrained with the parameters trust_remote_code=True and dtype=torch.bfloat16 to load the model. Then call .eval() to switch to inference mode. When loading, specify the local model path to ensure the code can correctly read the custom model structure.

4. Construct Input: Use the tokenizer.apply_chat_template method to assemble the dialogue prompt. Feed the user message in the dialogue format. When the task requires deep reasoning, set enable_thinking=True to activate the thinking mode. The model will perform internal reasoning before generating a response.

5. Inference Generation: Call the model.generate method to generate text. This method supports the use_cache parameter to speed up the inference process. In scenarios involving long text generation, you can enable the MTP acceleration mechanism to improve generation efficiency. Adjust generation parameters such as max_new_tokens based on the task complexity.

6. Service Deployment: In production environments, you can use the LightLLM or SGLang framework to provide API services. Note that you must currently use the official PR branch version of the framework, as the standard release version may not yet fully support this model architecture. Confirm the framework version compatibility before deployment.

5. Pros and Cons Analysis

6. Comparative Analysis with Similar Tools

Selection Recommendations: For scenarios with limited resources and the need to process long documents or perform complex Agent tasks, SenseNova Loop-9B-Preview achieves capabilities close to a 27B model with only 9B parameters, offering a clear cost advantage and is suitable for long context understanding and multi-step task execution requirements. If deployment ecosystem maturity and community support are of high importance, and the tasks are primarily text-based, Qwen3-8B, with its mature vLLM/Ollama ecosystem and rich fine-tuning toolchain, is more suitable for rapid deployment in production environments. For medium to large teams with ample computational resources seeking top-tier performance, Qwen3.5-27B demonstrates more comprehensive performance in complex reasoning and multimodal understanding, making it the preferred choice for high-demand tasks.

7. Editor's Summary

SenseNova Loop-9B-Preview demonstrates a clear direction of technological innovation at the model architecture level. The Looped Transformer cyclic architecture expands the effective computational depth from 32 layers to 56 layers without increasing the parameter scale, through parameter sharing and multi-round iterations. This design approach provides a feasible path for small-parameter models to approach the capabilities of large models. The introduction of the mHC feature interaction mechanism is not a simple engineering optimization, but rather addresses feature degradation during multi-round loops from the perspective of information manifold constraints. Ablation experiments show that this mechanism improves GPQA Diamond by 4.42 points and TeamBench by 13.34 points, verifying its practical value in enhancing reasoning capabilities and Agent task execution.

In terms of practical value, this model achieves performance comparable to a 27B-level model with approximately 9B parameters. Its cost control, with a single forward pass requiring about 14.6B computations, makes it attractive in scenarios with limited GPU resources. The combination of 256K ultra-long context and multimodal input capabilities provides clear application potential in real-world business scenarios such as long document processing, automated office tasks, and rich media content creation. Official evaluations show that it performs exceptionally well in multi-step Agent tasks like AutomationBench, offering a new model option for building automated workflows.

This model is suitable for development teams sensitive to inference costs and requiring long context processing and Agent task execution capabilities, as well as research institutions looking to explore multimodal applications within a controlled budget. Its deployment ecosystem and fine-tuning toolchain are still in the early stages of construction, but the cyclic architecture technical route and open-source strategy lay the groundwork for future iterations. As the MTP acceleration mechanism is implemented and speculative decoding solutions like Dspark and Dflash2 are advanced, its performance in inference efficiency is worth watching.

8. FAQ

Q: What is the memory requirement for SenseNova Loop-9B-Preview?
A: The model has approximately 9B parameters and requires about 18GB of GPU memory when loaded in bfloat16 precision. The computational load for a single forward pass during inference is approximately 14.6B tokens. It is recommended to use GPUs with 24GB or more memory (such as NVIDIA RTX 3090/4090 or A10/A100) for a smooth inference experience. If GPU memory is limited, consider using quantization techniques or CPU inference (which will be slower).

Q: How to enable the model's thinking mode?
A: When constructing the dialogue input, assemble the prompt using tokenizer.apply_chat_template, and then set the parameter enable_thinking=True to activate thinking mode. This mode is suitable for tasks requiring deep thinking, such as mathematical reasoning and logical analysis. The model will perform internal reasoning and validation before generating the final response, but this will increase inference time accordingly.

Q: What input modalities does the model support?
A: The model supports dual-modal input of text and images, with text as the output. Users can upload images along with textual descriptions in a conversation, and the model can understand the image content and generate relevant text responses. It should be noted that the model does not support image generation, only image understanding and text output.

Q: What is the maximum context length of the model? How long of a document can it process in practice?
A: The model is trained to support a maximum context length of 262,144 (256K) tokens. In practice, it can process documents of approximately 200,000 to 300,000 characters at once, depending on the efficiency of the tokenizer in splitting the text. When processing very long documents, it is recommended to enable use_cache for acceleration, while also noting that GPU memory usage will increase with the length of the input.

Q: Can LightLLM and SGLang be used directly for deployment in a production environment?
A: Currently, you need to use the PR branch version provided by the official repository, as the standard release version may not yet fully support the model's loop architecture. It is recommended to first obtain the latest code from the GitHub repository and confirm the PR merge status, or directly use the Transformers library for inference deployment. Once the official releases a stable version with full support, you can then migrate to a service-oriented framework.

9. Project Links

• GitHub repository: https://github.com/OpenSenseNova/SenseNova-Loop
• HuggingFace model library: https://huggingface.co/sensenova/SenseNova-Loop-9B-Preview