MiniCPM-V 4.7 – An Open-Source Edge Multi-Modal Large Model by Skyline AI and Others

1. What is MiniCPM-V 4.7

MiniCPM-V 4.7 is an open-source edge multi-modal large model series jointly developed by Skyline AI and the OpenBMB community, featuring two versions: 35B-A3B (MoE architecture) and 1B. The flagship 35B-A3B version is the world's first open-source MoE multi-modal model that can run seamlessly and without loss on mobile devices. Leveraging the Leading Expert Selection (LES) technology, it achieves a real-time generation speed of over 13 tokens/s on an iPhone with only about 3GB of memory, and outperforms models in its class across 34 authoritative benchmark tests. The 1B version is designed for entry-level mobile devices, providing high-quality multi-modal understanding capabilities with minimal resource consumption, thereby reaching a broader audience of end users.

Technical Positioning and Domain: MiniCPM-V 4.7 belongs to the domain of edge multi-modal large models. Its core application areas include visual understanding, document parsing, mathematical reasoning, and long video analysis on mobile devices. Its unique positioning lies in the deep integration of the Mixture-of-Experts (MoE) architecture with edge-side inference, enabling capabilities comparable to cloud-based models without relying on cloud computing power. This opens up a new path for the efficient deployment of multi-modal large models on terminal devices. The model is targeted at resource-constrained environments such as smartphones, tablets, automotive systems, and robots, filling the gap in deploying high-performance multi-modal models on edge devices.

Development Background: This model was jointly developed by Skyline AI and the OpenBMB community. Skyline AI is a domestic technical team specializing in edge-side AI large models, while OpenBMB is an open-source large model community initiated by the Natural Language Processing Lab at Tsinghua University. Both parties have accumulated extensive technical experience in edge model compression, efficient inference, and open-source ecosystem building. Previously, they had launched the MiniCPM series of language models and the MiniCPM-V series of multi-modal models. The motivation behind developing MiniCPM-V 4.7 stems from the long-standing challenge in edge multi-modal inference: the contradiction between large model weights and terminal memory capacity. Typically, MoE models are in the tens of gigabytes, while mobile device memory is generally between 8-16GB. How to truly run large models on edge devices has become a core technical challenge.

Core Value: MiniCPM-V 4.7 addresses three major pain points in deploying MoE multi-modal models on edge devices: excessively large model weights, insufficient memory capacity, and slow inference speed. The 35B model with 65.6 GiB of weights can run smoothly with only about 3GB of memory, thanks to the support of LES technology and optimized weight access, which is just one-quarter of the memory on a 16GB device. This breakthrough enables high-quality multi-modal AI capabilities without relying on cloud services, allowing users to enjoy low-latency, privacy-secure local inference experiences even without an internet connection, providing a feasible technical foundation for scenarios such as mobile office work, offline education, and automotive vision.

2. Key Features

• Edge-side MoE Architecture: The flagship version of MiniCPM-V 4.7 employs a 35B-A3B MoE architecture, meaning a total parameter count of 35B and activated parameters of 3B. This design allows the model to maintain the knowledge capacity of a large model while only activating a small number of expert parameters per inference, significantly reducing computational overhead and laying the architectural foundation for smooth edge-side operation. Compared to dense models of similar scale, the MoE architecture demonstrates higher parameter efficiency on multimodal tasks, which is the structural reason why this model outperforms its peers on 34 benchmarks.

• Look-Ahead Expert Scheduling (LES): This is the core technological innovation of MiniCPM-V 4.7. Due to the 65.6 GiB MoE weights being too large to fully reside in memory, most expert weights are stored on SSD and loaded when needed. Traditional approaches must wait for disk reads while routing layer by layer, relying on results from the previous layer. LES leverages the high similarity between adjacent layer inputs to predict the required experts for the next layer based on the current layer's routing features and pre-fetch them in the background. It also prioritizes the computation of experts that are already loaded and selected by real routing, enabling "compute while read." Real-world testing shows a 5x speedup with zero loss in output quality, marking a key breakthrough in running ultra-large MoE models on edge devices.

• General Visual Understanding: The model possesses general visual capabilities such as object recognition in images, multi-image comparison, and visual question answering. It scores 91.72 on the MMBench benchmark, outperforming its peers. Its multi-image comparison ability is particularly strong, allowing it to analyze multiple images simultaneously and perform associative reasoning, which is applicable to real-world scenarios like product comparison and scene difference analysis. This capability is supported by the deep alignment between the visual encoder and the language model, maintaining high visual-semantic understanding accuracy even under resource-constrained edge conditions.

• Document and OCR Parsing: Supports the recognition of text, tables, and chart structures in images and PDFs, achieving an OCRBench score of 91.2 and a CharXiv Reasoning score of 75.6. The model not only identifies textual content but also understands the layout structure and logical relationships of documents, outputting structured parsing results. This capability is especially important for mobile office scenarios, allowing users to directly capture physical documents or import PDF files and perform text extraction, table reconstruction, and chart interpretation locally, without the need to upload files to the cloud for processing.

• Math and STEM Reasoning: Excels in mathematical reasoning and answering scientific questions, scoring 80.22 on MMMU and 88 on MathVista. The model can understand mathematical formulas, geometric figures, and scientific charts, and perform multi-step reasoning. This capability is attributed to targeted optimization of STEM data during training, enabling the model to handle tasks requiring rigorous logical reasoning—such as educational tutoring and academic research assistance—on edge devices.

• Long Video Understanding: Supports understanding of long temporal video content, capable of processing multi-frame information and performing global reasoning, achieving a LongVideoBench score of 68.9. The model can perform content summarization, key event identification, and question answering on long videos recorded on edge devices, suitable for scenarios requiring extended video analysis, such as security patrols and content creation material management. Processing videos directly on edge devices avoids the bandwidth and privacy issues associated with uploading massive video data to the cloud.

• Low-Cost Custom Fine-Tuning (SRFT): Open-sources the self-replay fine-tuning (SRFT) technique, enabling the model to retain its original capabilities while learning new tasks, thus avoiding the catastrophic forgetting problem in traditional fine-tuning. Developers can adapt the model to specific domains with only a small amount of data, without needing to retrain the entire model, significantly lowering the threshold and cost for enterprise-level customization and development.

3. How to Use

1. Acquire Model Resources: Obtain the model code and inference scripts from the GitHub repository (OpenBMB/MiniCPM-V), and download the corresponding version of the model weights from the official model collection page on Hugging Face. It is recommended to select either the 35B-A3B or 1B version based on your target device, and verify the file integrity by checking the SHA checksum.

2. Online Quick Experience: If you wish to experience the model's capabilities before deployment, you can directly access the official Web Demo on Hugging Face Spaces (search for MiniCPM-V-4.7-Demo). No environment installation is required to test the model's core features such as visual understanding and document parsing online. This method is ideal for technical evaluation and functional verification.

3. Install Mobile Application: Download the iOS or Android client installation package from the MiniCPM-V-Apps repository on GitHub, and install it directly on your phone or tablet. The app includes a built-in model inference engine, allowing offline use without additional configuration. This is the most convenient way for individual users to experience the model, with Apple iPhone and iPad users achieving optimal performance.

4. Select Inference Framework for Deployment: For developers requiring deep integration, choose one of the following frameworks—llama.cpp, Ollama, MLX, or vLLM/SGLang—to load the model based on your hardware platform and complete the local inference service deployment. Among these, llama.cpp is suitable for CPU/GPU heterogeneous environments, MLX is optimized for Apple chips, and vLLM/SGLang is appropriate for high-concurrency scenarios on the server side. Specific loading commands and configuration parameters for each framework can be found in the deployment documentation of the GitHub repository. It is recommended to first verify the deployment correctness on a small-scale test set.

5. Enable LES Acceleration (Optional): When running the 35B-A3B version on the edge side, enable the forward expert scheduling feature as described in the open-source documentation to significantly reduce SSD read waiting time and achieve smooth operation with approximately 3GB of memory. The LES feature is recommended for low-memory devices, but you must ensure that the device's SSD read speed meets the requirements.

6. Switch Visual Compression Mode: Switch between 4x compression and 16x compression modes based on your task requirements. In 16x mode, key metrics remain almost lossless, making it suitable for interactive scenarios with high real-time demands. The 4x mode preserves more visual detail and is appropriate for tasks requiring strict visual accuracy.

7. Hardware Environment Requirements: The 35B-A3B version is recommended to run on mobile devices with memory of at least 8GB and SSD storage (such as iPhone 15 Pro and above, iPad Pro with M-series chips). The 1B version has more relaxed hardware requirements and can run smoothly on mid-range or low-end phones. For server-side deployment, NVIDIA GPU or domestic acceleration cards (such as Huawei Ascend and Hygon, which have achieved Day-0 compatibility) are required.

4. Pros and Cons Analysis

5. Comparative Analysis with Similar Tools

Selection Recommendations: For developers requiring on-device deployment of multimodal models and high performance, the MiniCPM-V 4.7 35B-A3B version is currently the most competitive option, particularly demonstrating clear advantages on Apple iOS devices and domestic chip platforms. Its LES technology addresses the performance bottlenecks of MoE models on on-device execution, achieving a good balance between memory usage and generation speed. If the development scenario has clear requirements for domestic information innovation environments, MiniCPM-V 4.7's Day-0 compatibility ensures smooth deployment on platforms such as Huawei Ascend and Hygon.

For entry-level devices or cost-sensitive scenarios, the 1B version of MiniCPM-V 4.7 shows stronger multimodal understanding capabilities at the same parameter scale, outperforming Qwen3.5-0.8B. If developers have made significant investments in the Alibaba Cloud technology stack or need system integration with the Tongyi Qianwen product series, the Qwen series remains a viable consideration. Overall, MiniCPM-V 4.7 exhibits a clear technical advantage in on-device multimodal scenarios, making it particularly suitable for mobile applications, offline smart devices, and domestic terminal products.

6. Editor's Summary

MiniCPM-V 4.7 has achieved groundbreaking technological innovation in the field of end-side multimodal large models, serving as a model for others to follow. Its core value lies not in the breakthrough of a single metric, but in systematically addressing the "weight volume - memory usage - inference latency" triangle dilemma in the deployment of MoE models on the edge. The forward-looking expert scheduling (LES) technology leverages the structural characteristics of input similarity between adjacent Transformer layers, using predictive prefetching instead of stop-and-wait reading to decouple SSD read latency from the critical inference path. This design approach demonstrates high technical value. Weight access optimization further reduces storage and transmission overhead through per-layer mixed-precision quantization and expert weight layout reorganization. These combined technologies present a complete edge deployment solution, rather than isolated optimization points.

In terms of practical value, this model compresses 65.6 GiB of MoE weights to run within approximately 3GB of memory, enabling flagship-level multimodal inference capabilities on mobile devices with 16GB of memory. It also maintains an interactive generation speed of over 13 tokens per second. This achievement means that AI assistants with capabilities such as photo-based object recognition, document parsing, and mathematical reasoning can operate entirely offline on personal devices, offering significant value in terms of privacy protection, usability in offline environments, and data security compliance. The leading performance on 34 benchmark tasks also indicates that this performance advantage is not achieved at the expense of model capabilities.

In terms of target users, this model is suitable for three categories of users: first, engineers with development needs for edge-side AI applications, who can quickly build product prototypes using open-source code and pre-trained weights; second, industry users with strict data privacy requirements (such as healthcare, finance, and government), who can avoid compliance risks associated with data leakage by deploying the model locally; third, solution integrators in vertical fields such as education, automotive, and robotics, who can leverage its multimodal capabilities and chip ecosystem compatibility to build industry-specific solutions.

In terms of development potential, the continuous investment by Baowei Intelligence and OpenBMB in edge-side models has established a clear technical evolution path. As mobile device computing power and storage capacity continue to improve, the deployment space for edge-side multimodal models will expand further. The SRFT self-replay fine-tuning technology lowers the threshold for industry customization, while the Apache-2.0 license removes legal barriers for commercial applications. MiniCPM-V 4.7 has set a technical benchmark in the edge-side multimodal model space, which will serve as an important reference for similar products for a considerable period of time.

7. Application Scenarios

• Mobile Office and Document Processing: Users can directly capture paper documents or import PDF files on their phones or tablets, and the model performs text recognition, table extraction, chart understanding, and document Q&A in real-time locally. The entire processing workflow does not require uploading files to the cloud, enhancing mobile office efficiency while protecting sensitive business information. For users who frequently travel or work remotely, this capability removes the dependency on network conditions for document processing.

• Offline Educational Tutoring: Students can use features such as photo-based question search, math reasoning, and science Q&A on devices without internet connectivity. The 35B model's on-device execution enables high-quality AI tutoring services to reach areas with weak network infrastructure, while also addressing privacy concerns related to minors using cloud-based AI services. With performance scores of 80.22 on MMMU and 88 on MathVista, the model is capable of meeting the subject tutoring needs of most primary and secondary schools, as well as some higher education institutions.

• In-Vehicle and Robotics Vision: Leveraging Day-0 chip compatibility capabilities, the model can be deployed on in-vehicle terminals and robot control units, enabling real-time environmental recognition, obstacle understanding, and human-robot visual interaction. In in-vehicle scenarios, local inference avoids the impact of network latency on safety-critical tasks; in robotics scenarios, the model's STEM reasoning capabilities enhance the device's understanding and decision-making quality in physical environments.

• Video Content Understanding and Moderation: The model's long-video understanding capability supports content summarization, key-frame Q&A, and compliance review for long videos recorded locally. This is applicable to scenarios such as security patrols, content creation material management, and compliance checks. Security patrol personnel can quickly review critical events in surveillance footage on-site using mobile devices; content creators can instantly analyze material content at the shooting location, improving post-production efficiency.

• Offline Smart Assistants and Accessibility Services: The model can function as a fully offline smart assistant, providing assistive features such as object identification through photos, text reading, and scene description for visually impaired users. All processing is completed locally, offering fast response times and independence from network connectivity, thereby providing a more reliable technical solution for accessibility services compared to cloud-based alternatives.

• Intelligent Upgrades for Domestic Terminals: With Day-0 compatibility for domestic chips such as Huawei Ascend and HYGI, this model can be used for AI capability upgrades in domestic computers, tablets, and in-vehicle systems. In government and party organizations, state-owned enterprises, and other units with clear supply chain security requirements, the model provides a technical path that does not rely on overseas computing power or cloud services.

8. FAQ

Q: What hardware requirements must be met for MiniCPM-V 4.7 to run on mobile devices?
A: The 35B-A3B version is recommended to run on mobile devices with memory of at least 8GB and SSD storage. Tested devices include the iPhone 17 Pro Max and iPad Pro M4. The device storage must reserve at least 70GB of space to store the model weights. The 1B version has lower hardware requirements and can run smoothly on mid-to-low-end Android phones with 4GB or more of memory.

Q: Does LES (Look-ahead Expert Scheduling) technology support all inference frameworks?
A: LES is currently integrated mainly into the officially recommended on-device inference pipeline. For compatibility with llama.cpp and MLX, please refer to the release notes of the corresponding versions. Server-side frameworks such as vLLM/SGLang already keep the model resident in GPU memory and do not require LES to be enabled. Developers should consult the official documentation to confirm whether the version of the framework they are using includes LES support.

Q: In what scenarios can the 16x visual compression mode cause visible quality degradation?
A: The 16x mode may experience slight performance drops compared to the 4x mode in tasks such as high-density text recognition, small object detection, and detailed structure understanding of charts. However, official benchmarks show that key metrics remain almost lossless. It is recommended to use the 16x mode for real-time interactive tasks (e.g., video Q&A, multi-turn conversations), and to switch back to the 4x mode for scenarios such as document parsing and detailed visual Q&A to ensure optimal results.

Q: Can MiniCPM-V 4.7 be used in commercial projects? Is a paid license required?
A: Yes. The model is licensed under the Apache-2.0 license, with both code and weights open-sourced. It allows for commercial use, modification, and redistribution without requiring payment for a license. However, the original copyright notice must be retained, and a copy of the license must be included with any distribution. Specific terms are subject to the original Apache-2.0 license text.

Q: What is the difference between SRFT self-replay fine-tuning and conventional SFT?
A: Conventional SFT (supervised fine-tuning) can lead to model forgetting of its original capabilities when adapting to new tasks, a phenomenon known as catastrophic forgetting. SRFT introduces a self-replay mechanism during fine-tuning, allowing the model to review its previously learned representative capabilities while training on new tasks. This enables developers to customize the model for specific domains at a lower cost without worrying about degradation of its foundational abilities after fine-tuning.

Q: What is the size of the model's context window? Can it handle very long documents?
A: Please refer to the official technical report to confirm the exact length of the context window. For handling very long documents, it is recommended to use segmented input and summary concatenation, leveraging the model's multi-image understanding capability to achieve global comprehension across pages. For extremely long videos, the model supports sampling key frames and performing overall inference.

9. Project Links

• Official GitHub repository for MiniCPM-V: https://github.com/OpenBMB/MiniCPM-V (includes code, weight download links, and deployment documentation)
• Official Demo on Hugging Face Spaces: https://huggingface.co/spaces (search for "MiniCPM-V-4.7-Demo" to experience it online; this is the official Web demo entry)
• Official model collection page on Hugging Face: https://huggingface.co/collections/openbmb/minicpm-v-47 (provides weight files for the 35B-A3B and 1B versions)