MiniCPM-OCR – OpenBMB's Open-Source OCR Series Model

1. What is MiniCPM-OCR

MiniCPM-OCR is a new series of open-source OCR models released by OpenBMB, including the document parsing-specific model MiniCPM-OCR 1.0 (1.3B) and the industry's first parsing result discriminative model, MiniCPM-OCR Jev (2B). The former achieves state-of-the-art (SOTA) results on two authoritative benchmarks, Real5-OmniDocBench and OCRBench v2, while the latter provides a quantitative measure of OCR result reliability through outputting quality scores ranging from 0 to 1. This series of models is designed for privacy-sensitive scenarios such as finance, taxation, legal affairs, and healthcare, supporting on-device local execution and secondary fine-tuning, ensuring parsing accuracy while maintaining data security and deployment flexibility.

Technical Positioning and Domain: Belongs to the field of optical character recognition (OCR) within the intersection of computer vision and natural language processing. It focuses on document parsing, structured information extraction, and parsing quality discrimination. MiniCPM-OCR 1.0 is positioned as an efficient document parsing engine, while Jev innovatively defines a niche category known as the "OCR quality inspection model," together forming a complete "parsing + inspection" closed loop.

Development Background: Developed by the OpenBMB team, which has accumulated substantial technical expertise in the MiniCPM series of multimodal models. MiniCPM-OCR is trained based on the MiniCPM-V 4.6 multimodal model, reusing the large-scale, high-quality document pre-training data from the MiniCPM-V/o series, enabling the model to have a solid foundation in image and document understanding before specialized training.

Core Value: Addresses three major pain points of traditional OCR systems: insufficient parsing accuracy for complex layouts, inability to quantify the reliability of results, and high costs for manual verification. The introduction of the Jev model makes it possible for the first time to predict which pages require manual verification, enabling automatic routing based on score thresholds. Up to 85% of pages can be exempt from manual checks, significantly reducing human labor costs.

Technical Features: Employs the LLaVA-UHDv4 efficient visual patching scheme to avoid computational expansion caused by high-resolution images; DSpark decoding acceleration improves end-to-end efficiency by 55%, achieving a throughput of 1.217 pages per second; Jev follows the System-1 fast thinking approach, capable of outputting quality scores with a single forward pass, offering extremely low latency suitable for high-frequency quality inspection gateways.

2. Key Features

• Document Parsing and Structuring: Restore documents such as invoices, contracts, reports, and medical records to their layout structure, outputting Markdown or field-based data. Supports end-to-end conversion from raw images to structured information, enabling direct integration with business systems and reducing manual data entry.

• Complex Layout Recognition: Covers seven high-frequency complex scenarios, including multi-column newspapers, wide financial tables, handwritten notes, complex matrix formulas, and documents with stamps or occlusions. Through training on large-scale real-world data and optimization with reinforcement learning, the parsing quality on long-tail samples significantly outperforms general OCR models.

• Automatic Image Correction: Integrates automatic upright design, automatically detecting and correcting rotated or inverted document images. This feature allows parsing without manual alignment, saving substantial preprocessing time in batch scanning scenarios.

• Cross-page Content Merging: Merges cross-page information from long documents for output compatible with business system calls. For multi-page contracts, reports, and similar scenarios, the model can identify continuous tables and paragraphs across pages, ensuring the integrity of the output structure.

• Parsing Quality Discrimination (Jev): Outputs a 0–1 quality score for the parsing results of any OCR system, determining whether the results are trustworthy. Jev is trained on block-level correctness labels from real human annotations, and the scores can be directly used as business thresholds, achieving an AUROC of 93.6, surpassing 35B-level discriminative models.

• Human Verification Routing: Automatically filters results based on score thresholds, allowing high-scoring outputs to be directly accepted without manual review, while low-scoring ones are routed to human verification. On multiple systems such as GLM-OCR and PaddleOCR-VL, AUROC remains stable between 91.9 and 93.4. The quality inspection capability is plug-and-play and not tied to any specific OCR system.

• Multi-model Optimization: After parsing with multiple models, Jev scores each region and combines the best results, achieving an overall score improvement of up to 0.25. This feature allows multiple OCR engines to work in parallel, with Jev objectively selecting the optimal results to enhance system-level accuracy.

• Key Information Extraction (KIE): Extracts critical field information from complex scenarios such as documents with stamps or occlusions and receipts. By combining layout analysis with semantic understanding, it accurately extracts text content obscured by stamps, meeting the structured data needs of government and financial scenarios.

3. How to Use

1. Environment Requirements: Supports local deployment for inference and requires a Python 3.8+ environment. The model is compatible with multiple chip configurations. For edge devices or low-computation environments, the GGUF/INT4 quantized version is recommended. It is advised to have at least 8GB of GPU memory for a smooth inference experience.

2. Model Download: Download the model weights using the Hugging Face CLI, either openbmb/MiniCPM-OCR-1.0 or openbmb/MiniCPM-OCR-Jev. Use the command huggingface-cli download openbmb/MiniCPM-OCR-1.0 to complete the download.

3. Inference Framework Installation: The model is initially compatible with four inference frameworks: vLLM, SGLang, llama.cpp, and Ollama. Choose one based on your needs. The general launch method for OpenBMB series models is vllm serve openbmb/<model name> --port 8000. After launching, the model can be accessed via API endpoints.

4. Quantization Version Selection: Choose the appropriate quantization format based on your hardware configuration. For edge deployment, it is recommended to use the GGUF/INT4 version to reduce resource consumption. Multi-chip compatibility supports different hardware platforms; refer to the official repository documentation for the full list of supported quantization formats.

5. Inference Invocation: Input document images, and the model automatically performs alignment correction, layout analysis, and structural parsing, outputting results in Markdown or structured fields. The Jev model accepts outputs from any OCR system and returns quality scores and confidence information.

4. Pros and Cons Analysis

5. Comparative Analysis with Similar Tools

Selection Recommendations: For privacy-sensitive scenarios such as finance and healthcare, where parsing accuracy is extremely critical, the combination of MiniCPM-OCR 1.0 and Jev offers a clear advantage—local deployment ensures data security, while Jev's automatic routing mechanism minimizes manual verification costs to the lowest possible level. If the team already has a mature PaddleOCR technology stack and the primary use case involves general documents, the mature ecosystem and fine-tuning toolchain of PaddleOCR-VL-1.6 may be more user-friendly.

For enterprise-level systems requiring multi-model collaboration, MiniCPM-OCR's Jev multi-model selection capability provides unique architectural value—it can achieve system-level accuracy improvements through Jev scoring without replacing the existing OCR engine. In scenarios relying on cloud services, a trade-off between data security and convenience is necessary. MiniCPM-OCR's local deployment solution is more attractive in industries with strict compliance requirements.

6. Editor's Summary

The MiniCPM-OCR series of models demonstrates a clear path of technological innovation in the OCR domain. From a technical perspective, its core breakthrough lies not in the improvement of a single metric, but in establishing a complete closed-loop of "parsing + quality inspection." The SOTA performance of MiniCPM-OCR 1.0 on two benchmarks confirms its solid foundational parsing capabilities, while the Jev model innovatively quantifies the reliability of OCR results, addressing the fundamental issue that has long existed in the industry: whether machine outputs can be trusted. The DSpark decoding acceleration technology increases throughput to 1.217 pages per second while maintaining quality, making the model practically usable in high-concurrency production environments.

From a practical value standpoint, Jev's human verification分流 mechanism directly addresses the most expensive aspect of OCR implementation in enterprises—human labor costs. A maximum of 85% inspection exemption rate means that with the same level of human investment, significantly more documents can be processed than before. This value is especially pronounced in large-scale document processing scenarios such as financial invoices and legal contracts. The lightweight 1.3B parameters combined with multiple quantized versions allow the model to run on edge devices, providing a feasible technical solution for industries with strict data security requirements.

The target users include three categories: first, developers who require high-precision document parsing and can quickly integrate based on open-source weights; second, technical decision-makers focused on OCR quality inspection and workflow optimization, for whom Jev's plug-and-play characteristics lower the adoption threshold; third, enterprises with privacy and compliance requirements, as the local deployment option avoids data leakage risks. The innovations in technical architecture and product design of the MiniCPM-OCR series provide new directions for the OCR field.

7. Application Scenarios

• Financial Document Processing: Automatic recognition and structured input of bank statements, checks, and receipts. High-quality documents verified by Jev are directly archived, reducing manual data entry at the counter. In bank back-end operations handling tens of thousands of documents daily, this solution can reduce verification manpower to approximately 15% of the original, while maintaining high input accuracy.

• Legal Contract Review: Law firms and legal departments can batch parse contract texts. Local execution ensures that documents remain within the jurisdiction, and combined with Jev's ability to skip verification for up to 85% of pages, significantly reduces verification time. For due diligence reports that can be hundreds of pages long, improved parsing efficiency directly translates to shorter project delivery cycles.

• Medical Record Digitization: Hospitals convert paper medical records and test reports into structured electronic data. Private deployment meets patient privacy compliance requirements. MiniCPM-OCR's capability to recognize handwritten notes and documents with stamps or obstructions makes the digitization of historical paper records feasible.

• Government Document Management: Recognition and key information extraction (KIE) of official documents, credentials, and stamped files, with automatic extraction of critical fields obscured by stamps. In scenarios such as real estate registration and household registration management, this solution significantly enhances the efficiency of document digitization while ensuring sensitive information remains within the government intranet.

8. FAQ

Q: What is the difference between MiniCPM-OCR 1.0 and the Jev model?
A: MiniCPM-OCR 1.0 is a document parsing model responsible for converting images into structured text; Jev is a discriminative model for parsing results, responsible for evaluating the quality of the parsed output. Both can be used independently or together to form a "parsing + quality inspection" workflow.

Q: What OCR system outputs does the Jev model support?
A: Jev is not tied to a specific OCR system and can accept parsing results from any OCR engine as input. It has been validated on systems such as GLM-OCR and PaddleOCR-VL, achieving a stable AUROC of 91.9 to 93.4.

Q: What are the minimum hardware requirements for on-device deployment?
A: The official recommendation is to use the GGUF/INT4 quantized version for on-device deployment, with a minimum of 8GB of GPU memory. Specific hardware requirements depend on the input image resolution and the number of concurrent requests. In low-computation environments, the input image size can be further reduced.

Q: Does the model support document parsing in Chinese?
A: Yes. The model is trained on MiniCPM-V 4.6 and has multilingual document understanding capabilities. It includes specialized optimizations for Chinese scenarios such as invoices, contracts, and medical records, and covers complex cases like handwritten notes and stamp occlusions.

Q: How can MiniCPM-OCR be fine-tuned further?
A: The model is fully open-sourced and supports fine-tuning with synchronized updates. You can use the fine-tuning toolchain from the OpenBMB series of models to adapt the model to your domain using your own annotated data. Specific fine-tuning scripts and example code can be found in the GitHub repository.

Q: How should the quality score threshold for Jev be set?
A: The scores are trained based on block-level correctness labels from real human annotations and can be used directly as business thresholds. It is recommended to set the threshold based on the actual fault tolerance requirements of the scenario. For example, in financial scenarios, a threshold of 0.8 can be used for automatic pass, while in general scenarios, a threshold of 0.7 is recommended.

9. Project Links

• GitHub Repository: https://github.com/OpenBMB/MiniCPM-OCR
• HuggingFace Model Library: https://huggingface.co/openbmb/MiniCPM-OCR-1.0
• HuggingFace Model Library: https://huggingface.co/openbmb/MiniCPM-OCR-Jev