LFM2.5-24B-A2B – Liquid AI's Open-Source Edge Inference Model

1. What is LFM2.5-24B-A2B

LFM2.5-24B-A2B is an open-source weight inference model launched by Liquid AI, part of the flagship product line in the LFM2.5 series. This model employs a 40-layer Mixture of Experts (MoE) architecture, with a total parameter count of 24B, but only activates 2.45B parameters per token. It supports a context window of up to 131K tokens and 16 languages. Before generating responses, LFM2.5-24B-A2B creates an explicit chain-of-thought (CoT), optimized specifically for edge-side Agent scenarios. It achieves a LiveClaw score of 48.09, surpassing Qwen3.6-35B-A3B, which has 50% more parameters, and reaches a decoding speed of 184 tok/s on Apple M5 Max, 1.7 times faster than similar MoE models. This model is designed for inference and tool calling in local private deployment scenarios, and it is recommended to use it with retrieval augmentation for knowledge-intensive tasks.

Technical Positioning and Domain: LFM2.5-24B-A2B is a sparse-activation MoE model in the field of natural language processing, focusing on edge-side agent inference and tool calling scenarios. It has a unique positioning in the direction of "large capacity, low computational cost," suitable for private deployment on consumer-grade hardware.

Development Background: This model was developed by the Liquid AI team, iterating upon the main architecture of the previous LFM2-24B-A2B. The team has ongoing technical accumulation in MoE sparse activation and hybrid linear architecture. This release continues their research direction in efficient inference on edge devices.

Core Value: The model achieves Agent inference performance comparable to 35B-level models with an activated parameter count of less than 3B. It significantly outperforms peer models on tool calling benchmarks such as LiveClaw and AutomationBench, addressing the pain points of limited capabilities in edge-side models for tool usage and multi-step reasoning tasks, while keeping inference costs low.

Technical Features: It uses a hybrid architecture combining 30 layers of dual-gated short convolution and 10 layers of GQA in an alternating stack, balancing local feature extraction with long-range dependency modeling; it doubles the vocabulary size to 128K through in-place vocabulary expansion, significantly improving tokenization efficiency for non-Latin scripts; and it integrates a three-phase training process with five-phase post-training using reinforcement learning within a real Agent framework, enabling the model to acquire stable Agent capabilities.

2. Key Features

• On-device Agent Inference: The model is designed for fast, private agent execution on personal devices, enabling continuous operation of planning, tool calling, and task execution workflows on hardware such as MacBook, with data never leaving the device. Compared to cloud-based calls, this mode offers structural advantages in privacy protection and response latency.

• Pure Inference Mode: Automatically generates explicit chain-of-thought (CoT) reasoning before answering, trading low activation costs for improved answer quality. With only 2.45B parameters activated per token, the additional computational overhead from the CoT is kept within an acceptably low range, allowing the model to perform complex reasoning tasks on consumer-grade devices.

| - Function Calling: Natively supports a four-step tool calling process. The model can output structured function call instructions using the `<|tool_call_start|

• Multi-format Deployment: Provides native versions in BF16, FP8, GGUF, ONNX, and MLX-4bit, supporting various hardware environments including NVIDIA GPU, CPU, Apple Silicon, and edge devices. The GGUF format is compatible with local inference tools like llama.cpp and LM Studio, the MLX format is optimized for Apple Silicon, and the FP8 version is tailored for high-throughput service scenarios.

• Long Context Handling: Supports a context window of up to 131,072 tokens, capable of handling complex inputs such as long documents and multi-turn agent trajectories. The context window has expanded from 32K in the previous generation to 131K, and with specialized long-text training, the model maintains stable output quality in long-sequence scenarios.

• Multilingual Support: Covers 16 languages including English, Chinese, Japanese, Korean, and Arabic, with a 128K vocabulary optimized for tokenization efficiency in non-Latin scripts. On-the-spot vocabulary expansion technology significantly improves token efficiency for languages like Chinese and Japanese, reducing the number of tokens required for long-text processing.

• Fast Inference: Measured decoding speed reaches 184 tok/s on Apple M5 Max, approximately 1.7 times faster than similar MoE models; inference throughput exceeds 21K tok/s on a single NVIDIA B300 GPU. Thanks to the efficient computation of linear layers enabled by the dual-gated short convolution structure, the model demonstrates speed and memory usage advantages over MoE models of similar scale.

3. How to Use

1. Environment Requirements: The model supports multiple hardware platforms, including NVIDIA GPU, AMD, Qualcomm, Apple Silicon, and pure CPU environments. The Python environment requires the installation of transformers>=5.0.0, or the deployment of inference frameworks such as vLLM, SGLang, llama.cpp, or LM Studio. It is recommended that the development environment has at least 16GB of memory, with actual requirements depending on the deployment format.

2. Download the Model: Download the LiquidAI/LFM2.5-24B-A2B repository from Hugging Face, and select the appropriate weight version based on your hardware conditions: NVIDIA GPU users can choose the native BF16 or FP8 format; Apple Silicon users are recommended to use MLX-4bit; CPU environments should use the GGUF quantized version; for edge devices, the ONNX export format is available.

3. Load the Model: Use the Hugging Face Transformers library to load the model. The core code is as follows:

   from transformers import AutoModelForCausalLM, AutoTokenizer
   model = AutoModelForCausalLM.from_pretrained(
       "LiquidAI/LFM2.5-24B-A2B",
       torch_dtype="bfloat16"
   )
   tokenizer = AutoTokenizer.from_pretrained("LiquidAI/LFM2.5-24B-A2B")
   

4. Construct Prompts and Set Parameters: Use tokenizer.apply_chat_template() to automatically apply the ChatML format template. When generating responses, it is recommended to use the officially suggested parameters: temperature=0.1, top_k=50, repetition_penalty=1.05. For tasks involving chain-of-thought reasoning, set max_new_tokens to 4096 or higher to reserve sufficient reasoning space.

| Configure Tool Calling: Write the list of tools in JSON format into the system prompt. When the model needs to call a tool, it will output the function call instruction marked with `<|tool_call_start|

6. Integrate with Agent Frameworks: Start a local service using the OpenAI-compatible interface, and then integrate with intelligent agent frameworks such as OpenClaw, Hermes Agent, Pi, or OpenCode. This allows the model to function as the underlying inference engine for these frameworks, driving complete workflows.

4. Pros and Cons Analysis

5. Comparative Analysis with Similar Tools

The following comparison is conducted horizontally among three models with similar positioning: LFM2.5-24B-A2B (Lingma), Qwen3.6-35B-A3B (Ali), and DeepSeek-V2-Lite (DeepSeek), as well as Llama 3.1 8B (Meta).

Selection Recommendations: For users who need to run a local private Agent on a MacBook or consumer-grade GPU, LFM2.5-24B-A2B's low activated parameter count and high inference speed make it the more efficient choice at present, especially showing strong competitiveness in tool calling and multi-step workflow scenarios. Its multi-format deployment design also makes it more usable on Apple Silicon and CPU devices than most similar models.

For applications primarily focused on knowledge question answering and document understanding, Qwen3.6-35B-A3B leads by approximately 12% on knowledge-intensive benchmarks such as GPQA Diamond, and its more mature ecosystem makes it suitable for deployment environments prioritizing knowledge-based tasks. If development resources are limited and complete Chinese documentation support is needed, DeepSeek-V2-Lite and Llama 3.1 8B offer deeper community resources and can be considered as alternative references.

6. Editor's Summary

LFM2.5-24B-A2B demonstrates a high level of technical completeness in its design philosophy for edge-side MoE models. With a total parameter count of 24B, only 2.45B is activated in its sparse architecture, combined with a hybrid layer design featuring dual-gating short convolutions and GQA, achieving a practical balance between inference efficiency and model capability. From the empirical data, the model's edge-side Agent performance outperforms or significantly exceeds that of competitors with 50% more parameters on benchmarks such as LiveClaw and AutomationBench, indicating that its architectural choices are specifically optimized for tool calling and multi-step reasoning scenarios.

In terms of practical value, the model's support for multiple deployment formats (native, FP8, GGUF, ONNX, MLX) covers the full spectrum of hardware, from cloud GPUs to edge devices. It offers direct usability for Apple Silicon users, which to some extent reduces the engineering barriers for private edge-side deployment. The open weights and commercial-use license also provide flexibility for enterprises to fine-tune the model based on private data. It is worth noting, however, that the model still lags behind traditional 35B-level models in knowledge-intensive tasks, and it is recommended to use it in conjunction with retrieval augmentation in high-demand scenarios such as finance and healthcare.

The target audience includes technically proficient developers and enterprise users who prioritize data privacy. Application areas span local Agent development, offline assistant deployment, and business process automation. From the perspective of model architecture and training process planning, the model has the potential to further enhance its knowledge capacity while maintaining low activation costs in future iterations. It is therefore worth keeping an eye on its future version updates and the development of its community ecosystem.

7. Application Scenarios

• On-device Private Agent: Operates locally on personal devices such as MacBook, performing the entire process of planning, tool calling, and task execution throughout the day. The model performs all inference locally, and conversation data never leaves the device, making it suitable for work scenarios with strict data privacy requirements, such as internal enterprise information processing and personal assistant applications.

• Offline/Edge Assistant: Deployed in environments with unstable or no network connection using GGUF, ONNX, and MLX formats, including automotive systems, field operation terminals, and embedded devices. The model can generate text, create summaries, and handle basic Q&A without requiring internet access, providing reliable offline inference capabilities in low-connectivity environments.

• Intelligent Tool Orchestration: Acts as the driving model for frameworks such as OpenClaw, Hermes Agent, and OpenCode, executing multi-step workflows like scheduled report generation, file organization, and code manipulation. The model natively supports a four-step tool calling process, enabling the chaining of multiple APIs or local tools to accomplish complex tasks and reducing the need for manual intervention.

• Business Process Automation: Leverages its advantages on benchmarks such as ToolSandbox and τ³-Bench Banking to handle business process conversations that require system queries and multi-round confirmations, such as banking, order processing, and recruitment. The model connects to business systems through a function calling mechanism, achieving an automated closed-loop for querying, verifying, and confirming actions.

8. FAQ

Q: What hardware requirements does LFM2.5-24B-A2B have?
A: The model supports multiple hardware configurations. NVIDIA GPU users can use the native BF16 format, with a recommended VRAM of no less than 24GB; the FP8 format can reduce VRAM requirements to approximately 16GB. Apple Silicon users are recommended to use the MLX-4bit version, which can run smoothly on M-series chips with 16GB of memory. For pure CPU environments, the GGUF quantized version can be used, but the inference speed will be significantly slower than GPU-based solutions.

Q: Does the model support Chinese? How is the performance?
A: Yes, it supports Chinese and is one of the key optimized languages. The model's vocabulary has been expanded from 65K in the previous generation to 128K, significantly improving tokenization efficiency for non-Latin scripts such as Chinese and Japanese. The number of tokens required for the same Chinese text is notably reduced compared to the previous generation. For Chinese scenarios, it is recommended to set max_new_tokens to a higher value to cover the output space for chain-of-thought reasoning.

Q: How to deploy the model's tool calling capability locally?
| A: Write the list of tools in JSON format into the system prompt. When the model needs to call a tool, it will output function calling instructions marked with `<|tool_call_start|

Q: How to choose between LFM2.5-24B-A2B and Qwen3.6-35B-A3B?
A: Both models have their own strengths. LFM2.5-24B-A2B performs better in Agent inference efficiency and tool calling capabilities (LiveClaw 48.09 vs. 38.19, AutomationBench over 3 times better), and has fewer active parameters and faster on-device deployment. However, Qwen3.6-35B-A3B leads in knowledge-based question answering (GPQA 83.69 vs. 74.42) and web navigation (WebArena 37.62 vs. 26.75). For knowledge-intensive scenarios, choose Qwen3.6-35B-A3B. For tool calling and on-device Agent scenarios, choose LFM2.5-24B-A2B.

Q: Can the model be used for commercial purposes? Are there any restrictions?
A: Both the base and post-training models of LFM2.5-24B-A2B are open-sourced with weights freely available for download, fine-tuning, and commercial use, with no additional licensing restrictions. Enterprises can deploy and fine-tune the model privately based on its weights, making it suitable as the underlying inference engine for commercial products.

Q: Can the model run on a pure CPU environment? What is the speed like?
A: Yes. The model can be run in a pure CPU environment using the GGUF format with llama.cpp or LM Studio. However, the speed depends on the hardware and quantization level. Compared to GPU environments, CPU inference speed will be significantly slower. It is recommended for offline batch processing scenarios where response latency is not critical. For real-time interactive experiences, it is recommended to use Apple Silicon or NVIDIA GPU.

9. Project Links

• HuggingFace Model Repository: https://huggingface.co/LiquidAI/LFM2.5-24B-A2B (model weights and version notes, including full deployment format introduction)