Model Inference

How to Use Xinference for Custom Inference of Phi-4

by
How to Use Xinference for Custom Inference of Phi-4

Click 👇🏻 to follow, article from “ A couple of days ago, Microsoft open-sourced the 14B dense model phi-4, which is called the strongest 14B model available. How strong is the model? What others say doesn’t count; only trying it yourself will tell. However, when you want to get hands-on, you find that most current … Read more

Using CPU for Inference of Llama Structure Large Models

by
Using CPU for Inference of Llama Structure Large Models

1. Review of Llama Model Basics The Llama model is built on the Transformer architecture, featuring multiple layers of attention mechanisms that enable deep semantic analysis and feature extraction of input text. This allows it to excel in natural language processing tasks such as text continuation, summarization, and machine translation. Its design philosophy aims to … Read more