Phi-3 and ONNX
(Open Neural Network Exchange)
Phi-3 and ONNX (Open Neural Network Exchange) are revolutionary for local AI development. Phi-3 offers powerful capabilities for constructing AI models, while ONNX facilitates seamless model interoperability across various platforms and frameworks. Together, they empower developers to create, deploy, and scale AI solutions.
Phi-3
– Small but powerful language model that outperforms models of the same size in capacity and performance.
– Phi-3 version (mini, small, medium and vision) that can be trained based on different parameters.
– You can deploy the language model to various locations, including local development, internal infrastructure, edge locations, and IoT, even in areas with limited or no internet connectivity. It can be used for language tasks, math and reasoning, coding, and function calling.
– Phi-3 models are available on Azure AI Model Catalog, GitHub, Hugging Face, ONNX Runtime, Ollama, and VSCode AI Toolkit.
– Define standard file format, set of operators, building blocks of machine learning and deep learning models
– Train your model using your favourite framework (TensorFlow, Keras, PyTorch, Chainer). After training, convert your model to ONNX format and then load and run the model using ONNX Runtime. ONNX provides hardware acceleration when you run inference using compatible runtimes and libraries.
– Run/Deploy ONNX models using ONNX runtime to the platform of your choice
– Cross-platform ( run on Windows, Mac, iOS, Android and Web browsers
– Optimizes latency, throughput and memory utilization in CPU, GPU, NPU
– Offer model offline training
Install Phi-3 models locally.
1. Open the terminal and do the following: cd D:\Data\Phi3Models
2. git lfs install (https://git-lfs.com/)
3. git clone https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx (This will take some time to download)