Microsoft released the family of Phi-3.5 artificial intelligence (AI) models on Tuesday, as the successor of the Phi-3 models introduced in April. The new release comprises Phi-3.5 Mixture of Experts (MoE), Phi-3.5 Vision, and Phi-3.5 Mini models. These are instruct models, so they will not work as a typical conversational AI but will require users to add specific instructions to get the desired output. The open-source AI models are available to download from the tech giant’s Hugging Face listings.
The release of the new AI models was announced by Microsoft executive Weizhu Chen in a post on X (formerly known as Twitter). The Phi-3.5 models offer upgraded capabilities over the predecessor, but the architecture, dataset and training methods largely remain the same. The Mini model has been updated with multilingual support, and the MoE and Vision models are new inclusions in the AI model family.
Coming to technicalities, the Phi-3.5 Mini has 3.8 billion parameters. It uses the same tokeniser (a tool that breaks down text into smaller units) and a dense decoder-only transformer. The model only supports text as input and supports a context window of 1,28,000 tokens. The company claims it was trained using 3.4 trillion tokens between June and August, and its knowledge cut-off is October 2023.
Leave a Reply