On December 1, Alibaba Cloud held a press conference for Tongyi Qianwen, and opened the open source Tongyi Qianwen 72 billion parameter model Qwen-72B. Qwen-72B has achieved the best results among open source models in 10 authoritative benchmarks, becoming the industry's most powerful open source large model. Its performance exceeds the open source benchmark Llama 2-70B and most commercial closed source models. In the future, high-performance applications at the enterprise level and scientific research level will also have the option of open source large models.
Tongyi Qianwen also open sourced the 1.8 billion parameter model Qwen-1.8B and the large audio model Qwen-Audio. So far, Tongyi Qianwen has open sourced four large language models with 1.8 billion, 7 billion, 14 billion, and 72 billion parameters, as well as two large multi-modal models for visual understanding and audio understanding, achieving "full size, full modality" Open source. The intensity is unparalleled in the industry.
is the strongest open source model in the industry, filling the gap in China's LLM open source field.
Qwen-72B is trained based on high-quality data of 3T tokens. It continues the strong performance of Tongyi Qianwen pre-training model and won the best open source model in 10 authoritative benchmark evaluations. Excellent results, surpassing closed source GPT-3.5 and GPT-4 in some evaluations. In the
English task, Qwen-72B achieved the highest score among open source models in the MMLU benchmark test; in the Chinese task, Qwen-72B dominated C-Eval, CMMLU, GaokaoBench and other benchmarks, with scores exceeding GPT-4; in mathematical reasoning, Qwen-72B It leads other open source models in GSM8K and MATH evaluations; in terms of code understanding, Qwen-72B's performance in HumanEval, MBPP and other evaluations has been greatly improved, and its coding capabilities have made a qualitative leap.
In the top 10 authoritative evaluations, Tongyi Qianwen's 72 billion parameter model won the best score among open source models
Tongyi Qianwen's 72 billion open source model's partial results surpassed the closed source GPT-3.5 and GPT-4
Qwen-72B, which can handle up to 32k For long text input, it has achieved results beyond ChatGPT-3.5-16k on the long text understanding test set LEval. The R&D team has optimized Qwen-72B’s command compliance, tool usage and other skills so that it can be better integrated with by downstream applications. For example, Qwen-72B is equipped with a powerful system command (System Prompt) capability. Users can customize the AI assistant with just one prompt word, asking the large model to play a certain role or perform a specific reply task.
Users can create their own AI assistant with just one prompt word
Previously, there was no high-quality open source model in China's large model market that could compete with Llama 2-70B. Qwen-72B fills the domestic gap. With the advantages of high performance, high controllability and high cost performance, it provides a choice that is no less than that of commercial closed-source large models. Based on Qwen-72B, large and medium-sized enterprises can develop commercial applications, and universities and scientific research institutes can carry out scientific research such as AI for Science.
From 1.8B to 72B, Tongyi Qianwen took the lead in realizing full-size open source
If we say Qwen-72B "reaches upwards", it raises the size and performance ceiling of large open source models; another open source model Qwen-1.8 at the press conference B "explores the bottom" and becomes the smallest Chinese open source large model. It only requires 3G of video memory to infer 2K text content and can be deployed on consumer-grade terminals.
has parameter scales ranging from 1.8 billion, 7 billion, 14 billion to 72 billion, making Tongyi Qianwen the first "full-size open source" large model in the industry. Users can directly experience the Qwen series model effects in the Moda community, call the model API through the Alibaba Cloud Lingji platform, or customize large model applications based on the Alibaba Cloud Bailian platform. Alibaba Cloud's artificial intelligence platform PAI has deeply adapted to the entire series of Tongyi Qianwen models and launched services such as lightweight fine-tuning, full-parameter fine-tuning, distributed training, offline reasoning and verification, and online service deployment.
Alibaba Cloud is the first technology company in China to open source large self-developed models. Since August, it has open sourced Qwen-7B, Qwen-14B and the visual understanding model Qwen-VL. Several models have successively appeared on the HuggingFace and Github large model lists, and have been favored by small and medium-sized enterprises and individual developers. The cumulative downloads have exceeded 1.5 million, and more than 150 new models and new applications have been spawned. At the press conference, many developer partners showed up and shared their practices of using Qwen to develop exclusive models and specific applications.
Alibaba Cloud CTO Zhou Jingren said that the open source ecosystem is crucial to promoting the technological progress and application implementation of China's large models. Tongyi Qianwen will continue to invest in open source, hoping to become "the most open large model in the AI era" and promote it together with partners. Large model ecological construction.
Tongyi Qianwen's base model continues to evolve, leading the industry in multi-modal exploration
Tongyi Qianwen's exploration of multi-modal large models is also one step ahead of the industry. On the same day, Alibaba Cloud opened up the large audio understanding model Qwen-Audio for the first time.
Qwen-Audio can perceive and understand various speech signals such as human voices, natural sounds, animal sounds, music sounds, etc. Users can input a piece of audio and ask the model to provide an understanding of the audio, and even perform literary creation, logical reasoning, story continuation, etc. based on the audio. Audio understanding can give large models near-human hearing capabilities.
Tongyi large model can "listen" and "see". Tongyi Qianwen open sourced the large visual understanding model Qwen-VL in August, which quickly became one of the best practices in the international open source community. This conference also announced a major update of Qwen-VL, which has greatly improved the basic capabilities of general OCR, visual reasoning, and Chinese text understanding. It can also handle images of various resolutions and specifications, and can even "look at pictures and answer questions." Regardless of authoritative evaluation results or real-person experience effects, Qwen-VL's Chinese text understanding capabilities greatly surpass GPT-4V.
Tongyi Qianwen closed source model is also continuing to evolve. The Tongyi Qianwen 2.0 closed source model released a month ago has recently been upgraded to version 2.1, with the context window length extended to 32k, code understanding and generation capabilities, and mathematical reasoning. Abilities, Chinese and English encyclopedia knowledge, and hallucination induction resistance are increased by 30%, 10%, nearly 5%, and 14% respectively. Users can experience the latest version of the closed-source model for free on the Tongyi Qianwen APP.
