Chinchilla (DeepMind): A Challenger To The GPT3 Model Developed By DeepMind

Chinchilla AI is a large natural language model developed by DeepMind. The original version was released in March 2022 and its technology is based on the same principles as other similar models, such as GPT-3, with the difference being in the training parameters and data size. DeepMind claims that for computational efficiency in training, the model size and training data set size should scale equally: as the model size increases by a factor of two, the training data set size should also double. Chinchilla AI tests this assumption by using the same computational budget as Gopher but with 70B parameters and 4 times more data to train a more optimized model, Chinchilla. The results show that Chinchilla uniformly and significantly outperforms Gopher, GPT-3, Jurassic-1, and Megatron-Turing NLG on a wide range of downstream evaluation tasks. Chinchilla has an average accuracy of 67.5% on the MMLU benchmark test, over 7% higher than Gopher.

Interestingly, OpenAI’s ChatGPT based on the GPT-3 model has taken the world by storm since its release, reaching 100 million users in just 2 months. Chinchilla AI currently doesn’t have any standout products, and it’s clear that DeepMind needs to prove its previous boasts to the public.

Price: None
Tag: AI Chatbot