Gopher DeepMind: A Natural Language Model that Approaches Human-Level in Complex Reasoning and Specialist Fields
What is Gopher
Gopher is a large natural language model developed by DeepMind and released on December 8th, 2021. During training, Gopher utilized 2,800 billion parameters, surpassing its main competitor GPT-3 with 1,750 billion parameters. DeepMind claims that Gopher outperforms existing large natural language models in many tasks, especially in answering questions related to logic reasoning and humanities subjects, with answers approaching human-level accuracy.
Gopher was trained using a modified language model architecture called Retrieval-Enhanced Transformer (RETRO). RETRO uses a retrieval mechanism on the internet-level for pre-training, enabling it to compare produced text to the paragraphs on which RETRO is dependent for generation. This feature is similar to human memory, which is extremely helpful in understanding how large language models can cause downstream harm and how to solve it.
Price: Unknown
Tag: Model
