OpenAI is set to release its fourth-generation language model, GPT-4, which will be the largest and most advanced model yet, with 100 trillion parameters.
This jaw-dropping parameter count is made possible by recent artificial intelligence supercomputing advances, which include the cerebral brain scale chip cluster hardware.
This technology can run a neural network with 120 trillion connections, making GPT-4's massive size possible.
Exploring the Sparsity Advantage of GPT-4 in Generating Outputs
Compared to the average human brain, which has approximately 86 billion neurons, GPT-4 is designed to consider a wider range of options when generating outputs.
This is because, at the core of its design, GPT-4 is both very large and very sparse, which is surprising given the history of OpenAI and its tendency to build dense models like GPT-3.
This sparsity means that many of the neurons in the model will be inactive, reducing the amount of computational power required to run the model.
GPT-4: A Multi-Format, Multi-Language Model for Versatile Output Generation
GPT-4 is also expected to be a multi-language model, capable of accepting a variety of inputs, including text, audio, image, and possibly even video.
This is an exciting development because it would allow GPT-4 to generate outputs in a variety of formats, making it more versatile than previous models.
Reducing Training Costs: OpenAI's Strategies for GPT-4
Another interesting aspect of GPT-4 is its expected training cost, which is around $6 million, significantly lower than its predecessor's $12 million.
This suggests that OpenAI is finding ways to reduce the cost of training large language models, possibly through improved optimization at the software level, reducing computing power required, or partnering with supercomputer company Cerebras.
The Future of AI: GPT-4 and Self-Healing Robots
GPT-4 is set to be released in January or February 2023, and with extremely large language models that have more connections than the human brain, artificial intelligence could be closer than we currently expect.
In other AI news, Northwestern University researchers have created a quadruped robot that can heal itself when damaged.
The X-shaped robot, just over 4 inches in length, moves using compressed air that is pushed through its body, enabling it to undulate and lift its four legs. The robot is coated in a self-healing layer of sensors made from a transparent rubbery material that tracks its motion.
When the sensor detects damage, it reacts immediately, sealing itself to repair the damage. In the future, such devices with self-healing parts could be used in hazardous environments, and sensors could be integrated into wearable devices to detect damage.
Revolutionary Machine Learning System Boosts AI Efficiency and Effectiveness
Finally, researchers at Texas A&M University have developed a new machine learning system that reduces the time and energy required for deep learning model training by 100,000 times.
This breakthrough could make deep learning models more efficient and effective for a range of applications, from addiction treatment to solving real-world tasks that involve physical data centers.
As these developments show, AI technology is advancing at a rapid pace, and the possibilities for its applications are limitless.