The announcement of Dolly, a ChatGPT clone, marked another stride forward in the realm of Open Source AI.
Databricks, the enterprise software company, has developed a new Large Language Model (LLM) called Dolly, which is a ChatGPT clone. This marks another advancement in the Open Source GPT Chat technology. The new clone is named after the well-known sheep that was the first mammal to be cloned.
The Open Source AI movement continues to expand, and the release of Dolly LLM is its latest development. This movement aims to democratize access to the technology, preventing its monopolization and control by large corporations.
The Open Source AI movement is fueled by concerns that businesses may be hesitant to entrust sensitive data to a third party that controls the AI technology.
Dolly refers to the democratization of the ChatGPT magic through open models
Today we are introducing Dolly, a cheap-to-build LLM that exhibits a surprising degree of the instruction following capabilities exhibited by ChatGPT. Whereas the work from the Alpaca team showed that state-of-the-art models could be coaxed into high quality instruction-following behavior, we find that even years-old open source models with much earlier architectures exhibit striking behaviors when fine tuned on a small corpus of instruction training data. Source
Dolly, the latest ChatGPT clone, was developed using an open source model created by the non-profit research institute EleutherAI, as well as the Stanford University Alpaca model. These models were in turn created from Meta’s open-source LLaMA model, which consists of 65 billion parameters and was trained on publicly available data.
According to a Weights & Biases article, despite being smaller in size, LLaMA can outperform many of the top language models, including OpenAI GPT-3, Gopher by Deep Mind, and Chinchilla by DeepMind.
An academic research paper titled “SELF-INSTRUCT: Aligning Language Model with Self Generated Instructions PDF” inspired the development of a superior autogenerated question and answer training dataset. This approach was deemed superior to the existing limited public data.
Databricks has made the first of several announcements aimed at empowering organizations to utilize the capabilities of large language models.
As they state: “We strongly believe in the transformative potential of artificial intelligence to enhance the productivity of both individuals and organizations. We invite you to join us on this journey and stay tuned for more updates in the coming weeks.”
Just a reminder, Google’s Bard AI was also recently released, coinciding with the launch of Microsoft Copilot’s Generative AI features.
— Advertisement on InTech.am —
