Surging demand for large language models fuels the meteoric rise of startup Dify...

Surging demand for large language models fuels the meteoric rise of startup Dify

Surging demand for large language models fuels the meteoric rise of startup Dify

Rebbeca Ren

posted on 5 hours agoEditor : Boyuan Wang

For the first time, Chinese developers stand at the same starting point as their counterparts in Silicon Valley.

The launch of ChatGPT 3.5, a large language model (LLM)-powered chatbot developed by OpenAI, has ignited a new wave of interest in artificial intelligence (AI) across China.

Companies, startups, and developers are increasingly considering the integration of LLMs (whether it's OpenAI's GPT or Baidu's ERNIE) into their products, such as marketing tools, text-to-image tools, content creation tools, AI assistants, and more. Therefore, the demand for developing and managing LLMs is on the rise, as not every player has the resources, expertise, or energy to handle the complexities of using such novel technology.

Recently, Dify, a small startup team offering LLMOps services, has seen a massive surge in its user base, reaching an impressive 18,000 in less than a month. This early success has earned them significant traction in the Chinese tech community.

Dify, or "do it for you," is an open-source LLMOps platform designed to simplify AI application development. 

For those who may not be familiar with the term: LLMOps, an abbreviation for "Large Language Model Operations," is a specialized field within the broader domain of MLOps (Machine Learning Operations). It focuses on managing and deploying LLMs, encompassing tasks such as fine-tuning, deployment, maintenance, and more. LLMOps plays a vital role in facilitating the development of efficient AI applications.

"After seeing ChatGPT3.5 go viral, we started to realize that there was a gap in the market where the demand for accessing and using LLMs would grow, but there is hardly any easy-to-use LLMOps platform available," Zhang Luyu, founder of Dify, told PingWest.

Open-source projects like Langchain offer LLM integration and agent-based capabilities but demand a high level of expertise from developers and lack operational features, while traditional MLOps providers like Scale offer expensive and non-generic solutions, the founder explained.

Recognizing the need for a more user-friendly solution in the sector, Zhang and his team embarked on product development in March. Their goal was to create a service that simplifies the process of creating, fine-tuning, and managing LLM applications, allowing users to concentrate on their core objectives without being overwhelmed by technical intricacies.

At present, the team has implemented support for the OpenAI GPT series and is working on integrating additional services such as Azure OpenAI Service, Claude, Hugging Face Hub, and Hugging Face Hub models. Moreover, they have plans to integrate plugins into app orchestration, allowing access to AI applications with plugin capabilities through an API or WebApp. 

Beyond that, Dify offers a wide range of templates based on LLMs, covering data processing, marketing, writing, and beyond, for users to reuse or customize.

Zhang's expertise lies in DevOps, a collaborative approach that combines development and operations to streamline software delivery through automation, continuous integration, and efficient processes. Previously, he served as the product director of Tencent's CODING, a DevOps platform designed to simplify the software development lifecycle and enhance collaboration among developers, project managers, and operations teams. Before joining Tencent, he founded Feie Work, a testing process management and collaboration tool for software development. 

The founder attributes his team's early success primarily to its ability to swiftly respond to market needs rather than relying on marketing or other factors. "We identified new demands ahead of anyone and filled them just in time," he said.

Additionally, Dify's visualization tools lower the threshold for building products powered by LLMs, which helps it appeal to a wider audience: even people without any coding background can build a product by dragging and dropping components. Yes, it's like a low-code/no-code platform in LLMOps.

Without disclosing further details, the startup said it had recently completed an angel investment round. Several investment institutions, including Chinese and overseas, have reached out to the company to explore investment opportunities.

The release of ChatGPT 3.5 represents a transformative moment, unlocking new possibilities for developers around the world. Zhang recognizes this as a significant milestone, as it allows Chinese developers to be on an equal footing with their international counterparts from Silicon Valley and beyond.

He highlighted that while the US excels at making innovative breakthroughs, China has a strong track record of efficiently building products on top of those advancements.

Dify is a testament to his words. In less than three months, this relatively mature set of services has been rolled out, outpacing both its Chinese and US counterparts. It's worth noting that the startup only has 16 or 17 members, with the majority coming from CODING.

Dify's repository on GitHub

With the vision of serving the global market, Dify has taken a strategic step by open-sourcing all of its codes on GitHub and has garnered significant attention with 4,700 stars. Notably, the startup reveals that 30% of the 46,558 lines of code were generated with the assistance of ChatGPT, highlighting the invaluable contribution of the language model to their development process.