OpenAI Has Privately Launched A New Developer Platform ‘Foundry’
source link: https://www.theinsaneapp.com/2023/02/openai-foundry.html
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
OpenAI Has Privately Launched A New Developer Platform ‘Foundry’
OpenAI quietly launched a new developer platform, ‘foundry,’ that allows customers to run its newer machine-learning models like GPT-3.5 on dedicated resources.
The screenshots shared by early access users describe the Foundry offering as “designed for cutting-edge customers with larger workloads.” The documentation states, “Foundry enables inference at scale with full command over the model configuration, performance profile.”
Users can monitor individual instances with the same dashboards and tools that OpenAI uses for optimizing and building models. Foundry will also provide some degree of version control. Customers can decide whether to upgrade to the latest model releases or not. It will also offer “more robust” fine-tuning options for OpenAI’s most recent models.
Foundry will also offer service-level commitments, such as uptime or on-calendar technical support. Rents will be made on dedicated compute units that have three-month or one-year commitments. Each model instance will need a certain number of compute units, as shown below in the chart.
These instances won’t come cheap. A lightweight version of GPT 3.5 will run at $78,000 monthly or $264,000 for a one-year commitment. This is a reasonable price compared to Nvidia’s DGX Station, which costs around $149,000 per unit.
Twitter users who were keenly observant noticed that the instance pricing chart’s text-generating model has a 32k maximum context window. The context window is the amount of text the model considers before it generates additional text. Longer context windows allow the model “remember” more text. GPT-3.5 is OpenAI’s latest text-generating machine. It has a full context window of 4k.
After a multi-billion dollar investment by Microsoft, OpenAI is pressured to make a profit. According to reports, the company expects to make $200 Million in 2023 and $1 Billion by 2024.
It is largely due to high computing costs. The cost of training state-of-the-art AI models can run into millions, and operating them is generally much more expensive. OpenAI CEO Sam Altman says that ChatGPT, OpenAI’s viral chatbot, costs a few cents per conversation. This is not a small amount when considering that ChatGPT has multi-million daily active users.
OpenAI launched ChatGPT Plus, a pro version of ChatGPT. It costs $20 per month. They also partnered with Microsoft to create Bing Chat, a controversial chatbot that has captured the attention of mainstream media.
According to reports, OpenAI will soon launch a mobile ChatGPT app and also integrate its AI language technology into Microsoft applications like Word, PowerPoint, and Outlook.
Related Stories:
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK