You Can Now Run a GPT-3 Level AI Model On Your Laptop, Phone, and Raspberry Pi
source link: https://slashdot.org/story/23/03/14/050225/you-can-now-run-a-gpt-3-level-ai-model-on-your-laptop-phone-and-raspberry-pi
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
You Can Now Run a GPT-3 Level AI Model On Your Laptop, Phone, and Raspberry Pi
Follow Slashdot stories on Twitter
binspamdupenotthebestofftopicslownewsdaystalestupid freshfunnyinsightfulinterestingmaybe offtopicflamebaittrollredundantoverrated insightfulinterestinginformativefunnyunderrated descriptive typodupeerror
Sign up for the Slashdot newsletter! or check out the new Slashdot job board to browse remote jobs or jobs in your area.
You Can Now Run a GPT-3 Level AI Model On Your Laptop, Phone, and Raspberry Pi 17
Posted by BeauHD
on Tuesday March 14, 2023 @09:00AM from the what-will-they-think-of-next dept.Typically, running GPT-3 requires several datacenter-class A100 GPUs (also, the weights for GPT-3 are not public), but LLaMA made waves because it could run on a single beefy consumer GPU. And now, with optimizations that reduce the model size using a technique called quantization, LLaMA can run on an M1 Mac or a lesser Nvidia consumer GPU. After obtaining the LLaMA weights ourselves, we followed [independent AI researcher Simon Willison's] instructions and got the 7B parameter version running on an M1 Macbook Air, and it runs at a reasonable rate of speed. You call it as a script on the command line with a prompt, and LLaMA does its best to complete it in a reasonable way.
There's still the question of how much the quantization affects the quality of the output. In our tests, LLaMA 7B trimmed down to 4-bit quantization was very impressive for running on a MacBook Air -- but still not on par with what you might expect from ChatGPT. It's entirely possible that better prompting techniques might generate better results. Also, optimizations and fine-tunings come quickly when everyone has their hands on the code and the weights -- even though LLaMA is still saddled with some fairly restrictive terms of use. The release of Alpaca today by Stanford proves that fine tuning (additional training with a specific goal in mind) can improve performance, and it's still early days after LLaMA's release. A step-by-step instruction guide for running LLaMA on a Mac can be found here (Warning: it's fairly technical).
Do you have a GitHub project? Now you can sync your releases automatically with SourceForge and take advantage of both platforms.
Do you have a GitHub project? Now you can automatically sync your releases to SourceForge & take advantage of both platforms. The GitHub Import Tool allows you to quickly & easily import your GitHub project repos, releases, issues, & wiki to SourceForge with a few clicks. Then your future releases will be synced to SourceForge automatically. Your project will reach over 35 million more people per month and you’ll get detailed download statistics.
Sync Now
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK