8

Ask HN: Best offline LLM in case OpenAI goes rogue?

 8 months ago
source link: https://news.ycombinator.com/item?id=38778330
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

Ask HN: Best offline LLM in case OpenAI goes rogue? Ask HN: Best offline LLM in case OpenAI goes rogue? 11 points by logicallee 24 minutes ago | hide | past | favorite | 3 comments

I have unique requirements: I'm moderating a subreddit with positive potential. I've moderated forums before, reaching thousands, so I have experience.

I want to set the stage with a few bots ("bot" would be in their name and flair) having positive conversations matching the tone I want to set in the community. I also want to have my bots give feedback to people who have good observations but need to adjust the tone a bit to match the subreddit.

However, I am studying a lot and only have limited amount of time to start the community. I am concerned about the failure case of OpenAI going rogue and starting to disobey its instructions to be positive and solution-oriented. This could have lasting effects on the tone in my community. (For example, positive, solution-oriented people will unsubscribe and stop visiting, people might engage in flame wars and then end up blocking each other, and after that the community would fall apart.)

Since I have seen some pretty spectacular failure cases, I would like to be one step ahead of the game and set up a known-good, airgapped LLM in case OpenAI goes rogue.

Does such an LLM exist, able to hold a basic positive conversation and run on commodity GPU? Which one is the best one? I don't think it needs to engage in higher level reasoning, the ability to string together simple coherent sentences with positive sentiments would be enough as a fall-back, and I can also use it as a filter to make sure OpenAI does not begin to use negative words.

What's the best offline LLM I can run on commodity hardware?


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK