I am aware of how to host models, mainly for shitty discord bots with abliterated models to joke around with with friends. But, my 6700XT cannot get reasonable TPS out of a lobotomised version Qwen. I can't exactly afford a 5090, and Moore's law hasn't really held true for a while now.

Show Post

I think the issue we're gonna have for pretty much ever is the compute required. Open weight doesn't mean accessible, if you want something Opus 5.5 level you're going to need to: 1. Wait for the chinese models to catch up 2. Invest INSANE amounts of money for hardware capable of running it.

Show Post

I feel the same, we'll not be able to get rid of AI unfortunately so if at least I can avoid feeding demons in the process… But the newer models have increasingly high requirements I can't afford (at a good speed), and that's ignoring the services like a search engine that I will have to pay… :(

Show Post

Optical 🅱️edia 🅱️ad

Show Post

cynically: they don't want us to understand anything about actual LLMs, they want us to use THEIR "AI" "product" with all its guardrails and sycophancy and safety harnesses and worship it like a creator-god-in-a-datacenter (and continue to give them money for the privilege of begging it for slop)

Show Post