Since we're doing random seed discourse, my hot take is that we shouldn't set seeds at all. If changing the seed meaningfully changes your results, there's a bigger issue.

Show Post

do you have stuff like softmax in mind?

Show Post

That plus backpropagation/gradient descent, yeah. And beyond the mathematical machinery, there's all the formal computer science (not to mention hardware) underlying LLMs. I'm just really annoyed by the dismissive reductionism that pretends nothing interesting is going on because dot products.

Show Post

Is this because it is a "stochastic parrot"? I have no idea, but to claim that either LLMs can't do math without tool calls or that it's some straightforward application of fancy autocomplete is inaccurate. There's something interesting, novel, and unexpected going on

Show Post

if you will go back to the thread Olivia has lifted the quote from out of context, then you'll see that others have demonstrated this isn't happening by using open source models that can be shown to not be doing this

Show Post