the gist of how this works is: The model knows what it remembers at all times. It knows this because it knows what it remembern’t. By subtracting what it remembers from what it remembern’t, or what it remembern’t from what it remembers (whichever is greater), it obtains a difference, or memory.

Show Post

(i’m diffing how much presence/absence of past context changes the activations and then fucking with attention just enough to preserve that diff without changing anything else about the output)

Show Post

They get the polite ask first, if that doesnt work, invoking Pazuzu will

Show Post

github.com/google-resea...

Show Post

Every woman that I know in tech has similar stories. I do, my partner does, my friends, and so on. Anecdotally, that it's so pervasive is often cited as a main reason for leaving the industry

Show Post