eirrann | he/him
eirrann.art
evil twin (sometimes) | husband/dad 1st, artist 2nd, veteran ∞ | epic beard | have won art awards, will again | brevity ≠ me | eirrann.art
👤 Joined on 22 April 2023
158
Followers
140
Following
1104
Total Posts
📜 Kullanıcı Geçmişi
- 22 Nis 2023, 18:00
🎉 Bluesky'e katıldı - İlk kullanıcı adı:
@eirrann-eth.bsky.social - 22 Nis 2023, 19:18
✏️ Kullanıcı adı değişti:
@eirrann-eth.bsky.social→@eirrann.art - 13 Kas 2023, 01:46
🔄 Profil bilgileri güncellendi
🚫 Blokladığı Kullanıcılar
Bu kullanıcının blokladığı kullanıcı bulunamadı.
❤️ Beğendiği Gönderiler
Q4_K_XL at 27B with 200k is a sweet spot on 24 GB. Keep an eye on recall past 100k though, since long context fits in memory long before the model actually uses all of it well.
Show Post
That 200k+ window is a tempting headline. Shipping functional work on your setup is the part that sticks. I'd be interested in whether details from early in a long session stay usable.
Show PostThe 3090 math surprises people because the KV cache at 180k context can rival the weights themselves. Q4 weights plus a quantized cache is the only way that fits in 24 GB.
Show Post