Angelika Romanou
@agromanou.bsky.social did:plc:b6wriy4nqf3aqpromaz4o4cn
PhD candidate at EPFL doing research in #NLProc 👩🏻💻 https://agromanou.github.io/
Bluesky’a katılma tarihi:
- Takipçi
- 532
- Takip edilen
- 316
- Post
- 19
- Sıralamaya girdiği gün
- 0
Özet
Angelika Romanou (@agromanou.bsky.social), 17 Kasım 2024 tarihinde katılan bir Bluesky kullanıcısı. 532 takipçisi var, 316 hesabı takip ediyor ve 19 post paylaştı.
Sıralama geçmişi
Bu hesabın postlarının Debya günlük listesine girdiği en iyi günler.
Bu hesap henüz günlük sıralamaya girmedi.
Son beğendiği postlar
Bluesky ağındaki herkese açık beğeniler.
Keynote talk: Apertus: Democratizing Open and Compliant LLMs for Global Language Environments. Imanol Schlag introduces Apertus, a fully open suite of LLMs with a focus on compliance, transparency, and multilingual representation training across 1000+ languages. 🌍🤖
Postu göster5/ Looking closer, feature sharing has limits: in Hindi & Arabic, overlap stays low even at 341B tokens. This may be due to richer agreement systems (e.g., verbs agreeing w/ subjects & objects) forcing BLOOM to keep language-specific features—or simply data scarcity!
Postu göster4/ In #multilingual models, cross-language feature overlap starts low and rises with training. At 6B tokens in BLOOM, most detectors are language-specific or for punctuation; by 341B tokens shared crosslingual features emerge, capturing syntactic abstractions over token patterns.
Postu göster3/ Which features matter early but fade, and which gain importance later? In Pythia, token-level detectors drop out, while higher-level grammatical features—like plural-noun detectors and nouns formed from verbs (e.g., runner from run)—strengthen by 286B tokens.
Postu göster2/ We align critical checkpoints for a task with sparse crosscoders, measure each feature’s causal role, and introduce RelIE to compare their influence across checkpoints. This lets us trace how internal features shift—and when they matter—in models like Pythia, OLMo, and BLOOM.
Postu göster
Blokladığı hesaplar
Bluesky’da bloklar herkese açık kayıtlardır.
Bu hesap kimseyi bloklamıyor.