Kolja Bauer
koljabauer.bsky.social
ELLIS PhD Student in Generative AI @ Ommer Lab (Stable Diffusion)
👤 Joined on 22 November 2024
269
Followers
333
Following
4
Total Posts
📜 Kullanıcı Geçmişi
- 22 Kas 2024, 09:30
🎉 Bluesky'e katıldı - İlk kullanıcı adı:
@koljabauer.bsky.social
🚫 Blokladığı Kullanıcılar
❤️ Beğendiği Gönderiler
This was joint work with @koljabauer.bsky.social, @stefanabaumann.bsky.social, @itsbautistam.bsky.social, @kindsuss.bsky.social, and Björn Ommer. Thanks for the awesome collaboration!
Show PostWe don’t see this specialized model as the end goal. Instead, we hope tasks like this can become useful ingredients for training future multimodal models to reason directly from visual examples — not just language. Paper: arxiv.org/abs/2607.02402 Code: github.com/CompVis/set...
Show PostOur VICIS task naturally extends beyond semantic concepts. For example, the “shared concept” can also describe transformations, allowing the model to learn visual analogies from examples. More broadly: examples can define the representation space.
Show PostSurprisingly, VLMs have a really hard time solving these tasks. Even very capable VLMs often ignore the visual context and instead latch onto the most salient object in the query: The same query paired with a different context should lead to a different interpretation, but often does not.
Show PostWe train a model specifically for this task. Given • a set of example images • a query image it infers the shared concept, predicts a concept-specific embedding space, and projects the query into that space. This retains only the relevant signal from the query.
Show Post