I'm a Research Scientist at Tavus, working on multimodal and generative human-centered AI systems.
I did my PhD in AI at the University of Amsterdam, working with Marcel Worring and Yuki Asano on multimodal foundation models, at the Informatics Institute. I was part of MultiX Amsterdam and AIMLab groups. During my PhD, I also spent time as a Research Scientist Intern in Meta, working on image generation and in-context learning.
I am a member of the ELLIS, co-organized WiML social and ICLR workshops, and served as a reviewer for leading conferences, including CVPR, ICCV, NeurIPS, ICLR, and ICML.
I obtained my master degree in Artificial Intelligence at the KU Leuven. Before that, I spent some time as a Software Engineer in Netcetera and I was an undergraduate student in Computer Science and Engineering at the FCSE at University ”Ss. Cyril and Methodius” in Skopje.
For collaborations and research discussions around multimodal AI, feel free to reach out!
My research focuses on multimodal generative AI, with the goal of building systems that can see, listen, and respond in a natural, human-centered way. During my PhD, I studied how multimodal foundation models learn from context: enabling in-context and few-shot learning, extending the context length of vision-language models, and steering diffusion models. I've also worked on detecting AI-generated images and on multiomodal AI for medical imaging.
We present LATTE - Latent Trajectory Embedding - a novel approach for AI-generated image detection, which models the evolution of latent embeddings across several denoising timesteps.
We present ArtRAG - novel training-free framework that integrates structured knowledge into a RAG pipeline for multi-perspective artwork explanation.
We propose TULIP, a generalizable method able to upgrade the token length to any length for CLIP-like models.
We present Context Diffusion, an in-context-aware image generation framework capable of learning from a variable number of visual context examples and prompts.
We present Self-Context Adaptation (SeCAt), a self-supervised approach that unlocks few-shot open-ended classification with small visual language models.
We propose a method for bridging large-scale vision and language models to perform multimodal few-shot learning. The model meta-learns visual prefixes from frozen visual backbone, which are used as prompts to a large language model.
We introduce a novel method for open-ended VQA suited for small, domain-specific, medical datasets. We employ parameter-efficient strategies for efficient tuning of the LMs.
We define a meta-learning approach for multimodal few-shot learning, to leverage its strong ability of accruing knowledge across tasks (predecessor of the ICLR 2023 work).
We present a probabilistic latent variable model for chest X-Ray report generation. We extend the VTI model by providing a fully Transformer-based definition and explore the trade-off between an LSTM- or Transformer-based decoder for generation of medical text.
We introduce LifeLonger, a benchmark for continual disease classification on the MedMNIST collection, by applying existing state-of-the-art continual learning methods.
We introduce a label-guided Transformer model, and show that it is possible to automatically generate relevant and readable ECG descriptions with a data-driven captioning model.
We propose Variational Topic Inference (VTI), a probabilistic latent variable model for automatic report generation. We introduce a set of topics as latent variables to guide sentence generation by aligning image and language modalities in the latent space.