I'm a Research Scientist at Tavus, working on multimodal and generative human-centered AI systems.
I did my PhD in AI at the University of Amsterdam, working with Marcel Worring and Yuki Asano on multimodal foundation models, at the Informatics Institute. I was part of MultiX Amsterdam and AIMLab groups. During my PhD, I also spent time as a Research Scientist Intern in Meta, working on image generation and in-context learning.
I am a member of the ELLIS, co-organized WiML social and ICLR workshops, and served as a reviewer for leading conferences, including CVPR, ICCV, NeurIPS, ICLR, and ICML.
I obtained my master degree in Artificial Intelligence at the KU Leuven. Before that, I spent some time as a Software Engineer in Netcetera and I was an undergraduate student in Computer Science and Engineering at the FCSE at University ”Ss. Cyril and Methodius” in Skopje.
My research centers around multimodal foundation models - with focus on designing efficient approaches for multimodal understanding and generative tasks. I'm interested in better understanding what large-scale models learn, and how to exploit that through in-context learning and prompting. Some of my work also includes automated linguistic interpretation of images, as well as its applications in the medical domain.
We present LATTE - Latent Trajectory Embedding - a novel approach for AI-generated image detection, which models the evolution of latent embeddings across several denoising timesteps.
We present ArtRAG - novel training-free framework that integrates structured knowledge into a RAG pipeline for multi-perspective artwork explanation.
We propose TULIP, a generalizable method able to upgrade the token length to any length for CLIP-like models.
We present Context Diffusion, an in-context-aware image generation framework capable of learning from a variable number of visual context examples and prompts.
We present Self-Context Adaptation (SeCAt), a self-supervised approach that unlocks few-shot open-ended classification with small visual language models.
We propose a method for bridging large-scale vision and language models to perform multimodal few-shot learning. The model meta-learns visual prefixes from frozen visual backbone, which are used as prompts to a large language model.
We introduce a novel method for open-ended VQA suited for small, domain-specific, medical datasets. We employ parameter-efficient strategies for efficient tuning of the LMs.
We define a meta-learning approach for multimodal few-shot learning, to leverage its strong ability of accruing knowledge across tasks (predecessor of the ICLR 2023 work).
We present a probabilistic latent variable model for chest X-Ray report generation. We extend the VTI model by providing a fully Transformer-based definition and explore the trade-off between an LSTM- or Transformer-based decoder for generation of medical text.
We introduce LifeLonger, a benchmark for continual disease classification on the MedMNIST collection, by applying existing state-of-the-art continual learning methods.
We introduce a label-guided Transformer model, and show that it is possible to automatically generate relevant and readable ECG descriptions with a data-driven captioning model.
We propose Variational Topic Inference (VTI), a probabilistic latent variable model for automatic report generation. We introduce a set of topics as latent variables to guide sentence generation by aligning image and language modalities in the latent space.