Demonstration venue · read-only. Every page can be browsed; the buttons that would change it are switched off. Create an account to run TaxoReview on your own data. A simulation from public publication records: the people listed were not invited and did not agree to review.
? Tutorial for this page

Yizhang Jin

dblp:361/2599 · DBLP profile ↗
← Back
1ranked-venue papers
1as first author
1since 2021 · last 2024
0009-0008-6892-2964ORCID · reported

Domains — the database's venue-derived domains; a paper can count in several

Graphics, computer vision, multimedia, augmented reality and games · 1 · 1 first-author · 1 since 2021

Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.

Artificial intelligence
1 paper
Vision and language · 67% Language models and text generation · 33%

Topics — the 3 heaviest of 3, each with the papers that count most for it

TopicWeightPapersLastEvidence papers
Natural language and speech › Language models and text generation
instruction following
0.812024
LLaVA-VSD: Large Language-and-Vision Assistant for Visual Spatial Description · ACM Multimedia 2024
Computer vision › Vision and language
vision-language model
0.812024
LLaVA-VSD: Large Language-and-Vision Assistant for Visual Spatial Description · ACM Multimedia 2024
Computer vision › Vision and language › image captioning › description generation
visual spatial description
0.812024
LLaVA-VSD: Large Language-and-Vision Assistant for Visual Spatial Description · ACM Multimedia 2024

Methods — techniques the papers use, named apart from their topics

large language model refinement · 0.8LoRA fine-tuning · 0.8
YearPublicationVenuePosition
2024 LLaVA-VSD: Large Language-and-Vision Assistant for Visual Spatial Description
abstract
Visual Spatial Description (VSD) aims to generate texts that describe the spatial relationships between objects within images. Traditional visual spatial relationship classification (VSRC) methods typically output the spatial relationship between two objects in an image, often neglecting world knowledge and lacking general language capabilities. In this paper, we propose a Large Language-and-Vision Assistant for Visual Spatial Description, named LLaVA-VSD, which is designed for the classification, description, and open-ended description of visual spatial relationships. Specifically, the model first constructs a visual spatial instruction-following dataset using given figure-caption pairs for the three tasks. It then employs LoRA to fine-tune a Large Language and Vision Assistant for VSD, which has 13 billion parameters and supports high-resolution images. Finally, a large language model is used to refine the generated sentences, enhancing their diversity and accuracy. LLaVA-VSD demonstrates excellent multimodal conversational capabilities and can follow open-ended instructions to assist with inquiries about object relationships in images.
Yizhang Jin, Jian Li 0062, Jiangning Zhang, Jianlong Hu, Zhenye Gan, Xin Tan 0002, Yong Liu 0032, Yabiao Wang, Chengjie Wang 0001, Lizhuang Ma
ACM Multimedia1