HOME / 커뮤니티 / 공지사항


[YONSEI CS BK21 Seminar] The recent trend in visual-language joint learning_2022.02.17_16:00

관리자 2022.01.25 10:23 조회 93
  • 제목: The recent trend in visual-language joint learning
  • 강사: Caren Han 교수님 (The University of Sydney)
  • 일시: 2022년 2월 17일 (목요일) 오후 4시
  • 링크: 각 연구실 방장을 통해 이메일 공지

Joint modeling of visual and textual representations is a challenging task. It aims to measure the visual-semantic correspondence between an image and a text caption. It is tough mainly because the image lacks semantic context information as in its corresponding text caption, and the text representation is very limited to fully describe the details of an image.

With recent progress in visual and textual representations, Vision-Language Pretraining (VLP) has achieved impressive performance on many multimodal downstream tasks. In this talk, I will present an overview of the major advances achieved in the topic of PLMs for visual linguistic tasks. As the preliminaries, I will present the general task definition and briefly describe the mainstream architectures of PLMs for visual linguistic tasks. As the core content, I will discuss how to adapt existing VL-PLMs to model different input data and satisfy special properties. I will further summarize several important fine-tuning strategies for visual language downstream tasks. Finally, I will present several future directions. I hope the talk would give some insight for researchers who work in any joint modeling or cross-modal learning work.

Dr. Caren Han is a Lecturer (Assistant Professor in U.S. Systems) at the School of Computer Science, the University of Sydney. During her Ph.D., until 2016, she introduced a novel artificial intelligence-based architecture that enables integrating human expertise and machine learning models and applied this to the medical and security domain. This led her to successfully investigate more than $5 million from international and national grants from the Australian Government, the Korean Government, the U.S. air force and Naval research center, and Hyundai Motors Company. She also investigated the proposed hybrid expert systems and machine learning model and received a $3 million grant from the Korean Government (2017).

From 2019, she leads the University of Sydney Natural Language Processing Group (usydnlp - https://usydnlp.info/). She has been working on proposing deep learning-based natural language processing and text mining algorithms to solve multi-modality and multi-disciplinary. In 2020, her models were recognized in various international top-tier conferences, including the International Conference on Neural Information Processing 2020 (Best Paper Award), the 28th International Conference on Computational Linguistics (Best Area Paper Award), the Annual Meeting of the Association for Computational Linguistics (ACL), International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), Association for the Advancement of Artificial Intelligence (AAAI) and the International Conference on World Wide Web (WWW), as well as the International Journal of Medical Informatics (IJMI). Dr. Han is the winner of the Australian Young Achiever Award in 2017, and also the winner of the University of Sydney, Teacher of the Year 2020, Dean’s Outstanding Teaching Award 2018-2021.