Mastering Retrieval-Augmented Generation: Unlocking Advanced Retrieval Techniques

Overview

In this tutorial, we explore the critical role of Retrieval-Augmented Language Models (RALMs) in enhancing large language model's knowledge retrieval capabilities. Starting with an overview of their importance, we’ll trace the evolution of core retrieval techniques from 2014 to present, covering milestones including word2vec, S-BERT, ICT, DPR, REALM, ColBERT, GAR, HyDE, and LLM Embeddings. Participants will gain a foundational understanding of how each method contributed to retrieval advancements and acquire practical insights into leveraging these techniques to improve model efficiency and precision in real-world applications.

Click here for our slide

Presenters

Members from NLP Lab, NCHU, Taiwan

Yao-Chung Fan

Sin-Syuan Wu

Yen-Hsiang Wang

Che-Wei Chang

Reading List

We encourage participants to go through the following readings before attending the tutorial:

• Latent Retrieval for Weakly Supervised Open Domain Question Answering (Lee et al., 2019)
• REALM: Retrieval-Augmented Language Model Pre-Training (Guu et al., 2020)
• Dense Passage Retrieval for Open-Domain Question Answering (Karpukhin et al., 2020)
• Condenser: a Pre-training Architecture for Dense Retrieval (Gao & Callan, 2021; 2022)
• ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT (Khattab & Zaharia, 2020)
• PAQ 65 Million Probably-Asked Questions and What You Can Do With Them (Lewis et al., 2020)
• Generation-Augmented Retrieval for Open-Domain Question Answering (Mao et al., 2020)
• Precise Zero-Shot Dense Retrieval without Relevance Labels (Gao et al. 2022)
• Query Rewriting for Retrieval-Augmented Large Language Models (Ma et al., 2023)
• Improving Text Embeddings with Large Language Models (Wang et al., 2023)