A + B: A General Generator-Reader Framework for Optimizing LLMs to Unleash Synergy Potential

TY - GEN

T1 - A + B

T2 - Findings of the 62nd Annual Meeting of the Association for Computational Linguistics, ACL 2024

AU - Tang, Wei

AU - Cao, Yixin

AU - Ying, Jiahao

AU - Wang, Bo

AU - Zhao, Yuyue

AU - Liao, Yong

AU - Zhou, Pengyuan

N1 - Conference code: 62

PY - 2024

Y1 - 2024

N2 - Retrieval-Augmented Generation (RAG) is an effective solution to supplement necessary knowledge to large language models (LLMs). Targeting its bottleneck of retriever performance, “generate-then-read” pipeline is proposed to replace the retrieval stage with generation from the LLM itself. Although promising, this research direction is underexplored and still cannot work in the scenario when source knowledge is given. In this paper, we formalize a general “A + B” framework with varying combinations of foundation models and types for systematic investigation. We explore the efficacy of the base and chat versions of LLMs and found their different functionalities suitable for generator A and reader B, respectively. Their combinations consistently outperform single models, especially in complex scenarios. Furthermore, we extend the application of the “A + B” framework to scenarios involving source documents through continuous learning, enabling the direct integration of external knowledge into LLMs. This approach not only facilitates effective acquisition of new knowledge but also addresses the challenges of safety and helpfulness post-adaptation. The paper underscores the versatility of the “A + B” framework, demonstrating its potential to enhance the practical application of LLMs across various domains.

AB - Retrieval-Augmented Generation (RAG) is an effective solution to supplement necessary knowledge to large language models (LLMs). Targeting its bottleneck of retriever performance, “generate-then-read” pipeline is proposed to replace the retrieval stage with generation from the LLM itself. Although promising, this research direction is underexplored and still cannot work in the scenario when source knowledge is given. In this paper, we formalize a general “A + B” framework with varying combinations of foundation models and types for systematic investigation. We explore the efficacy of the base and chat versions of LLMs and found their different functionalities suitable for generator A and reader B, respectively. Their combinations consistently outperform single models, especially in complex scenarios. Furthermore, we extend the application of the “A + B” framework to scenarios involving source documents through continuous learning, enabling the direct integration of external knowledge into LLMs. This approach not only facilitates effective acquisition of new knowledge but also addresses the challenges of safety and helpfulness post-adaptation. The paper underscores the versatility of the “A + B” framework, demonstrating its potential to enhance the practical application of LLMs across various domains.

UR - http://www.scopus.com/inward/record.url?scp=85205323691&partnerID=8YFLogxK

U2 - 10.18653/v1/2024.findings-acl.219

DO - 10.18653/v1/2024.findings-acl.219

M3 - Article in proceedings

AN - SCOPUS:85205323691

T3 - Proceedings of the Annual Meeting of the Association for Computational Linguistics

SP - 3670

EP - 3685

BT - The 62nd Annual Meeting of the Association for Computational Linguistics

A2 - Ku, Lun-Wei

A2 - Martins, Andre

A2 - Srikumar, Vivek

PB - Association for Computational Linguistics (ACL)

Y2 - 11 August 2024 through 16 August 2024

ER -