LLMR: Knowledge Distillation with a Large Language Model-Induced Reward
arXiv:2409.12500v2 Announce Type: replace-cross Abstract: Large language models have become increasingly popular and demonstrated remarkable performance in various natural language processing (NLP) tasks. However, these models are typically computationally expensive and difficult to be deployed in resource-constrained environments. In this paper, we propose LLMR, a novel knowledge distillation (KD) method based on a reward function induced from large language models. We conducted experiments on…