공대생 도전 일지

https://arxiv.org/abs/2310.05029 Walking Down the Memory Maze: Beyond Context Limit through Interactive ReadingLarge language models (LLMs) have advanced in large strides due to the effectiveness of the self-attention mechanism that processes and compares all tokens at once. However, this mechanism comes with a fundamental issue -- the predetermined context windowarxiv.org 이 논문은 트리 구조를 통해 짧게 요약해..

https://arxiv.org/abs/2309.08112 Empowering Private Tutoring by Chaining Large Language ModelsArtificial intelligence has been applied in various aspects of online education to facilitate teaching and learning. However, few approaches has been made toward a complete AI-powered tutoring system. In this work, we explore the development of a full-fledarxiv.org 오 LLM이 선생님이 된다!Memory를 활용하여 아는 것, 모르는 ..

https://arxiv.org/abs/2307.07924 ChatDev: Communicative Agents for Software DevelopmentSoftware development is a complex task that necessitates cooperation among multiple members with diverse skills. Numerous studies used deep learning to improve specific phases in a waterfall model, such as design, coding, and testing. However, the deep leaarxiv.org 이 논문도 이전에 보았던 마인크레프트 Agent와 비슷하게 Long term, S..

항목내용논문의 주제LLM의 협상 대화에서의 다면적 능력을 체계적으로 평가.연구 목표- 협상 대화에서 LLM의 이해, 주석, 파트너 모델링, 응답 생성 능력을 평가.- LLM을 활용한 협상 시스템의 가능성과 한계를 탐구.데이터셋CRA, DND, CA(CaSiNo), JI(Job Interview) 등 총 4개 데이터셋 사용.- Multi-Issue Bargaining Task(MIBT) 기반으로 협상 시나리오 설계.평가 방식- 태스크 설계: 35개 태스크로 세분화(이해, 주석, 파트너 모델링, 응답 생성).- 시간 단계: 협상 시작(Start), 진행(During), 종료(End)로 구분.- 객관적(정답 존재) 및 주관적(심리 상태 추론) 평가로 나눔.비교 모델GPT-4, GPT-3.5, Mistral-7..

https://arxiv.org/abs/2402.12354 LoRA+: Efficient Low Rank Adaptation of Large ModelsIn this paper, we show that Low Rank Adaptation (LoRA) as originally introduced in Hu et al. (2021) leads to suboptimal finetuning of models with large width (embedding dimension). This is due to the fact that adapter matrices A and B in LoRA are updated warxiv.org기존 LoRA가 A,B 모두 같은 학습률을 가졌다면 여기서 A,B는 다른 학습률을 가져..

https://arxiv.org/abs/2106.09685 LoRA: Low-Rank Adaptation of Large Language ModelsAn important paradigm of natural language processing consists of large-scale pre-training on general domain data and adaptation to particular tasks or domains. As we pre-train larger models, full fine-tuning, which retrains all model parameters, becomes learxiv.org Efficient parameter tuning 방식인 LoRA!기존 weight는 그대..

https://arxiv.org/abs/2402.13550 Are LLMs Effective Negotiators? Systematic Evaluation of the Multifaceted Capabilities of LLMs in Negotiation DialoguesA successful negotiation requires a range of capabilities, including comprehension of the conversation context, Theory-of-Mind (ToM) skills to infer the partner's motives, strategic reasoning, and effective communication, making it challenging fo..

https://arxiv.org/abs/2305.16291 Voyager: An Open-Ended Embodied Agent with Large Language ModelsWe introduce Voyager, the first LLM-powered embodied lifelong learning agent in Minecraft that continuously explores the world, acquires diverse skills, and makes novel discoveries without human intervention. Voyager consists of three key components: 1) anarxiv.org이 논문도 마인크레프트 환경에서 LLM이 세부적인 목표를 정하고,..

https://arxiv.org/abs/2301.12314 Progressive Prompts: Continual Learning for Language ModelsWe introduce Progressive Prompts - a simple and efficient approach for continual learning in language models. Our method allows forward transfer and resists catastrophic forgetting, without relying on data replay or a large number of task-specific parametearxiv.org 이 논문의 특징에 대해 크게 모르겠네요결국 Soft prompt tuni..

https://arxiv.org/abs/2310.12823 AgentTuning: Enabling Generalized Agent Abilities for LLMsOpen large language models (LLMs) with great performance in various tasks have significantly advanced the development of LLMs. However, they are far inferior to commercial models such as ChatGPT and GPT-4 when acting as agents to tackle complex tasks in tharxiv.orgAgent Instruction dataset을 통해서 사고 과정을 학습하고..