공대생 도전 일지

모델을 굽기 위해 데이터를 수집하면서 토큰 수 확인은 필수기에 한번 가지고 왔습니다.from datasets import list_datasets, load_dataset# 데이터셋 불러오기dataset = load_dataset("nvidia/ChatQA-Training-Data","synthetic_convqa")# 데이터셋 분할 정보 확인print(dataset)일단 데이터 불러오기!import pandas as pdimport tiktokendf = pd.DataFrame(dataset["train"])df이제 DataFrame으로 변경하고 데이터 형식 확인하기여기선 다른 이름이 많은데 저는 특정 column만 골라서 사용할 겁니다.import mathdef tokenize_in_batches(d..

https://arxiv.org/abs/2312.03720 Negotiating with LLMS: Prompt Hacks, Skill Gaps, and Reasoning DeficitsLarge language models LLMs like ChatGPT have reached the 100 Mio user barrier in record time and might increasingly enter all areas of our life leading to a diverse set of interactions between those Artificial Intelligence models and humans. While many stuarxiv.org 이 논문은 ChatGPT Turbo 3.5를 사용하..

https://arxiv.org/abs/2210.03629 ReAct: Synergizing Reasoning and Acting in Language ModelsWhile large language models (LLMs) have demonstrated impressive capabilities across tasks in language understanding and interactive decision making, their abilities for reasoning (e.g. chain-of-thought prompting) and acting (e.g. action plan generation) haarxiv.org Agent 관련 논문에서 빠질 수 없는 것이 ReAct기에...한 번 읽어..

https://arxiv.org/abs/2405.05248 LLMs with Personalities in Multi-issue Negotiation GamesPowered by large language models (LLMs), AI agents have become capable of many human tasks. Using the most canonical definitions of the Big Five personality, we measure the ability of LLMs to negotiate within a game-theoretical framework, as well as methodarxiv.org 이 논문은 LLM을 성격적 요소로 나눠서 협상을 진행하고, 그 결과를 보는 논..

https://arxiv.org/abs/2404.01037 ARAGOG: Advanced RAG Output GradingRetrieval-Augmented Generation (RAG) is essential for integrating external knowledge into Large Language Model (LLM) outputs. While the literature on RAG is growing, it primarily focuses on systematic reviews and comparisons of new state-of-the-art (SoTA)arxiv.org 이 논문은 RAG 기술을 체계적으로 비교하며, 검색 정확도와 답변 유사성이라는 명확한 지표를 통해 RAG 시스템의 성..

https://arxiv.org/abs/2403.10081 DRAGIN: Dynamic Retrieval Augmented Generation based on the Information Needs of Large Language ModelsDynamic retrieval augmented generation (RAG) paradigm actively decides when and what to retrieve during the text generation process of Large Language Models (LLMs). There are two key elements of this paradigm: identifying the optimal moment to activate thearxiv.o..

https://arxiv.org/abs/2402.05131 Financial Report Chunking for Effective Retrieval Augmented GenerationChunking information is a key step in Retrieval Augmented Generation (RAG). Current research primarily centers on paragraph-level chunking. This approach treats all texts as equal and neglects the information contained in the structure of documents. We proarxiv.org 일단 제가 찾던 논문 중 하나입니다!Chunking을..

https://arxiv.org/abs/2406.17526 LumberChunker: Long-Form Narrative Document SegmentationModern NLP tasks increasingly rely on dense retrieval methods to access up-to-date and relevant contextual information. We are motivated by the premise that retrieval benefits from segments that can vary in size such that a content's semantic independencearxiv.org 이 논문은 LLM을 통해 청크를 분리하네요그런데 이렇게 되면 리소스가 너무 과하..

https://arxiv.org/abs/2406.07348 DR-RAG: Applying Dynamic Document Relevance to Retrieval-Augmented Generation for Question-AnsweringRetrieval-Augmented Generation (RAG) has recently demonstrated the performance of Large Language Models (LLMs) in the knowledge-intensive tasks such as Question-Answering (QA). RAG expands the query context by incorporating external knowledge bases to enhaarxiv.org..

https://arxiv.org/abs/2402.01737 Assistive Large Language Model Agents for Socially-Aware Negotiation DialoguesWe develop assistive agents based on Large Language Models (LLMs) that aid interlocutors in business negotiations. Specifically, we simulate business negotiations by letting two LLM-based agents engage in role play. A third LLM acts as a remediator agent tarxiv.org 이 논문에는 LLM을 중재자 혹은 판단..