공대생 도전 일지

https://aclanthology.org/2024.acl-long.536/ Dodo: Dynamic Contextual Compression for Decoder-only LMsGuanghui Qin, Corby Rosset, Ethan Chau, Nikhil Rao, Benjamin Van Durme. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.aclanthology.orgacl 2024 long에 붙은 논문입니다. 기존 방법들(sparse attention, 커널 등)은 nlp에서 일관적인 효과가 나지 않거나, 대형 llm에 적용이..

https://arxiv.org/abs/2310.01732 Nugget: Neural Agglomerative Embeddings of TextEmbedding text sequences is a widespread requirement in modern language understanding. Existing approaches focus largely on constant-size representations. This is problematic, as the amount of information contained in text often varies with the length of tarxiv.org고정 길이 임베딩은 문장 길이와 정보량이 달라도 동일한 크기로 압축해야 해서 긴 텍스트에서 정보..

https://arxiv.org/abs/2006.03236 Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language ProcessingWith the success of language pretraining, it is highly desirable to develop more efficient architectures of good scalability that can exploit the abundant unlabeled data at a lower cost. To improve the efficiency, we examine the much-overlooked redundancyarxiv.orgTransformer가..

https://arxiv.org/abs/2511.06411 SofT-GRPO: Surpassing Discrete-Token LLM Reinforcement Learning via Gumbel-Reparameterized Soft-Thinking Policy OptimizationThe soft-thinking paradigm for Large Language Model (LLM) reasoning can outperform the conventional discrete-token Chain-of-Thought (CoT) reasoning in some scenarios, underscoring its research and application value. However, while the discre..

https://openreview.net/forum?id=rkIw2GqYEt Probing to Refine: Reinforcement Distillation of LLM Reasoners via...Distilling robust reasoning capabilities from large language models (LLMs) into smaller, computationally efficient student models remains an unresolved challenge. Despite recent advances, distilled...openreview.net https://aclanthology.org/2024.emnlp-main.811/ LLM Task Interference: An..

https://aclanthology.org/2025.emnlp-main.36/ CODI: Compressing Chain-of-Thought into Continuous Space via Self-DistillationZhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, Yulan He. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.aclanthology.orgemnlp 2025 main에 붙었네요 기존 CoT는 토큰 사용량이 너무 많았고, Coconut는 단계적 치환을 통해 latent로 바꾸는데 stage간 망각 가능성을 제..

https://arxiv.org/abs/2412.06769 Training Large Language Models to Reason in a Continuous Latent SpaceLarge language models (LLMs) are typically constrained to reason in the language space, where they express the reasoning process through a chain-of-thought (CoT) to solve complex problems. However, the language space may not always be optimal for reasoningarxiv.orgCOLM 2025 에 붙었습니다.기존 LLM의 추론은 언..

https://aclanthology.org/2024.emnlp-main.811/ LLM Task Interference: An Initial Study on the Impact of Task-Switch in Conversational HistoryAkash Gupta, Ivaxi Sheth, Vyas Raina, Mark Gales, Mario Fritz. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.aclanthology.org https://arxiv.org/abs/2502.05167 NoLiMa: Long-Context Evaluation Beyond Literal Matc..

https://ieeexplore.ieee.org/document/10681073 KDPII: A New Korean Dialogic Dataset for the Deidentification of Personally Identifiable InformationThe rapid growth of social media in the era of big data and artificial intelligence has raised significant safety concerns related to the communication of sensitive personal information. In modern society, awareness of the importance of preserving priv..

https://arxiv.org/abs/2505.12540 Harnessing the Universal Geometry of EmbeddingsWe introduce the first method for translating text embeddings from one vector space to another without any paired data, encoders, or predefined sets of matches. Our unsupervised approach translates any embedding to and from a universal latent representatioarxiv.org텍스트 임베딩은 검색, 분류, 클러스터링 등 다양한 곳에 쓰이지만 다른 임베딩 모델은 같은 텍스..