공대생 도전 일지

Latent Reasoning, Soft Thinking 논문 정리 3

https://arxiv.org/abs/2511.06411 SofT-GRPO: Surpassing Discrete-Token LLM Reinforcement Learning via Gumbel-Reparameterized Soft-Thinking Policy OptimizationThe soft-thinking paradigm for Large Language Model (LLM) reasoning can outperform the conventional discrete-token Chain-of-Thought (CoT) reasoning in some scenarios, underscoring its research and application value. However, while the discre..

Multi-turn, Long-context Benchmark 논문 5

https://openreview.net/forum?id=rkIw2GqYEt Probing to Refine: Reinforcement Distillation of LLM Reasoners via...Distilling robust reasoning capabilities from large language models (LLMs) into smaller, computationally efficient student models remains an unresolved challenge. Despite recent advances, distilled...openreview.net https://aclanthology.org/2024.emnlp-main.811/ LLM Task Interference: An..

Latent Reasoning, Soft Thinking 논문 정리 2

https://aclanthology.org/2025.emnlp-main.36/ CODI: Compressing Chain-of-Thought into Continuous Space via Self-DistillationZhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, Yulan He. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.aclanthology.orgemnlp 2025 main에 붙었네요 기존 CoT는 토큰 사용량이 너무 많았고, Coconut는 단계적 치환을 통해 latent로 바꾸는데 stage간 망각 가능성을 제..

Latent Reasoning, Soft Thinking 논문 정리 1

https://arxiv.org/abs/2412.06769 Training Large Language Models to Reason in a Continuous Latent SpaceLarge language models (LLMs) are typically constrained to reason in the language space, where they express the reasoning process through a chain-of-thought (CoT) to solve complex problems. However, the language space may not always be optimal for reasoningarxiv.orgCOLM 2025 에 붙었습니다.기존 LLM의 추론은 언..

https://aclanthology.org/2024.emnlp-main.811/ LLM Task Interference: An Initial Study on the Impact of Task-Switch in Conversational HistoryAkash Gupta, Ivaxi Sheth, Vyas Raina, Mark Gales, Mario Fritz. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.aclanthology.org https://arxiv.org/abs/2502.05167 NoLiMa: Long-Context Evaluation Beyond Literal Matc..

https://ieeexplore.ieee.org/document/10681073 KDPII: A New Korean Dialogic Dataset for the Deidentification of Personally Identifiable InformationThe rapid growth of social media in the era of big data and artificial intelligence has raised significant safety concerns related to the communication of sensitive personal information. In modern society, awareness of the importance of preserving priv..

https://arxiv.org/abs/2505.12540 Harnessing the Universal Geometry of EmbeddingsWe introduce the first method for translating text embeddings from one vector space to another without any paired data, encoders, or predefined sets of matches. Our unsupervised approach translates any embedding to and from a universal latent representatioarxiv.org텍스트 임베딩은 검색, 분류, 클러스터링 등 다양한 곳에 쓰이지만 다른 임베딩 모델은 같은 텍스..

https://arxiv.org/abs/2504.04150 Reasoning on Multiple Needles In A HaystackThe Needle In A Haystack (NIAH) task has been widely used to evaluate the long-context question-answering capabilities of Large Language Models (LLMs). However, its reliance on simple retrieval limits its effectiveness. To address this limitation, recent sarxiv.org https://aclanthology.org/2025.naacl-long.267/ Multiling..

https://kimjaehee0725.github.io/publications/ PublicationsSelected publications and manuscripts in natural language processing and trustworthy AI.kimjaehee0725.github.io리뷰 받는 중 인 것 같은데 여기서 찾았습니다기존 Inference 방법론들은 API환경에서 제한적이거나 성능 감소가 매우 컸음 토큰 수준에서 치환을 통해 fine-tuning을 진행하고 이를 통해 성능 평가를 했을 때 80%의 성능을 유지함을 보여줌 의료, 금융, 교육과 같은 API 응용 환경에서 민감한 데이터를 보호할 수 있음토큰 암호화를 통해 사람이 읽을 수 없는 언어로 학습을 진행한다. API 사용이..

https://arxiv.org/abs/2505.18332 An Attack to Break Permutation-Based Private Third-Party Inference Schemes for LLMsRecent advances in Large Language Models (LLMs) have led to the widespread adoption of third-party inference services, raising critical privacy concerns. Existing methods of performing private third-party inference, such as Secure Multiparty Computation (Sarxiv.orghidden state를 per..