공대생 도전 일지

https://arxiv.org/abs/2403.10949 SelfIE: Self-Interpretation of Large Language Model EmbeddingsHow do large language models (LLMs) obtain their answers? The ability to explain and control an LLM's reasoning process is key for reliability, transparency, and future model developments. We propose SelfIE (Self-Interpretation of Embeddings), a frameworkarxiv.org 이 논문은 Sparse Autoencoder(SAE)와는 다르게 추가..

https://arxiv.org/abs/2101.03961 Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient SparsityIn deep learning, models typically reuse the same parameters for all inputs. Mixture of Experts (MoE) defies this and instead selects different parameters for each incoming example. The result is a sparsely-activated model -- with outrageous numbers of pararxiv.org  1. 문제 ..