공대생 도전 일지

2024.11.08 - [인공지능/XAI] - Sparse Autoencoder를 통한 LLM의 Bias 줄이기 - 성에 따른 직업 4조금씩 잡혀가는 모습입니다... import osfrom setproctitle import setproctitlesetproctitle("")os.environ["CUDA_VISIBLE_DEVICES"] = "0"import torchfrom tqdm import tqdmimport plotly.express as pximport pandas as pdimport numpy as np# Imports for displaying vis in Colab / notebooktorch.set_grad_enabled(False)# For the most part I'll try ..

2024.11.07 - [인공지능/XAI] - Sparse Autoencoder를 통한 LLM의 Bias 줄이기 - 성에 따른 직업 3  SAE는 7번 레이어에 붙어있습니다.11레이어까지 있으니까 한 번 쭉 확인해봅시다....JobGender DominanceFemale PercentageCosine Similarity with WomanCosine Similarity with Manskincare specialistFemale98.20.8852370.851516kindergarten teacherFemale96.80.8798330.844357childcare workerFemale94.60.9086590.854653secretaryFemale92.50.8668990.826608hairstylistFem..

2024.11.05 - [인공지능/XAI] - Sparse Autoencoder를 통한 LLM의 Bias 줄이기 - 성에 따른 직업 2 이번에는 SAE의 Feature에서 COS 유사도를 구하는 것이 아니라 그 이후의 레이어에서 확인해 보도록 하겠습니다. JobGender DominanceFemale PercentageCosine Similarity with WomanCosine Similarity with Manskincare specialistFemale98.20.6511860.556518kindergarten teacherFemale96.80.6494910.514379childcare workerFemale94.60.716210.519848secretaryFemale92.50.5833850.5458..

https://arxiv.org/abs/2310.12823 AgentTuning: Enabling Generalized Agent Abilities for LLMsOpen large language models (LLMs) with great performance in various tasks have significantly advanced the development of LLMs. However, they are far inferior to commercial models such as ChatGPT and GPT-4 when acting as agents to tackle complex tasks in tharxiv.orgAgent Instruction dataset을 통해서 사고 과정을 학습하고..

https://arxiv.org/abs/2311.05997 JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language ModelsAchieving human-like planning and control with multimodal observations in an open world is a key milestone for more functional generalist agents. Existing approaches can handle certain long-horizon tasks in an open world. However, they still struggle whenarxiv.orgJARVIS-1은 멀티모달..

Agent-Pro2024.11.27 - [인공지능/논문 리뷰 or 진행] - Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimization - 논문 리뷰 GITM2024.11.26 - [인공지능/논문 리뷰 or 진행] - Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory - 논문 리뷰Agent-Pro와 GITM은 각각 특정 환경에서의 AI 에이전트 학습 및 적응을 목표로 하지만, 접근 방식과 적용 범위에서 차이를 보인다.Agent-Pro..

2024.11.05 - [인공지능/XAI] - Sparse Autoencoder를 통한 LLM의 Bias 줄이기 - 성에 따른 직업 1 이번에는 구글의 잼마 2 27b 모델입니다.https://huggingface.co/google/gemma-2-27b google/gemma-2-27b · Hugging FaceThis repository is publicly accessible, but you have to accept the conditions to access its files and content. To access Gemma on Hugging Face, you’re required to review and agree to Google’s usage license. To do this, plea..

https://arxiv.org/abs/2402.17574 Agent-Pro: Learning to Evolve via Policy-Level Reflection and OptimizationLarge Language Models (LLMs) exhibit robust problem-solving capabilities for diverse tasks. However, most LLM-based agents are designed as specific task solvers with sophisticated prompt engineering, rather than agents capable of learning and evolving throarxiv.org 여태까지는 한 판의 게임을 어떻게 이길까, 목..

https://arxiv.org/abs/2309.07870이 논문은 LLM의 한계를 넘기 위해 다양한 방법을 사용했습니다.Agent는 LLM의 능력을 오로지 쓴다기 보다는 장, 단기 기억을 LLM혹은 BERT류를 통해 저장하고, 상태에 따라 다양한 API를 쓰면서 다중 에이전트를 통해 토론 및 동적인 스케쥴링을 진행하여 LLM의 기능을 강화했습니다.이러한 메모리와 API활용을 통해 LLM의 능력을 더 활용하였고, 새로운 시스템을 활용할 수 있도록 프레임워크도 개방하였습니다. 항목내용논문 제목AGENTS: An Open-source Framework for Autonomous Language Agents주요 목표- LLM의 한계를 보완하여 더 복잡한 작업을 수행할 수 있는 에이전트 시스템 설계- 연구자, ..

DEPS2024.11.27 - [인공지능/논문 리뷰 or 진행] - Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents - 논문 리뷰 Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents - 논https://arxiv.org/abs/2302.01560 Describe, Explain, Plan and Select: Interactive Planning with Large Langua..