https://arxiv.org/abs/2406.17526 LumberChunker: Long-Form Narrative Document SegmentationModern NLP tasks increasingly rely on dense retrieval methods to access up-to-date and relevant contextual information. We are motivated by the premise that retrieval benefits from segments that can vary in size such that a content's semantic independencearxiv.org 이 논문은 LLM을 통해 청크를 분리하네요그런데 이렇게 되면 리소스가 너무 과하..
