2023-10-24 Tue 18.23pm [[Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf]] >%% >```annotation-json >{"created":"2023-10-25T02:02:24.867Z","updated":"2023-10-25T02:02:24.867Z","document":{"title":"Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf","link":[{"href":"urn:x-pdf:2de32134837176858d7e63e358ae5891"},{"href":"vault:/z.Attachments/-Papers and books/Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf"}],"documentFingerprint":"2de32134837176858d7e63e358ae5891"},"uri":"vault:/z.Attachments/-Papers and books/Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf","target":[{"source":"vault:/z.Attachments/-Papers and books/Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf","selector":[{"type":"TextPositionSelector","start":12634,"end":12793},{"type":"TextQuoteSelector","exact":"Information in external context isnot immediately visible to the LLM processor, however, it can be brought into main context throughappropriate function calls.","prefix":"ry (i.e. disk storage) in OSes. ","suffix":" In practice, the underlying sto"}]}]} >``` >%% >*%%HIGHLIGHT%% ==Information in external context isnot immediately visible to the LLM processor, however, it can be brought into main context throughappropriate function calls.== * >%%LINK%%[[#^1akzb0e33w4h|show annotation]] >%%COMMENT%% > >%%TAGS%% > ^1akzb0e33w4h >%% >```annotation-json >{"created":"2023-10-25T01:47:39.082Z","updated":"2023-10-25T01:47:39.082Z","document":{"title":"Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf","link":[{"href":"urn:x-pdf:2de32134837176858d7e63e358ae5891"},{"href":"vault:/z.Attachments/-Papers and books/Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf"}],"documentFingerprint":"2de32134837176858d7e63e358ae5891"},"uri":"vault:/z.Attachments/-Papers and books/Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf","target":[{"source":"vault:/z.Attachments/-Papers and books/Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf","selector":[{"type":"TextPositionSelector","start":4772,"end":4929},{"type":"TextQuoteSelector","exact":"In MemGPT, we treat context windows as a constrained memory resource, and design a memoryhiearchy for LLMs analogous to memory tiers used in traditional OSes","prefix":"ely utilize its limited context.","suffix":" (Patterson et al., 1988).Applic"}]}]} >``` >%% >*%%HIGHLIGHT%% ==In MemGPT, we treat context windows as a constrained memory resource, and design a memoryhiearchy for LLMs analogous to memory tiers used in traditional OSes== * >%%LINK%%[[#^0dznpbjd9kui|show annotation]] >%%COMMENT%% > >%%TAGS%% > ^0dznpbjd9kui >%% >```annotation-json >{"created":"2023-10-25T01:48:19.045Z","updated":"2023-10-25T01:48:19.045Z","document":{"title":"Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf","link":[{"href":"urn:x-pdf:2de32134837176858d7e63e358ae5891"},{"href":"vault:/z.Attachments/-Papers and books/Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf"}],"documentFingerprint":"2de32134837176858d7e63e358ae5891"},"uri":"vault:/z.Attachments/-Papers and books/Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf","target":[{"source":"vault:/z.Attachments/-Papers and books/Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf","selector":[{"type":"TextPositionSelector","start":5265,"end":5481},{"type":"TextQuoteSelector","exact":"To provide a similar illusion of longer context length (analogous to virtual memory),we allow the LLM to manage what is placed in its own context (analogous to physical memory) viaan ‘LLM OS’, which we call MemGPT. M","prefix":"y when accessedby applications. ","suffix":"emGPT enables the LLM to retriev"}]}]} >``` >%% >*%%HIGHLIGHT%% ==To provide a similar illusion of longer context length (analogous to virtual memory),we allow the LLM to manage what is placed in its own context (analogous to physical memory) viaan ‘LLM OS’, which we call MemGPT. M== * >%%LINK%%[[#^iextwl1tba|show annotation]] >%%COMMENT%% > >%%TAGS%% > ^iextwl1tba >%% >```annotation-json >{"created":"2023-10-25T01:52:28.934Z","updated":"2023-10-25T01:52:28.934Z","document":{"title":"Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf","link":[{"href":"urn:x-pdf:2de32134837176858d7e63e358ae5891"},{"href":"vault:/z.Attachments/-Papers and books/Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf"}],"documentFingerprint":"2de32134837176858d7e63e358ae5891"},"uri":"vault:/z.Attachments/-Papers and books/Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf","target":[{"source":"vault:/z.Attachments/-Papers and books/Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf","selector":[{"type":"TextPositionSelector","start":6728,"end":6935},{"type":"TextQuoteSelector","exact":"MemGPT’s multi-level mem-ory architecture delineates between two primary memory types: main context (analogous to mainmemory/physical memory/RAM) and external context (analogous to disk memory/disk storage).","prefix":"y to achieve unbounded context. ","suffix":"Main context is the standard fix"}]}]} >``` >%% >*%%HIGHLIGHT%% ==MemGPT’s multi-level mem-ory architecture delineates between two primary memory types: main context (analogous to mainmemory/physical memory/RAM) and external context (analogous to disk memory/disk storage).== * >%%LINK%%[[#^ozjagf9pi5i|show annotation]] >%%COMMENT%% > >%%TAGS%% > ^ozjagf9pi5i >%% >```annotation-json >{"created":"2023-10-25T01:55:35.833Z","updated":"2023-10-25T01:55:35.833Z","document":{"title":"Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf","link":[{"href":"urn:x-pdf:2de32134837176858d7e63e358ae5891"},{"href":"vault:/z.Attachments/-Papers and books/Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf"}],"documentFingerprint":"2de32134837176858d7e63e358ae5891"},"uri":"vault:/z.Attachments/-Papers and books/Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf","target":[{"source":"vault:/z.Attachments/-Papers and books/Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf","selector":[{"type":"TextPositionSelector","start":8300,"end":8567},{"type":"TextQuoteSelector","exact":"n LLM-based conversational agents, a significant portion of maincontext tokens is generally used to hold a ‘system message’ or ‘preprompt’ that dictates the natureof the interaction to the system, while the remainder of the tokens can be used to hold conversationdata","prefix":")as the system’s main context. I","suffix":" (Touvron et al., 2023; SillyTav"}]}]} >``` >%% >*%%HIGHLIGHT%% ==n LLM-based conversational agents, a significant portion of maincontext tokens is generally used to hold a ‘system message’ or ‘preprompt’ that dictates the natureof the interaction to the system, while the remainder of the tokens can be used to hold conversationdata== * >%%LINK%%[[#^a49oyc6xigq|show annotation]] >%%COMMENT%% > >%%TAGS%% > ^a49oyc6xigq >%% >```annotation-json >{"created":"2023-10-25T01:56:37.663Z","updated":"2023-10-25T01:56:37.663Z","document":{"title":"Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf","link":[{"href":"urn:x-pdf:2de32134837176858d7e63e358ae5891"},{"href":"vault:/z.Attachments/-Papers and books/Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf"}],"documentFingerprint":"2de32134837176858d7e63e358ae5891"},"uri":"vault:/z.Attachments/-Papers and books/Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf","target":[{"source":"vault:/z.Attachments/-Papers and books/Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf","selector":[{"type":"TextPositionSelector","start":10036,"end":10251},{"type":"TextQuoteSelector","exact":"Resursive summariza-tion (Wu et al., 2021b) is a simple way to address overflowing context windows, however, recursivesummarization is inherently lossy and eventually leads to large holes in the memory of the system","prefix":"of days (or potentially hours). ","suffix":"(as we demonstrate in Section 3)"}]}]} >``` >%% >*%%HIGHLIGHT%% ==Resursive summariza-tion (Wu et al., 2021b) is a simple way to address overflowing context windows, however, recursivesummarization is inherently lossy and eventually leads to large holes in the memory of the system== * >%%LINK%%[[#^yhfwgbecjr8|show annotation]] >%%COMMENT%% > >%%TAGS%% > ^yhfwgbecjr8 >%% >```annotation-json >{"created":"2023-10-25T01:57:45.892Z","updated":"2023-10-25T01:57:45.892Z","document":{"title":"Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf","link":[{"href":"urn:x-pdf:2de32134837176858d7e63e358ae5891"},{"href":"vault:/z.Attachments/-Papers and books/Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf"}],"documentFingerprint":"2de32134837176858d7e63e358ae5891"},"uri":"vault:/z.Attachments/-Papers and books/Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf","target":[{"source":"vault:/z.Attachments/-Papers and books/Charles Packer 2023 - MemGPT- Towards LLMs as Operating Systems.pdf","selector":[{"type":"TextPositionSelector","start":10429,"end":10898},{"type":"TextQuoteSelector","exact":"In our experiments on multi-session chat and document analysis, we further divide main contextinto three components: system instructions, which hold the base LLM instructions (e.g., informa-tion describing MemGPT functions and control flow to the LLM), conversational context, whichholds a first-in-first-out (FIFO) queue of recent event history (e.g., messages between the agent anduser), and working context, which serves as a working memory scratchpad for the agent.","prefix":"o be used in long-term settings.","suffix":" Systeminstructions are read-onl"}]}]} >``` >%% >*%%HIGHLIGHT%% ==In our experiments on multi-session chat and document analysis, we further divide main contextinto three components: system instructions, which hold the base LLM instructions (e.g., informa-tion describing MemGPT functions and control flow to the LLM), conversational context, whichholds a first-in-first-out (FIFO) queue of recent event history (e.g., messages between the agent anduser), and working context, which serves as a working memory scratchpad for the agent.== * >%%LINK%%[[#^a8y2p0sube5|show annotation]] >%%COMMENT%% > >%%TAGS%% > ^a8y2p0sube5