Sentient’s memory pipeline is a sophisticated system that manages user data, ensuring a personalized, context-aware experience while maintaining full user control over their information. This process is centered around creating, updating, and managing a knowledge graph, which organizes user data in a structured and relational manner. Here’s a breakdown of how Sentient processes and maintains user memories.

Graph Creation

The memory pipeline begins with the creation of a user’s knowledge graph. This graph acts as a central repository of all user-related data, structured in a way that allows Sentient to make intelligent, contextually-aware decisions during interactions.

1. Personality Test Integration:
   • When a new user sets up Sentient, they’re prompted to complete a personality test based on the MBTI (Myers-Briggs Type Indicator) framework.
   • The test results identify the user's personality type from one of the 16 MBTI profiles.
   • A brief description of the identified personality type is added to the knowledge graph. This foundational data helps Sentient tailor its responses and behavior according to the user’s personality traits.
2. Social Media Data Integration:
   • Users can opt to enrich their knowledge graph by importing data from various social media platforms. Importantly, all data stays local to ensure privacy and security.
   • Depending on the platform, data is converted into a structured format suitable for insertion into the graph:
     • LinkedIn: Sentient scrapes the user's LinkedIn profile to extract professional and educational information. Irrelevant data, such as profile pictures and metadata, is discarded. The relevant data is summarized into a concise paragraph that highlights key experiences and qualifications, which is then added to the graph.
     • Reddit: Sentient reviews the user’s activity on Reddit, focusing on the subreddits they engage with most. From this, the system identifies the user's interests, which are categorized and added to the graph.
     • Twitter: The last 10 tweets from the user’s profile are analyzed to understand their current interests and focus areas. These insights are transformed into interest nodes within the graph.
3. Source-Based Data Tagging:
   • Each piece of information added to the graph includes a source tag indicating where the data originated (for example, the current source tags that are being used are LinkedIn, Reddit and Twitter).
   • This tagging allows users to manage their data at a granular level. For example, if a user wants to remove only their LinkedIn data without affecting other stored information, Sentient can easily identify and delete the relevant nodes.

Graph Structure

The user’s knowledge graph is structured with the user as the root node at the center. From this root, several category nodes are attached, each representing a different area of the user's life (e.g., personality traits, professional background, interests).

• Facts and Relationships: Each category node holds specific facts related to that domain. Additionally, Sentient can draw relationships between facts across different categories, allowing for a richer, interconnected understanding of the user.
• Unstructured to Structured Data: Unstructured data from personality tests and social media profiles is converted into a structured format through Neo4j queries, which insert and manage the graph data.

Memory Updates

Sentient is designed to continuously evolve its understanding of the user through dynamic memory updates. This is handled by the memory orchestrator, which governs how new information is integrated into the graph.

1. Triggering Memory Events:
   • Whenever a user mentions a fact about themselves during a conversation (e.g., “I started a new job at XYZ Corp.”), a memory event is triggered.
   • Sentient searches the relevant category in the graph to determine if this fact already exists.
2. Handling Existing Memories:
   • If the fact already exists in the graph, Sentient can:
     • Update: Replace the old memory with the new information.
     • Delete and Replace: If the information is outdated, the old memory can be deleted, and the updated fact is inserted.
   • If the fact does not exist, Sentient will simply create a new memory node and attach it to the appropriate category.
3. CRUD Operations on the Knowledge Graph:
   • Sentient supports full CRUD operations (Create, Read, Update, Delete) on the user’s knowledge graph. These actions ensure that the data remains current, relevant, and accurate.
   • The memory orchestrator determines the appropriate sequence of operations to execute based on the context of the conversation and the nature of the new information.