PgVector Knowledge Base

Last modified: November 19, 2024

Introduction

The PgVector Knowledge Base module contains operations to interact with a PostgreSQL database that has the pgvector extension installed. It lets you easily store vectors and perform cosine similarity calculations from your Mendix app. This way, you can leverage knowledge bases to enhance your app functionality by performing operations based on (embedding) vectors and vector similarity. In the context of generative AI, large language models (LLMs), and embeddings, this is a key component in natural language processing (NLP) patterns such as retrieval augmented generation (RAG), recommendation algorithms, and similarity search operations.

Typical Use Cases

This module is particularly powerful for Mendix apps that use large language models in generative AI contexts. The PgVector Knowledge Base module allows these apps to securely use private company data in the app logic. For example, this might be essential when constructing prompts.

When there is a need for a separate private knowledge base outside of the LLM infrastructure, this module provides a low-code way to store discrete pieces of data (commonly refered to as chunks) in the private knowledge base and retrieve relevant information for end-user actions or app processes.

Retrieval Augmented Generation

A common NLP pattern is retrieval augmented generation (RAG), where the goal is to have LLMs construct answers to questions or provide on-demand information about private knowledge base data. In order to make this work, discrete pieces of information from the knowledge base are sent along with user questions to the LLM. The retrieval operations from this module are designed for this step in such use cases.

Also without invoking LLMs directly with the retrieved information, the similarity search logic from the retrieval operation can be leveraged in combination with embedding models to create a semantic search in a Mendix app. This can be used for fuzzy search capabilities, suggestions, or simple recommendation systems.

Features

With the current version, Mendix supports inserting data chunks with their vectors into a knowledge base (population) and selecting those records from that moment onwards (retrieval). Apart from cosine similarity search, which is executed based on the vector only, custom filtering is possible using key-value labeling (metadata) to support an additional traditional search component.

Prerequisites

You should have access to your own (remote) PostgreSQL database server with the pgvector extension installed. For more information, see the Setting up a Vector Database section.

Dependencies

Installation

Follow the instructions in How to Use Marketplace Content to import the PgVector Knowledge Base module into your app.

Configuration

After you install the PgVector Knowledge Base module, you can find it in the App Explorer, in the Marketplace modules section. The connector provides a domain model and several activities that you can use to connect your app to a database and let it act as a knowledge base. To implement an activity, use it in a microflow. To ensure that your app can connect to an external database, you must also configure the Encryption module.

General Configuration

You must perform the following steps to integrate a Mendix app integrate a PgVector knowledge base:

  1. Add the module role PgVectorKnowledgeBase.Administrator to your Administrator user role in the security settings of your app. Optionally, map GenAICommons.User to any user roles that need read access directly on retrieved entities.
  2. Add the DatabaseConfiguration_Overview page (USE_ME > Configuration) to your navigation, or add the Snippet_DatabaseConfigurations to a page that is already part of your navigation.
  3. Set up your database configurations at runtime. For more information, see the Configuring the Database Connection Details section in Setting up a Vector Database.

General Operations

After following the general setup above, you are all set to use the microflows and Java actions in the USE_ME > Operations folder in your logic. Currently, ten operations (microflows and Java actions) are exposed as microflow actions under the PgVector Knowledge Base Operations category in the Toolbox in Mendix Studio Pro. These can be split into three categories, corresponding to the main functionalities: managing data chunks in the knowledge base (for example, (Re)populate), finding relevant data chunks in an existing knowledge base (for example, Retrieve), and deleting chunk data or a whole knowledge base (for exapmle, Delete Knowledge Base). In many occasions, metadata in a MetadataCollection can be provided to enable additional filtering.

Additionally, there is one activity to prepare the connection input, which is a required input parameter for all operations, under USE_ME > Connection, and exposed separately in the Toolbox in Studio Pro.

Create PgVector Knowledge Base Connection

All operations that include knowledge base interaction need the connection details to the knowledge base. Adhering to the GenAI Commons standard, this information is conveyed in a specialization of the GenAI Commons Connection entity (see the Technical Reference section). After instantiating the PgVectorKnowledgeBaseConnection based on custom logic and/or front-end logic, this object can be used for the actual knowledge base operations.

(Re)populate Operations

In order to add data to the knowledge base, you need to have discrete pieces of information and create knowledge base chunks for those. You can use the operations for Chunks and KnowledgeBaseChunks in the GenAI Commons module. After you create the knowledge base chunks and generate embedding vectors for them, the resulting ChunkCollection can be inserted into the knowledge base using an operation for insertion, for example the (Re)populate Knowledge Base operation.

A typical pattern for populating a knowledge base is as follows:

  1. Create a new ChunkCollection. See the Initialize ChunkCollection section.
  2. For each knowledge item that needs to be inserted, do the following:
  3. Call an embeddings endpoint with the ChunkCollection to generate an embedding vector for each KnowledgeBaseChunk
  4. With the ChunkCollection, use (Re)populate Knowledge Base to store the chunks.

(Re)populate Knowledge Base

This operation handles the following:

  • Clearing the knowledge base if it does exist
  • Creating the empty knowledge base if it does not exist
  • Inserting all provided knowledge base chunks with their metadata into the knowledge base

The population handles a whole collection of chunks at once, and this ChunkCollection should be created using the Initialize ChunkCollection and Add KnowledgeBaseChunk to ChunkCollection operations.

Insert

In cases where additional records need to be added to an existing knowledge base, the Insert operation can be used. This operation handles a collection of chunks that need to be inserted into the knowledge base. It behaves similarly to the (Re)populate operation, except that it does not delete any data.

Replace

The Replace operation is intended to be used in scenarios in which the chunks in the knowledge base are related to Mendix objects (in other words, data in the Mendix database). It can be used to keep the knowledge base in sync when data in your Mendix app database changes, which needs to be reflected in the knowledge base. The operation handles a collection of chunks: it will remove the knowledge base data for the Mendix objects the chunks refer to, after which the new data is inserted. For example, this operation can be used before a Mendix object gets committed to keep the knowledgebase in sync with the change.

Retrieve Operations

Currently, four operations are available for on-demand retrieval of data chunks from a knowledge base. All operations work on a single knowledge base (specified by the knowledge base name) on a single database server (specified by the DatabaseConfiguration). The details for this are captured in the PgVectorKnowledgeBaseConnection. Apart from a regular Retrieve, an additional operation was exposed to Retrieve Nearest Neighbors, where the cosine similarity between the input vector and the vectors of the records in the knowledge base is calculated. In both cases it is possible to filter on metadata.

A typical pattern for retrieval from a knowledge base uses GenAI Commons operations and can be illustrated as follows:

  1. Use Initialize MetadataCollection with Metadata to set up a MetadataCollection for filtering with its first key-value pair added immediately.
  2. Use Add Metadata to MetadataCollection (iteratively) to create a collection of the necessary metadata.
  3. Do the retrieval. For example, you could use Retrieve Nearest Neighbors to find chunks based on vector similarity.

For scenarios in which the created chunks were based on Mendix objects at the time of population and these objects need to be used in logic after the retrieval step, two additional operations are available. The Java actions Retrieve & Associate and Retrieve Nearest Neighbors & Associate take care of the chunk retrieval and set the association towards the original object, if applicable.

A typical pattern for this retrieval is as follows:

  1. Use Initialize MetadataCollection with Metadata to set up a MetadataCollection for filtering with its first key-value pair added immediately.
  2. Use Add Metadata to MetadataCollection (iteratively) to create a collection of the necessary metadata.
  3. Do the retrieval. For example, you could use Retrieve Nearest Neighbors & Associate to find chunks based on vector similarity.
  4. For each retrieved chunk, retrieve the original Mendix object and do custom logic.

Retrieve

Use this operation to retrieve knowledge base chunks from the knowledge base. Additional selection and filtering can be done by specifying the optional input parameters for offset and a maximum number of results, as well as a collection of metadata or a Mendix object. If a metadata collection is provided, this operation only returns chunks that conform with all of the metadata in the collection. If a Mendix object is passed, only knowledge base chunks that were related to this Mendix object during insertion will be retrieved.

Retrieve & Associate

Use this operation to retrieve knowledge base chunks from the knowledge base and set associations to the related Mendix objects (if applicable). Additional selection and filtering can be done by specifying the optional input parameters for offset and a maximum number of results, as well as a collection of metadata. If a metadata collection is provided, this operation only returns knowledge base chunks that are conform with all the metadata in the collection.

Retrieve Nearest Neighbors

Use this operation to retrieve knowledge base chunks from the knowledge base where the retrieval and sorting are based on vector similarity with regard to a given input vector. Additional selection and filtering can be done by specifying the optional input parameters: minimum (cosine) similarity (0–1.0), maximum number of results, and a collection of metadata. If a metadata collection is provided, this operation only returns chunks that conform with all of the metadata in the collection.

Retrieve Nearest Neighbors & Associate

Use this operation to retrieve knowledge base chunks from the knowledge base and set associations to the related Mendix objects (if applicable). In this operation the retrieval and sorting are based on vector similarity with regard to a given input vector. Additional selection and filtering can be done by specifying the optional input parameters: minimum (cosine) similarity (0–1.0), maximum number of results, as well as a collection of metadata. If a metadata collection is provided, this operation only returns knowledge base chunks that are conform with all of the metadata in the collection.

Delete Operations

When a whole knowledge base, or part of its data, is no longer needed, this can be handled by using a delete operation. If, however, the knowledge base is still needed, but the data needs to be replaced, see (Re)populate Operations or Replace operations instead. For cases where the chunks in the knowledge base were based on Mendix objects during insertion, chunks can be deleted using the original Mendix object as a starting point in two additional Delete for List operations.

Delete Knowledge Base

Use this operation to delete a complete knowledge base at once. After execution, the knowledge base including its data will no longer exist in the vector database.

Delete for Object

In scenarios where the chunks in the knowledge base are related to Mendix objects (in other words, data in the Mendix database), deletion of Mendix data typically needs to result in the removal of its related knowledge base chunks from the knowledge base. For this, the Delete for Object operation can be used. The Delete for Object operation accepts any kind of Mendix object, and it removes all the knowledge base chunks related to the provided Mendix object at the time of insertion.

Delete for List

This operation is meant to be used in a similar scenario to the one described for the Delete for Object operation, but handles a list of Mendix objects in a single operation. Executing this operation removes all the knowledge base chunks related to the provided Mendix objects at the time of insertion.

Technical Reference

The module includes technical reference documentation for the available entities, enumerations, activities, and other items that you can use in your application. You can view the information about each object in context by using the Documentation pane in Studio Pro.

The Documentation pane displays the documentation for the currently selected element. To view it, perform the following steps:

  1. In the View menu of Studio Pro, select Documentation.

  2. Click the element for which you want to view the documentation.

Showcase Application

For more inspiration and guidance on how to use these operations in your logic and how to combine it with use cases in the context of generative AI, Mendix highly recommends downloading the GenAI Showcase App from the Marketplace. This application contains various examples in the context of generative AI, some of which use the PgVector Knowledge Base module for storing embedding vectors.

Read More