KnowledgeStack

A RAG agent with Text-to-SQL capabilities that can answer questions within a workspace (upload docs or authenticate connectors to platforms like Notion and Confluence) and execute queries safely against connected databases.

1What does KnowledgeStack do?

Motivation: New hires onboarding within a company often struggle with finding the proper documentation that they need to understand projects, software system design, and navigate databases to find the relevant data they need.

Project Solution: LLMs have strong text generation capabilities. Combining LLM models with retrieval-augmented generation (RAG) allows for company-specific responses grounded in documentation. Additional Text-to-SQL capabilities allow the agent to 1) leverage documentation as context to navigate databases and 2) provide fast ad-hoc data analysis to end users.

Project Features:

• Google OAuth for users to log in with their email and get started by creating/joining a workspace.
• Workspaces for users to either manually upload documents or connect documentation platforms such as Notion and Confluence. PostgreSQL databases can also be connected in workspaces for agent readonly access. (Other database engines may be supported in the future.)
• Document syncing pipeline to store chunks in a vector database, searchable on user queries.
• A chatbot with an agent loop to call document/database tools and provide grounded responses to users with clickable citations.

2Technology Stack & Architecture

Application Development:

• Next.js and React for the web application.
• NestJS and Node for the backend API server.
• PostgreSQL and pgvector for the application database (storing users, workspaces) and vector database (document retrieval, query relevance).
• Prisma for the object-relational mapper on top of the application database.
• OpenAI API for the LLM chatbot models and chunk embedding.
• Google OAuth for application login, Notion & Confluence OAuth to connect external document sources.

Developer Tools:

• pnpm for package and workspace management.
• Docker Compose to run the application database, demo tenant database, and frontend/backend of web application.

System Context diagram

• Person
• Software System
• Outside the scope

KnowledgeStackSoftware System4 containers

Container diagram

• Person
• Container
• Outside the scope

WebContainer5 components

Component diagram

• Person
• Component
• Container

proxyComponent1 symbol

Symbol	What it does
proxyapps/web/proxy.ts:9	export default function proxy(request: NextRequest): NextResponseSends a browser that carries no ks_session cookie to /signin. A cookie that names an expired session still reaches a page, and that page reads 401 from /api/auth/session and redirects itself.

Sign-in PageComponent1 symbol

Symbol	What it does
SignInPageapps/web/features/signin/signin-page.tsx	export function SignInPage(): ReactNodeLinks to /api/auth/google, which the API answers with a redirect to Google. It renders with no session. The API never renders an error page of its own, so every grant failure arrives as ?error= in the location.

Page FeaturesComponent11 symbols

Symbol	What it does
AskPageapps/web/features/ask/ask-page.tsx	export function AskPage({ chatId }: { chatId: string | null }): ReactNodeOpens a chat, posts a question, and reads the Server-Sent Events stream frame by frame. A chatId of null opens an empty chat and creates the row on the first question. It posts to /chat/:chatId/turns and reads Server-Sent Events frame by frame. A stream that ends with no done frame is a failure, and the page says so rather than showing a partial answer as complete.
Composerapps/web/features/ask/composer.tsx	export function Composer({ question, onQuestionChange, onSubmit, isRunning, isLoadingChat, modelIds, modelId, onModelChange, usage }): ReactNodeThe question box and the model picker. The parent owns the question text and the model id. isRunning disables the submit control, so one chat never has two turns in flight. modelIds comes from GET /chat/models, never from a constant in the browser.
TurnListapps/web/features/ask/turn-list.tsx	export function TurnList({ turns, activeTurnIndex, activeCitationIndex, onCitationSelect }): ReactNodeDraws each tool step as a bullet while the model still writes. The parent owns the selection. A click on a citation calls onCitationSelect with the turn index and the citation index. The component holds no state of its own, so a re-render never loses the open citation.
CitationRailapps/web/features/ask/citation-rail.tsx	export function CitationRail({ turn, activeTurnIndex, activeCitationIndex, onSelect }): ReactNodeShows each cited chunk, its heading path and its cosine similarity. A click on [n] opens it. A turn of undefined renders an empty rail. The citation index is the number the model wrote in brackets, so the rail and the answer can never disagree.
QueryResultTableapps/web/features/ask/query-result-table.tsx	export function QueryResultTable({ result }: { result: QueryResult }): ReactNodeDraws the rows a SELECT returned, under the tool steps. Draws the rows one SELECT returned, under the tool steps. It formats and colours the SQL behind a disclosure. Every value arrives as a string, because DatabaseConnectionPool stringifies each column before it leaves the API.
ContextRingapps/web/features/ask/context-ring.tsx	export function ContextRing({ usage }: { usage: ContextUsage }): ReactNodeShows the context size the API reported after a compaction. Draws the fraction the API reported. The ring turns to its high state at fraction >= 0.8. The chat compacts at whichever limit it reaches first, the turn count or the token count, so the ring shows the larger of the two fractions.
SourcesPageapps/web/features/sources/sources-page.tsx	export function SourcesPage(): ReactNodeLists the sources and their documents, and runs a sync pass on demand. Lists the sources of the active workspace and the documents under each one. POST /sources/:sourceId/sync answers as soon as the pass starts, so the page polls the document list rather than waiting on the request.
UploadDropZoneapps/web/features/sources/upload-drop-zone.tsx	export function UploadDropZone({ fileExtensions, isBusy, busyMessage, onFiles }): ReactNodeTakes a dropped file and posts its text to /sources/uploads. Reads each dropped file as text in the browser and hands the text to onFiles. fileExtensions of null accepts every extension the API accepts. The API stores the text and never the original bytes, so a binary file that reaches this zone stores unreadable text.
ConnectorListapps/web/features/sources/connector-list.tsx	export function ConnectorList({ providers, sources, isBusy, onConnect }): ReactNodeStarts the OAuth grant for Notion or Confluence. Shows one row per provider with the sources already connected to it. A provider whose environment variables are missing arrives with that fact from GET /sources/connectors, and the row names the missing variables instead of starting a grant that would fail.
DatabasesPageapps/web/features/databases/databases-page.tsx	export function DatabasesPage(): ReactNodeRegisters a business database, tests a stored one, and deletes one. The password reaches POST /database-connections once and never returns in a read, so the page cannot show a stored password and never tries.
WorkspacesPageapps/web/features/workspaces/workspaces-page.tsx	export function WorkspacesPage(): ReactNodeCreates a workspace, joins one by code, and selects the active one. A select writes activeWorkspaceId on the session row, so every other open page must reload. It publishes workspaceChangedEvent for that reason.

api-clientComponent3 symbols

Symbol	What it does
sendRequestapps/web/lib/api-client.ts:25	export async function sendRequest(path: string, init?: RequestInit): Promise<Response> export async function requestJson<T>(path: string, init?: RequestInit): Promise<T> export async function sendJson(path: string, method: "POST" | "DELETE", body?: unknown): Promise<Response>One fetch wrapper. It sends the same origin, so the browser attaches ks_session by itself. A 401 throws RedirectError, and any other failure throws ApiError carrying the status, so a caller never reads response.ok. requestJson parses the body, and sendJson returns the response unread because most write routes answer { status: "ok" }.
usePageRequestapps/web/lib/use-page-request.ts	export function usePageRequest(): { notice: Notice | null; setNotice: (notice: Notice | null) => void; busyMessage: string; isBusy: boolean; reportFailure: (error: unknown, fallback: string) => void; runRequest: (busyText: string, run: () => Promise<unknown>, successText?: string) => Promise<boolean> }Holds the loading and error state of one page request. Holds the loading state and the notice of one page. runRequest returns true when run resolved and false when it threw, so a caller reloads its list only on true. reportFailure sends the browser to /signin on a RedirectError and shows fallback on anything else.
workspaceChangedEventapps/web/lib/workspace-changed-event.ts	export const workspaceChangedEvent = "knowledgestack:workspace-changed"Tells every open page that the active workspace changed. The name of the window event that says the active workspace changed. Every page that lists workspace data listens for it and reloads. A page that reads workspace data and does not listen shows the data of the previous workspace.

API Forward RouteComponent1 symbol

Symbol

What it does

forwardRequestapps/web/app/api/[...path]/route.ts:4

async function forwardRequest(request: Request, context: { params: Promise<{ path: string[] }> }): Promise<Response>Rebuilds the request against API_INTERNAL_URL and passes the cookie through unread. dynamic = "force-dynamic", so Next.js never caches an answer. The API resolves the signed-in person from the forwarded cookie header, so this route never reads it. A missing API_INTERNAL_URL throws at request time, never at build time.

APIContainer8 components

Component diagram

• Container
• Component

AuthModuleComponent9 symbols

Symbol	What it does
AuthControllerapps/api/src/auth/auth.controller.ts:46	startSignIn(@Res() response: CookieResponse): void completeSignIn(@Query() query: unknown, @Res() response: CookieResponse): Promise<void> findAccount(@CurrentSession() session: RequestSession): Promise<FindAccountResponse> deleteSession(@CurrentSession() session: RequestSession, @Res() response: CookieResponse): Promise<void>Five routes: /auth/google, the callback, /auth/session and /auth/signout. The callback sets the ks_session cookie. startSignIn and completeSignIn carry @Public(), so they run with no session. The cookie is httpOnly and sameSite: lax. A refused grant or a state value that does not match redirects to /signin?error=, never to a JSON error.
SessionGuardapps/api/src/auth/session.guard.ts:28	async canActivate(context: ExecutionContext): Promise<boolean>Registered under APP_GUARD, so a route of any module that carries no @Public() never runs for a signed-out browser. It resolves the session from the cookie and writes it on the request. A closed route answers 401. Downstream code therefore never checks whether a session exists.
SessionServiceapps/api/src/auth/session.service.ts:38	createSession(userId: string, activeWorkspaceId: string | null): Promise<{ token: string }> findSession(token: string): Promise<RequestSession | null> selectWorkspace(sessionId: string, workspaceId: string): Promise<void> deleteSession(sessionId: string): Promise<void>Creates and reads a session. The row holds the SHA-256 hash of the cookie, so a database dump signs nobody in. findSession returns null for an unknown or expired token and never throws. createSession returns the only copy of the token that ever exists, so a lost token cannot be recovered.
TokenServiceapps/api/src/auth/token.service.ts:39	getAccessToken(sourceId: string): Promise<string> refreshAccessToken(sourceId: string): Promise<string>Reads the OAuth token of a source and refreshes it when the provider expires it. It returns plaintext, so a caller must never log the value. A refresh the provider refuses marks the credential as needing re-authorization and throws ReauthRequiredError, which the caller turns into a message for the person rather than a 500.
OAuthRegistryapps/api/src/auth/oauth.registry.ts:37	findMissingVariables(provider: OAuthProviderName): string[] createClient(provider: OAuthProviderName): OAuthClientMaps a provider name to its client, so one callback route serves Google, Notion and Confluence. findMissingVariables returns the environment variables that are absent, and an empty array when the provider is ready. createClient throws when a variable is missing, so a caller asks first.
GoogleOAuthClientapps/api/src/auth/google.oauth.ts:34	exchangeCode(code: string): Promise<{ externalId: string; email: string; name: string }>Exchanges the Google code for the email and the name of the person. externalId is the stable Google subject, which is the value User.externalId keys on. An email that changes at Google therefore updates the row rather than creating a second one.
NotionOAuthClientapps/api/src/auth/notion.oauth.ts:26	exchangeCode(code: string): Promise<OAuthGrant> refreshToken(refreshToken: string): Promise<OAuthGrant>Exchanges the Notion code, and refreshes the token. One Notion grant covers one Notion workspace, so the grant produces exactly one source. The grant carries the workspace id, which is the value the sources_notion_identity_key index keys on.
ConfluenceOAuthClientapps/api/src/auth/confluence.oauth.ts:60	exchangeCode(code: string): Promise<OAuthGrant> listSpaces(accessToken: string, cloudId: string): Promise<Space[]>Exchanges the Confluence code, and lists the spaces the grant covers. One Confluence grant covers several spaces, so the grant produces one source per space. listSpaces returns the spaces the grant covers, and the caller writes one Source row for each.
EncryptionServiceapps/api/src/encryption/encryption.service.ts:13	encrypt(plaintext: string): string decrypt(encrypted: string): stringFrom EncryptionModule, which this module imports. AES-256-GCM over every token before it reaches Postgres. AES-256-GCM over every OAuth token and every database password before it reaches Postgres. The key is TOKEN_ENCRYPTION_KEY, 32 bytes base64url, validated at boot. The returned string packs the initialization vector, the authentication tag and the ciphertext. decrypt throws when the tag does not verify, so a tampered row fails loudly and never returns wrong plaintext.

WorkspacesModuleComponent2 symbols

Symbol	What it does
WorkspacesControllerapps/api/src/workspaces/workspaces.controller.ts:27	listWorkspaces(@CurrentSession() session): Promise<ListWorkspacesResponse> createWorkspace(@Body() body: unknown, @CurrentSession() session): Promise<CreateWorkspaceResponse> joinWorkspace(@Body() body: unknown, @CurrentSession() session): Promise<JoinWorkspaceResponse> leaveWorkspace(@Param("workspaceId") workspaceId: string, @CurrentSession() session): Promise<StatusResponse> selectWorkspace(@Param("workspaceId") workspaceId: string, @CurrentSession() session): Promise<StatusResponse>Four routes: list, create, join by code, select one. Reads each body as unknown, parses it with the request schema of that route, and answers 400 with the first issue. The service therefore receives typed values and re-parses nothing.
WorkspacesServiceapps/api/src/workspaces/workspaces.service.ts:12	listWorkspaces(userId: string): Promise<Workspace[]> createWorkspace(userId: string, name: string) joinWorkspace(userId: string, code: string) leaveWorkspace(userId: string, workspaceId: string): Promise<void> isMember(userId: string, workspaceId: string): Promise<boolean>Creates a workspace with an eight-character join code, and writes the membership row that a join needs. createWorkspace writes the workspace, its eight-character join code and the membership row of the creator in one transaction, so a workspace never exists with no member. joinWorkspace on a code that matches nothing throws a 404, and a second join by the same person is a no-op rather than a duplicate row.

SourcesModuleComponent3 symbols

Symbol	What it does
SourcesControllerapps/api/src/sources/sources.controller.ts:35	listSources(@ActiveWorkspaceId() workspaceId: string): Promise<ListSourcesResponse> saveUploads(@Body() body: unknown, @ActiveWorkspaceId() workspaceId: string): Promise<SaveUploadsResponse> syncSource(@Param("sourceId") sourceId: string, @ActiveWorkspaceId() workspaceId: string): Promise<StatusResponse> deleteSource(@Param("sourceId") sourceId: string, @ActiveWorkspaceId() workspaceId: string): Promise<StatusResponse> startAuthorization(@Param("provider") provider: string, @ActiveWorkspaceId() workspaceId: string): StartAuthorizationResponse completeAuthorization(@Query() query: unknown)Nine routes. GET /sources/oauth/callback serves every provider, because the state value carries the provider name. @ActiveWorkspaceId() answers 403 when the person opened no workspace, so workspaceId is always a workspace the person belongs to. syncSource answers as soon as the pass starts and never waits for it.
SourcesServiceapps/api/src/sources/sources.service.ts:34	saveUploads(workspaceId: string, files: readonly UploadedFile[]): Promise<SaveUploadsResponse> syncSource(workspaceId: string, sourceId: string): Promise<void> deleteSource(workspaceId: string, sourceId: string): Promise<void> startAuthorization(workspaceId: string, provider: SourceProvider): string completeAuthorization(code: string, state: string): Promise<void>Stores an uploaded file as a row, creates one source per Confluence space, and deletes a source with the files uploaded to it. It rejects a file name that is not letters, digits, spaces, dots, hyphens or underscores, and rejects an extension outside the accepted list, because the name came from a person. deleteSource removes the rows before the source, so a failed delete leaves rows the next pass sweeps rather than files the next upload imports again.
SourceRepositoryapps/api/src/sources/source.repository.ts:12	listSources(workspaceId: string) saveUploadSource(workspaceId: string): Promise<{ id: string }> saveCredential(workspaceId: string, provider: SourceProvider, credential): Promise<{ id: string }> saveOAuthSources(workspaceId: string, provider: SourceProvider, credentialId: string, resources): Promise<void> deleteSource(workspaceId: string, sourceId: string): Promise<number>Every read and write of sources and source_credentials. saveOAuthSources writes the credential and every source of that grant in one transaction, so a source never points at a credential that was never stored. Each provider has its own partial unique index, so a second grant updates the same row instead of adding one.

ChatModuleComponent5 symbols

Symbol	What it does
ChatControllerapps/api/src/chat/chat.controller.ts:38	listModels(): ListModelsResponse listChats(@ActiveWorkspaceId() workspaceId: string, @CurrentSession() session: RequestSession): Promise<ListChatsResponse> createChat(@ActiveWorkspaceId() workspaceId: string, @CurrentSession() session: RequestSession): Promise<CreateChatResponse> readChat(@Param("chatId") chatId: string, @ActiveWorkspaceId() workspaceId: string, @CurrentSession() session: RequestSession): Promise<ReadChatResponse> deleteChat(@Param("chatId") chatId: string, @ActiveWorkspaceId() workspaceId: string, @CurrentSession() session: RequestSession): Promise<StatusResponse> createTurn(@Param("chatId") chatId: string, @Body() body: unknown, @ActiveWorkspaceId() workspaceId: string, @CurrentSession() session: RequestSession, @Res() response: StreamingResponse): Promise<void>Six routes under /chat. POST /chat/:chatId/turns sends the stream headers before the first search, so a stream that ends without a done frame is a failure. createTurn writes one Server-Sent Event per frame and closes the response itself, so it returns void and Nest never serializes a body. A connection the browser closes ends the run. Every other route answers a plain JSON body.
ChatServiceapps/api/src/chat/chat.service.ts:31	listChats(workspaceId: string, userId: string): Promise<Chat[]> createChat(workspaceId: string, userId: string): Promise<Chat> readChat(workspaceId: string, userId: string, chatId: string): Promise<{ chat: Chat; turns: Turn[]; usage: ContextUsage }> deleteChat(workspaceId: string, userId: string, chatId: string): Promise<void> streamTurn(workspaceId: string, userId: string, chatId: string, question: string, modelId: string): AsyncGenerator<ChatAnswerEvent>Owns the chats and turns rows. It builds the messages from the stored turns, passes every frame to the controller, and writes what the run produced. streamTurn writes the turn row before the first model call and completes it after the last frame, so a crash leaves a turn marked running rather than a missing turn. onModuleInit fails every turn still marked running from a previous process. Every method scopes by workspaceId and userId, so one person never reads the chat of another.
AgentLoopapps/api/src/chat/agent.loop.ts:11	streamAnswer(workspaceId: string, messages: readonly ChatMessage[], modelId: string, requestSummary: string): AsyncGenerator<ChatAnswerEvent>Runs the OpenAI Responses API with every declared tool. Every fault the model itself can make returns a correction the model reads, not an error. The caller owns messages and never writes to it during the run. The loop stops after 50 model calls. A bad tool name and unparsable arguments each return a correction rather than a throw. It compacts first when the context is already full, and yields a compaction frame when it does. pseudocodetext if measureUsage(summary, messages).fraction >= 1: summary, messages = compact(...) yield compaction, contextfor modelCallCount in 0 .. 49: response = openai.responses.create(model, input, tools) for call in response.toolCalls: args = readToolCall(call) # unparsable args return a correction, never a throw output = registry.run(session, call.name, args) input.push(output) if response has no toolCall: yield done stop
CompactionServiceapps/api/src/chat/compaction.service.ts:15	measureUsage(summary: string, messages: readonly ChatMessage[]): ContextUsage compact(summary: string, messages: readonly ChatMessage[], modelId: string): Promise<{ summary: string; messages: ChatMessage[]; turnCount: number } | null>Compacts a chat at 100 turns or 256000 tokens. It summarizes the oldest nine tenths and keeps the newest tenth as written. The two limits are independent, and whichever the chat reaches first starts the compaction. compact returns null when the chat is too short to split, and the caller then sends it unchanged. The summary carries no result number, because a number from an earlier search does not match the current one.
ChatRepositoryapps/api/src/chat/chat.repository.ts:18	createTurn(chatId: string, question: string, modelId: string): Promise<number | null> completeTurn(chatId: string, turnIndex: number, turn): Promise<void> failTurn(chatId: string, turnIndex: number, errorMessage: string): Promise<void> recordCompaction(chatId: string, summary: string, compactedTurnCount: number): Promise<void> failEveryRunningTurn(errorMessage: string): Promise<number>Every read and write of chats and turns. It never builds a wire type. createTurn returns the new turn index, or null when the chat does not belong to that workspace and person. The turn index is dense and starts at zero, so a caller can address a turn by position.

DatabaseConnectionsModuleComponent4 symbols

Symbol	What it does
DatabaseConnectionsControllerapps/api/src/database-connections/database-connections.controller.ts:25	listDatabaseConnections(@ActiveWorkspaceId() workspaceId: string): Promise<ListDatabaseConnectionsResponse> createDatabaseConnection(@Body() body: unknown, @ActiveWorkspaceId() workspaceId: string): Promise<CreateDatabaseConnectionResponse> testDatabaseConnection(@Param("databaseConnectionId") id: string, @ActiveWorkspaceId() workspaceId: string): Promise<TestDatabaseConnectionResponse> deleteDatabaseConnection(@Param("databaseConnectionId") id: string, @ActiveWorkspaceId() workspaceId: string): Promise<StatusResponse>Four routes: list, register, test one, delete one. The password arrives once on POST and never leaves in a read, so no response type carries it. A test answers with the reachability and the time of the check, never with the connection error of the driver in full.
DatabaseConnectionsServiceapps/api/src/database-connections/database-connections.service.ts:28	listDatabaseConnections(workspaceId: string): Promise<DatabaseConnection[]> createDatabaseConnection(workspaceId: string, request: CreateDatabaseConnectionRequest): Promise<CreateDatabaseConnectionResponse> testDatabaseConnection(workspaceId: string, databaseConnectionId: string): Promise<TestDatabaseConnectionResponse> deleteDatabaseConnection(workspaceId: string, databaseConnectionId: string): Promise<void> executeSql(workspaceId: string, name: string, sql: string, params?: readonly unknown[]): Promise<SqlRows>Opens a connection before it stores one, and stamps status and lastCheckedAt on every check. A row that exists is a row that answered at least once. executeSql resolves the row by workspace and name, so a name from one workspace never reaches the database of another.
DatabaseConnectionPoolapps/api/src/database-connections/database-connection.pool.ts:36	checkConnection(settings: Omit<ConnectionSettings, "id">): Promise<void> executeSql(settings: ConnectionSettings, sql: string, params?: readonly unknown[]): Promise<SqlRows> closePool(databaseConnectionId: string): Promise<void>One pg.Pool per row, opened with default_transaction_read_only=on and statement_timeout=60000, and every statement runs inside BEGIN READ ONLY. Every statement commits, and a failure rolls back and rethrows. Every value returns as a string: a null becomes an empty string, a date becomes an ISO string, and an object becomes JSON. A pool that fails to open throws ConnectionFailedError, which the caller reports as unreachable rather than as a 500. pseudocodetext client = pool(settings).connect() # a failure here throws ConnectionFailedErrortry: client.query("BEGIN READ ONLY") result = client.query(sql, params) rows = stringify every value client.query("COMMIT") return { columns, rows }catch: client.query("ROLLBACK") rethrowfinally: client.release()
DatabaseConnectionRepositoryapps/api/src/database-connections/database-connection.repository.ts:23	listDatabaseConnections(workspaceId: string) createDatabaseConnection(workspaceId: string, databaseConnection) updateDatabaseConnection(databaseConnectionId: string, databaseConnection) recordConnectionCheck(databaseConnectionId: string, isReachable: boolean) deleteDatabaseConnection(workspaceId: string, databaseConnectionId: string): Promise<boolean>Every read and write of database_connections, with the password encrypted. (workspaceId, name) is unique, so a second registration under the same name updates the row. deleteDatabaseConnection returns false when the row belongs to another workspace. The password decrypts only for a connection attempt.

SyncModuleComponent10 symbols

Symbol	What it does
SyncServiceapps/api/src/sync/sync.service.ts:12	sync(sourceId: string): Promise<void>One pass over one source. It stamps lastSeenAt with the pass start time, re-indexes only what the provider edited later, and deletes a row only after the provider confirms the document is gone. It stamps the pass start time rather than the current time, so a document listed late in a long pass is not mistaken for stale. Every deletion candidate is confirmed with the provider first, because a listing that omits a live page must never delete it. An attachment is listed and never indexed, because no connector returns its text yet. pseudocodetext syncStartedAt = now()cursor = nonerepeat: page = connector.listDocuments(cursor, pageSize = 100) for document in page.documents: row = upsertObservedDocument(document, sourceId, syncStartedAt) if document is an attachment: continue if row.lastIndexedAt is null or row.lastIndexedAt < document.externalUpdatedAt: body = connector.fetchDocumentById(document.externalId) chunks = createChunks(body) vectors = embedChunks(title + headingPath + text for each chunk) reindexDocument(row.id, chunks with vectors, syncStartedAt) # array order sets chunkIndex cursor = page.nextCursoruntil cursor is nonefor stale in listStaleDocuments(sourceId, syncStartedAt): if not connector.checkDocumentExists(stale.externalId): # one at a time; Notion allows about 3 per second mark for deletiondeleteDocuments(marked)recordSyncCompletion(sourceId, syncStartedAt)
ConnectorResolverapps/api/src/sync/connector.resolver.ts:10	resolveConnector(sourceId: string): Promise<DocumentConnector>Reads the stored settings of a source, decrypts its token, and builds the connector for its provider. It refreshes an expired token before it returns, so the caller never handles a 401 from the provider. A source whose credential needs re-authorization throws, and the pass stops rather than deleting every document it cannot list.
createChunksapps/api/src/chunking/chunking.ts:338	export async function createChunks(body: DocumentBody, options?: Partial<ChunkOptions>): Promise<Chunk[]>Parses Markdown to mdast, keeps the heading path, packs siblings to 512 tokens, splits a long table by row and repeats its header, and re-fences each piece of a split code block. Cuts one fetched document body into the chunks that get embedded. The default ceiling is 512 tokens and the floor is 64. The returned order is the contract: the repository assigns chunkIndex from the array position. Each chunk carries its heading path, and the caller prepends the title and that path before it embeds, so the stored text stays raw. pseudocodetext parse the Markdown to an mdast treewalk the tree and keep the heading path above each nodepack sibling nodes until the next one would pass 512 tokenssplit a long table by row and repeat its header on each piecere-fence each piece of a split code blockdrop a piece under 64 tokens into its neighbourreturn the chunks in reading order
ConnectorFactoryapps/api/src/connectors/connector.factory.ts:31	createConnector(options: ConnectorOptions): DocumentConnectorFrom ConnectorModule. Builds one connector from a complete set of options. It reads no environment variable and no database row, so every value arrives from the caller. An unknown provider throws, which makes a new provider a compile error at the call site rather than a silent no-op.
UploadConnectorapps/api/src/connectors/upload.connector.ts:23	listDocuments(options?: ListDocumentsOptions): Promise<DocumentPage> fetchDocumentById(externalId: string): Promise<DocumentBody> checkDocumentExists(externalId: string): Promise<boolean> deleteDocument(externalId: string): Promise<void>Reads an uploaded file from the uploaded_files table. The one connector that can delete a document. The externalId is the file name, which is unique within the workspace, so no path ever reaches the file system. SourcesService uses the delete to remove the rows of a source it manages.
NotionConnectorapps/api/src/connectors/notion.connector.ts:341	listDocuments(options?: ListDocumentsOptions): Promise<DocumentPage> fetchDocumentById(externalId: string): Promise<DocumentBody> checkDocumentExists(externalId: string): Promise<boolean>Lists and fetches a Notion page, 100 per page, cursor paged, and renders its blocks to Markdown. Notion allows about three requests per second across the integration, so the caller must not run these calls in parallel. It cannot delete, because the grant is read-only.
ConfluenceConnectorapps/api/src/connectors/confluence.connector.ts:235	listDocuments(options?: ListDocumentsOptions): Promise<DocumentPage> fetchDocumentById(externalId: string): Promise<DocumentBody> checkDocumentExists(externalId: string): Promise<boolean>Lists and fetches a Confluence page of one space, and renders its storage format to Markdown. Lists and fetches the pages of one space and renders the Confluence storage format to Markdown. One source covers one space, so spaceIds holds exactly the space of that source. It cannot delete, because the grant is read-only.
DocumentRepositoryapps/api/src/documents/document.repository.ts:14	upsertObservedDocument(document: DocumentRef, sourceId: string, lastSeenAt: Date): Promise<{ id: string; lastIndexedAt: Date | null }> reindexDocument(documentId: string, chunks: readonly DocumentChunkValues[], indexedAt: Date): Promise<void> listStaleDocuments(sourceId: string, syncStartedAt: Date): Promise<{ id: string; externalId: string }[]> deleteDocuments(documentIds: readonly string[]): Promise<number>From DocumentModule. Writes a document row and replaces every chunk of it, with raw SQL for the vector(1536) column. reindexDocument deletes every chunk of that document and writes the new ones in one transaction, and it assigns chunkIndex from the array position, so the caller owns the order. It writes the vector(1536) column with raw SQL, because Prisma Client cannot read or write an Unsupported field. listStaleDocuments returns the documents the pass did not stamp, which is a candidate list and never a delete list.
SourceSyncRepositoryapps/api/src/documents/source-sync.repository.ts:26	findConnectorSettings(sourceId: string): Promise<ConnectorSettings> recordSyncCompletion(sourceId: string, syncStartedAt: Date): Promise<void>From DocumentModule. Reads the connector settings of a source and stamps lastSyncedAt when a pass finishes. It stamps the pass start time, not the finish time, so the next pass never skips a document edited while this one ran.
UploadedFileRepositoryapps/api/src/connectors/uploaded-file.repository.ts:7	listUploadedFiles(workspaceId: string, afterFileName: string | undefined, take: number): Promise<{ fileName: string; updatedAt: Date }[]> findUploadedFile(workspaceId: string, fileName: string): Promise<{ text: string; updatedAt: Date } | null> saveUploadedFiles(workspaceId: string, files: readonly { fileName: string; text: string }[]): Promise<void> deleteUploadedFile(workspaceId: string, fileName: string): Promise<void>From ConnectorModule. Every read and write of uploaded_files. listUploadedFiles pages by file name, so a pass reads a stable order and never repeats a row. saveUploadedFiles upserts on (workspaceId, fileName), so a second upload of one name replaces the text rather than adding a row.

ToolsModuleComponent5 symbols

Symbol	What it does
ToolRegistryapps/api/src/tools/tool.registry.ts:36	has(name: string): boolean buildDisplayText(name: string, args: Record<string, unknown>): string run(session: ToolSession, name: string, args: Record<string, unknown>): Promise<string>Reads the @Tool methods of every provider once at startup, so a new tool needs only that decorator. The agent and any future protocol server call the same method. run returns the text the model reads. A tool that fails returns a correction rather than throwing, because the model wrote the arguments. has answers before a call, so an unknown name never reaches a method.
DocumentToolsapps/api/src/tools/document.tools.ts:22	searchDocumentChunks(session: ToolSession, args: Record<string, unknown>): Promise<string>Holds search_document_chunks. It embeds the query with the chunk model, ranks every chunk of the workspace by cosine distance, and returns the nearest six. It cites each returned chunk on the session, so the number in the answer and the number in the rail come from one place. It never reads a chunk outside session.workspaceId. The query must use the same embedding model as the chunks, or the distances mean nothing.
DatabaseToolsapps/api/src/tools/database.tools.ts:10	listDatabases(session: ToolSession): Promise<string> listTables(session: ToolSession, args: Record<string, unknown>): Promise<string> describeTables(session: ToolSession, args: Record<string, unknown>): Promise<string> executeSql(session: ToolSession, args: Record<string, unknown>): Promise<string>Holds list_databases, list_tables, describe_tables and execute_sql. describe_tables returns the columns, the keys and five live sample rows. The model writes the SQL, so execute_sql treats it as untrusted text. It strips a trailing semicolon, rejects a second statement, and rejects anything that does not start with SELECT or WITH. It wraps the statement in an outer LIMIT 1000 before it runs. Every rejection returns a sentence the model reads and corrects, never an exception. The answer is cut to 20000 characters. pseudocodetext name = args.nameif name is empty: return "call list_databases first"sql = trim(args.sql), strip trailing semicolonsif sql is empty: return "send one SELECT statement"if sql contains ";": return "one statement, no semicolon"if sql does not start with SELECT or WITH: return "this database is read-only"rows = executeSql(workspaceId, name, "SELECT * FROM (" + sql + ") AS query LIMIT 1000")emit rows to the browserreturn the table, cut to 20000 characters
ToolSessionapps/api/src/tools/tool.session.ts:10	cite(chunk: Omit<Citation, "index">): Citation emit(event: ChatAnswerEvent): void drainEvents(): ChatAnswerEvent[]Carries the workspace id and the collected citations through one answer. cite assigns the next index and returns the citation, so a tool never picks its own number. drainEvents returns the queued events and empties the queue, so the agent loop reads each event once.
EmbeddingServiceapps/api/src/embedding/embedding.service.ts:9	embedChunks(texts: readonly string[]): Promise<number[][]> embedQuery(text: string): Promise<number[]>From EmbeddingModule, which this module imports. text-embedding-3-small at 1536 dimensions. embedChunks batches under 2048 inputs and 300000 tokens per request, and returns the vectors in the order of texts. embedQuery must use the same model, because a query embedded by another model does not compare against the stored chunks.

PrismaModuleComponent2 symbols

Symbol	What it does
PrismaServiceapps/api/src/prisma/prisma.service.ts:7	onModuleDestroy(): Promise<void>Extends the generated PrismaClient and closes the pool on onModuleDestroy. It is the only client in the API. A second instance would open a second pool against the same database, so every module injects this one.
AppConfigapps/api/src/config/app-config.ts:5	readonly deploymentName: string readonly encryptionKey: Buffer readonly host: string readonly logLevel: LogLevel readonly openaiApiKey: string readonly port: number readonly requestBodyLimit: string readonly webAppUrl: stringFrom AppConfigModule. Reads and validates every environment variable once at boot. A missing value stops the boot, so no service reads process.env. encryptionKey must decode to exactly 32 bytes from base64url, and webAppUrl must be an HTTP or HTTPS origin. A malformed value throws as well as a missing one, so no service checks a value again.

app-dbContainer12 symbols

Symbol	What it does
Workspaceapps/api/prisma/schema/workspace.prisma:1	One team. It owns every source, document, chunk, database connection and chat below it. slug and joinCode are each unique across the table. Every row below it cascades on delete.
Userapps/api/prisma/schema/workspace.prisma:24	One person, keyed by the Google account. One person, keyed by externalId, which is the stable Google subject. An email that changes at Google updates this row rather than creating a second one.
WorkspaceMembershipapps/api/prisma/schema/workspace.prisma:41	Joins a user to a workspace. The join code writes this row. (workspaceId, userId) is unique, so a second join writes no row. A delete of either side cascades, so a membership never points at a workspace or a user that is gone.
UserSessionapps/api/prisma/schema/workspace.prisma:57	One signed-in browser. It holds the SHA-256 hash of the ks_session cookie, never the cookie. tokenHash is unique. activeWorkspaceId is nullable and sets to null when that workspace is deleted, so the person lands with no workspace open rather than on a dangling id.
Sourceapps/api/prisma/schema/document.prisma:21	One upload area, one Notion workspace, or one Confluence space. It carries lastSyncedAt. Each provider carries its own partial unique index, so a second grant updates the same row: Notion keys on the credential, Confluence on the external id and the space, and upload on the external id. The credential relation is Restrict, so a credential in use cannot be deleted.
UploadedFileapps/api/prisma/schema/document.prisma:82	The text of one uploaded file. (workspaceId, fileName) is unique, so the file name is the external id the upload connector reads. The text is stored, never the original bytes, so a re-index needs no file system.
SourceCredentialapps/api/prisma/schema/document.prisma:54	The OAuth token of one source, encrypted with AES-256-GCM. The OAuth token of one grant, encrypted with AES-256-GCM. (workspaceId, provider, externalId) is unique. (id, workspaceId, provider) is unique as well, because Source joins on all three and that join must not cross a workspace.
Documentapps/api/prisma/schema/document.prisma:82	One page or file. lastSeenAt and the provider edit time decide whether a pass re-indexes it. One page or one file. (sourceId, externalId) is unique. lastSeenAt and lastIndexedAt decide the work of a pass: the provider edit time against lastIndexedAt decides a re-index, and lastSeenAt against the pass start time decides a deletion candidate.
DocumentChunkapps/api/prisma/schema/document.prisma:109	One chunk with its heading path and its vector(1536) embedding, under an HNSW vector_cosine_ops index. Prisma Client cannot read the column, so the search and the write both use raw SQL. (documentId, chunkIndex) is unique and chunkIndex is dense from zero. prisma migrate dev proposes to drop the HNSW index in every migration it generates, and the drop must be deleted before the migration is committed.
Chatapps/api/prisma/schema/chat.prisma:7	One conversation. It holds the compaction notes, so a browser that misses the frame loses nothing. The summary and the compacted turn count are stored fields, so a reload recovers both after a browser missed the compaction frame.
Turnapps/api/prisma/schema/chat.prisma:26	One question and its answer, with the display texts, the citations and the query results. The turn index is dense from zero within a chat. A turn stays marked running until it completes or fails, and failEveryRunningTurn at boot closes the ones a crash left open.
DatabaseConnectionapps/api/prisma/schema/database_connection.prisma:12	One registered business database, with the password encrypted and status from the last check. One registered business database. (workspaceId, name) is unique, which is why the tools address a database by name and never by id. The password is encrypted, and status and lastCheckedAt come from the last check.

3Future Work (Currently Being Implemented)

• Use change streams to update the database index and keep information current.
• Replace the fixed chunk citation limit with adaptive document retrieval.
• Let users share chats with other workspace members.
• Add search across chats.
• Store uploaded files in cloud storage.
• Add multimodal support for PDF documents and images.
• Add rate limits before the public release.
• Generate chat titles through a separate large language model call.
• Load more chats as the user scrolls.