Backup/feat db seed #235
Backup/feat db seed #235
Conversation
Replaces the upsertAgent function with findExistingAgent, which only checks for the existence of the agent instead of inserting or updating. Updates the seed logic to only upsert intent stakes if the agent exists.
Replaces generic test account names with more realistic names for the PRIVY_TEST_ACCOUNTS array in db-seed-demo.ts. This improves clarity and realism for demo and testing purposes.
This update extends the demo DB seeding script to support seeding files and links for demo users, including associating intents with these resources. It introduces new types and logic for files and links, updates the summary output, and allows users to have shared intents and resource-backed intents.
Expanded DEMO_INDEXES to include detailed membership configuration, join policies, and invitation codes. Added support for default intent sources (files/links) and ensured users are assigned to multiple indexes with specific permissions, prompts, and auto-assign flags. Refactored membership seeding logic to use new configuration, and improved intent seeding to attach default sources. Updated database upsert logic to handle new permissions and membership fields.
Introduces a Logger abstraction and injects it throughout the seeding process to provide consistent, contextual log messages. Logging now respects CLI options for JSON output and silent mode, improving script observability and user experience.
WalkthroughAdds two npm scripts for DB utilities, a new TypeScript CLI that deterministically seeds demo data (users, indexes, intents, files, links, memberships, stakes, connection events) with Privy integration and flags, and exports a closeDb() helper to gracefully terminate the Postgres client. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor Dev as Developer
participant NPM as npm
participant CLI as db-seed-demo CLI
participant Seed as runSeed()
participant DB as PostgreSQL
participant Privy as Privy API
Dev->>NPM: npm run db:seed:demo -- --force [--json|--silent]
NPM->>CLI: node -r ts-node/register ./src/cli/db-seed-demo.ts
CLI->>Seed: parse flags, validate env
rect rgba(180,220,255,0.15)
Seed->>Privy: ensurePrivyIdentity()
Seed->>DB: upsert users / indexes / files / links
Seed->>DB: upsert intents / memberships / stakes / events
Seed-->>CLI: summary (counts, ids)
end
CLI-->>Dev: output (JSON or human-readable)
CLI->>DB: closeDb() -> client.end({ timeout: 5 })
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Poem
Pre-merge checks and finishing touches❌ Failed checks (2 warnings)
✅ Passed checks (1 passed)
✨ Finishing touches
🧪 Generate unit tests
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
protocol/package.json(1 hunks)protocol/src/cli/db-seed-demo.ts(1 hunks)protocol/src/lib/db.ts(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
protocol/src/lib/db.ts (1)
protocol/generate-mock-data.js (1)
db(6-6)
protocol/src/cli/db-seed-demo.ts (3)
protocol/src/lib/privy.ts (1)
privyClient(5-8)protocol/src/lib/schema.ts (6)
files(78-87)indexLinks(226-226)users(13-27)intents(29-44)intentIndexes(90-94)userConnectionEvents(96-105)protocol/src/lib/db.ts (1)
closeDb(16-18)
🔇 Additional comments (1)
protocol/package.json (1)
15-16: Scripts follow existing ts-node patternThe new
db:flushanddb:seed:demoscripts line up with the existing ts-node execution style and should be easy to integrate. Looks good to me.
| const agentId = await findExistingAgent(); | ||
| if (agentId) { | ||
| logger.info('🤖 Found existing demo agent; will refresh stakes.'); | ||
| } else { | ||
| logger.info('🤖 Demo agent not found; skipping stake updates.'); | ||
| } |
There was a problem hiding this comment.
Seed the demo agent so stakes actually get created
On a fresh database findExistingAgent() returns null, which drops straight into the “skipping stake updates” branch. That leaves agentId unset, so the later block never calls upsertIntentStake, and the run completes with agentId: null in the summary. As a result the deterministic dataset promised here ships without any demo stakes on first run. Please upsert DEMO_AGENT (e.g., via a helper similar to the other upsert functions) whenever the lookup misses, and only skip staking if that upsert fails; otherwise we never create the stakes we defined in DEMO_STAKES.
Also applies to: 1236-1241
🤖 Prompt for AI Agents
In protocol/src/cli/db-seed-demo.ts around lines 1086-1091 (also apply the same
change at 1236-1241): currently when findExistingAgent() returns null the code
logs and skips stake updates leaving agentId unset; instead, when the lookup
misses call the appropriate upsert helper to create/upsert DEMO_AGENT (mirroring
the pattern used for other upsert functions), set agentId to the returned id,
and only log/skip staking if that upsert fails (handle and log the error).
Ensure subsequent calls that upsertIntentStake use the newly assigned agentId.
| export async function closeDb(): Promise<void> { | ||
| await client.end({ timeout: 5 }); | ||
| } |
There was a problem hiding this comment.
Reset dev cache after closing the client
In dev mode we stash the drizzle instance on globalThis.__db. After closeDb() runs the cached instance still points at the now-terminated client, so the next import/reuse in the same process will try to run queries against a closed connection (“Client is closed” errors). Please clear the global cache when we shut the client down so a fresh connection can be created on the next load.
export async function closeDb(): Promise<void> {
await client.end({ timeout: 5 });
+ if (process.env.NODE_ENV === 'development') {
+ globalThis.__db = undefined;
+ }
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| export async function closeDb(): Promise<void> { | |
| await client.end({ timeout: 5 }); | |
| } | |
| export async function closeDb(): Promise<void> { | |
| await client.end({ timeout: 5 }); | |
| if (process.env.NODE_ENV === 'development') { | |
| globalThis.__db = undefined; | |
| } | |
| } |
🤖 Prompt for AI Agents
In protocol/src/lib/db.ts around lines 16 to 18, closeDb currently awaits
client.end but does not clear the cached drizzle instance on globalThis.__db;
update closeDb to first await client.end({ timeout: 5 }) and then clear the
global cache (e.g., if (globalThis.__db) delete globalThis.__db or set
globalThis.__db = undefined) so subsequent reloads in the same process will
create a fresh connection.
Replaces hardcoded demo user and index definitions with a new QA_USER_DETAILS structure, enabling more flexible and maintainable configuration of demo users, indexes, files, links, and intents. Updates DEMO_INDEXES, DEMO_USERS, and related seed data to use this new structure, and refreshes sample stakes and connection events to match the new user keys and intent keys.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
protocol/src/cli/db-seed-demo.ts (4)
1264-1274: Don’t overwrite prompt/autoAssign on existing memberships unless explicitly providedThe update branch always sets prompt to NULL and autoAssign to true via defaults, even when the caller didn’t specify them. This clobbers existing settings on every run.
Apply this diff:
- const updateAssignments = [sql`"permissions" = ${permissionsArray}`, sql.raw('"updated_at" = NOW()')]; - - if (capabilities.indexMembersHasPrompt) { - updateAssignments.push(promptValue !== null ? sql`"prompt" = ${promptValue}` : sql`"prompt" = NULL`); - } - - if (capabilities.indexMembersHasAutoAssign) { - updateAssignments.push(autoAssignValue ? sql`"auto_assign" = TRUE` : sql`"auto_assign" = FALSE`); - } + const updateAssignments = [sql`"permissions" = ${permissionsArray}`, sql.raw('"updated_at" = NOW()')]; + + if (capabilities.indexMembersHasPrompt && options && Object.prototype.hasOwnProperty.call(options, 'prompt')) { + updateAssignments.push(promptValue !== null ? sql`"prompt" = ${promptValue}` : sql`"prompt" = NULL`); + } + + if (capabilities.indexMembersHasAutoAssign && options && Object.prototype.hasOwnProperty.call(options, 'autoAssign')) { + updateAssignments.push(autoAssignValue ? sql`"auto_assign" = TRUE` : sql`"auto_assign" = FALSE`); + }
1042-1067: Consider single-statement UPSERT for indexes to reduce round tripsCurrent pattern INSERT ... DO NOTHING followed by unconditional UPDATE costs two queries. A single INSERT ... ON CONFLICT (id) DO UPDATE SET ... would be simpler and faster.
Also applies to: 1088-1094
1333-1363: Require at least two intents before creating a stakeA stake over a single intent is likely meaningless. Skip unless both referenced intents resolve.
Apply this diff:
- if (intentList.length === 0) return; + if (intentList.length < 2) return;
1002-1029: Optional: Avoid printing access tokens in non-JSON outputEven for local dev, printing bearer tokens can leak via logs. Consider hiding tokens behind a flag or truncate them.
Also applies to: 1614-1645
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
protocol/src/cli/db-seed-demo.ts(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
protocol/src/cli/db-seed-demo.ts (4)
protocol/generate-mock-data.js (1)
db(6-6)protocol/src/lib/privy.ts (1)
privyClient(5-8)protocol/src/lib/schema.ts (6)
files(78-87)indexLinks(226-226)users(13-27)intents(29-44)intentIndexes(90-94)userConnectionEvents(96-105)protocol/src/lib/db.ts (1)
closeDb(16-18)
🔇 Additional comments (3)
protocol/src/cli/db-seed-demo.ts (3)
974-991: Nice: capability checks avoid hard schema couplingDetecting optional columns (prompt, link_permissions, auto_assign) keeps the seeder resilient across migrations. LGTM.
Also applies to: 1281-1291, 1400-1584
1415-1420: Seed/upsert the demo agent when missing so stakes are createdOn a fresh DB, this path skips staking and leaves agentId null. Upsert DEMO_AGENT when not found and use that id for subsequent stake upserts.
Apply this diff:
- const agentId = await findExistingAgent(); - if (agentId) { - logger.info('🤖 Found existing demo agent; will refresh stakes.'); - } else { - logger.info('🤖 Demo agent not found; skipping stake updates.'); - } + let agentId = await findExistingAgent(); + if (!agentId) { + logger.info('🤖 Demo agent not found; creating it so stakes can be seeded.'); + agentId = await upsertAgent(DEMO_AGENT, logger); + } else { + logger.info('🤖 Found existing demo agent; will refresh stakes.'); + }Add this helper somewhere near the other upsert fns:
async function upsertAgent( def: typeof DEMO_AGENT, logger: Logger ): Promise<string> { const agentId = stableId(`agent:${def.key}`); const now = new Date(); try { await db.insert(agents).values({ id: agentId, name: def.name, // Adjust fields to match your schema description: def.description as any, avatar: def.avatar as any, }); } catch (error) { if (!isUniqueViolation(error)) throw error; await db .update(agents) .set({ name: def.name, // Adjust fields to match your schema description: def.description as any, avatar: def.avatar as any, updatedAt: now as any, }) .where(eq(agents.id, agentId)); } return agentId; }
1122-1144: indexLinks schema includes all required columns
Verified thatlinksTable(exported asindexLinks) definesid,userId,url,lastSyncAt,lastStatus,lastError,createdAt, andupdatedAt, so theupsertLinkinsert/update fields align.
Introduces the indexKeys property to intent definitions and user details, allowing specific targeting of index IDs during seed operations. This enables more granular control over which indices intents are associated with, improving demo data relevance.
Introduces a new 'demo-everything' index to the demo seed configuration, updating all user and intent definitions to include it. This enables a unified feed for sharing updates across the network and ensures all demo users and intents are visible in the new index.
Introduces 'network-sync' and 'growth-support-loop' intents to COMMON_INTENTS and updates QA_USER_DETAILS to include these shared intents for relevant test accounts. Adds several new demo stakes and connection events to enrich the demo data for testing cross-team scenarios and user interactions.
Introduces optional triggering of context brokers for each seeded intent via a new --with-brokers CLI flag. Adds mock embedding generation for intents without embeddings, refactors intent upsert logic to return creation status, and updates autoAssign defaults for QA user details.
Introduces CLI options for controlling broker concurrency, bulk upserts for files and links, and fast concurrent user seeding. Improves seed performance and flexibility by allowing batched database operations and parallel processing.
Summary by CodeRabbit
New Features
Chores