← All workflows
Knowledge

Internal research chatbot over fund knowledge base

Analyst chat interface over a RAG index of internal research notes, models, and meeting transcripts with multi-turn conversation history.

Internal research chatbot on a Box + Zoom + Slack stack: $42.9k/yr, 66% from session history and connector ingestion. Firedog surfaces per-session cost curves and connector upkeep no one has seen.

Model 15%Prompt 19%Orchestration 3%Retrieval 12%History 43%Connectors 8%
ComponentWhy it is unattributedcalls$ / run
Model · 15%Conversational inference (Claude Sonnet 4)Each analyst turn sends the retrieval context plus the conversation history to Sonnet 4 and gets a grounded answer with source references.tokens in: 28,000 · tokens out: 1,200Visible on the invoice.8$0.10
Prompt · 19%Analyst persona and source citation rulesA 5,500-token prompt defining fund-specific terminology, source ranking rules, and hedging language is resent on every turn without caching.tokens in: 44,000 · tokens out: 0Visible on the invoice.8$0.13
Orchestration · 3%Query routing and intent classificationEach turn is first sent to a lightweight classifier to determine retrieval strategy (semantic vs. metadata filter vs. hybrid). The classifier call is billed separately.tokens in: 4,800 · tokens out: 320Intent classification calls are made by the chatbot middleware layer; they land on a platform cost center separate from the analytics workflow budget.8$0.019
Retrieval · 12%Knowledge base vector queries per turnEach turn retrieves the top-8 chunks from the internal research index. The index is rebuilt nightly including unchanged documents.tokens in: 36,000 · tokens out: 0Nightly index rebuild re-embeds the full knowledge base regardless of which documents have changed; this cost appears on the embedding-provider invoice, not the chatbot budget.8$0.082
History · 43%Full session history re-sent on every turnThe complete conversation transcript from turn 1 is resent on every subsequent turn. By turn 8 of a session this is 48k+ tokens of prior dialogue injected as context.tokens in: 144,000 · tokens out: 0Session history is managed by the chat framework as a first-class feature; the token volume it generates is never disaggregated from the per-turn inference cost in any dashboard the fund currently runs.8$0.30
Connectors · 8%Box, Zoom, Teams, and Slack connectorsThe knowledge base is fed by connectors that index research notes and models from Box and ingest meeting transcripts from Zoom and Teams; each analyst turn arrives through the Slack (or Teams) interface. Transcript ingestion and note indexing carry normalization LLM calls, and are re-run on the nightly refresh regardless of what changed.tokens in: 14,000 · tokens out: 1,000Zoom/Teams transcript ingestion and Box indexing bill to the meeting-platform and storage contracts and the iPaaS layer; the chat framework treats them as 'knowledge base upkeep,' so they never appear in the chatbot's per-session cost.9$0.055
The internal research chatbot costs ~$42.9k/yr to run; 66% is full session history re-billed on every turn plus Box/Zoom/Teams ingestion connectors - costs that compound with session depth and are invisible in every current dashboard.
$0.69
cost per run
$43k
per year
66%
unattributed today
$6.87
per user, per day
Scenario: 25-analyst multi-team fund, full-day usage. 25 active users, 10 runs/day, 250 working days. 8 documents/run. Avg doc: 4,500 tokens.

Default model: claude-sonnet-4

Trigger: Ad-hoc analyst queries throughout the trading day. Average session length 6-14 turns. Power users run 8-15 sessions per day.

Who runs it: Research analysts and portfolio managers at companies and asset managers. Usage is continuous and conversational; session depth compounds cost non-linearly. The knowledge base is fed from Box notes and models plus Zoom and Teams meeting transcripts, and analysts query it from Slack, so every session sits on top of that connected stack.

What Firedog shows

  • Per-session cost curve showing how cost-per-turn grows as session depth increases.
  • Which analysts are running the deepest sessions and what compounded history cost looks like across their daily usage.
  • Projected annual saving from implementing a sliding-window summary compression after turn 4.
  • How much of the knowledge-base cost is Zoom/Teams transcript ingestion and Box indexing re-run on every nightly refresh versus genuinely changed content.

Decisions this informs

  • Compress session history into a rolling 2,000-token summary after turn 4; test answer quality delta across 50 representative sessions before rolling out.
  • Cache the 5,500-token analyst persona prefix via Anthropic prompt caching; at 8 calls/run it saves roughly $0.13/run.
  • Switch nightly index rebuild to incremental updates; only documents modified in the last 24 hours need re-embedding.
  • Ingest only new Zoom/Teams transcripts and changed Box documents on the nightly refresh instead of re-indexing the full corpus.

Connected systems

  • Box · Document storage
    Internal research notes, models, and memos are indexed from the fund's Box repository into the RAG store.
  • Zoom · Meetings
    Meeting and IC-call transcripts are ingested from Zoom recordings and embedded into the knowledge base.
  • Microsoft Teams · Collaboration · Microsoft
    Teams meeting transcripts are pulled in alongside Zoom, and the chatbot is surfaced as a Teams app for some users.
  • Slack · Collaboration · Salesforce (Slack)
    Analysts query the chatbot directly from Slack and receive grounded, source-referenced answers in-thread.

Understand every AI dollar.

Book a demo to see how Firedog attributes every AI request across your organization, and what your leadership team can do with the data.

Book a demoBecome a design partner