Everyone's tried ChatGPT for work questions. And everyone's noticed the same thing: it gives plausible-sounding answers that are completely wrong about your specific business. It doesn't know your customers, your revenue numbers, your inventory levels, or your internal processes. It can't, because it doesn't have access to any of that data.
But you can build something that does. A private AI assistant connected to your actual business systems — CRM, accounting, support desk — that answers questions from real, live data using a technique called tool use (or function calling). The AI model understands the question and formats the response, but the answers come from your systems. Here's how it works, and what's involved in building one.
What does "connected to your data" actually mean?
When someone on your team asks "which invoices are overdue by more than 30 days?", the AI assistant needs to do several things: understand the question, figure out which system has the answer (your accounting software), query that system's API, format the results, and present a human-readable response.
This is fundamentally different from ChatGPT, which generates answers from its training data. Your assistant retrieves answers from live systems. The AI model is the brain that understands the question and formats the response, but the data comes from your CRM, accounting software, support desk, or database.
The technical term for this pattern is tool use or function calling. The AI model doesn't contain your data — it has access to tools that can query your data on demand.
How the architecture works
A business AI assistant has three layers:
The interface. This is where your team types questions. It can be a web chat widget, a Slack bot, or a simple browser-based interface. The interface sends the user's question to the backend and displays the response. Nothing complicated here.
The orchestration layer. This is the core of the system. It receives the question, sends it to an AI model (GPT-4, Claude, or similar) along with a description of the available data sources. The model decides which data source(s) to query, the orchestration layer executes those queries via API calls, sends the results back to the model, and the model generates a natural-language response.
The connectors. These are API integrations to your business tools. A connector to Salesforce that can look up deals, contacts, and pipeline data. A connector to QuickBooks that can pull invoices, revenue, and expense data. A connector to your support desk that can search tickets. Each connector translates the AI's request into the specific API calls that tool requires.
What your team can actually ask
Once the system is connected, the questions your team can ask are limited only by what data the connectors can access. Real examples from systems we've built:
- "What's our MRR as of today?" — pulls from Stripe or your billing system
- "Show me all deals in the pipeline over $50K" — pulls from Salesforce or HubSpot
- "Which clients haven't been invoiced this month?" — cross-references CRM and accounting
- "What's the average resolution time for support tickets this week?" — pulls from Zendesk or Freshdesk
- "Who on the sales team closed the most deals last quarter?" — aggregates CRM data
- "Give me a summary for the board meeting" — pulls from multiple sources and formats a report
The last example is the most powerful. The assistant can pull data from multiple systems in a single query, something that would take a human analyst 30 minutes of tab-switching and spreadsheet-building. If your team currently relies on someone to manually build reports from multiple data sources, this is the same problem solved from a different angle.
What about data privacy and security?
This is the question we get first in every conversation, and rightly so. Here's how we handle it:
Your data never leaves your infrastructure. The AI model receives the question and a description of available tools. It returns a structured request ("query the CRM for deals over $50K"). Your backend executes that query against your own API. The raw data stays in your systems. Only the formatted response goes back to the user.
Access controls mirror your existing permissions. If a sales rep shouldn't see finance data, the assistant enforces the same rules. Each user's session is scoped to the connectors and data they're authorized to access.
All queries are logged. Every question asked, every data source queried, and every response generated is logged with timestamps and user identity. You have a full audit trail.
You choose the AI provider. If your compliance requirements prohibit sending data to external APIs, we can run open-source models on your own servers. The orchestration layer works the same regardless of which model powers it.
How long does it take to build?
A basic assistant with 2-3 data source connections typically takes 2-4 weeks. The breakdown:
- Week 1: Set up the orchestration layer and interface. Connect the first data source (usually the CRM, since that's where most questions start).
- Week 2: Add 1-2 more connectors (accounting, support desk). Tune the system prompt so the model understands your business terminology.
- Weeks 3-4: Testing with real users, handling edge cases, refining responses. This is where the assistant goes from "works in demos" to "works in real life."
The ongoing cost is primarily AI API usage (typically $50-200/month depending on query volume) plus hosting ($30-50/month for the backend).
When this isn't the right solution
An AI assistant works best when your team asks varied, ad-hoc questions about business data. If the questions are always the same five questions every Monday, a scheduled report or dashboard is simpler and cheaper. If the data isn't accessible via API (locked in spreadsheets or legacy systems with no interface), the first step is getting the data into a queryable system, not building an AI on top of it.
The sweet spot is teams that currently rely on an analyst, an operations person, or "the one person who knows where everything is" to answer data questions. That's the person whose time you're freeing up — and the person who's currently a single point of failure for your company's access to its own information. This is the kind of system we build through our AI-ready software service.