An AI Answer Can Be Accurate and Still Be Incomplete

A project lead asks whether a customer is ready for launch. The AI checks the project tracker, finds every milestone marked complete, and returns a confident yes. The answer accurately reflects the tracker. But the signed acceptance is in another system, and that system was unavailable when the question ran.

This hypothetical example exposes a practical problem with AI that works across business systems: a correct statement about the information retrieved can still support the wrong decision.

On October 1, Progress Software announced new Agentic RAG capabilities, including a Smart Agent for multistep retrieval. Progress says it can plan queries and retrieve information across indexed content, applications connected through Model Context Protocol, and web sources. The company described the release as available that day. These are vendor statements, not capabilities we have independently evaluated.

The announcement makes a broader operating question timely. As retrieval spans more systems, teams need to define what counts as enough evidence to answer. Our recommendation is to make source coverage part of the answer's acceptance criteria.

Start with the decision and its required evidence

Connecting another application gives an agent another place to look. It does not tell the agent which evidence the business considers essential.

For the launch question, the required evidence might include completed delivery tasks, customer acceptance, and confirmation that the support handoff is ready. Those requirements should come from the owner of the launch decision. They should not emerge accidentally from whichever connector responds fastest.

Start with a small set of recurring questions. For each one, document the systems required, the specific records needed, and what to do when the evidence is missing or contradictory. This is practical workflow design, and it should happen before treating a cross-system answer as ready for operational use.

Some questions can tolerate partial coverage. A request for background reading can return useful documents while saying one library could not be searched. A recommendation to confirm a launch needs a stricter standard. The consequences should determine how much missing evidence is acceptable.

Distinguish no record from no access

An empty result can mean several things. The record may not exist. The user may lack permission to see it. A connector may have timed out. The query may have used the wrong customer identifier.

Those conditions require different responses. Reporting that acceptance has not been received is a business assertion. Reporting that acceptance could not be checked describes a limitation of the lookup. Conflating them can send staff chasing a customer for something the customer already supplied.

The integration should preserve enough status information to make that distinction. The answering layer should carry the relevant limitation forward in plain language. A useful response might say: delivery tasks are complete; customer acceptance could not be checked; launch readiness remains unconfirmed.

That response gives the project lead a specific next step. A vague warning that the answer may contain errors leaves the person to reconstruct the whole investigation.

Access restrictions also need to remain intact. An incomplete answer is not a reason to retry through a more privileged account. Route the missing check to an authorized person or approved workflow.

A citation does not show the whole search

A source link helps a reader inspect the evidence used for a statement. It does not establish that every required source was checked.

For operational questions, we recommend showing a short coverage note alongside the conclusion. Name the required checks that succeeded and any that remain unresolved. Include the relevant record identifiers or links so a reviewer can follow the same case across systems.

Be especially careful when matching records. A customer name can appear on several projects, subsidiaries, or renewals. Joining the right documents to the wrong engagement produces a convincing answer with perfectly real citations. Use stable identifiers where available, and escalate ambiguous matches rather than silently choosing one.

The goal is a response a busy owner can evaluate quickly. Keep technical diagnostics in the execution record. Put the business consequence of missing evidence in the answer itself.

Test the missing-source cases deliberately

A demonstration with every connector working will not show how the workflow behaves when coverage breaks.

Before rollout, evaluate representative questions with a required source unavailable, a permission denied, an ambiguous account match, and conflicting records. Define the expected outcome for each case before running it. Some should produce a qualified answer. Others should stop the recommendation and request a specific check.

Also test whether the workflow mistakes an empty search for proof that something never happened. That is particularly relevant when the requested decision depends on the absence of a record, such as whether an invoice was already paid or a request was previously approved.

Record whether the system preserved uncertainty and routed the unresolved work correctly. As we discussed in the exception path, a workflow needs a useful destination for cases it cannot complete. The destination should have an owner, the evidence already gathered, and the exact missing check.

Budget for the complete answer

Live retrieval introduces a tradeoff. Checking more sources can add waiting time, usage charges, and exposure to upstream outages. Checking too few can shift the cost into manual verification or an incorrect business action.

Set a retrieval budget around the decision: required sources, acceptable wait, and the point at which the workflow should return an incomplete status. Optional background research can stop earlier than a required approval check. Track the cost of resolved questions alongside the work handed back to staff, consistent with measuring AI value at the end of the workflow.

For an initial pilot, pick one recurring cross-system question and name its business owner. Agree on the required evidence, test the failure cases, and measure how often staff can accept the result without repeating the lookup.

If your team is connecting AI to several business systems, talk with Revival Group about defining the evidence and exception handling for one production workflow.