At 23:56, a test user reported that the columns in a table were misaligned. A minute later, the same symptom was reported again. At 23:58, a third report widened the scope: the problem affected multiple tables in the HR module.
None of those reports was remarkable on its own. Together, they exposed the problem.
The team did not need three tickets. It needed one trustworthy signal: what the user saw, where it happened, whether the reports described the same defect, and enough context for someone to make a decision without starting an interrogation.
That is the uncomfortable side of AI-assisted delivery. Producing a change is becoming cheap. Establishing what that change did in the hands of a real user is not.
We encountered this while building Ezzo, our multi-tenant ERP platform. The lesson is broader than one product: when software delivery accelerates, feedback must become a control system—not a mailbox.
Table of contents
Open Table of contents
The new bottleneck is knowing
Before coding agents, implementation capacity constrained most teams. A feature crossed the desk slowly enough that the author could hold much of it in their head.
That ratio has changed. An agent can modify a user interface, an API, a data model, translations, background jobs, and tests during one session. The surrounding organization still reviews, tests, supports, and learns at human speed.
This does not mean AI-generated code is uniquely defective. Humans have always shipped bugs. The risk is that more behavior now enters the system per unit of human attention.
Traditional feedback makes that imbalance worse. A message such as “the invoice button does nothing” is a signal, but it is poor evidence. By the time engineering receives it, the route, browser state, failed request, console error, permissions, locale, and screen state may be gone. Support becomes a lossy translation layer between the person who experienced the problem and the person expected to fix it.
Adding a longer form does not solve this. Automatically creating a GitHub issue for every submission is worse: it converts ambiguity, duplicates, configuration mistakes, and misunderstandings into backlog noise.
The problem is not ticket creation. It is the distance between experience, evidence, judgment, action, and closure.
Capture the moment, not the memory
Ezzo now exposes a lightweight “Report a bug” control inside the authenticated product. A user describes what happened and can optionally attach the current page or screen without leaving the workflow.
The reporting panel stays inside the product: describe the problem, add visual evidence if it helps, and keep working.
The product adds a bounded technical envelope automatically: the route, application version, locale, theme, viewport, browser, recent console errors, and failed network requests. Request and response bodies are deliberately excluded. Sensitive headers and obvious secrets are scrubbed. A screenshot is optional, because useful evidence can also contain business data.
That last detail matters. The capture is not a decorative attachment added after the fact; it represents the state the user was trying to explain.
The report is persisted in Ezzo before any model or external API is involved. Screenshot storage is tenant-scoped. If an external tool or the AI step fails, the report still exists. Automation is downstream of the fact—not the owner of it.
The queue turns three isolated complaints into a visible pattern. An operator can compare timing, wording, severity, and functional area before deciding what deserves engineering attention.
A report is not yet engineering work
Once the report exists, AI proposes a severity, a functional area, and a confidence level. It can summarize evidence and surface likely duplicates. It cannot silently declare a customer-visible problem to be a confirmed defect.
That decision belongs to a human operator who can ask questions the model cannot settle from telemetry alone:
- Is the behavior reproducible?
- Is it a defect, a permission issue, or expected configuration?
- Does it block a critical business workflow?
- Is it the same problem as an existing report?
- Is the captured evidence appropriate to send outside the product boundary?
Only an approved escalation becomes a GitHub issue. GitHub remains where engineering works; Ezzo remains where the product relationship lives.
This boundary is the design, not administrative ceremony. AI compresses the time required to read and classify the queue. The operator decides whether the signal deserves amplification.
The same principle applies on the return path. Closing an engineering task is not the end of the product conversation. The meaningful outcome must reach the original reporter. The loop closes where it began.
The feedback journey
The whole system is easier to understand as a journey than as an implementation checklist. Each step hands responsibility to the next without confusing automation with authority.
Seven steps, one purpose: preserve what happened, apply judgment, act deliberately, and return the outcome to the person who raised the signal.
What about Datadog, Sentry, LogRocket, and FullStory?
An informed reader should challenge this design immediately: modern observability products already capture browser errors, failed requests, user sessions, and replayable context.
They do—and that is why building a general-purpose session-replay platform inside Ezzo would be wasteful.
Datadog Session Replay connects replay with Real User Monitoring data. Sentry Session Replay reconstructs the interaction around errors and performance problems. LogRocket and FullStory provide mature replay and investigation workflows.
The overlap is real, but the responsibility is different:
| Question | Strongest owner |
|---|---|
| What did the browser do before the failure? | RUM, error tracking, and session replay |
| How many users or sessions are affected? | Observability and product analytics |
| Is this behavior a defect in our business workflow? | Product-aware human triage |
| Should this become engineering work? | The explicit escalation gate |
| Did the reporter receive a meaningful outcome? | The product’s own feedback record |
The Ezzo loop does not replace replay. It owns the decisions replay cannot make: tenant-aware access, product context, duplicate handling, approval before backlog creation, synchronization with engineering, and closure with the reporter.
If a replay platform satisfies the privacy, retention, and cost constraints, its session link should become evidence on the Ezzo report. Buy the reconstruction engine. Keep the product decision and accountability loop close to the product.
The management lesson
AI changes the economics of implementation, but it does not remove the obligation to understand outcomes.
The wrong metric is how many more changes a team can merge. The useful metric is how quickly the organization can turn an unexpected outcome into reliable evidence, a deliberate decision, corrected behavior, and visible closure.
That loop should survive imperfect reports, duplicate submissions, unavailable tools, and uncertain model output. It should use AI where AI compresses attention and preserve human authority where context changes the decision.
The teams that benefit most from coding agents will not be the teams that generate the most code. They will be the teams that shorten the distance between what the software did and what the organization learned from it.