# What to measure after you automate a process in Dubai
Six numbers tell you whether an automation is working: median first reply time, the 90th percentile of that same reply time, real resolution rate (not deflection), cost per handled conversation, count of silent failures and how long each one ran before anyone noticed, and the AED value of enquiries that still got no answer. You can pull all six in one evening from a chat export and a spreadsheet. No dashboard, no BI tool, no vendor report. Most owners I do this with have been reading a vendor's containment percentage for months and have never once counted how many of those conversations ended with the customer getting what they asked for.
The green dashboard problem
An automation reports on itself. That is the whole issue. It logs what it did, not what happened to the customer afterwards, and those two things separate faster than anyone expects.
On our own stack we found two sync bugs between payment confirmation and CRM stage. Roughly AED 15,000 of paid bookings had been sitting in the CRM marked unpaid for weeks. The dashboard was green the entire time, because from the automation's point of view every step it knew about had completed. Nothing errored. Nothing alerted. The gap was between two systems that each believed they were fine.
So measure outcomes the machine cannot fake, and measure them from outside the machine.
1. Median and 90th percentile first reply, per channel
Report the median. The mean is the number that lets a bad month look acceptable, because one thread answered after three days pulls the average of a hundred rows by more than half an hour on its own.
Then put the 90th percentile next to it. After automation this is the metric that earns its keep. The median goes to seconds almost immediately, which is why owners stop looking. The p90 is where the broken tenth lives: the enquiries that arrived in a format the flow did not parse, the ones that hit a rate limit, the ones where the customer sent a voice note and the automation sat there. A median of 40 seconds with a p90 of 14 hours is not a fast system. It is a fast system with a hole in it.
Targets differ by channel because buyer behaviour differs by channel. WhatsApp: under five minutes, and realistically under one once a machine is answering, because someone comparing three clinics messages all three inside two minutes. Instagram Direct: under fifteen. A missed call: a WhatsApp back inside sixty seconds of the ring. A web form: under an hour. If you have never measured the baseline, the manual version of this count is laid out in our missed lead audit method, and it costs an evening and nothing else.
2. Deflection rate versus resolution rate
Deflection is the share of conversations that never reached a human. Resolution is the share where the customer got the answer or the outcome they came for. A conversation can be deflected and unresolved at the same time, which is the single most common way an automation report flatters itself.
The published industry range for deflection sits around 50% to 80%. Real resolution for a general-purpose bot lands closer to 65% to 70%. Narrowly scoped agents, ones that handle a short list of things a specific business actually gets asked, run above 85%. That gap between 80% deflected and 68% resolved is the part that shows up as bad reviews rather than as a red number in a report.
Deflection is easy to make look good, and here is how it happens without anybody lying. Count every conversation where no human replied as deflected, and you have counted the customer who asked twice, got a generic answer, and left. Close threads automatically after 24 hours of silence and mark them resolved. Route only simple FAQ traffic through the automation and send everything hard to a human before the machine touches it, then report on the traffic that remained.
Resolution has to be judged from the transcript, by a person, on a sample. There is no automatic way to do it honestly. Read fifty conversations and code each one: the customer got what they asked for; the customer got handed to a human and that human closed it; the customer left mid-conversation; the customer asked the same question twice. The fourth code is the one to watch. Repeat questions inside one thread are the clearest signal that an answer was delivered and did not land.
3. Cost per handled conversation
The formula is short. Build cost divided by the number of months you plan to amortise it over, plus the monthly support fee, divided by conversations handled that month.
Run it on the floor of our own published build range, AED 6,000 to 12,000. A single simple process at AED 6,000 to build, amortised over 24 months, is AED 250 a month. Add AED 1,200 a month to run and monitor it, and the fixed cost is AED 1,450.
- 120 conversations a month: AED 12.08 each
- 400 conversations a month: AED 3.63 each
- 1,000 conversations a month: AED 1.45 each
The comparison worth making is against what those same first replies cost in salary. A coordinator on AED 7,000 costs about AED 40 an hour loaded against 176 working hours. Eight minutes per conversation across 400 conversations is 53 hours, or roughly AED 2,130, which is AED 5.33 each. So the machine is cheaper at that volume, though not dramatically, and at 120 conversations a month it is more expensive per conversation than the coordinator.
That is the honest read, and it is why per-conversation cost alone is a weak argument for automating. The coordinator sleeps, takes Fridays, and answers a 03:00 Saturday enquiry on Monday morning. The difference that matters is not the AED 1.70 per conversation. It is the conversations that happen at hours when nobody is on shift.
Exclude message fees and software licences from this number and track them separately. They scale with volume, your build cost does not, and mixing the two makes the trend unreadable.
4. Silent failures and time to detect
This is the metric nobody sets up and everybody needs. Two figures: how many silent failures happened in the last quarter, and how many days each one ran before a human noticed.
The failure modes are boring and they repeat. An OAuth token expires on a Thursday evening. Somebody renames a field on a CRM form and the mapping stops writing. A vendor deprecates an API version with an email nobody read. A staff member adds a dropdown value no rule knows about. A date arrives as 03/04 instead of 2026-04-03. None of these throw an error the owner sees, because in most stacks the failure is an absence: a record that was never created, a message that was never sent.
Alert on absence, not on error. That single change catches almost all of it. If no deal has been created in six working hours, something is wrong. If payment links went out and no confirmation came back for two days, something is wrong. Errors alert themselves. Silence does not.
Then run a weekly canary, which takes four minutes. Message your own business number from a phone that is not in your CRM, at an odd hour, as if you were a customer. Check three things: a reply arrived, a record appeared in the CRM with the correct source tag, and whatever the flow is supposed to produce next actually appeared. If any of the three is missing you have found the failure before a customer did. Put it in the calendar on a fixed day. Nobody does this from memory.
5. Unanswered enquiries, priced in AED
Automation reduces this number. It does not zero it. Count the enquiries in the window that received no substantive reply at all, from any source including the ones the automation never saw, then multiply by your own close rate on answered enquiries and your average ticket.
Twenty-two unanswered enquiries, a 14% close rate and an AED 4,000 average ticket is AED 12,320 of unworked pipeline in one month. Treat it as the top of the range rather than a confirmed loss, since unanswered threads skew toward duplicates and browsers. Our working figure for a typical Dubai SMB is around AED 4,100 a month lost to slow and unanswered leads, alongside roughly AED 2,000 a month of salary time spent on work a machine should be doing.
The reason this stays on the list after automation is that new channels appear. An Instagram requests tab, a staff member's personal WhatsApp, comments under an ad, the old number nobody decommissioned. The automation covers the front door. The leak moves to the windows.
6. Payback, without the vendor arithmetic
Payback in months equals build cost divided by the sum of monthly hours saved times hourly rate, plus monthly leak recovered, minus the monthly run fee.
Run it honestly and year one usually does not pay back on hours alone. Thirty hours a month at AED 45 is AED 1,350, or AED 16,200 a year, against an AED 8,000 build, a step up from the AED 6,000 floor above once you add proper error handling, plus AED 14,400 of run fee. That is minus AED 6,200 in year one. Any ROI slide that says otherwise is counting hours nobody was actually spending.
Recovered leak is what turns the sum positive. Recover a third of AED 4,100 a month and you add AED 16,400 a year, which moves minus 6,200 to plus 10,200. So the labour saving pays the run fee and the recovered leak pays for the build. If the process you automated does not touch revenue at all, expect a longer payback and decide with your eyes open.
Metrics that look good and mean nothing
Messages sent. It measures activity, not outcome, and it goes up when the automation is confused.
Deflection or containment on its own. Covered above. Without a resolution number beside it, it is a measure of how many customers gave up quietly.
Average response time. The mean hides the tail. It is the number that makes an owner argue with the data instead of acting on it.
Uptime percentage. An automation can be 100% up and writing to the wrong field for three weeks. Uptime measures the process running, not the process working.
Hours saved, as estimated at build time. That is a projection someone made in a proposal, and it stays in reports forever without ever being re-measured. Count it again at month three from real data or drop it.
Satisfaction scores with a 4% response rate. Fifteen ratings from four hundred conversations, mostly from people at the two extremes. Read transcripts instead.
Number of automations live. Nine flows, three of which quietly stopped in March.
The evening version: 100 conversations, 90 minutes
Export the last 100 conversations from the channel the automation runs on. WhatsApp Business exports a thread as a timestamped text file: open the chat, tap the contact name, Export chat, without media. Build seven columns.
- Timestamp in
- First reply timestamp
- Human takeover timestamp, blank if none
- Outcome code: resolved by machine, escalated and closed, escalated and dropped, customer abandoned, machine gave a wrong or empty answer
- Repeat question in thread: yes or no
- Channel
- Ticket value if it converted
Then four cells. Median minutes to first reply. =PERCENTILE(range,0.9) for the tail. Resolution rate as resolved-by-machine plus escalated-and-closed, divided by 100. Deflection rate as 100 minus the count with a takeover timestamp, divided by 100. Put the two rates side by side and the gap between them is your actual answer.
At around fifty seconds a row this is 90 minutes of work. The rows coded "wrong or empty answer" are worth reading in full, because three or four of them usually share one root cause that takes an hour to fix.
What this looks like on our own business
We run the studio's enquiries on the same contour we sell. The agent replies in under a minute, qualifies, quotes, books the slot, sends the payment link and writes the deal into the CRM with its source tag, with no human touching the thread. That system currently runs the studio at roughly AED 67,600 net per month at around 5x ROAS.
The metrics above exist because we needed them, not because they read well. Absence alerting came from the AED 15,000 of paid bookings sitting marked unpaid. The weekly canary came from a token that expired on a Thursday. Reading transcripts by hand came from a period where the report looked excellent and the bookings did not move.
One side effect worth knowing about. Once every conversation carries a timestamp and an owner, per-person performance stops being an impression. You can see which of your people reply inside five minutes, which ones qualify completely, and which ones let Thursday-evening threads slide. That comes free with the plumbing, and for most owners it turns out to be worth more than the hours saved.
If you want a process wired with the alerting and the reporting built in rather than bolted on later, that is what the process automation work covers, from AED 6,000 to set up plus AED 1,200 a month. If you would rather have the six numbers above measured on your actual data before you commit to anything, a growth audit starts at AED 3,000 and the process map it produces is yours whoever builds the thing afterwards.
