Project Overwatch Logo

Small Conversations = Big DQ Impacts

Aaron Reese: 10th October 2025

Introduction

Data quality issues often emerge through seemingly simple conversations between business analysts and subject matter experts. What starts as a straightforward question about a business process can quickly reveal layers of complexity, exceptions, and edge cases that weren't immediately obvious. This dialogue demonstrates how a simple requirement about sending emails to Neighbourhood Officers uncovers multiple data quality challenges that need to be addressed.

The Conversation

SME

When a new Damp and Mould call comes in we need to send an email to the Neighbourhood Officer (N.O.)

BA

How do we know who that is?

SME

There is an officer code on the property; the system user account is linked to the officer code and we get the email from the user account

BA

Hold on... (does some typing, running a query against the database). I can see we have 1742 properties that don't have an N.O. assigned to them

SME

Ah no, some of those ARE errors, the others are flats; for those we assign the N.O. at the block level.

BA

Hold on.. (more typing). OK I can see that there are 3 blocks that don't have an N.O. assigned, of those that do, I can also see 8 flats that have a DIFFERENT N.O.

SME

Yes, when the tenant has a vulnerability warning of "No Male Visitors" and the assigned N.O. is male, we have to assign an override N.O. of the correct gender - Same if it is "No Female Visitors".

DANGER! Will Robinson

What started as a simple business requirement that is relying on a convention (Officer Code -> email address) and a 2 minutue conversation has actually uncovered at least TWELVE business events where the email either doesn't get sent or is sent to the wrong person.

  1. Officer Code is not linked to a user account (maybe the position is vacant)
  2. The linked user account is disabled (user has left)
  3. The user account does not have a gender assigned
  4. User account does not have a valid email address (maybe we only allow internal emails for this purpose)
  5. A non-flat property does not have an officer code assigned
  6. A block property does not have an officer code assigned
  7. A flat tenant gets a new vulnerability warning (no Male/Female visitors) and the gender of the block officer is of the incorrect gender, but an override officer is not assigned
  8. A non-flat tenant gets a vulnerability warning and the assigned officer is of the wrong gender
  9. A flat tenant has the vulnerability removed and the override officer is not removed from the property
  10. The user assigned to an officer code is updated with a different gender which causes one of the above conditions to be triggered
  11. The tenancy is terminated, as a result the override is no longer required
  12. A new tenancy is started with the vulnerability already in place which triggers the gender mismatch

Design Time Vs Run Time

For most organisations, These business events are going to be identified, managed and corrected when we suddently start getting disrepair claims because we haven't followed our defined process under Awaabs Law. This means lots of running about, crisis meetings, expensive report development, financial penalties and reputational damage (not to mention the actual human misery of living in a damp and mouldy home); all of which could have been avoided if we were proactive about data quality.

Project Overwatch let's you take these business events and codify them into a data quality system at design time. The cost of capturing these events (or at least the data discrepancies that they cause) is trivial compared to the cost of post-event BI reporting and remediation. For most Housing Management Systems, these checks could be written in a couple of hours, rather than a couple of months for fancy (and unnecessary) Board reports. As a result of having the checks in place before the process goes live, downstream consequences are avoided, and the organisation can focus on delivering great services to their tenants.