Data Observation Toolkit
Introducing the Data Observation Toolkit (DOT)
The Data Observation Toolkit (DOT) is an open-source, community-informed toolkit capable of automated monitoring and detection of inconsistent or problematic data in a relational database. DOT is designed such that it can sit as close as possible to the point at which CHW-gathered data syncs with the server and enters the database - at which point a series of tests can be applied.
At its core, DOT uses two powerful data integrity and validation libraries—DBT and Great Expectations. DOT builds off out-of-the-box tests from both libraries to support classic data quality scenarios and common scenarios related to the community health domain - such as specific protocols for the community case management of childhood diseases (malaria, pneumonia, and diarrhea) and maternal, newborn, and child health.
DOT also provides a simplified UI as a management layer where tests can be easily configured and results are saved to a DOT database so that data integrity over time can be tracked. One of the most advantageous features of DOT is that it can be deployed to monitor multiple databases and it comes with a Docker build for easier deployment.