How do you manage the quality of data?
In the company I am working on, we have a set of devices which send us some events. These events are programatically sent by the developers but we find more than often that the values of the events are wrong or missing. This generates very important data quality problems. Sadly, I can not force the developers to do it properly, yet.
I was wondering if there is some standard method to control this problem. First I thought on develop some parser with regexps to ensure the proper syntax of the values. This works but I need more control. I would like even to verify the values with the database to enure the correctness of values in relation on the context.
I thought that may be I could use something like antlr[1] to ensure the syntax and semnatic based on a certain grammar. But I am not sure if this is an overkill.
Does any one of you faced this problem before ?, how did you solve it?
[1] - https://www.antlr.org/
Regards
2 comments
[ 4.5 ms ] story [ 17.9 ms ] thread