Andy Brownsword asks if this thing is a number:
Data validation is key when ingesting from external sources. As we can’t always be certain of data quality we inevitably find bad data which needs to be handled. Here I wanted to look at a couple of options for validating numeric data.
Here’s the scenario – we’ve got data which may have been received via a flat file or passed into our database, and it should be a numeric value. How can we weed out the valid from invalid data?
Read on for the wrong answer (at least, the wrong answer given our expectations as developers or data platform specialists), followed by a good answer.