Data Quality Part 1: What Do We Mean by Data Quality?

Posted: April 17th, 2023

Authors: Gene Y. Aditya S.

When making a measurement, we are looking to answer a question. We want and need our measurement to be robust at the levels that address the underlying question. That concept seems simple and obvious, but it gets into the weeds very quickly. We (Gene and Aditya) are going to spend the next 6 articles trying to unpack this simple but twisty subject. We’re going to use some silly examples, some real-life stories, and try to make this topic a little clearer and a little more accessible.

For the purpose of this article series, we are talking about how to structure a “small” measurement event. (Small in terms of the number of measurements.) A large data set lends itself to all sorts of statistical data analysis (means and modes and medians and ranges), estimates of uncertainty and variability, and perhaps even trends and correlations. Small data sets, not so much, or not at all. For this situation, the data need to be “externally” qualified and supportable; that is, any assessments of data quality need to be done by the use of indicators other than the measurement itself. A good example of a small measurement event is a stack test. Stack tests are typically three runs, completed all in one day. There are only three results per pollutant. We don’t know the true value for our measurement, and we don’t have enough data to do an in-depth statistical analysis. Any indication of data quality must come from outside these three runs. Outside measurements might include analysis of a known standard, a spiked sample, blanks of various composition, multiple analysis, and/or serial dilutions. Procedurally, we must use tools with known and acceptable performance. These indicators become the components of a data quality discussion. The components we’re discussing here are how we define the measurement program to answer the underlying question. When performing a small test, it’s necessary to set up and define “data quality objectives” (DQOs). You’re probably asking yourself: Huh, what does that mean? To define DQOs, we need to select a set of external measurements, specifications, and/or characteristics that will assess measurement performance and improve the likelihood that we get data that can address our underlying question. And again: Huh, what does that mean?

How about an example: Let’s say I want to build a deck in the yard behind my house. We’ll use several specific activities about this deckbuilding to demonstrate different approaches to measuring and determining length or distance.

First, I need to know how long an extension cord I need to get to the furthest point on the deck so I can use my saw and my sander. I can pace off the distance to the nearest plug, it’s 8 steps, my stride is about 5 feet, I need a 50‑foot extension cord.

Next, I want to buy the decking material. So, I need to know the size of the deck. I drop a rock at the extreme corners, and I grab a tape measure, and I roughly measure the size of the deck. It’s going to be 22’ x15’. I only measure to the nearest foot because I’m just specifying my material buy list. And I’m going to buy extra because I know I’m going to make a mistake or find a bad board, so I don’t need an “exact” number.

Now, I’ve got the material and I need to start cutting. I have a flooring pattern in mind, so I now need to measure (and cut) to the nearest 1/8” (or thereabouts). Same tape measure, more careful use. Mark and cut.

Finally for this discussion, I want the gaps between the boards to be pretty close to consistent. For this, I could use a caliper, or maybe a thickness gauge.

I measured length, distance, and width using three different tools and 4 different procedures. Do the measurements guarantee a perfect deck? Maybe. It depends on the quality of the measurements, which in turn depends on the tools and procedures used to make them. And how did this address data quality or DQOs? Each of my measurements (extension cord length, buying material, etc.) had a different underlying question (does the cord reach, did I buy enough material, etc.). Since the questions were different, the data quality requirement for each is different as well. So we used different tools and different procedures (approximate by eye, use a tape measure roughly, etc.).

Next time: The Components of Data Quality. Until then, feel free to contact either of us: