In January my brother and I were laying laminate flooring in his house. Each time we needed to trim a plank, we stood reverently by his table saw and incanted the familiar carpenter’s adage, “Measure twice, cut once. (Amen.)” My brother said, “It’s the damnedest thing. You can repeat and repeat a measurement, and then find out it is still wrong.” As an electrical engineer (he’s working on the 3rd edition of his book on digital signal processing), his observation comes from dozens of real-life technical projects.
In the behavioral sciences as well as in program evaluation and performance assessment we attempt to measure fairly abstract things—like social class, anxiety, customer loyalty, community need, awareness of services, and so on. Measuring these is difficult. But even in the “hard” sciences measurement is a continuous challenge. [Read more...]
You are absolutely right.
Validity of a test is more important than Reliability/ Repeatability.
“There are no such things as hard facts and figures. There are no such things as hard facts and figures. There are no such things…”
Are you not saying this as a Hard fact? Repeating it thrice for emphasizing the hardness.
Of course you are correct, Dheeraj! On one level, my message is that people shouldn’t be mystified by figures, attributing more precision to them than they really have. (Especially business profit and loss statements!) On another level my message is that “everything is relative–including the idea that everything is relative!” Like Godel’s theorem. As for the 3 repetitions, I offer them more as incantations than emphases–though I suppose those boil down to the same thing, huh?