Measurement Mini Series, Part 2: Understanding reliability of measurement in People Analytics Artwork

People Analytics Deconstructed

Are you responsible for understanding an employees’ experience? Have you tried to incorporate people analytics in your organization but have struggled? Have you ever wondered what it means to have a data culture? Would you like to make more data-driven decisions? These are the kinds of discussions you can expect to hear on People Analytics Deconstructed. Co-hosts Ron Landis and Jennifer Miller are co-founders of Millan Chicago, a data science consulting company dedicated to helping organizations make the most out of their data. Each week, they will ‘deconstruct’ modern and contemporary topics in the People Analytics space.

All Episodes

People Analytics Deconstructed

Measurement Mini Series, Part 2: Understanding reliability of measurement in People Analytics

March 11, 2022 • Millan Chicago • Season 1 • Episode 10

In the second episode of a special three part mini-series about measurement, co-hosts Ron Landis and Jennifer Miller discuss how reliability of measurement is critical in People Analytics.

In this episode, we had conversations around these questions:

What is reliability?
Why is it is important to consider the reliability of measures?
What are the different types of reliability?
How do I determine the reliability of a measure?
What are some of the steps that HR professionals can take to assess the reliability of the assessments used in their organization?

Key Takeaways:

In order to measure something accurately, we must ensure that our measures are first reliable. One of the most critical questions when selecting or developing a new measure or metric is whether that measure provides consistent scores.
In the People Analytics field, the term reliability is defined as the consistency of a measure. In essence, we want to know whether what we are measuring is consistent across time (I.e., test-retest reliability), content (I.e., internal consistency), and/or raters (I.e., observer reliability).
Different types of reliability evidence require different types of data collection efforts, but the central analysis is the same. Ron and Jennifer talked about ways by which we could operationally assess the reliability of any measures that we use.
In addition, they also talked about some rules of thumb for what is "reliable enough."
At the end of the episode, Jennifer and Ron recommend steps for folks just starting out in this space all the way to the more advanced HR professional.