Gauging Pay Survey Quality - Internet & Traditional Sources

Much has been said about the Internet and its role as the great equalizer.  One of the ways that the Internet has indeed leveled the proverbial playing field - much to the chagrin of many Human Resource professionals - is by providing univeral access to salary survey information (notice that I didn't say good salary survey information).  Today, nearly any motivated employee can gain access to salary  data which suggests that he or she is underpaid - often by tens of thousands of dollars.  With employees now aggressively searching out and bringing forward their own sets of competitive pay data, the Human Resources department can find itself on the defensive with respect to the organization's pay policies.

One way to meet this challenge is by focusing on pay data quality and establishing a set of quality criteria that all pay data must meet (whether brought to the table by Human Resources or an employee) before it will be considered relative to the organization's compensation practices.  Armed with a formal policy on pay data quality, the Human Resources professional is positioned to educate the employee on the topic and have a productive discussion about the specific information at hand.

Data of questionable quality exists both on the Internet and in traditional hard-copy format, but the good news is that the same quality standards should apply regardless of the source of the pay information.  Here are some of the standards I recommend for assessing pay survey quality, presented for your consideration in developing your own policy.

Survey details its methodology.  The survey data provider should detail its process for contacting or soliciting participants, collecting and analyzing data, and checking data quality and validity.  A lack of transparency around survey process and method makes me wonder what they are trying to hide.

Survey data is collected from an independent, verifiable source.  I would consider the Human Resources or Payroll departments of an employer to be independent, verifiable sources of pay data.  Surveys based on data submitted by recruiting firms or by individual employees (both of which stand to gain by inflating pay rates) are not considered reliable by most compensation professionals.

Survey publishes participants.  A list of participating organizations, as well as a demographic profile which outlines information like their size, location, industry and other characteristics, is an important quality criteria for surveys.  If you don't know where and from whom the data is drawn, how can you determine whether it truly represents your organization's labor market? 

Survey provides job descriptions adequate for matching.  Proper job matching is the foundation for successful job pricing.  Matching by job title alone is fraught with peril given the wide variances in titling practices that exist among different organizations and industries.  Survey job descriptions should include education and experience specifications, as well as a listing of job purpose and typical responsibilities.

Survey reveals effective date of data.  Its important to know how current the data is so that its validity can be ascertained.  In today's dynamic employment market, data that is too old -- regardless of whether or not an aging factor has been applied to bring it forward -- may no longer reflect the realities of a particular job's competitive value.

Survey notes sample sizes.  How many organizations and employees are represented in a particular piece of data?  Obviously, data based on 200 employees is more meaningful than data based on 5 employees -- but without this information it is impossible to determine whether a markedly high or low pay figure is valid, or simply the skewed result of a small sample size.

With formal pay data quality policy in hand, my advice to the HR professional who is approached by an employee with their own competitive information is as follows:

1. Thank them for their efforts in bringing the data forward.
2. Explain (and/or provide a hard copy of) the organization's policy on pay survey data and quality criteria.
3. Sit down with the employee and review their information together to ascertain whether it meets the organization's criteria.

To the extent that the employee's information meets the quality criteria (and it represents a valid match for his/her job), it ought to be taken seriously and considered in light of current pay practices for the position and the particular employee.  To the extent that it does not, the occasion provides a teaching moment for the employee (although the astute HR professional may wish to go double check the data used historically to set pay practices for the position, just to ensure that the organization is standing on firm ground).

Comments

[Matt Johnson]

Very nice article. I find this to be a constant struggle with organizations. I like your comment about using the employee's inquiry as a "teaching moment".

I've been searching Technorati for HR related blogs and yours is one of the nicest I've found.

I'll keep checking back and maybe I can link to your blog from mine at johnsonhr.blogspot.com