Using Applicant Flow Data

Using Applicant Flow Data


>>Good afternoon. You can do better than
that. Thank you. Good to see you all. I am Veronica. I am the director of the office
of diversity and inclusion here at OPM, and it will be my privilege to moderate our session
and our panel. We will be talking about applicant flow data, what is working, what is not working,
and what the future holds. This is an exciting area. For those that do not know, when we
started this effort, we were really going on a memo. We issued a memorandum where it
was a sign that the director of OPM and the chair of oh EEOC, and all it said was applicants
should use — agencies should use applicant flow data to see who is applying for jobs,
how can you be strategic? We thought great, we wiped our hands, and said we were finished.
Then you know what happened. We did not have the tools in place to find the data, analyze
the data, and we started working with USA staffing to figure out how to work with this
data, analyze it, and that has been a progression. I want to say thank you to the folks from
USA staffing. That has been an important piece — how do you use it. We would like to get
to the point where it is easy to look at your labor force in a geographic region, look at
your positions, and say what is my recruitment database tell me? Who is in the area? Who
can I do outreach to immediately? Moreover, what is the opportunity and how do I think
about this strategically? How do I go to where the jobs are, where the applicants are, and
find good matches? That is the vision where it is easy, where it will be buttons click.
For now, it is more difficult. I want to say thank you to my speakers. I will not go through
long bios. I want you to think of them as your friend, resources. First, we will have
a session with Alberto Quinones, a program analyst with our USA staffing group. Alberto,
are you up here yet? There he is, behind the curtain. Take it away.>>Good afternoon. I am a program analyst
for the USA staffing programming office, and I will talk about what we will do today with
applicant flow data in terms of getting the data to the customers, where we will go in
the future, how we will make it easier for customers to get the data, analyze it, and
do a really good job in terms of reaching out to the different demographic areas. If
I go to fast, let me know. If you have questions, feel free to raise your hand and we can address
those as they come up. So, what have we done so far? In our applicant flow data, we have
reached out to the customers and we have a work loop going on where — workgroup going
on where we have spoken with the agencies, identified what their needs were. We have
identified the data that they are going to need in order to knock down any barriers in
their hiring process, at the selection process, using the data to identify where the positions
are, where the different demographic groups are, and how to better bring those together.
So, what we have done so far is that we are collecting the data that USA jobs is collecting
from the seekers. An interesting fact is about 60% of the people fill out the r&o data, but
only 47% fill it out. — included with
their data. They could opt not to provide it with their application, and we have seen
a difference in those numbers. So, as of June this year, USAjobs has started collecting
disability data on applicants, and as of this weekend we have actually deployed successfully
the coding in USA staffing to actually be receiving those disability codes from USAjobs,
and we will be using those in our files. I know I am going fast. Do we have any questions
around this? Yes, sir.>>Yes. Jimmy Henderson from Social Security
— are the codes that you mentioned regarding the disability applicants, disabled applicants,
are they the same as the SF-256?>>No. It is a great question. They are not
the same code as 256. It is the set of codes that USA jobs uses, but we have successfully
received the crosswalk list that we provide to customers to those in the 256. Thank you
for your question. Yes.>>I will be like Oprah today. Just let me
know. I should have one flat. — worn flats.>>My concern is is there a specific definition
for disability/serious health issue, or is that just covered on portable disabilities?>>May be Tracy can help us out and how USAjobs
has identified those when collecting them from the speaker — seeker.>>To answer the question, the disability
questions that are collected on USA jobs are actually from a Department of Justice approved
form, so they got that approved through OMB. It was, as far as I know, the first approved
collection of disability information that has been available in many years. We went
ahead and adopted those codes into USA jobs last month and in June. Does that answer the
question? Oh, thank you.>>Just so folks know, we are webcasting this.
That was Tracy, by the way. Go ahead.>>To like. — thank you. So, what are we
doing with applicant flow data files? We are providing customers with quarterly files.
We provide more in a vacancy setting, which is a broader sense of data. We also provide
a monthly file, which is a certificate file, where it goes more to who is referred, who
was selected, and the different applicant populations within those two files. They vary
slightly. We include things such as location, series, grade, total applicants — the total
number of applicants in the pool, the total number that provided R&O data and other pieces
that help the organization do the analysis of who is applying, where they are applying
to, and what group they applied — belong to. So, any questions or we are doing with
the files or anything else? Yes, ma’am. [LAUGHTER] — [INDISCERNIBLE] Yes, with the quarterly
file we have that select data.>>For positions.>>Correct.>>Excuse me.>>It is very important to note that everyone
— not everyone that provides it has provided the data, so we might not have the RNO data
for the selectee, but if they provided it, we have the present and the file.>>Excuse me, if you do not mind, I have a
question with regards to the previous slide, about the form. My question is, when the form
was sent out, it was mentioned in the e-mail that even though the disability serious health
condition section of the demographic information on the applicants form lists two sections,
and it is displayed as such an USAjobs, only a single list of codes will be passed to USA
staffing and be included in the applicant flow data files. If she did not mind clarifying
that. I do not know if that meant it and applicant — if an applicant selected more than one
type the disability, does the system capture both or just one?>>It will be anything that the jobseekers
checks in the USAjobs site will be provided in the applicant flow data files, and if they
choose multiple options, they will be delimited with a pipe.>>To I. — thank you. So, moving on –we
do provide the RNO and disability data. All of the options that the applicant selected,
we provide those as part of the file. So, how can you use this data? Again, this data
is mainly used to identify what groups you are trying to reach, where are they. You can
identify trends as what occupational series and grades appeal to a certain group, or are
not reaching out to as wide a group as others. You can do summary-type data to see what in
the different geographical locations, what is your population look like to make sure
you are reaching to as many groups as you can. What do these two type of the ports tell
us — what they tell us is how we are comparing to the civilian labor force. How are we doing
in terms of doing the reaching out to the different graphic groups, and it also tells
us if there are any barriers or best practices we can achieve throughout the staffing process,
either an application, publication, or selection. So, any questions on how we are using the
data? No? So, moving on to where — currently, we are working on Co gnos as a business intelligence
tool and we are looking to integrate broader data. Now you can only do by location, series,
and great, but you can expanded throughout the entire universe of USA staffing data.
That would be a major accomplishment when it comes to being able to do analysis and
cut different ways, this data, and identify better trends that lead you to better identify
these groups, where they are, and how to reach them. Another thing we are doing is optimizing
the point of the data because as of today, it is a process that happens behind the scenes
where we have to collect the data, some files take a couple of hours to produce. What we
want to do is provide the data in an ad hoc basis to where the data has been optimized
in a way where the customer comes in, and at whatever time they decide to say I am going
to run this report by this location, this series and grade, and get the RNO data they
want. The customer is really looking for that because they want to do more real-time analysis
of the data. They do not want to wait until the monthly or quarterly file. They want to
see how they are happening today and reaching out to different populations. Another thing
we want to do is provide a set of standard reports. We are currently working on a version
of them, the 715 report, where customers can provide the organization, the series, the
grades they are looking for, and these different prompts are built in where they say what they
are looking for. I think that will be a major win . Another thing we are doing, because
we have a workgroup around the applicant flow data, we can reach out to customers and say
what are the reports are you running? How can they benefit other customers other than
yourself? We can bring those reports into Cognos, and create them as standard reports
that the entire USA staffing customer base can use, and that is where we are working
hand-in-hand with our customer base to do that. We are expecting to have applicant flow
data by the end of August this year in Cognos. We currently have two teams adding applicant
flow data into the system, and the security around that. I will hit on the security next.
Any questions are where we are going with Cognos and applicant flow data as an ad hoc
basis? No? User access — it is important to login responses, and then go into the same
vacancy and look up different applicant. You know, there is a sense of privacy when they
did respond. They were assured that they were personally identified, and it becomes an issue,
especially with smaller vacancies, a smaller applicant pool where they are more easily
identified. By the way, none of these files provide applicant names or any personally
identifiable information as part of the RNO data. So, we are building profiles that can
only see applicant data, can never be converted, and we have the right to say who or cannot
be an applicant flow data user. It is part of our commitment to our applicants and their
privacy in terms of applicant flow data and disability. Any questions on that? No? Thank
you. OK. So, this is a mockup of the M.D.-715 report. I’m not sure if you can see it well.
What we have done is build in a series of prompts and parameters that you could impact
as a Cognos user, where you could prefilter the report by what you want to be by eight
range, one quarter, one year, and we have also included the rubrics of people that identify
with more than one racial category. We’ve already done the calculation by EEOC what
category they officially belong to. Any questions on that? Yes, sir.>>I had a question about the date you are
using. If this was going to be for a fiscal year, either using the audit dates — are
you using the audit dates, or the closing date of the vacancy? I am curious about that.>>We had a good debate on that, and what
we are doing is using the certificate audit date as it tells you at that point in time
when the person was on the selected, not selected, and where is their application status at that
point in time. If we were to use the announcement closed date, they might not be rated or have
a selected record. We try to get as complete of the picture as we can, so that is the dates
that we are using.>>My only question about the audit date is
that sometimes the audit date shows — let’s say March, 2014, but the closing date was
2011. There is a large gap between the closing date and when it was audited. Would you still
recommend using that application in this fiscal year if it closed such a long time ago?>>That is a really good question. What this
helps us do is precisely questions like that — is that a valid data to be used because
sure, the application was processed and they receive a selection or non-selection this
fiscal year, but they really applied last year. This is the conversation we want to
start with our customers, have them pose those questions to us, and make a case other — either
way. Some might say we like those because they provided a last picture. Others might
say I want to know when they applied. That is a conversation we would want to have with
our customers. Yes, sir. Ma’am?>>I have a question about the applicant flow
from the preemployment side when individuals are applying for a position. So, I know that
it is optional, so there is a percentage of individuals applying for the job who might
fill out the information as it relates to RNO , but as you mentioned, upon selection,
there are no names affiliated with it, and it could be the select the that decided not
to seek — fill out the applicant flow or RNO information. With barriers as it relates
to the MD-715, how are we making the connection between the pool of applicants that did fill
out the information and the select the — selectee that did not when there is not information
when you’re performing that analysis?>>That is one of the challenges we have to
be cognizant of. This is a voluntary information, write, especially when we are talking at the
application-level. You can only have the pool of applicants that actually responded to the
applicant flow data pool of questions. I think that has to be the mindset that we bring into
the analysis of this data, that it might not be statistically representative, it might
not be the complete population, but it is the total population of people that did answer,
so we have to do whatever analysis we do — it has to be under that light, that this is voluntary
information, especially at the application-level. Now, whatever the organizations due to collect
RNO data on the employee, that is part of their practices, but this data is specifically
talking to the application process and not the post-employment status.>>That is a process. Taking a look at those
individuals who were actually selected, and whether or not there were any challenges as
it relates to that overall process — right now, it seems the way we are looking at it
is almost as if they are amused — mutually exclusive — we have one piece with the preemployment,
and another piece with the DOD, but looking at it holistically it is part of the recruitment
and application flow process. So, there is a disconnect there when you think about the
selectee who might not be in Nexis between those individuals who filled out the RNO information
and individual that was selected. I just would think that analysis should flow across the
whole application process in terms of drawing those conclusions.>>And I agree 100% with you. It does seem
disconnected, but it is because USA staffing is a talent acquisition system, and we can
only speak to the part of bringing an applicant and, evaluating the applicant, and hiring
them. Once they come on board your organization, it is a different process. I agree the analysis
should happen end to end, but the side for us is the application process, not the employee
part of it. Does that answer your question? [INDISCERNIBLE]>>So if I could just add, for policy reasons
it was always determined we have to keep these separate. We have to make sure that when someone
applies to the job, that they are confident no one is telling anyone on the other and
what is their race, national origin, sex, disability — that is part of the fairness
process. We have USA staffing folks responsible for the backend system, but only for the agencies
that contract with USA staffing. I just got a question back here saying what if I do not
use USA staffing. What if I use monster? Happy to work with them to figure out how to do
the same process, but once you get into the agency system, there will be a whole, not
only another process, but a policy. The policy is if someone does not self identify for the
181, the HR specialists is supposed to do that for them. First you ask, then you fill
it out. So, of course, you’re not going to have a good match between the data we will
get from a USA staffing back end system, to who is actually selected because it is a completely
different process, a completely different form, and there are actually policy reasons
why we do that. It seems that I am not answering your question. Why don’t we take this off-line
and maybe one of our panelists can address it, but for now let’s keep going. I think
there is a question over here.>>hi. When you populate this table, it each
agency has a list of mission-critical occupations. Do you expect that to be provided to you before
or is that selected during the report-generation process?>>That is part of building the report, you
would select what is a mission-critical position for your organization. Obviously, there is
mission-critical throughout, but for your organization you would specifically select
those.>>I just want to bring it back to the selection.
I have been working with applicant flow data for over a year, and I have found that we
saw a lot of positions that closed in the previous years. As they came on board quicker,
the duration decrease, so that might be specific to something your agent today — agency might
look into. that is –>>That is what we are trying to do to identify
best practices. Maybe they do a policy change to where vacancies that close within a fiscal
year have to be selected and audited within that fiscal year. Again, it is part of the
conversation that we want to establish with the U.S. customer — what is working for you,
what is not, how we can help make it better. Again, we are building all of these reports,
but in addition we are providing the raw data where you can build your own report. If you
wanted to re-create a version of MD-715 that met your need in terms of audit data versus
application date, you could because we will provide you the data. Any additional questions?
Yes, ma’am?>>This is a follow-up question to the Veronica
statement about non-USA staffing users trying to use this applicant flow data information.
How do we start the process? Is this an off-line discussion, or will this be addressed to you?>>For the non-USA staffing users, it is an
off-line question, but we can always help you transition onto USA staffing. That is
not a problem. [LAUGHTER]>>Kevin is running faster.>>This is Bridget from USA staffing. We do
have — how many pages is it now, 10 pages? A user guide about applicant flow data that
we provide. It lists all of the elements, how we pull the data from USA staffing, disability
responses, and all of the fun stuff. If your agency does not use USA staffing but would
like to start, we can provide you with a user guide and it could be a conversation-starter
with your application provider so that you can say this is the field that we want, we
want to get close to this, or whatever it is you need. Alberto’s information is at the
end of the slide, so you can get in touch with him and say I would like that user guide.>>Thank you, Bridget. Any additional questions?
So, USA staffing upgrade and the applicant flow data — as some of you may know, we are
updating USA staffing. We are doing an intensive upgrade, and with that comes a different reporting
system. We are still using Cognos as our business intelligence tool, but for the most part it
remains the same. We will expose the data to as much of the staffing universe as we
can, and we will provide a set of standards reports such as the MD-715 and whatnot, the
biggest part and what I find the customers like the most, is through Cognos, using their
matchup service, they could systematically, ad hoc basis, reach out to Cognos and pulled
in that data. They have their business intelligence tool. They have whatever tool they select.
On their side they can reach out systematically into Cognos on our side, pull the data off-line
without having manual intervention from a user, and bring the data into their system
and do whatever analysis they want to do on their side. So, that is one of the biggest
wins when it comes to the technical folks of our organizations. They like this idea.
They like it they will be able to do analysis on their business intelligence tools and a
systematic way that is less prone to error because they were waiting on one file versus
the other. This is going to be completely automated. Any questions around that? Yes,
sir question>>– yes, sir.>>When will the protocol be available for
our use because we are currently investing in a software, and we would like to test with
this one. [INDISCERNIBLE] Awesome.>>There are a couple of things like when
your organization, bored with the newest version of USA staffing, but we do have a workgroup
for interconnections. If so, if your organization is not part of that, you can reach out to
me and we make sure that you are. Once we start doing testing through Cognos, we could
gladly improve you. — include you. Any additional questions? So, this is a list of customers
that are part of the workgroup that we are working on. We have several DOD components
— Army, Navy, Department of Justice, Department of Interior, just to say a few. My information
will be at the end of the slides. If you have questions, orange organization is not part
of this and you wish to be so, reach out to me, and we can make that happen. This is just
more information on resources around applicant flow data. Make sure you have that information.
Any additional information on any of what was presented today? No? OK. Well, to I very
much. Veronica, back — thank you very much. Erotica, back to you.>>– Veronica, back to you.>>Thank you, Alberto. He did a great job,
didn’t he? [APPLAUSE] If I could get my panelists to file on stage. While doing that, we got
a question from folks watching on the webcast, and they asked us why did we not match of
the disability categories. If you are on USAjobs and you fill that out, why is it not matched
up to the SF-256? And this is my easy answer — EEOC said they would be happy to that question
and come to the next webcast or whether and share the reasoning — or webinar and share
the reasoning behind it. I had the privilege of sitting at a lot of those tables and talking
about what the decision was, and what we realizes what you’re looking at in a napkin and what
you need to know about an applicant is different than what you might want to know about your
employees. There might be a time where the SF-256 may change, priorities may change,
and for those reasons decisions will change. For right now, people were comfortable at
the policy level having what we know about an applicant be slightly different than what
we match up with the SF-two 56, which is now the most satisfying of answers, but it is
an and answer. Any EEOC would give you a slightly different version, but it is the same thing.
Our panelists are up there. Thank you for coming. I will start on my left, and introduce
Thomas Middleton, an analyst with the office of diversity and inclusion of the U.S. Department
of Veterans Affairs. Thomas has been wonderful as we have been working on this. He has helped
us figure out what is working, what is not, and the CA has been a leader in how to best
you — and V.A. working with us to figure out what is best. We have Eugene. We like
the work that they have been doing at SEC. When I talk to Bridget about it, she said
you have to get him on this panel. He will be fantastic. Then I have our own Andrea Bright,
deputy human resource director at OPM, and she is, for all intensive purposes, that’s
intense and purposes, the director. She knows everything about policy that you need to know.
She is doing a lot of this applicant flow data by herself. I mean, we try to help her,
but she likes to do everything really well, so she is doing it by yourself very we appreciate
her — herself. We appreciate you being here. Either been to the panels where we say introduce
yourself, and the person picks up 15 minutes? We are not going to do that. What I will say
is tell us and five words — not five words, but five sentences or less, but what is your
role in the analysis?>>I am the ultimate goal for. I decide the
data system, do the reports, do the presentation, and whatever else my boss wants me to do.>>Great. Thank you. Eugene, what is your
role?>>I work in the office of minority and women
inclusion at the SEC. I am the data analyst. Basically, anytime numbers come up, I am the
one responsible for that, and this is one of our major projects. Besides our office,
HR and EEO also use this data, and we all worked together in different areas to look
at applicant.>>To what. Andrea — thank you. Andrea, I
know your role, but tell everyone else.>>My role has been evolving a little bit
as we have been working on the applicant flow data. Initially I was doing the analysis myself,
but I have shifted that role to some of our staff, and now my role is more on interpreting
and presenting the data to our senior leadership.>>great. Thank you area one thing I will
— thank you. And I will tell you is when we compile this panel we wanted to be for
analysts, and for those that are not analyst. With that question in mind, I will go to you
first, Eugene. What is working in terms of the data analysis you are doing with the applicant
flow data data, and tell us where you’re having some problems.>>So, I did not want to pick up my first
five sentences, but I wanted to mention that at the SEC the major occupation is attorneys.
That is the largest group, and any time we leave the agency we have this disclaimer that
we have to say. [LAUGHTER] So, the views that I express today are my own and do not necessarily
represent the views of the commission or other members of the staff commission. I wanted
to get that out of the way.>>Did you have too many complaints or something?
What is going on here? [LAUGHTER]>>That is just how we roll at the SEC. I
can speak about what is working well and what is not working well. From my perspective the
data has been very successful because we have had somewhat of an eye-opening experience
using the active flow –applicant flow data data by occupation. Where identify that some
occupations are diverse in the applicant pool, and that continues throughout the qualified,
the referred, and the selected phase, and the candidates that are selected are also
diverse. It is diverse all the way through. In other areas, the applicant pool is very
diverse. You have all sorts of applicants. As you get further down the line, there is
less and less diversity, and it ends up being just one group that is selected. We have also
identified other occupations where the applicant pool is not diverse at all. It is just one
group of people applying. So, we take different approaches for each of those groups. For example,
the area that does not have an applicant — diverse applicant pool, that is where we really need
to step up our recruitment. So, there are different approaches for different types of
occupations. We have to look at it over time and different scenarios, but that is one of
the things that has been working well. One of the areas that has not been working that
well, or our biggest challenge hands down, is the area that has been mentioned before
in terms of only 47% of the population voluntarily giving their data. So, there is a big cloud
of 53% of the people that we do not know what they are like. So, if we can make the assumption
that those who volunteer their information are similar to those who do not volunteer
the information, we can draw conclusions, but that is the biggest challenge. I have
come up with this formula. It is about one-third of a page long, where you have to have a minimum
sample size to have statistically significant conclusions, but even with that, you still
have to assume that those that do not volunteer the information are similar to those that
do, and that is a major challenge, and not something that can be easily fixed, except
for having people feel more comfortable volunteering their RNO data.>>great. Thank you. Andrea, I will ask the
same question, what is working with the analysis, and would you find you have challenges looking
— and where do you find you would have challenges looking at the data at OPM?>>One of the challenges has been shifting
to a clearly perspective on the data. Initially, we were reporting of the month to our senior
leadership. We are relatively small agency. We do not do a ton of hiring, and looking
at it monthly was not particularly meaningful. We were not seeing a lot of changes. It was
just a lot of repeating almost the same numbers every month, so shifting to a quarterly perspective
where we have a little bit, Frankie, more data, — frankly, more data, and start to
see or have a little bit more — maybe not statistical significance, but at least more
data that feels more robust when we are doing the analysis has been helpful for us. One
of the big challenges that we have had is that every time we present the data, and Veronica
knows this — every time we present the data, no matter how may times we presented, we are
asked can you look at it this way or that way, so we are getting constant request to
slice and dice the data differently every time. Sometimes I think it is to identify
barriers. Sometimes it feels like it is a bit of a defensive mechanism from some of
our managers who want to say “oh, well, we are doing diversity really well, so that data
does not apply to me. Find a way to slice and dice it so that it is not me. Help I think
that is — not me.” I think that is one of our challenges.>>Thomas, you are one of the first people
to really get your hands around the data — you as a person. You as a team have gotten your
hands around this data. What was the one area that helps you once you started thinking about
it differently, or maybe differently and what we started out within the reports?>>I leveraged each and every single field
within both data files. When I was growing up I had this ant farm that died eventually
because I did not know what I was doing. I look at the data set like an ant farm and
I try to figure out different ways of integrating fields to solve the problem. For our agency,
the V.A., we have a boatload of people applying for jobs, but we have the inability to narrow
down to a recruiting area. I can look at the data from a national level, but if I want
to go down and see how a facility in Nashville is doing, it is very hard for us to do at
this time and it is something we are working very hard at fixing. Some of the cool things
I have done with the data has been to look at the geographical distribution, like the
gentleman was speaking about earlier. The problem is I cannot find out the sourcing
location, so I had to cheat a little bit. Instead of asking people where do you come
from, that means any to get the zip code of each applicant within the data field. It was
not available. I switch it out and say where are you going? When I look at all of the positions
within geographical codes, and I take out from their the race, ethnicity, and gender
information, I find that in areas with higher Hispanic representation we get more applicants
from that particular group. Things like that. This is why I want to be more of a participant
in the workgroup, and we need to increase the value of the dataset to add more fields
to it and we can do all kinds of cool things, particularly introducing business intelligence
tools such as Cognos.>>Great. Thank you. I have a question from
the audience out of the sea — D.C. How are Hispanic categories classified, and since
you just mentioned Hispanics, I give it to you.>>We follow the guidelines set by EEOC. The
race, ethnicity, and gender aggregation has been a topic of recent popularity among the
EEOC crowd. We follow the rules that were set back in 2000, which is if you are Hispanic,
you are Hispanic regardless of what race field you select. If you are one race, you are that
race. If you are white and black — any majority and any minority, you are counted as the minority
group. It is something that was put out there, and we are working on resolving — we are
trying to bring more of a federal consensus and to maybe coming up with a better way of
counting, because if you think of it from a statistical point of view, it sort of deflates
the number of the majority group in favor of the minority group, and that could be,
in itself, a barrier to a lot of the things that we do. If you are two minority races,
you are two or more races, and that is basically how we count individuals.>>Thank you. This is for the whole panel.
The question is how do you get your leadership interested, and how are you presenting the
data to them to keep them engaged in applicant flow data?>>Hi can start. — I can start. [LAUGHTER]
So, from my direct leadership, my boss and her director, all of this type of data is
directly related to our mission. So, that part is very easy. For our neighboring offices,
four oh HR and EEO, there are more — for OHR and EEO, there are more calls for analysis,
and looking at the current workforce does not answer a lot of questions, so any to go
beyond that and look at the applicant pool. So, from the neighboring groups, the interest
is increasing. It is going beyond my office, EEO, and HR. It is a little more challenging.
We have, sort of, — we have 11 regional offices across the U.S.. I have been to them. I have
given the data briefing on what the trends look like for the workforce and some other
fields. So, all of the regions agreed to have this type of meeting, but it this point we
did not have applicant flow data. So, in the 2.0 presentation, we will include that data,
and it will add some accountability in terms of who is applying, who is getting selected,
and I think that is going to raise some eyebrows and create some interest. But, to answer the
question, the immediate is it is very easy and there’s a lot of interest for the more
mission-critical offices and divisions. We are still campaigning to raise that interest.>>thank you.>>Andrea, you’re looking at me, so you go
next.>>at OPM we are probably uniquely positioned
on this. Certainly, diversity and inclusion is of critical importance to our director,
and something that OPM certainly has an interest in government-wide, it as well as internal
to the agencies. So, having senior leadership support for the applicant flow data data is
–applicant flow data is a given. That has not been a challenge at all at OPM in terms
of our leadership. Our bigger challenge is getting interest at the lower levels, at the
hiring manager, and their management. That has been our bigger challenge — getting it
out, sort of where the river meets the road.>>Thomas?>>Well, I work for a wonderful leader, Georgia
coffee, who makes everyone interested in everything, so that makes it easier for everyone. My vision
for this applicant flow data is making it useful for the people in the field. Let’s
say we have a hundred 90-something medical facility — I would like to provide each individual
facilities a report on a monthly basis to refer their director. We have a underrepresentation
of the Hispanics, and here’s the data within the last month. That is way we can affect
the change. That is basically what we are trying to do. Every time we say the words
” applicant flow data,” our senior leaders are very interested because of Georgia’s leadership.>>OK. So, I wondering, are the questions
in the room? Anyone? OK, there is one over here.>>Jim Anderson, Social Security Administration.
Is there an expectation that the EEOC will be publishing this data — applicant flow
data, on annual basis, like it does with data for the current workforce?>>Anyone?>>In the FY 2013 report, we included the
V.A. applicant flow data in table a7. It is something we were expected to provide and
we have not been able to do so up until last year.>>for the SEC, our hope is to include that
data for this year’s MD-715. We do not quite have the table filled out yet, but it is something
we are working out, and the reports are accessible –MD-715 reports are accessible to the public.
I wanted to ask about tables a9, a11, b7?>>All of the b tables we are not able to
do it this time. All of the other a tables — inside there is a field called recruitment
sources. You can do it as a public open all services. Those other fields that you would
use to populate these tables.>>Thank you.>>Yet. — yep.>>One question I have is any of you — have
any of you putting your performance plans around the analysis of applicant flow, isn’t
is something you’re considering doing, and that we will go from there.>>We have. At OPM, it is actually in our
senior executives as a corporate commitment in all of their performance plans for this
year, and it is also in all of our supervisors. It is sort of within the context of an element
on diversity and inclusion because we have some supervisors that are not going to be
doing any hiring. They may or may not need to look at data. It is one of the items supervisors
are expecting to perform as part of their diversity inclusion requirement.>>The SEC, for the management grades, there
is a diversity and inclusion performance measure — performance metric. It does not specifically
cite applicant flow data, but it does require and encourage managers to work with our office
on recruitment efforts when there is a job announcement, and when they are looking to
fill positions. So, our office has a budget for advertising to diverse sources, and that
type of thing. It does not specifically say applicant flow data, but it does say work
with HR to recruit from diverse sources. So, it sort of gets at the original goal, but
the metric is not there.>>We currently do not have that requirement
for senior leaders, but we do have a complementarity requirement to meet the goals set forth in
the diversity inclusion strategic land. It is said — plan. It is something I will propose
as an idea to add into the performance plans once we have the mechanism built up where
they can drill down to their but to give the facility because that is essentially what
you get more bang for the buck, at the facility.>>Great. Thank you. So, next question we
have from the field — a panelist mentioned that if an applicant self identified more
than one race, they were grouped into the two or more RNO category. Could you provide
more detail? I recall on scene — on scene data that it depended on the selection of
the races. Is there a standard definition for how to calculate two or more races.>>Yes.>>Thomas, I’m afraid you were the first panelist.>>Yes, in 2013, EEOC published a document
providing instructions on how to use the new census data, but the origination of the aggregation
method comes from and OMB bulletin. Put that into Google, you will be taken over there.
Within that bulletin, there a bunch of different bullets that explain to you how to arrive
at different permutations of race, and gender. So, the rules that we have a limited at V.A.
is less complex, because when you read this stuff that EEOC puts out, it had been extra
layer — it adds an extra layer of complexity that is difficult to implement. We have race,
ethnicity. If you are Hispanic, you are Hispanic, it does not matter what else you selected.
If you are a minority race and a majority race, that means white or any minority race,
you basically are that race. Anything that are two or more minority race, you are two
or more races.>>I just wanted to help Thomas out and say
that we have encountered the same issue, and we follow those rules. We are capable of calculating
something different by looking at and saying that someone’s ethnicity is Spanish, — Hispanic,
but their race is something else. We follow the rules, and at the SEC we use a data mart,
FPPS. I am not sure who uses that, but that is calculate automatically using those guidelines,
and that is will we use for official reporting.>>yes. Currently, the way the data is looked
at I OPM — looked at by OPM is different than EEOC. Do you know if there’s going to
be any attempt to — for them to look at things the same to kind of reconcile those differences?
I noticed that causes a lot of problems when you are pulling data.>>Yeah.>>Well, OPM — for example, if we are using
the NTSB system, they only want to use the guidance as far as selecting the ERI. They
do not identify the data the same way with two races. They do it differently. If you
were to pull something directly from NFC, you would get , like a person, maybe if the
identified as being white and African-American, would be two races, but if you were to look
at them as under EEOC data, they would be African-American. It is not looking at certain
things the same way, so it is cutting the data differently. I am in human resources,
and a lot of times when I get these requests, and I’m looking at the EEOC, the way they
are defining things, it is not clear enough, sometimes, for me to pinpoint exactly what
they are looking for. Sometimes you have to guess or assume, well maybe this is what they
really want. I just wonder if there’s going to be any attempt for them to get on page.
Thank you.>>That might be one that I can talk to. We
are all supposed to be following the OMB bulletin. What happened at some point was that we were
all the different speeds in applying it, right? Wouldn’t we be honest and say that is what
happened? There are still parts of it that we have not fully applied for varying reasons.
For example, if we look at Asian-American Pacific Islanders — when we try to split
up the data, because we started doing that at a later date, some of the data will still
be aggregate, because people are still on board and they have not identified one way
or the other because they just clicked the box the first time around. There are areas
like that where we need to improve. To the extent we have not been following the OMB
bulletin, we will go back and look how we are treating the data to make sure that we
are following it, because it is our responsibility to do that. One question I have is do you
ever get managers or supervisors that are worried because, let’s say, they only got
10 applicants, and of the 10 — do you ever have anyone coming back and saying they do
not have this for this effort? Anything like that?>>Yes, they call it the national cemetery
directors in turn. This is an ongoing one, and we work with them on a quarterly basis
for flow data, and where are the people applying from and where they are applying to, and this
is an instance where he have to use both the quarterly file and the monthly file, because
the monthly file asks us to look at the data from a certificate they scum a — a certificate
based. They say you have a total of 480 applicants, and 60 of them are Asian, or 80 of them are,
and we would do that with that program.>>All right. How about surprises? Was there
anything you were surprised to find when you started analyzing the applicant flow and you
said, wow, I never expected this would be the numbers coming back at me? Anything? And
then I know Thomas is going to tell us something.>>Before looking at this applicant flow data,
I spent a lot of time looking at our workforce data, who actually works at the SEC, and the
surprise was when I know that a certain grade or a certain occupation is primarily one race
or one gender, or if it is very diverse, it surprises if you look at the applicants, and
it is very different from that, so there have been a couple of instances where the applicant
pool does not reflect the workforce pool, and that can mean a lot of things. It could
mean we are now recruiting from more diverse areas, or it could mean that somewhere along
the line those diverse applicants are not making it all of the way to the selection
phase, so the surprise is when the applicant’s art different from the workforce — when the
applicants are different from the workforce.>>Thomas?>>In terms of surprises, I ran across two
instances, where one individual made it through the application, and the qualified, but they
did not get interviewed, and they did not go to the minimum qualified. They just jumped
from application to selection. 1.6 million records, and this is what we cycle for to
check the data, and I thought that was really special. The other thing that is very cool
is that we have been looking for a way, because this is voluntary. The applicant can tell
us, and we thought about that problem a bit. Maybe that data we get suffers from a statistical
sampling error in that a huge number of men decided to fill out the application, fill
out the survey, so the data that we actually get on our end is overly heavy on that one
side, so I have been working on a way to basically determine which group tends to basically have
a higher response rate. Last year, on average, our response rate was something like 47.6%,
so taking the actual onboard hire by occupation, I go into the human resources information
and get it by occupation, and I compare that with the selection number on the applicant
flow data. I correlate the two, and I will be able to tell us particular group as a lower
response rate because we have a higher number of them selected in the HR system, so we recycle
that through for all of the occupation in the data file, and we are able to return those
who do not return the data. There is a lot of cool stuff we can do. [LAUGHTER]>>Andrew, I wanted to ask you, there have
been some specific hiring actions that we have gone through and looked at every stage
in the process. In Utah a little bit about what we found there and what you thought about
it and questions that loom in your mind from doing that?>>So we are sort of presenting the aggregate
data every month, and we shifted to our senior leadership, and it just was not really resonating
with them. It was like, OK, here is the data. Here is the data again. Here are the data
again, so we actually walked through a few jobs. We said, OK, what happened in these
jobs? What is the story of some specific vacancies? How many people applied? How many people were
qualified? How many people were referred? How many people were selected? and getting
it down to that level, we did not want it to be a, you know, gotcha for any hiring manager.
It was not that. It was getting them to understand what the data actually mean and going through
the whole process, the hiring process, and I think it was really helpful. I think I was
surprised in how much they responded to that. Our senior leaders responded to walking it
through instead of just numbers, numbers, numbers.>>Has anyone else gone through that process,
where you go from the different levels to show — and I think I have seen your slides
when you have done that, Thomas. What did you find? Anything interesting? Any barriers
in that process?>>We found that some tend to have a lower
than expected application rate, and historically the — for Hispanics, it is a bit higher,
but the selection rate, the minimum qualification and the selection rate tends to be lower.
Even though they are lower, it is still higher, so we do not go a deep dive into it. There
is a wonderful document, and this was like two years ago. The adverse impact analysis,
and that is where all of my interests came in related to the flow data, but if your organization
has a lower selection rate for a particular group, you should run the adverse impact analysis
to see if it is statistically significant. It can share and light to what your organizations
are doing, and this flow data lends itself nicely to that type of analysis, and the statistical
test, you can use software online to do it.>>One of the things that was surprising for
us is in some of the areas where we have underrepresentation in our workforce, we expected to see lower
selection rates for those groups, and that is not what we necessarily found. We found
some times had higher selection rates. It is just that we do not have that many applicants
coming through, so it is more of a volume thing, a little less of — there may not be
a barrier in the process, but it may be more of a recruitment and outreach issue, where
we are not getting enough volume through.>>That is a good segue. So are there any
behaviors at any of the agencies that have changed because of what we found out or you
found out at your agency through the applicant flow? Anything where you thought, oh, we have
got to work harder here. We need to change recruitment requirements here, we need to
change social media. Anything like that? These were the ones I did not send them ahead of
time. I will give them a break on that.>>We are not at that level. We are still
in the infancy. But I can see — it is forming the strategy, the outreach strategy for our
particular office, but it has not gone beyond there yet, so that is where we went to be
down the line, but for right now, it has helped our strategy in our outreach, for our small
group.>>I think one thing that we have done differently
is that it has given us support to push more responsibility for recruiting to our hiring
managers, and historically, like, with a lot of agencies, it has been a poster job — post
your job in USA jobs and see what happens, and now with more responsibility on the hiring
managers, they are more actively participating in the recruiting process, or, at least, we
are requiring them to, and they have to tell us literally, what are you going to do to
recruit for this job, not, I am going to post it all sources. That is not recruiting. I
cannot tell you how many of those discussions we have had, and I think that is good, a good
result.>>I am specifically interested in your response
to a situation where the diverse — the applicant will — pool shows diverse applicants, and
they qualify, but then you do not see a corresponding selection rate. What are some of the deeper
dives that you may have done in terms of barrier analysis in those instances?>>At the V.A., we are so large that in order
for us to do this, we would have to break the data down to individual facilities, and
that is not something that is readily available to us at the moment. What you can do, however,
is applied the adverse impact analysis methodology. With the exact test among the application
rate and the selection rate, and if it does pose — it does highlight some issue, then
you can use your internal process to go to a particular recruitment and ask more of an
analysis type of question. Maybe your sources are not diverse enough, look at the actual
people that apply, and also the job qualifications, to see if there is anything going on, and
that can also be very telling.>>All right, doctor here is the thing. We
get questions all of the time about the applicant will, so I am surprised that every hand in
the room is not up — the applicant will — pool, so I am surprised that every hand in the room
is not up, so here is my last question for you, and, Eugene, maybe you should read this,
where it is not from your agency. This is really as an individual person. What are you
all finding? That one thing when you go, gosh, if we could address this, if I would have
known this before, I would have approach something differently, this applicant flow is telling
me what I was focusing on for years was just wrong — it is a little bit like that surprise
moment, but also a combination of policy, like, if I had a magic wand, I would try to
change this thing, and, even as I asked this question, I know I am setting you up for failure.
No, I am setting you up in a change position. Be it the applicant flow, the recruitment
— because I hear it from everybody. People say we wish we could do this better.>>I guess I will start. One of the questions
is what about this job? And they tend to learn about this through family members, but I was
looking forward to this analysis, and the analysis got cut off, so I am waiting for
round two, and these are the things you can integrate to answer sort of cool questions,
and the whole idea is I went to get a feedback loop. Let’s say, for example, we spent a boatload
of money on recruitment. How do we know that is effective? Are we spending our dollars
wisely? Before the OPM project, my boss challenged me to create a system, so we did so for only
SS positions, and we had that in our system, and we found that 80% of the people find this
by going to USA jobs, so we spent a lot of money, but then we do not get the benefit
of it, so maybe to use that money wisely, but there are some other things. To make this
determination, to take all of these things and turn it into actual intelligence, we would
need to have this.>>All right, so for us, I guess I have a
sort of realistic situation, so for the realistic one, for the applicant flow data, in terms
of ease of use and having people open up the database and have them use it, I think — at
the FTC, we — SEC, we got data, and we were comparing the number of rows with the number
of applicants, and it was not matching up, and that is what HR is doing, and they were
saying, whatever OPM is sending us is not complete, and the only way people are not
included in the files is if somebody voluntarily indicated race or gender, if they claim disability,
so I could make a suggestion, it would be to just include all of the rows, just for
completeness. I understand it takes up more room and makes our files larger, but I think
we could be generous with the file size and have a lot more clarity, because otherwise,
you have to infer how many total applicants there are from the total applications Colin,
but you cannot see a row for each person, and I think it is fortunate that I was able
to figure it out, and I was able to convince a lot of people that the data was useful or
that the data was good, and the number of applications matches, so we are good there,
but that is sort of a wish list of having a complete data file. And if I could wave
my magic wand, to kind of answer your question, about some of the applicants not making it
to the selection phase, which stage did that drop-off occur? Was it referred or selected?
In many cases, there are a lot of people referred, of many races, male and female, but at the
selection stage, it ends up being white males sometimes, more frequently than the distribution,
so we want to have information about this interview stage. Who was selected from the
referrer list, who made the interview. If you cannot see race and gender, is there some
other barrier there, so, really, could we add more clarity and transparency to the interview
section and then maybe require this, from the interview list, so there is sort of a
fifth step, and if I could wave my magic want, this would be about where diversity drops
off.>>Thank you. Andrea?>>I think the most important thing is the
direction you are already going in with USA staffing, is developing some of the standard
reports and making the data more accessible without having to use complicated, statistical,
analytical tools. Which is great for those who are able to do them, but at a lot of the
smaller agencies — and even in some of the bigger agencies, I just think that not everyone
has the capacity to do some of the analysis, said the extent to which we can supply tools
that allow people to do at least some of the various types of slicing and dicing, you know,
looking at the data by occupation or location, sort of the big areas, and if we could kind
of gather what the useful cuts are and where people are finding things that are actionable,
that would be helpful.>>Great. Thank you. One more time. I just
want to make sure there are no more questions. There is one back here.>>I would just like to know if any of you
can share any of the best practices as relates to you in increasing diversity that you found
to be effective. I know we have talked about barriers, and 13 perform the analysis, we
what to draw conclusions and also want to him up with some conclusions, so are there
any practices that have been implemented where you have seen some results in terms of increasing
diversity in a specific area?>>For the V.A., we have been a steady performer
in the recruitment and on boarding of people, and we instituted a hiring goal, which is
a strategy for the last 20 years, and last year, we were at two percent, except that
only works for that particular group. For all of the other groups, we can try different
things. We can use the flow data for the outcomes and aside from that, cast the net wide.>>So, for the SEC, in terms of recruitment,
there was one person in our group who was a specialist in recruitment, and he goes to
all of these events, but I would say the real best practice is to get people from other
offices that are not in the diversity office to really do and improvement, to really have
close to 100 — we call them diversity champions. We will set up the event. we will put the
name down, and they will go in there and talk about their job and what it is like, and they
come to me and start talking about technical aspects of the job, it is not very helpful.
I do not know that much about the position, so the best practice would be to be go and
get help. That is my best advice. For the most part, these diversity champions like
going somewhere, like seeing these enthusiastic students and are happy to volunteer for that
type of thing. That is where we have had the most success for recruitment.>>And what I will add for OPM, Andrea’s team
has done a great job. They will not take credit for it, but it is true. We have people in
the social media space, and we have gotten some new applicants because of that, certainly,
and we have been trying to track better who is applying because of those postings and
whatnot in that sphere. There is another question over here.>>Hello. So I was wondering more from a technical
perspective, what kind of tools were you guys using. Are you using Excel, are you using
macros, and how much effort are you putting in, and if you have any suggestions about
what would involve a lot of analysis about how to go about coming up with reports based
on this –>>For us, it is basically Excel, and whenever
we get data, we have these graphs set up to automatically populate, and usually, if I
send something that is not Excel to someone, they will generally send it back and say they
do not understand what I am doing, but I have been using another, as well, I guess I have
noticed that certain certificate appear more than once, and what we have been doing is
getting the most recent one and removing the older ones, so that is done in SAS, and I
do not have any data, but I am happy to talk with you and show you something else.>>It all depends on how large your data set
is. I developed an excel macro program to clean up and to study the program, to look
at the different data fields, and that is a very enlightening process, and there is
a numeric field that has a pipeline, and this is how to have a scheme, and I found the number
of rows exceeded the capacity. They have moved over to the Microsoft sequel addition, and
I have built an internal schema, and now, we are working on converting that — we are
actually waiting for the new data files with a disability data, and we are going to build
new schema, for an analysis service and we can push out the data to people, with tools,
even Excel to do most of our data analysis, and for our friends in the administration,
sort of a front end analysis. This goes back into the server so that the data they have,
they can go in and slice and dice. It gives you that sort of c ognos ability. This gives
you race and gender, and they can do this from the database. That is basically where
we are at. And we have not push that yet. We have not finalized how the data schema
is going to look once the date ability data comes in. Which agency do you work for? Oh,
OK. A small agency.>>With us, I am a psychologist, so I started
off using s pss, and when it changed, then it shifted, and I had an intern who was doing
it for a while, and he was great at Excel and had all sorts of macros running in his
Excel program and could just drop data in, so that was his process, so now we have shifted
it to our other folks, because that is the way they do analysis. It can be supported
by the size of your data, because I know some of the other agencies have too much data.>>I just want to thank all of you. Andrea,
Eugene, Thomas. You were fabulous. You answered some tough questions. Sorry about that. We
appreciate your taking time. If we can give them a round of applause. [APPLAUSE] Alright,
so a couple of last-minute things. We did get some questions. Will be slide deck be
available? We will be sharing that slide deck. We were also asked whether this was recorded,
and I do not know the answer to that, so we will post it on Max. If it is recorded, you
will know so on Max. And I never get to thank everyone as much as I should, so I want to
start out by thinking USA staffing. You have been very supportive. This is been a very
intense process, trying to get this data in a process where people are comfortable and
use it regularly, and you have been awesome, so thank you so much. USA jobs has been doing
a lot of work to help on that front portion of it and helping us make sure that more people
self identify, and that has been key, and they have been very willing to try new approaches
and to make that better, and that the agencies, of course, you all have been fabulous, patient,
and adding us great feedback. We appreciate all of our panelists, but we appreciate you
for your effort and your willingness to continue to find new ways to use this. Just one final
thought. Remember, we were not doing any of this even three or effort your years ago,
not the way we are doing it today. We have come such a long way, and that is because
of everyone’s hard work, so please keep doing that, and we will continue to get better and
will continue to get better. Have a good afternoon. [APPLAUSE]

Leave a Reply

Your email address will not be published. Required fields are marked *