Guidance for Customers on New York City Local Law 144 of 2021 Regarding the Use of Automated Employment Decision Tools
The NYC Department of Consumer and Worker Protection (“DCWP”) published for public comment a set of proposed rules (“Proposed Rules”) to provide guidance on interpretation of and compliance with the new law, and on November 4, 2022 held a public hearing.
The final rules have not yet been released. When the final rules are published, we will make any updates to this page. As such, this guidance is intended as assistance, not authority, based on our interpretation of the Proposed Rules, and it confers no rights or remedies.
Is Vervoe an AEDT and does this new law apply?
The new law defined an AEDT as a tool used “to substantially assist or replace discretionary decision making.” The Proposed Rules clarify the meaning of that phrase to be relying “solely on a simplified output (score, tag, classification, ranking, etc.) with no other factors considered, or to use a simplified output as one set of criteria where the output is weighted more than any other criterion in the set, or to use a simplified output to overrule or modify conclusions derived from other factors including human decision-making.”
Based on the clarification provided, the law applies to you if you use Vervoe's AI-powered automated candidate ranking functionality within your hiring process. From our perspective, we plan to be in full compliance with the law so that Employers can confidently use the data provided in the manner that best suits their needs and goals.
Is Vervoe working with an independent auditor and will we receive the results of the audits of our models?
Yes, we have contracted Holistic AI, a leading AI audit and risk management firm. Holistic AI will produce individual Employer results that will be required by the new law.
What specific testing is required for the bias audit of a Vervoe’s model?
The Proposed Rules clarified that where an AEDT generates a score for candidates (as is the case with Vervoe), the audit must calculate the average score and impact ratio for individuals in the EEOC component 1 category of race/ethnicity and gender, including the intersections of those groups. This is the approach that Holistic AI tested against. As a reference, the below sample chart was provided by DCWP as an example in the Proposed Rules.
What is required of you as an Employer of Vervoe?
As an Employer using an AEDT, in addition to ensuring that a bias audit of the AEDT was conducted within a year from the use of the AEDT, you have two primary obligations: notifying candidates who reside in New York City of the use of the tool in your process; and publishing a summary of the independent audit results and certain related information.
The Proposed Rules provide more clarity on the specifics of how firms can comply with these obligations.
- Notifying Candidates:
The Proposed Rules clarify that the requirement to notify candidates who reside in New York City 10 business days in advance of using an AEDT can be satisfied by including notice on the careers or jobs section of your website in a clear and conspicuous manner that you are using Vervoe at least 10 days prior to using in the recruiting cycle or including the notice in a job posting at least 10 business days prior to using Vervoe for that particular role.
- Please see this document on Notice Guidance. The notice must also include instructions on how to request an alternative selection process or accommodation.
- Guidance text for your use:
An automated decision tool may be used in the evaluation of candidates for the applied roles to evaluate the any of the following relevant job qualifications or characteristics:
- The skills that are required for the open job position.
- The proficiency of skills that the candidate has self-assessed.
- The proficiency of skills and role level that we can infer from the candidate’s current role.
If you would like to request an alternative selection procedure, please email HR (email@example.com) to log your accommodation request.
To request information on the type of data collected for the automated decision tool, source of such data, or our data retention policy, email HR (firstname.lastname@example.org) and the Data Protection Officer (email@example.com) or write to us at our company address XXX.
You can also find a copy of our data retention policy here (www.xyz.com/dataretention).
Please note: Samples provided in this guidance are for demonstration purposes only and not for live use. Speak with your General Counsel before implementation.
- Notably, the Proposed Rules seem to clarify that while firms are required to provide a way to request an alternative selection process, they are not required to actually provide one.
- Publishing Results:
Employers will be required to make (i) a summary of the audit results, which shall include the selection rates and impact ratios for all categories, (ii) the date of the most recent bias audit of such AEDT, and (iii) the period that the model was used publicly available on the careers or jobs section of their website in a clear and conspicuous manner. Holistic AI will provide the information required for each Employer. The Proposed Rules also clarify that posting a link to a separate website containing the required summary of results would satisfy this requirement so long as the link is also clear and conspicuous. This means that Employers will have the choice of publishing it themselves or linking to private websites hosted by Holistic AI to provide the results. The results are required to remain posted for at least 6 months after last using the model to which they apply.
Is January 1, 2023 still the date that the law goes into effect?
As of now, the law still goes into effect on January 1, 2023. Firms are required to meet the notice or publishing requirements 10 business days before they want to start directing candidates to the assessment in their recruiting processes.
13 December 2022 Update: The Department of Consumer and Worker Protection (DCWP) is working on rules for Local Law 144 of 2021 (automated employment decision tools). Due to the high volume of public comments, we are planning a second public hearing. In the meantime, we will not enforce Local Law 144 until April 15, 2023. Please continue to check this page for updates.