AITC Comment Letter Re: Colorado, DRAFT Proposed Reg 10-2-XX: Concerning Quantitative Testing of External Consumer Data and Information Sources, Algorithms and Predictive Models
October 26, 2023
The Honorable Michael Conway, Commissioner, Division of Insurance
Mr. Jason Lapham
Colorado Department of Regulatory Agencies
1560 Broadway, Suite 850
Denver, CO 80202
Re: DRAFT Proposed New Regulation 10-2-XX: Concerning Quantitative Testing of External Consumer Data and Information Sources, Algorithms and Predictive Models Used for Life Insurance Underwriting for Unfairly Discriminatory Outcomes
Dear Commissioner Conway and Mr. Lapham:
Thank you for the opportunity to provide public comments on the proposed rule “Concerning Quantitative Testing of External Consumer Data And Information Sources, Algorithms, And Predictive Models Used For Life Insurance Underwriting For Unfairly Discriminatory Outcomes.” (“Proposed Rule”). This comment letter is submitted on behalf of the American InsurTech Council (“AITC”). AITC is an independent advocacy organization dedicated to advancing the public interest through the development of ethical, technology-driven innovation in insurance, and a modern approach to regulation that is essential to this industry’s future growth.
AITC would like to thank the Division of Insurance for the inclusive process. We appreciate the Division’s efforts to put out a working draft and to publicly work through the process. We hope the process continues, but the Division should build in more time for stakeholders to review each proposal including several weeks between public hearings and the submission of written comments. Since this is really a first of its kind process, getting it wrong will make it difficult if not impossible for any corrective effort going forward.
We’d also note that there is considerable public interest in getting this process right. Every group of stakeholders: consumers, life insurers, and third-party technology providers are expecting the Division to develop a testing framework for potential unlawful discrimination that is credible, can be relied upon to produce reliable results, and satisfies professional standards for data modeling and testing in a regulatory context.
As others have noted throughout the stakeholder process, there are significant technical and other concerns with the Division’s decision to mandate use of Bayesian Improved First Name Surname Geocoding (“BIFSG”) as the exclusive methodology to test for potential unlawful discrimination. If the Division is committed to requiring exclusive use of BIFSG, sufficient time will be needed for qualified, independent research and testing to validate BIFSG as a credible methodology that satisfies the standards needed to support regulatory enforcement. We recommend requesting assistance from the American Academy of Actuaries and/or Society of Actuaries to conduct this analysis and testing.
After testing has been completed, a reasonable implementation date should be considered. We would also suggest that additional time be allowed for companies to conduct testing on their own data. Similar to the NAIC’s Market Conduct Annual Statement (MCAS) process and the first Own Risk Solvency Assessments (ORSA), we would urge the Division to treat the first year’s data as a confidential test of the process, which should allow tweaks to report time frames and details.
We also have a number of specific concerns:
1. Exclusive Use of BIFSG. AITC understands the interest in using BIFSG methodology, but we do not believe the Division should exclude insurance carriers from utilizing other alternatives that exist currently or may be developed in the future. Prescribing only one methodology, as technology and methodologies change rapidly, would effectively prevent insurance carriers from utilizing other alternatives that may provide more accurate results than BIFSG. It is difficult to understand how that could possibly benefit consumers.
We note as well that even the Rand Corporation acknowledges serious limitations with the accuracy of its own methodology:
“The C-statistic ranges from 0.5 (no predictiveness) to 1.0 (perfect predictiveness). BISG estimates are strongly predictive of self-reported race and ethnicity for the four largest racial and ethnic groups in the U.S. Predictive accuracy is measured using the C-statistic, also called the Concordance Statistic. C-statistics for the BISG methodology are 0.94 for Asian/Pacific Islander, 0.93 for Black, 0.94 for Hispanic, and 0.93 for White.”[1]
While these comparisons are relatively highly predictable when compared to the self-reported data (which also has its own error rate), it is still significantly inaccurate– indeed outside the proposed 5% error rate for insurer investigation. This raises serious questions about the validity of BIFSG as a tool to be used for regulatory enforcement, where significantly higher standards for accuracy are required. Insurers should be allowed and encouraged to develop more accurate predictive systems to analyze their book of business.
2. Omission of Provisions Regarding the Use of Previously Assessed Tools. C.R.S. 10-13-1104.9(3)(c)(II) requires that the regulation “must include provisions establishing…the ability of insurers to use external consumer data and information sources, as well as algorithms or predictive models using external consumer data and information sources, that have been previously assessed by the Division and found not to be unfairly discriminatory.” Language implementing the statutory intent should be added to the next version of the Draft Regulation.
3. Definition of External Consumer Data and Information Source. We believe the definition of “External Consumer Data and Information Source” is both pejorative and unreflective of what would be commonly understood as an external data source. As defined:
“External Consumer Data and Information Source” or “ECDIS” means, for the purposes of this regulation, a data source or an information source that is used by a life insurer to supplement or supplant traditional underwriting factors. This term includes credit scores, credit history, social media habits, purchasing habits, home ownership, educational attainment, licensures, civil judgments, court records, occupation that does not have a direct relationship to mortality, morbidity or longevity risk, consumer-generated Internet of Things data, biometric data, and any insurance risk scores derived by the insurer or third-party from the above listed or similar data and/or information source. ECDIS does not include traditional underwriting factors.”
The proposed list is overly broad. A number of factors included have a predictive impact on mortality assumptions, and in some cases (for example consumer use of biometric data) may result in better health and life expectancy AND therefore lower rates. We’d also note that the proposed list includes a number of common factors that seemed to be intended to pull anyone who underwrites risk into this proposal.
1. Regression models. Until testing on underwriting and pricing data is conducted, there is insufficient evidence to know that regression analysis will work and whether the factors included are sufficient. As recommended above, the American Academy of Actuaries and/or Society of Actuaries should be considered to conduct independent analysis to determine the impact and accuracy of the proposed testing.
2. Insurer Testing. The proposed difference of five (5) percent to require further insurer action seems to be a random placeholder. Given the error rate for BIFSG testing, it seems likely a higher number may be necessary. Once again, this is an area where testing of BIFSG on life insurer data will provide better information.
Another issue with the required testing is the presumption that insurer ECDIS data and ECDIS data held by a third-party vendor is precisely the same; or alternatively, that both parties have access to the data held by their counterparty. This may well not be the case. If it is not the case, it does not seem possible that the testing as proposed in the Proposed Rule would produce accurate results. Again, this is something that would be determined through independent testing and analysis of BIFSG.
The testing requirements also appear to be onerous, particularly for smaller and medium sized companies that may not be able to bear the annual expense of conducting each of the required tests. This would create an unlevel playing field and disadvantage many of the companies that currently served underserved markets. A principles-based approach would benefit carriers of every size by enabling them to choose a testing methodology that best suits their own business.
Thank you for the opportunity to provide public comments on this important document. We look forward to working with you. If you have any questions, please do not hesitate to contact us at the email addresses below.
Respectfully Submitted,
Scott R. Harrison, Co-Founder
sharrison@americaninsurtech.com
JP Wieske, Co-Founder
JPwieski@horizondc.com
The Honorable Thomas Mays, Co-Founder
tmays@americaninsurtech.com
Jack Friou, Co-Founder
jfriou@americaninsurtech.com
Teri Hernandez, Co-Founder
thernandez@americaninsurtech.com
[1] https://www.rand.org/health-care/tools-methods/bisg.html