5 Guiding Principles

State agencies should prioritize user research as an integral component in the procurement or development of automated systems. It's important to maintain the human element during the design of any service. Seek input and insights from user groups, diverse stakeholders, and domain experts to identify concerns, risks, and potential impacts associated with the system.

Automated systems must undergo pre-deployment user testing to identify potential risks and assess their intended functionality. Implement risk identification and mitigation strategies to ensure system safety and effectiveness, including addressing unintended consequences.

It's essential to confirm that the system continues to operate as intended; deviations should be addressed promptly. Adhere to domain-specific standards to ensure compliance and compatibility with industry best practices. Regularly evaluate system performance, ethical adherence, and the impact on outcomes and take corrective actions as needed.

State agencies should be prepared to halt the deployment of an automated system or remove it from use if it fails to meet safety or effectiveness standards.

Ensure that the design, development, and deployment of automated systems protect against inappropriate or irrelevant data use. Mitigate the risks associated with the reuse of data, preventing compounded harm.

GTA reserves the right to conduct an independent evaluation and report to confirm the safety and effectiveness of automated systems, including mitigation of potential harm. GTA will make evaluation results publicly available whenever appropriate, promoting transparency and accountability.

State agencies should adopt a set of ethical AI principles that prioritize fairness, transparency, accountability, and privacy in the design and deployment of AI systems for state services. Develop a set of ethical guidelines for AI system design and deployment. Users should be able to understand why a particular decision was made, building trust in the system. Assign specific individuals or teams to be responsible for monitoring AI systems for bias and corrective actions. Clearly defined accountability ensures that bias-related issues are addressed promptly.

Provide training and educational programs for agency staff about the concept of algorithmic bias and its potential impacts on decision-making processes within state services. Awareness is the first step in addressing bias effectively. Develop and implement strategies to address and mitigate algorithmic bias whenever detected, such as refining algorithms, adjusting data inputs, or retraining models. Regularly assess how AI systems affect different user groups. Understand any disparities or unintended consequences that may arise and take action to rectify them.

Carefully curate and vet the data used to train AI algorithms. Make sure the data is diverse and representative of all relevant demographic groups. This helps prevent biased outcomes caused by skewed data.

Continuously monitor AI systems to detect and rectify biases where they emerge. Regular assessments are essential to maintaining fairness and effectiveness over time.

Establish clear guidelines for data governance to maintain integrity and privacy.

Prioritize robust security measures and transparent data handling practices.

Ensure data accuracy, minimize storage, and dispose of obsolete data.

Maintain compliance with data protection laws, conduct regular audits, and involve the public in the decision-making process.

Ensure that individuals are informed about the use of automated systems and understand how these systems contribute to outcomes that can affect them.

Encourage designers, developers, and deployers of automated systems to provide plain language documentation that is easily accessible to the public. This documentation should include clear descriptions of system functionality and ownership, the role of automation, and explanations of outcomes.

Require that notice regarding the use of automated systems is kept current, and individuals impacted by the system should be notified of significant use cases or key functionality changes.

Ensure that individuals have access to information explaining how and why outcomes that affect them were determined by the automated systems, even when these systems are not the sole contributors to the outcome. Mandate that automated systems provide technically valid, meaningful, and useful explanations to affected individuals, as well as operators and stakeholders who need to understand the system. The level of detail in these explanations should align with the level of risk involved.

Promote the publication of summary information about automated systems in plain language. Assessments of the clarity and quality of notice and explanations should also be made public whenever possible to enhance transparency and public trust.

State agencies should establish and adhere to policies that emphasize human responsibility and ownership of the outcomes produced by AI systems used in state services. AI systems should not operate in isolation. State agencies should ensure that humans retain control over the operation of AI systems and that human decision-makers remain responsible for the final decisions made with the support of AI.

State agencies should prioritize transparency and accountability in the deployment of AI systems. Agencies should retain clear records of AI system use, their objectives, and the roles of individuals overseeing and interacting with these systems. Agencies should mandate that AI systems be designed and used in accordance with ethical principles that prioritize fairness, transparency, accountability, and privacy. Ethical considerations should be an integral part of AI system development and use.

Clearly define roles and responsibilities for individuals involved in AI system implementation. This includes specifying the duties of AI system operators, data stewards, and decision-makers.

Encourage collaboration between humans and AI systems to enhance decision-making processes. AI should be viewed as a tool that complements human expertise rather than a replacement for human judgment.

Promote user education to ensure that individuals interacting with AI systems understand the capabilities and limitations of these technologies. Users should be aware of how AI contributes to outcomes and that humans remain responsible for those outcomes. Agencies should invest in training and development programs to equip their staff with the skills and knowledge necessary to effectively use AI systems and make informed decisions.

State agencies should retain ownership and control over the data used to train AI models and the models themselves. This ownership ensures that AI systems serve the agency's mission and values.