25 Jun Health IT End-Users Alliance Responds to NIST AI Standards Pilot Invitation for Input
National Institute of Standards and Technology
100 Bureau Drive
Gaithersburg, MD 20899
Submitted electronically to ai-standards@nist.gov
RE: NIST AI Standards “Zero Drafts” Pilot Project: Invitation for Input
To whom it may concern:
The Health IT End-Users Alliance (the Alliance) appreciates the opportunity to provide input on the National Institutes of Standards and Technology (NIST) Artificial Intelligence (AI) Standards Zero Drafts Pilot Project.
The Alliance brings together health information professionals, physicians, hospitals, and other front-line healthcare providers and organizations that use health information technology (IT) to ensure that policy and standards development activities reflect the complex web of clinical and operational challenges facing those who use such technologies today. By working collaboratively, the Alliance is focused on priorities for how technology can best support clinical care and operations.
The Alliance appreciates NIST’s commitment to broadening participation in the development of standards for the AI community. We agree that the need for AI standards is urgent to keep up with the advancement of AI. The development of such standards requires a wide range of expertise and perspectives to ensure the standards function properly and benefit end-users and their workflows.
AI is rapidly transforming healthcare, with developments and innovations producing promising non-clinical and clinical benefits in various settings and specialties. Still, there are concerns about how AI models are developed, trained, used, and monitored, and the significant impact AI has on healthcare operations and health outcomes. The widespread adoption of AI in healthcare requires thoughtful oversight and governance frameworks to minimize risks and ensure the appropriate, safe, and ethical use of AI. Health IT end-users are at the forefront of AI use in healthcare and are well-equipped to collaborate on common principles to ensure the proper balance between innovation and use of AI with appropriate guardrails.
As healthcare organizations become more comfortable with AI and increase adoption and implementation of such tools, developers and policymakers can work with end-users to address resource constraints and hesitancy in deploying AI tools. By incorporating end-users in the design, testing, implementation, and monitoring of AI tools, AI can be developed and tailored to organizations in ways that fit their unique needs, integrate with existing workflows, promote trust and confidence in the use of such tools, and inform organization-specific governance frameworks for end-users.
The Alliance published a consensus statement in April 2025 reflecting on the current state of AI in healthcare including principles intended to guide policymakers as they work to ensure appropriate oversight of AI tools without hampering innovation. As end-users are often brought into the development cycle for technology during implementation, the Alliance’s consensus statement highlights the need for end-users to be engaged throughout the entire AI development lifecycle. We urge NIST to prioritize end-users and the recommendations put forward within the consensus statement as NIST and other agencies contemplate how to ensure proper oversight of AI while fostering innovation.
A copy of the consensus statement is attached for your convenience. Alliance feedback on the initial topics and scopes proposed by NIST can be found below.
Documentation about system and data characteristics for transparency among AI actors
Potential Scopes: Contents of model, data, and/or system cards; standardized mechanisms and practices for formatting, presenting, sharing, and/or accessing documentation; application of existing documentation practices for securing information technology supply chains to AI systems
Transparency from AI developers on the contents of the model, data used to train the model itself, performance measurements, evaluation data, and intended use are all critical components that can help end-users understand the tools they are working with. AMA surveys of more than 1,000 physicians have found a significant increase in calls for transparency between 2023 and 2024. (https://www.ama-assn.org/system/files/physician-ai-sentiment-report.pdf) The Alliance supports NIST’s inclusion of these components, as well as standardized mechanisms and practices for formatting, sharing, and accessing this documentation. Developers, in addition to policymakers, payers, and healthcare organizations, have a shared responsibility to prioritize transparency and promote end-user trust and confidence that AI tools are safe and effective to use over time and will not result in unintended bias.
Any documentation about system and data characteristics should be tailored in a way to best suit the needs of the end-users. We encourage NIST to collaborate with end-users to determine which transparency information is most helpful to better understand the unique needs of end-users, the settings in which they work, and how to best tailor this information. NIST should also collaborate with other federal agencies to ensure transparency requirements are aligned.
With this information provided in understandable ways and at appropriate levels, end-users can better consider their disclosure and consent processes and determine the best approaches to inform patients how AI is used in their care. Additionally, transparency can serve as a mechanism to clarify liability so that potential issues related to use of AI-enabled technologies can be identified and accountability apportioned appropriately.
Methods and metrics for AI testing, evaluation, verification, and validation (TEVV)
Potential Scopes: Application of well-established TEVV methods to TEVV for generative AI; approaches for translating heterogeneous benchmark scores into meaningful scores or rankings for a given use case; methods for preventing training data from becoming “contaminated” with canonical test outputs and detecting when contamination has occurred
Developing and applying established methods to evaluate AI tools throughout development, implementation, use, and post-market surveillance is critical to ensuring a tool is successful and works as intended. It is important to ensure that AI TEVV methods also include methods to gather qualitative feedback from end-users on their experiences using the tool, in addition to numerical data, to ensure scores and rankings are well-rounded, accurate, and reflective of an AI tool’s true performance and the impact to patient care and administrative workflows. The design, development, implementation, and ongoing surveillance of AI tools, as well as TEVV methods, must include efforts to identify and address biases that surface. Throughout these processes, the end-user community must be engaged in the design, development, implementation, and post-market surveillance of AI tools, including establishing appropriate feedback mechanisms.
Maps of concepts and terminology regarding AI system designs, architectures, processes, and actors
Potential Scopes: Clarification of the “AI stack”; reference architectures or design patterns for AI systems to establish shared understanding of AI system components and their relationships
The Alliance supports the idea of developing and clarifying an “AI stack” for AI applications, meaning the layers of technology and resources used to build AI applications, including the roles, responsibilities, and processes involved in each layer throughout the AI lifecycle. Including end-users and having an understanding of their roles and workstreams in each layer of the AI stack will be crucial to ensuring an accurate full picture of AI systems and will contribute to a greater shared understanding of system components and their relationships. End-users’ roles, responsibilities, and processes are directly impacted by the introduction of any new technology or AI tool and are among the individuals most knowledgeable about how both clinical and non-clinical processes and workflows may change as a result. AI must be designed to be an integrated component of existing workflows and should avoid exacerbating administrative burden by incorporating end-user experiences and feedback into AI stacks, concept maps, and implementation plans.
Technical measures for reducing risks posed by synthetic content
Potential Scopes: A taxonomy of approaches and terms to refer to these approaches; methods and metrics for evaluating and reporting the effectiveness of such measures
Creating a taxonomy of approaches for reducing risks and terms to refer to such approaches is helpful for developers, policymakers, payers, and end-users to fully understand the risk involved with AI tools. Additionally, establishing methods and metrics to evaluate these measures is critical to ensure risk is accurately measured and accounted for. Healthcare AI requires a risk-based approach to oversight, where the level of scrutiny and validation should be proportionally accounted to minimize the disparate harm and consequences the AI tool might introduce. Any method to determine risk and measures to reduce risk should consider and reflect the unique levels of risk associated with various AI tools.
The Health IT End-Users Alliance thanks NIST for the opportunity to provide input on the process and topics chosen for consideration in the pursuit of the creation of standards that meet the end-user community’s needs, and we are committed to being a partner in this effort.