On September 17th, 2024 the UC Berkeley Center for Long-Term Cybersecurity (CLTC) AI Security Initiative and the CITRIS Policy Lab hosted the first annual update workshop for the project “Risk Management-Standards Profile for Increasingly Multi- or General-Purpose AI.”
The workshop gave an overview of key guidance and draft updates in the draft Version 1.1 profile material. Draft updates to the Profile, as compared with the Version 1.0 Profile release last November, include alignment with the newly released NIST GenAI Profile (NIST AI 600-1), a new draft Quick Guide to support usability, and the results of testing the profile guidance against new foundation models GPT-4o, Claude 3.5, Gemini 1.5, and Llama 3.1.
CLTC and CITRIS Policy Lab researchers working on the profile update and who presented at the workshop included Tony Barrett, Jessica Newman, Brandie Nonnecke, Nada Madkour, and Evan R. Murphy.
During this workshop, participants were asked to consider questions including:
- What changes, if any, should we make to our terminology and scope?
- What should we add regarding best practices, resources, standards, evaluations, or benchmarks for developers of GPAI or foundation models?
- Are there substantive errors or omissions in the example applications of draft Profile guidance in Appendix 4?
- How can the draft Quick Guide be modified to help make the profile as actionable as possible?
- What other key items are missing from this first annual update of the Profile?
The draft Version 1.1 profile material is publicly available, and the research team invites feedback from anyone! (Please send input or feedback to anthony.barrett@berkeley.edu or jessica.newman@berkeley.edu.)
Comments by October 17th, 2024 would be most helpful. Our team will consider input and feedback as we work towards releasing the final Version 1.1 Profile in the coming months.
