What Happened
An AI Now Institute commentary argues that AI safety debates should draw more heavily on established risk-assessment methods and account for harms to people, the environment, and public assets—not only hypothetical future risks. It also asks who decides when a system is “safe enough,” arguing that this threshold requires democratic judgment [1]. This is a policy argument, not a new regulation or technical standard.
Why It Matters to Businesses
Businesses need to distinguish legal obligations, voluntary standards, and safety research. The EU AI Act establishes requirements that depend on a system’s use and the organization’s role. The NIST AI Risk Management Framework and OECD AI Principles offer governance approaches rather than universal compliance certificates. Research and evaluations can identify risks, but they do not determine an acceptable risk threshold for every deployment.
A safety program focused only on model behavior can miss operational harms: incorrect decisions, privacy exposure, insecure integrations, excessive resource use, and the effects of errors on customers or employees. Those outcomes belong in the same risk register as model misuse and loss-of-control scenarios [1].
Kimbodo Engineering Perspective
“Safe enough” is a deployment decision, not a model property. A low-stakes internal drafting tool and an agent that can change customer records need different permissions, testing, and approval gates. We would set thresholds with the business owner, security team, and affected stakeholders, then record the evidence and residual risk behind the decision. A benchmark score or a vendor’s safety claim cannot replace that judgment.
How We Would Implement It
- Inventory each AI use case, its users, data, connected tools, and potential effects; map applicable legal requirements and a framework such as NIST AI RMF.
- Define measurable failure conditions and launch criteria for accuracy, misuse, privacy, security, and human impact.
- Evaluate the full application—not just the model—using representative tasks, adversarial tests, and checks on tool permissions and data access.
- Apply least-privilege access, human approval for consequential actions, audit logs, and rollback paths.
- Monitor incidents and performance after launch; reassess when models, prompts, data, or regulations change.
Risks, Costs and Security
The main cost is sustained evaluation and oversight, not a one-time policy document. Testing must be repeated as systems change, while logging and review must protect sensitive data. Overly restrictive controls can also make a system unusable; weak controls can shift costs onto people affected by its errors. Governance should make those trade-offs explicit, assign an owner, and provide a way to stop or narrow deployment when evidence no longer supports it [1].
Where Kimbodo Comes In
Kimbodo builds and operates this in production for businesses — see our AI Cost & Governance practice, or Analyze My AI Costs.
