Machine learning principles. Secure deployment

Content

Protect information that could be used to attack your model
Monitor and log user activity

In today's article, we will discuss the third section of the National Cyber Security Centre publication "Secure Deployment". It will cover the principles of protection during the deployment phase of ML systems, including protection against evasion attacks, model extraction attacks, model inversion attacks, data extraction attacks, and availability attacks.

Protect information that could be used to attack your model

Knowledge of the model can help attackers organize a more effective attack. This knowledge ranges from "open box," where the attacker has complete information about your model, to "closed box," where the attacker can only query the model and analyze its output. Many attacks on ML models depend on the model's outputs given certain inputs. More detailed outputs increase the likelihood of a successful attack.

What can help implement this principle?

Ensure your model is appropriately protected for your security requirements

The model should provide only the necessary outputs to reduce the risk of attacks. Some attacker techniques may use the model's outputs to organize attacks, even though this is technically challenging. The increasing popularity of LLMs has revived prompt injection attacks. Limiting what can be input as a prompt can help but does not eliminate the possibility of an attack. It is recommended to implement access control to restrict access levels based on user authorization.

Use access controls across different levels of detail from model outputs

It is necessary to define the level of detail for each user. It is important to present information in a way that allows them to quickly assess system behavior without accessing detailed output information. There are two popular solutions for access control: Role-Based Access Control (RBAC) and Attribute-Based Access Control (ABAC).

It is advisable to restrict access to high-precision outputs only for certain authorized users (e.g., trusted developers). For roles requiring transparency, it is better to display data in summary or graphical format. If access to precise values is necessary, consider options with lower precision or slower data transmission. Configuration settings should be evaluated based on their benefits and associated security risks, with the most secure configuration embedded in the system as the only option.

Monitor and log user activity

Analyzing user queries to the model and identifying unusual behavior helps prevent attacks. Many of them occur through repeated requests at a rate not typical for ordinary users.

With the rise in popularity of LLMs, prompt injection attacks have emerged that allow bypassing API restrictions and interacting with models in unintended ways.

What can help implement this principle?

Consider automated monitoring of user activity and logs

A key point is understanding the differences between suspicious and expected behavior. The criteria for "suspicious" behavior vary for each system and should be based on its intended purpose.

With SAF tools, you can analyze user behavior, navigate through event flows, and customize actions for critical metrics quickly and conveniently. This is clearly demonstrated in this article.

You can find previous reviews of other sections of the article below ⬇️

Secure design

Secure development