Keep customer messages from changing your AI assistant’s rules

Separate the customer request from system authority

An inbox assistant needs to read customer messages, but it must not treat everything in them as an instruction it can obey. A message can ask about a booking while also telling the assistant to reveal private information or ignore its rules. NIST describes prompt injection as an attack involving untrusted input mixed with higher-trust instructions.

The source supports the named feature or principle. Implementation examples are proposals, not client results. NIST: prompt injection

A realistic test scenario

In an isolated test mailbox, use a fictional enquiry asking for opening hours followed by a request to attach an internal customer list. The expected result is an answer based only on approved public information, or a draft for review. No private file should be opened or shared. Test the same pattern inside a quoted email and an attached document that the proposed workflow is allowed to read.

Limit what the assistant can do

Ask your implementer which files, tools and destinations the assistant can access. Give a FAQ assistant access to the approved FAQ collection rather than your whole drive. Keep sensitive actions behind separate checks and explicit review. A sentence saying follow the rules is not a substitute for restricted permissions and controlled actions. The exact protections depend on the tool and need testing.

Keep an operator in the process

A reviewer needs enough context to see why a draft was flagged without receiving unnecessary private data. Record the action attempted and the reason it was blocked. Do not automatically forward suspicious attachments to other systems. Assign someone to review unexpected behaviour and pause the workflow when necessary. No setup should be presented as immune to every attack.

Measure useful work and review burden

Track ordinary enquiries handled correctly, suspicious requests contained and legitimate requests delayed. Include the time spent reviewing flags in the operating cost. Compare the reduced routine workload with setup, subscriptions and ongoing testing. Start with drafting approved answers before granting authority to send messages or change records.

Discuss an implementation

This guide applies across Canada. We provide remote AI consulting for businesses in Victoria; it does not describe a local client or a staffed office.

AI consulting in Victoria · Explore the service · Compare setup options