How Much Does AI Already Know About You and Your Family Office?
Safeguarding privacy is essential for family offices – so it was shocking when a senior executive learned how much a trial artificial intelligence model “knew” about the family. The source wasn’t a data leak, but a family member using a free AI app as a therapist, sharing sensitive details with a tool that could retain and learn from interactions. Depending on configuration, AI can store, connect, and infer information, creating privacy and security risks if not tightly governed. As “agentic” AI – systems that act autonomously – emerge, the risks grow.
Why governance must come first in the family office
Family office networks weren’t built for AI’s speed and data-gathering power. Files may be scattered across drives, inboxes and apps. While people “just know” where sensitive data lives, an AI agent with broad permissions will access everything allowed. Before introducing AI, catalog what data exists, where it sits, and who can access it. Then:
- Decide which tools to use and set a clear policy.
- Use a sandboxed dataset and tag sensitivity.
- Start in “read-only” mode, restrict connectors and disable model learning.
- Set retention/deletion rules and test access boundaries.
Dynamic risks require setting new parameters
LLMs are rapidly evolving, and Agentic AI will soon act on prompts, such as booking travel. These capabilities may require access to sensitive data – passport details, birthdates, inboxes. Family offices must require explicit approvals for payments, bookings, and transfers of personal or HR data, and maintain strong audit trails.
Educate your office team about using AI – and set clear guidelines
AI can streamline work, but only if everyone knows what not to share. Assume all inputs may be discoverable. Prohibit entering account numbers, medical histories, personal identifiers, and proprietary or HR files. If you wouldn’t want it public, don’t put it in a prompt.
Align personal and professional behaviors
Standardize approved tools, monitor for unapproved AI assistants, and hold vendors to the same standards – because third parties may expose your data inadvertently.
Rick Barragan is the Managing Director,
Los Angeles Market Manager, for
J.P. Morgan Private Bank.
[email protected] | (310) 860-3658
privatebank.jpmorgan.com/los-angeles
Source: “How much does AI already know about you and your family office,” Ileana van der Linde, head of cyber advisory, April 14, 2026
