What It Takes to Build an AI That Actually Knows You
a16z
Portable context 1:01
First Person Labs builds a model of a person called a second. It gathers memories, priorities, preferences, goals, values, commitments, and intent, then shares that context with other agents. You can use it with tools like Grokbot or Codex, so a prompt like buying a flight carries your seat, timing, and travel habits with it. The aim is not another agent, but a trusted source of your personal context.
Always-on understanding 4:01
The system watches the computer’s own signals: app switches, typing, pauses, Wi-Fi state, messages, and email. It builds meaning from that stream in the background and can surface things you did not ask for, like a weather change for an event or an old thread that now matters. The goal is reactive help and, more importantly, reflective help that connects months of events to your goals and preferences.
Confidential inference 13:00
Zero data retention is only a policy promise. Confidential inference is different because it can be proved with hardware checks, signature verification, and a trusted Nvidia chip. The speaker says the goal is to make this the default for personal data. OpenAI has also started offering it. The point is that you do not just take the provider’s word. You rely on a security setup that can be tested.
Local data control 14:00
The storage model keeps data local or inside a confidential container. You can encrypt first on the device, store in the cloud, and decrypt only after the user acts. That gives a chain from local encryption to secure decryption. The system still has weak spots. Some rules are deterministic, like never keeping passwords, while others depend on model reasoning and context. The speaker says you cannot write rules for every trick, but models improve at resisting them.
Embodied access matters 26:31
They want the system to feel less tied to a phone and more like a presence you can reach anywhere. The talk turns to Jarvis, Alfred, and the Culture series, where the AI is embodied and continuous across devices. The ideal is simple: one context that follows you through screens, doors, speakers, and other outputs, while the real work now is still the understanding layer.
AI-generated summary. It can be wrong or incomplete - check anything that matters against the original.

