The market is flooded with "AI for buildings" and "AI for sustainability". Most of it is a thin layer on top of a generic LLM, with your logo in the corner. Ask it for your Scope 3 and it hands you a confident, plausible, wrong number.
In a marketing demo, a hallucination is embarrassing. Here, it lands in an audited report or a slide for the board. In operations and compliance, the worst failure mode isn’t "I don’t know". It’s "I know", said with confidence, about something the model invented. Because somebody signs underneath it.
How we think about it — and what we’ve broken our face learning: Sofia cites the source on every answer. The table, the metric, the document the number came from. If she can’t anchor the answer in real data, she says she can’t — instead of guessing. Guardrails per role define what each agent can and can’t do. There’s human approval on every write. And an immutable audit log on every interaction: input, context, output, model, cost.
The hard part — and the boring part where most of the engineering actually lives — is teaching a model to say "I don’t know". Retrieval, grounding, refusal, logging. None of that makes for a good demo video. Sofia v1 was a wrapper over GPT-4. v2 isn’t, and rewriting her so she wasn’t one was half the work of the last few months.
Here’s the test for any AI somebody wants to sell you. Ask it for a number. Then ask: "where did that come from?". If it can’t show you the source line, it’s guessing.
The right question is never "what can this AI do?". It’s "what does it do when it doesn’t know?". If the answer is "make it up", walk away.