Baris Gultekin: belief

Transcript context

…Before going into your expectations for the year ahead, what are you seeing in terms of guardrails. And obviously one big pattern that I think is like very natural to you guys is sourcing answers back to the document or the sort of authoritative place from which it came. Beyond that though, we've got this whole constellation of different patterns, right? In terms of you can filter inputs for appropriateness, you can filter outputs, you can log things and post-process logs. You can, AWS has, I think, a really interesting new service called automated reasoning checks. where you can put a policy in, they convert with a language model, your natural language policy into a set of rules and values. And then they do like literal formal methods to ensure that at runtime, like the agent or whatever the system, whatever it gave you back, that it actually passes those like formal reasoning checks that were derived originally from a natural language policy. That's like pretty interesting and pretty cutting edge from what I've seen. But I think in most places, my sense is like the frontier model companies are doing a ton of this stuff. Anthropic has pushed this to probably farther than anyone when it comes to preventing you from using Claude to do certain things in the biosphere. But are people at the enterprise level actually doing much of it? Or are they just saying, this thing seems to work, we've got an eval set, it passes, and we'll go with that? I think the sophistication is increasing. And usually companies start with products that are more internally focused. While it's important, the bar is a little lower than something that's externally focused. At Snowflake, we offer products to check for guardrails, doing things like checking for hate speech, violence, and other violations. You can detect them and flag them and not have the model respond. But we, of course, also benefit from all of the great work that, as you called out, companies like Anthropic do on their own models as a baseline. The other thing that is also super interesting is as the models keep getting better, their adherence to instructions, of course, keeps getting better. So some of these also get codified as instructions to the agent. So not only do this and that, but also here is the policy that you need to comply with. And that tends to work quite well as well. Have you seen anything in in the interpretability realm being used for practical guardrail monitoring kind of purposes in enterprises so far?…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.