Simon Willison: recommendation

Transcript context

…So, so Simon and, and Colonel, thank you. And Simon has taken us to kind of the, the end, the end game of the spaces. We've been at this for two hours and four minutes and some, and I think we, we have maybe 15 more. But I think Simon, what you said, and we are expecting for a while now, is the vision part of GPT four, right? So definitely we know that GPT four, when they released, they announced that vision come very soon. They didn't say when, then there was like a leak. Somewhere that says the roadmap for open AI is such and touch. And then they talk about vision coming maybe next year. Then we saw Bing is actually rolling out 10%, 20% of their kind of instance inside Bing Chat that understands vision. Definitely not inside a usable plugin like ecosystem we have now. And definitely, definitely not via api, which is all of what we as developers want, right? We want division capabilities. We want to be able to access this via API to build actual products with it. And I think it's a great kind of point to start talking about. What else would you guys want to see next, either from code interpreter, we've, we've mentioned many, many stuff that we'd like like complete more complete access to the, to the container, the machine. We've talked about integration with plugins. What else would we want to make this like, incredible. I would love to hear from folks on stage and then folks in the audience who wants to also kind of tell us what they want. Feel free to raise your hand, come up and let's give this like 15 more minutes and then I think it's a good closing point. Yeah, I've got a really cheap one. If Code interpreter's running off a fine tuned model, which I think it is, let us use that fine tune model directly. Let us like use it via the api, but have our own function that de evaluates code, because then I can build my dream version of code interpreter with. All of the capabilities that I want and it wouldn't cause any harm to open AI to do that, you know, bill me for the use of the model. But let me go get, let me go wild in my own Kubernetes container doing network access and whatever. That would be really cool. A hundred percent. A hundred percent. Kyle, you had the Yeah, I think, I mean the, the really cool thing, so the, the data analyst piece, like if we, if we put the multimodal capabilities together, like you see sometimes when you're, when you're doing code interpret code interpreter with, with the model is that it will sometimes hallucinate what was in your plot. Like, oh look, this is what I thought. It doesn't know. Yeah, it doesn't know what it, yeah, it's like if it has had multimodal, like could it do real analysis and, and look at, you know, the actual chart that came out of it and then come up with new analyses. Simon was talking about the code interpreter as like a fine tuned model and I did some research because I was developing on it before they released the function model. And I noticed that like the new function model somehow understands the code interpreter task way better. So I think they have the fine tune model on it. Yeah, I think we're, we're coalescing around this idea that what we're seeing right now in code interpreter and in plugins model is the, the kind of a different fine tuned model. I, I've, I have a really dumb question here, right? Like, if this model is so important and we can sniff the, the network when we make requests inside of the web app why don't we build an unofficial api? Right. It should be doable. I think there's one, one step forward swyx here is that since you can dump its results into a text file, you can start maybe fine tuning like a llama or, or a X Gen from Salesforce, one of the open source models to also give us like this behavior. I think that would be important. Yeah. Like after I finished this infrastructure project, I wanted to release like a link chain wrapper around this. So like just open source code around. This infrastructure API where it's like an implementation of code interpreter, like I am using it on my Discord board. So open AI folks, if you're in the audience, we want access via API to this model specifically cryp law review. Hey, welcome. If you have a use case for code interpreter that we haven't covered, please share with us that, and if you want to give us your thoughts about next, also do that. Yeah, amazing spaces and yeah, the use case that I played around with is creating a a video from just a, a sample photo and just playing around with it. And the point that Simon made earlier about the model, self debugging and honing in on your intent and continuing to cycle through until it fulfills that intent is really the game changer. And Simon did a really great job of you know, laying that out. It's, it's, it's magic watching it work on the simplest and, you know, I imagine more, much more complicated tasks.…

Stored transcript either side of the excerpt. The highlighted words are the published quote; the surrounding text is unedited source, never generated.