app / likes
Spark
“Um, the only thing I like out of, out of Codex is the, is like Spark and like yeah.”
Public evidence record
Shawn Wang
Host · Latent Space
- Claims
- 500
- Episodes
- 64
- Shows
- 1
- Named items
- 20
Books, apps, and tools
The evidenced stack.
app / uses
Bank of America
“Yeah. I got, I got the tool. Uh, what, like, I hate, I use Bank of America. I hate bank, I hate the app. Mm-hmm. I hate the web. All banking websites just horrible.”
other / likes
Wiki approach
“I like the, the Wiki approach. Uh, my, I’m actually like, uh, you know, obviously I spent some my time at cognition, which, uh, you, you know very well.”
person / uses
Ted Chiang
“You were very excited because I read Ted Chiang over the holidays and I was very inspired by this short story called Understand, which apparently is, like, pretty old.”
app / uses
Overcast
“I used to use Overcast. So it would just link to the Overcast page.”
other / uses
Snip
“By the way, we use the snip count as a proxy for popularity, right? Because we have download counts, but for example, platforms like Spotify re-host our MP3 file.”
other / recommends
The Peel
“I think I strongly recommend Jack Bridger's Scaling DevTools, as well as Turner Novak's The Peel.”
other / recommends
Scaling DevTools
“I think I strongly recommend Jack Bridger's Scaling DevTools, as well as Turner Novak's The Peel.”
app / uses
AI News
“So that's what basically I use AI News for. I have a lot of prompts and a lot of steps and a lot of criteria and O1 just kind of checks through each kind of systematically.”
tool / built
Bolt.new
“It's funny because I built on top of the fork of Bolt.new that already has the multi LLM thing.”
other / uses
XML
“I use XML in other models as well, and it's just a really nice way to make sure that the thing that ends is tied to the thing that starts. That's the only way to do code fences where you're pretty sure example one start, example one end, that is one cohesive unit.”
app / uses
DevIn
“I used to tell people go to the DevIn demo and look at the four things that they offer and say each of those things is a startup.”
Claim ledger
What Shawn said.
62 transcript-backed records
01 / uncertainty
I, I don’t know, I don’t know, uh, you know, it’s, uh, people va-variously refer you as like CEO or, or, uh, I don’t know what that, that, that said previous role at Microsoft was.
“I, I don’t know, I don’t know, uh, you know, it’s, uh, people va-variously refer you as like CEO or, or, uh, I don’t know what that, that, that said previous role at Microsoft was.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
02 / uncertainty
Do you guys use stack diffs? I don’t know if, uh, that’s a, like, a merge queue stack diff type of thing.
“Do you guys use stack diffs? I don’t know if, uh, that’s a, like, a merge queue stack diff type of thing.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
03 / uncertainty
Uh, you, you, you just have a lot of comments. Uh, you, you, uh, I don’t know where you wanna start.
“Uh, you, you, you just have a lot of comments. Uh, you, you, uh, I don’t know where you wanna start.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
04 / uncertainty
I’m also calling back to there’s this like levels of EGI I don’t know if Opening Eye is still talking about this, but they used to talk about five levels of EGI and one of it was like, oh, it’s like an intern coding software patient.
“I’m also calling back to there’s this like levels of EGI I don’t know if Opening Eye is still talking about this, but they used to talk about five levels of EGI and one of it was like, oh, it’s like an intern coding software patient.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
05 / uncertainty
You make a pipe errors versus the standard, standard out. I don’t know. Okay. Whatever.
“You make a pipe errors versus the standard, standard out. I don’t know. Okay. Whatever.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
06 / uncertainty
There’s a lot of customization, but there’ll be multiple unicorns just doing this as a service. I don’t know.
“There’s a lot of customization, but there’ll be multiple unicorns just doing this as a service. I don’t know.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
07 / uncertainty
I was briefly part of the, that, that world is it salt? I don’t know. It’s actually really hard for humans to agree on what revenue is.
“I was briefly part of the, that, that world is it salt? I don’t know. It’s actually really hard for humans to agree on what revenue is.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
08 / uncertainty
Like, which it’s hard to tell because you put out the end results, but we don’t know the inputs that go into it.
“Like, which it’s hard to tell because you put out the end results, but we don’t know the inputs that go into it.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
09 / uncertainty
Your partnerships, uh, I mean, I don’t know if you ever hit of partnerships, but this is gonna be a bonanza for anyone on to do deals.
“Your partnerships, uh, I mean, I don’t know if you ever hit of partnerships, but this is gonna be a bonanza for anyone on to do deals.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
10 / uncertainty
Um, and I I, we have like Claude Coworks, it might be a GI, I don’t know if we, we have, uh, uploaded it yet, but one of the sessions was like a, like a Claude cowork thing.
“Um, and I I, we have like Claude Coworks, it might be a GI, I don’t know if we, we have, uh, uploaded it yet, but one of the sessions was like a, like a Claude cowork thing.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
11 / uncertainty
Uh, people are 10 x saying, everyone’s saying, you know, uh, maybe engineers are out of a job. I don’t know.
“Uh, people are 10 x saying, everyone’s saying, you know, uh, maybe engineers are out of a job. I don’t know.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
12 / uncertainty
I don’t know if you have any predictions that you, that you like to keep, you know, like, uh, one, one way to do this is you have your tests whenever a new model comes out that you run, uh, what’s something that you’re, you’re not quite happy with yet.
“I don’t know if you have any predictions that you, that you like to keep, you know, like, uh, one, one way to do this is you have your tests whenever a new model comes out that you run, uh, what’s something that you’re, you’re not quite happy with yet.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
13 / uncertainty
Like one of my, uh, so I interviewed ETA who was on, who was on that team. Uh, and he was like, yeah, I, I don’t know how they work.
“Like one of my, uh, so I interviewed ETA who was on, who was on that team. Uh, and he was like, yeah, I, I don’t know how they work.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
14 / uncertainty
Uh, I don’t know if you’ve, you’ve any conversations with him about anything in particular on AI building and AI infra.
“Uh, I don’t know if you’ve, you’ve any conversations with him about anything in particular on AI building and AI infra.”
- Speaker
- Shawn Wang
- Episode
- Artificial Analysis: Independent LLM Evals as a Service — with George Cameron and Micah-Hill Smith
- Publisher
- Latent Space
15 / uncertainty
Uh, so that’s the, that’s the last of the proprietary art numbers, I guess. I don’t know how you sort of classify all these.
“Uh, so that’s the, that’s the last of the proprietary art numbers, I guess. I don’t know how you sort of classify all these.”
- Speaker
- Shawn Wang
- Episode
- Artificial Analysis: Independent LLM Evals as a Service — with George Cameron and Micah-Hill Smith
- Publisher
- Latent Space
16 / uncertainty
I don’t know. Maybe it might be though. You put it like a JSON field, say, say confidence and maybe it spits out something.
“I don’t know. Maybe it might be though. You put it like a JSON field, say, say confidence and maybe it spits out something.”
- Speaker
- Shawn Wang
- Episode
- Artificial Analysis: Independent LLM Evals as a Service — with George Cameron and Micah-Hill Smith
- Publisher
- Latent Space
17 / uncertainty
Would that be the other way around in a normal capability environments? I don’t know.
“Would that be the other way around in a normal capability environments? I don’t know.”
- Speaker
- Shawn Wang
- Episode
- Artificial Analysis: Independent LLM Evals as a Service — with George Cameron and Micah-Hill Smith
- Publisher
- Latent Space
18 / uncertainty
We don’t know for a fact that it’s like eight runs and then with the LM judge on top.
“We don’t know for a fact that it’s like eight runs and then with the LM judge on top.”
- Speaker
- Shawn Wang
- Episode
- Artificial Analysis: Independent LLM Evals as a Service — with George Cameron and Micah-Hill Smith
- Publisher
- Latent Space
19 / uncertainty
There’s a little bit of debate over the accuracy of TileBench. I don’t know if you’re clued in to what’s going on.
“There’s a little bit of debate over the accuracy of TileBench. I don’t know if you’re clued in to what’s going on.”
- Speaker
- Shawn Wang
- Episode
- Artificial Analysis: Independent LLM Evals as a Service — with George Cameron and Micah-Hill Smith
- Publisher
- Latent Space
20 / uncertainty
All these nuances, like, once you dig into benchmarks, you’re like, I don’t know how anyone believes the numbers on all these things.
“All these nuances, like, once you dig into benchmarks, you’re like, I don’t know how anyone believes the numbers on all these things.”
- Speaker
- Shawn Wang
- Episode
- Artificial Analysis: Independent LLM Evals as a Service — with George Cameron and Micah-Hill Smith
- Publisher
- Latent Space
21 / uncertainty
I don’t know if you have anything to add there. Or we could just go right into showing people the benchmark and like looking around and asking questions about it.
“I don’t know if you have anything to add there. Or we could just go right into showing people the benchmark and like looking around and asking questions about it.”
- Speaker
- Shawn Wang
- Episode
- Artificial Analysis: Independent LLM Evals as a Service — with George Cameron and Micah-Hill Smith
- Publisher
- Latent Space
22 / uncertainty
Um, and the cost is not, it's not cheap, but I mean, we're in New York city right now and, uh, I don't know, I paid $8 for a coffee this morning, so, uh, the coffee is cheaper in Zurich than the New York city.
“Um, and the cost is not, it's not cheap, but I mean, we're in New York city right now and, uh, I don't know, I paid $8 for a coffee this morning, so, uh, the coffee is cheaper in Zurich than the New York city.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
23 / uncertainty
I think I'm just trying to make sure that I'm not making false predictions. Because, like, you can predict that things will be better generically, but how?
“I think I'm just trying to make sure that I'm not making false predictions. Because, like, you can predict that things will be better generically, but how?”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
24 / uncertainty
I don't know how big the prompt distribution needs to be because you are literally catering to everyone.
“I don't know how big the prompt distribution needs to be because you are literally catering to everyone.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
25 / uncertainty
I don't know that they do tree search. They never said they do. It's implied. Yes.
“I don't know that they do tree search. They never said they do. It's implied. Yes.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
26 / uncertainty
The web is increasingly closing to the bots and the scrapers, Twitter, Reddit, Quora, Stack Overflow. I don't know what else.
“The web is increasingly closing to the bots and the scrapers, Twitter, Reddit, Quora, Stack Overflow. I don't know what else.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
27 / uncertainty
We don't know what the small model size is, but yeah, it's probably in the double digits or maybe single digits, but probably double digits.
“We don't know what the small model size is, but yeah, it's probably in the double digits or maybe single digits, but probably double digits.”
- Speaker
- Shawn Wang
- Episode
- Latent.Space 2024 Year in Review
- Publisher
- Latent Space
28 / uncertainty
Because I think there's a lot of doubt or questions about where ML engineering stops and AI engineering starts.
“Because I think there's a lot of doubt or questions about where ML engineering stops and AI engineering starts.”
- Speaker
- Shawn Wang
- Episode
- Latent.Space 2024 Year in Review
- Publisher
- Latent Space
29 / uncertainty
Yeah, I think Steven Johnson, I think he's on your team. I don't know what his role is.
“Yeah, I think Steven Johnson, I think he's on your team. I don't know what his role is.”
- Speaker
- Shawn Wang
- Episode
- How NotebookLM Was Made
- Publisher
- Latent Space
30 / uncertainty
I would love, like, a, you know, like when you play Baldur's Gate, like, you roll, you roll like 17 on Charisma and like, it's like what race they are. I don't know.
“I would love, like, a, you know, like when you play Baldur's Gate, like, you roll, you roll like 17 on Charisma and like, it's like what race they are. I don't know.”
- Speaker
- Shawn Wang
- Episode
- How NotebookLM Was Made
- Publisher
- Latent Space
31 / uncertainty
I'm not sure that the Singapore government is aware that safety sometimes is a bad word in some AI circles because their work is associated with censorship.
“I'm not sure that the Singapore government is aware that safety sometimes is a bad word in some AI circles because their work is associated with censorship.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
32 / uncertainty
I don't know how we get them over the hump of going into industry and being successful engineers and I fear that we're going to create a whole bunch of certificates that don't mean anything.
“I don't know how we get them over the hump of going into industry and being successful engineers and I fear that we're going to create a whole bunch of certificates that don't mean anything.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
33 / uncertainty
I think there's a key term of sovereign AI as well that's kind of going around. I don't know what level this is at.
“I think there's a key term of sovereign AI as well that's kind of going around. I don't know what level this is at.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
34 / uncertainty
You want to sprinkle LLMs into a database. Because we know how to scale systems, we don't know how to scale agents that are quite hard to be reliable.
“You want to sprinkle LLMs into a database. Because we know how to scale systems, we don't know how to scale agents that are quite hard to be reliable.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
35 / uncertainty
I don't know if it's quotable, but you said you had something from Vercel, from Malte.
“I don't know if it's quotable, but you said you had something from Vercel, from Malte.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
36 / uncertainty
Yeah, yeah, I'm a little bit scared to fine tune especially for vision, because I don't know what I don't know for stuff like vision, right?
“Yeah, yeah, I'm a little bit scared to fine tune especially for vision, because I don't know what I don't know for stuff like vision, right?”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
37 / uncertainty
Maybe they're going to run L1 for five hours instead of five minutes, and then suddenly it works. So, I don't know.
“Maybe they're going to run L1 for five hours instead of five minutes, and then suddenly it works. So, I don't know.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
38 / uncertainty
Like, I always want to eval the eval. I don't know if that ever came up. Like, sometimes the evals themselves are wrong, and there's no way for me to tell you.
“Like, I always want to eval the eval. I don't know if that ever came up. Like, sometimes the evals themselves are wrong, and there's no way for me to tell you.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
39 / uncertainty
I don't know. I don't know if I got an invite. No. I can't just talk to you. Yeah, like, and then there was some speculation around October 1st.
“I don't know. I don't know if I got an invite. No. I can't just talk to you. Yeah, like, and then there was some speculation around October 1st.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
40 / uncertainty
I guess the best way to go through this, you know, you picked out 58 techniques out of your, I don't know, 4,000 papers that you reviewed, maybe we just pick through a few of these that are special to you and discuss them a little bit.
“I guess the best way to go through this, you know, you picked out 58 techniques out of your, I don't know, 4,000 papers that you reviewed, maybe we just pick through a few of these that are special to you and discuss them a little bit.”
- Speaker
- Shawn Wang
- Episode
- The Ultimate Guide to Prompting
- Publisher
- Latent Space
41 / uncertainty
Is there any well known brand that People can link to, uh, you know, I know about like AI influencers, like on Instagram or AI wrappers, but I don't know about brand, uh, identities.
“Is there any well known brand that People can link to, uh, you know, I know about like AI influencers, like on Instagram or AI wrappers, but I don't know about brand, uh, identities.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
42 / uncertainty
I was actually, we just did an interview with Yi Tay from Reka, I don't know if you're familiar with his work, but also another encoder-decoder bet, and one of his arguments was actually people kind of over-index on the decoder-only GPT-3 type paradigm.
“I was actually, we just did an interview with Yi Tay from Reka, I don't know if you're familiar with his work, but also another encoder-decoder bet, and one of his arguments was actually people kind of over-index on the decoder-only GPT-3 type paradigm.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
43 / uncertainty
I don't know, you're way better at prompting than me, so I wanted to capture how you prompted as well.
“I don't know, you're way better at prompting than me, so I wanted to capture how you prompted as well.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
44 / uncertainty
I don't know if we Okay, do you want to pick a multiple voices, emotion we also have Chinese language.
“I don't know if we Okay, do you want to pick a multiple voices, emotion we also have Chinese language.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
45 / uncertainty
I'm excited for OpenAI phone. I don't know if you would buy an OpenAI phone. I mean, I'm very locked into the iOS ecosystem.
“I'm excited for OpenAI phone. I don't know if you would buy an OpenAI phone. I mean, I'm very locked into the iOS ecosystem.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
46 / uncertainty
Hey, this is our long-awaited one-on-one episode. I don't know how long ago the previous one was.
“Hey, this is our long-awaited one-on-one episode. I don't know how long ago the previous one was.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
47 / uncertainty
Because I think that one of the burning rumors is, I don't know, nothing from Scott, I haven't talked to him at all about this, even though he's very friendly.
“Because I think that one of the burning rumors is, I don't know, nothing from Scott, I haven't talked to him at all about this, even though he's very friendly.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
48 / uncertainty
We still need to have, at the architecture level, some kind of variable inference length thing that lets you actually think in latent space, like you're talking about. I don't know if there's any papers that you're thinking about.
“We still need to have, at the architecture level, some kind of variable inference length thing that lets you actually think in latent space, like you're talking about. I don't know if there's any papers that you're thinking about.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
49 / uncertainty
I don't know if you have a view on it, but it's just, like, it should be known to be hard.
“I don't know if you have a view on it, but it's just, like, it should be known to be hard.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
50 / uncertainty
I don't, unfortunately, I don't know enough to push back on this, but on the surface of it, it seems to make sense.
“I don't, unfortunately, I don't know enough to push back on this, but on the surface of it, it seems to make sense.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
51 / uncertainty
I don't know if you know, the post money valuation, or the pre money valuation is public.
“I don't know if you know, the post money valuation, or the pre money valuation is public.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
52 / uncertainty
I, I, I don't know if you have commentary on whether this is obvious to you or this is the, this is the way, or they'll just be, they'll coexist.
“I, I, I don't know if you have commentary on whether this is obvious to you or this is the, this is the way, or they'll just be, they'll coexist.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
53 / uncertainty
I don't know how much, how many evals you want to do, right? Like, I do think Andre Capalti mentioned that.
“I don't know how much, how many evals you want to do, right? Like, I do think Andre Capalti mentioned that.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
54 / uncertainty
Sorry, I, I should have used that word. So like, you know, I don't know if you have any commentary on like ra, deep seek Snowflake Quinn all these proliferation of Moe e models that seem to all be spars op cycle because, you know, you, you were advisor on, on the spars op cycling paper.
“Sorry, I, I should have used that word. So like, you know, I don't know if you have any commentary on like ra, deep seek Snowflake Quinn all these proliferation of Moe e models that seem to all be spars op cycle because, you know, you, you were advisor on, on the spars op cycling paper.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
55 / uncertainty
You got the NIC health checks, GPU health check, this space health check, Docker D message. I don't know what that is.
“You got the NIC health checks, GPU health check, this space health check, Docker D message. I don't know what that is.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
56 / uncertainty
I don't know if, John, you had something that you were sort of burning to ask or.
“I don't know if, John, you had something that you were sort of burning to ask or.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
57 / uncertainty
Got it. And I don't know about like what you would highlight so far as a post-acquisition, but the most recent news is that you guys released DBRX.
“Got it. And I don't know about like what you would highlight so far as a post-acquisition, but the most recent news is that you guys released DBRX.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
58 / uncertainty
I don't know if that's useful or not, but I mean, that's useful to me as a, as someone who might be working with them.
“I don't know if that's useful or not, but I mean, that's useful to me as a, as someone who might be working with them.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
59 / uncertainty
I don't know what is in the comparison set for you, like human, you know, like, or whatever scale has skill spellbook.
“I don't know what is in the comparison set for you, like human, you know, like, or whatever scale has skill spellbook.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
60 / uncertainty
Jason, you are extremely famous, so I don't know what I'm going to do introducing you, but you're one of the Waterloo clan.
“Jason, you are extremely famous, so I don't know what I'm going to do introducing you, but you're one of the Waterloo clan.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
61 / uncertainty
Yep. And then just to recap as well, like the models you were using back then were like, I don't know, would they like BERT type stuff or T5 or I don't know what timeframe we're talking about here.
“Yep. And then just to recap as well, like the models you were using back then were like, I don't know, would they like BERT type stuff or T5 or I don't know what timeframe we're talking about here.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space
62 / uncertainty
You know, the fun thing you can do with a credit system, which is data for data, basically you can give people more credits if they give data back to you. I don't know if you've already done that.
“You know, the fun thing you can do with a credit system, which is data for data, basically you can give people more credits if they give data back to you. I don't know if you've already done that.”
- Speaker
- Shawn Wang
- Publisher
- Latent Space