High Signal Podcasts Evidence ledger
Method
Browse
How the evidence works

Public evidence record

Yi Tay

Published podcast speaker

Claims
16
Episodes
1
Shows
1
Named items
0

Claim ledger

What Yi said.

16 transcript-backed records

01 / belief

I think Hyungwon is without going into detail, I still spend a lot of time talking to Hyungwon, even like in the, even after we both are different places, about like very interesting algorithm, arithmetic ways to think about life.

“I think Hyungwon is without going into detail, I still spend a lot of time talking to Hyungwon, even like in the, even after we both are different places, about like very interesting algorithm, arithmetic ways to think about life.”
Speaker
Yi Tay
Publisher
Latent Space

02 / belief

I think the second thing I learned from Jason is more about like, from my you know, kind of like, from my own career, it's like, the importance of like marketing and PR.

“I think the second thing I learned from Jason is more about like, from my you know, kind of like, from my own career, it's like, the importance of like marketing and PR.”
Speaker
Yi Tay
Publisher
Latent Space

09 / belief

Like, I think Making those are important, and for the community to heal crime, but I think long context is important, it's just that you don't have a very good way to measure them properly now, and yeah, I mean, I think long context is definitely the future, rather than RAC, but I mean, they could be used in conjunction.

“Like, I think Making those are important, and for the community to heal crime, but I think long context is important, it's just that you don't have a very good way to measure them properly now, and yeah, I mean, I think long context is definitely the future, rather than RAC, but I mean, they could be used in conjunction.”
Speaker
Yi Tay
Publisher
Latent Space

10 / belief

Rather than like, I pay like, like one cent and then like get back a wrong answer. So I think that's like, that is actually very easy to show that RAC is better than long context because there are a lot of tasks that don't need this long context.

“Rather than like, I pay like, like one cent and then like get back a wrong answer. So I think that's like, that is actually very easy to show that RAC is better than long context because there are a lot of tasks that don't need this long context.”
Speaker
Yi Tay
Publisher
Latent Space

11 / evaluation

I think there's a couple, I agree with you fundamentally that like, it's actually quite easy to tell, like when you see a paper, okay, this one doesn't work, this one works, this one doesn't work.

“I think there's a couple, I agree with you fundamentally that like, it's actually quite easy to tell, like when you see a paper, okay, this one doesn't work, this one works, this one doesn't work.”
Speaker
Yi Tay
Publisher
Latent Space

12 / evaluation

all the changes, all the changes that, like, Swiglu was this, like, okay, Swiglu is probably one of my favorite papers of all time, just because of the divine benevolence, like, the Noam (Shazeer) actually wrote, like we owe this success to divine benevolence, like, that was, like, it's always a meme thing, right?

“all the changes, all the changes that, like, Swiglu was this, like, okay, Swiglu is probably one of my favorite papers of all time, just because of the divine benevolence, like, the Noam (Shazeer) actually wrote, like we owe this success to divine benevolence, like, that was, like, it's always a meme thing, right?”
Speaker
Yi Tay
Publisher
Latent Space

13 / recommendation

I think inefficiency misnomer was like, we found that a lot of people, like, they use params, like, especially, like, like, to the kind of, like, right, and then MOEs was not very hot, like, in the community at that time, right, but MOEs were, like, a thing long ago. So I think using active params, I'm comfortable with using active params to kind of approximate like cost of the model, but like in the efficiency misnomer paper, we actually made it quite clear that you should always look holistically about like, because you have serving, like additional serving costs, like fitting in the GPUs, like fitting on single node, and something like that.

“I think inefficiency misnomer was like, we found that a lot of people, like, they use params, like, especially, like, like, to the kind of, like, right, and then MOEs was not very hot, like, in the community at that time, right, but MOEs were, like, a thing long ago. So I think using active params, I'm comfortable with using active params to kind of approximate like cost of the model, but like in the efficiency misnomer paper, we actually made it quite clear that you should always look holistically about like, because you have serving, like additional serving costs, like fitting in the GPUs, like fitting on single node, and something like that.”
Speaker
Yi Tay
Publisher
Latent Space

14 / evaluation

I mean, at the end of the day, I think research is still Like fundamentally like, we, as an industry, RIS, you still write papers, your goal is to advance science and everything.

“I mean, at the end of the day, I think research is still Like fundamentally like, we, as an industry, RIS, you still write papers, your goal is to advance science and everything.”
Speaker
Yi Tay
Publisher
Latent Space
Search evidence