Joe Sepi

Transcript

Thank you. Hello. How is everyone doing? I am Joe Sepi and I am actually superexcited to be here. But I've also had very little sleep so it might not show through. But, yeah.

Thanks for coming. And thanks to JSConf for inviting me and thanks to everyone that has made this event really awesome. It's super great to be here.

And I wanted to kind of just take a quick moment to say, like, happy anniversary, tenth year of JSConf. Superexciting, but also a little bit sad since they're gonna take a little break. But let's stay positive and be excited about the tenth year and I'm really excited to see what happens next. There are some amazing people that are involved in this whole effort. So, I look forward to seeing what they bring next.

But also, happy birthday to Node. I don't know if everyone knows that this is ten years of Node. Whoo!

[ Applause ]

Ryan introduced Node here at JSConf in 2009. Shoutout to Jenn Shiffer, 2009. Happy belated birthday from I/O JS. I couldn't find the Node.js Tweet that said happy birthday, so I went with this one.

I'm here to talk to you about promises in Node Core. The Core APIs in Node and what state they are in terms of being promisified.

And I want to start with a little disclaim. I will try to make as few puns as possible. I'm going for zero. And technically I don't think this is a pun. And if you are keeping track, mark of zero.

I'm Joe Sepi. My little Twitter handle's down there. You can barely see it. But if you search, you'll find me.

I'm an open source engineer. I have the great fortune of being paid to work on open source exclusively pretty much.

And technically and in the community and as a part of the Foundation. Why does my phone keep buzzing? I told them, leave me alone. Okay. It's not doing it. Anyway.

So, I'm superfortunate and it's great. I love it. I work if a lovely company called IBM. We make typewriters and we have  we have a cloud. But it's really just someone else's computer.

So, don't be fooled. Not to be confused with how so, you know, IBM. This is my lovely family.

[ Laughter ]

This is also my lovely family. This is what I do in my spare time. That's Pat and Stella, and this is what I really enjoy doing. I play in another band, that's also Stella in the background. That's me up there as well in the back.

And I occasionally play in a Misfits cover band. So, funny story. All of this was to lead up to, you know, karaoke tonight. But then I realized that was my talk is after official karaoke.

However, I am always open for karaoke. So, if anyone wants to go, I'm here all week for the TC39 meetings. Ping me. My DMs are open, let's go do karaoke. Okay.

Cool. So, I digress. Which I do often if you know me.

I wanted to just quickly share a little bit of personal context around Promises. I've worked at a company called The New York Times a few years ago. And pretty much everywhere I've worked for the last  I think almost every job I've ever had  I've done some form of developer advocacy formally or informally.

And I started this weekly JavaScript thing. Which I managed to do every single week. Largely sourced by internal folks. Luckily at the Times we had so many great people like Jeremy and David Nolan and Mike and lots of great people.

But occasionally I would have some outside folks come in and talk to us about cool stuff. So, this is in like 2012? 2013?

Tom Hughes came in, Rebecca Murphey. Spike came by and my old friend Dominik was there too. So, Tom talked about sockets and streams in Node. What's the date on this again? Yeah, 2012. And so, I mean, Node was around for a few years.

It was still pretty new, pretty exciting. And the hockey stick had not really happened yet. Rebecca came and we had kind of an informal discussion, you know, a group discussion. This was one of the things she talked about.

Shoutout to yayQuery. I mean, does anyone know yayQuery? All right? Do some research. It's fantastic. Anyway. Spike came and talked to us about Render which was this really cool isomorphic framework where you could bring backbone app that would be frontend and backend with the same code.

It was kind of revolutionary at the same time. It was really cool. And Dominik came in to talk to us about this little spec he was working on.

And at the time there were a few libraries that were implementing Promises. But everybody kind of had their own flavor. And so, Dominik was working on, you know, developing a spec that would be something that the language could implement. And fortunately, Dominik continues to be amazing work in the spec space. Thank you, Dominik.

It was also a great opportunity for me to say that wrote for the "New York Times," which is pretty cool. I talked about callback hell in JavaScripts. And how we're working on it using, you know, the Promises async flow.

And this looks bad, but it's not actually that bad. It's fairly linear. But when you're nesting asynchronous flow in a variety of ways, they can get really, really, really painful and hard to debug. And, you know, I think it was one of the quiz questions last night, you know? Which of these async functions are gonna end first? If I'm not mistaken, almost everyone got it wrong.

Then I went to work for a company called Behance which was a lot of fun. And there were really smart people there. And everything we did there was kind of Promisebased and it kind of blew my mind for a few weeks or months. But once I got a hang of it, writing almost all my code using Promises, it was just a fantastic way to control your async flow. And you may know that Ryan was here last year talking about regrets in Node.

And one of the key things he talked about was not sticking with Promises. And granted, this is really early on, 2009 to 2010. He removed them in February. And as he says in the talk, you know, maybe it's okay that they didn't remain in Promises. Because not having the implementation in there allowed the community to kind of work through what might be the best implementation.

There were a variety of libraries like I said before, Q and R is SVP doing their own implementations of promises. And it allowed the community to find the right abstraction and we could go from there. It's a good model that I think we've used time and again in JavaScript.

But as he said, Node's many async APIs are aging pretty badly. And this is definitely a pain point in writing Node which has a lot of callbackbased APIs, obviously. Async is at the core.

So, one tip, if you are really trying to get your talk accepted at a conference is to reference one of the organizers in your talk proposal. I don't know if it worked or not, but I'm here in front of you now. So, one of those references is to this Tweet. And it really was kind much funny, but sad.

LOL at Node.js Core modules for still using the callback pattern. This is in November. And I have this bad habit of  maybe it's a good habit, actually  of finding something interesting that I would like to hear a talk about and submitting it.

Whether I know a lot about it or not. But this was really interesting to me and so I submitted a talk about how, you know, where we are and where we're trying to get to. And how you may be able to help. And fortunately for me, it got accepted. Here we are.

So, one of the responses is Myles who is superactive in the Node space, in a variety of spaces. I love Myles. And he shares with us that the FS API already has an experimental promise implementation. And this is one of the problems.

It's not maybe the prettiest because we needed to figure out how to namespace, essentially, Promise implementations of APIs. But he points this out and people are pretty excited about it. And he shares that, you know, unfortunately there are only two that were implemented at the time in late November.

And it's FS and DNS and they're experimental. Now, there are some puns on these slides. But I did not make them. So, I don't  I still don't think they count.

Keep it zero, Donny. And then it was also pointed out that there's a utility in the framework that you can basically wrap any callbackbased Node API using util.promisify.

As stated there, it's still a little bit horrible. But it gets somewhere. Why was it taken out and then has not been just like brought back into Node? And this was something Matteo said to me a couple of days ago at the Open JS Collaborator Summit which happened just before JSConf. Is that it's an insanely hard problem that's been there forever.

And that we've never really been able to find a great way to solve and find consensus on what the implementation is. For a variety of reasons.

So, the big problem is that Promises were primarily designed for browsers. And so, if you have, you know, something go wrong in your promise in a browser for a user, you're only affecting one user, you know, one tab. They might not even notice. They may think why is my Facebook reply not showing up or something? And they view the console, they may see some sort of error. But it probably doesn't even affect them that much in most cases.

But if something happens on the server, it could bring the whole server down. It likely will. And that affects obviously way more than one user, right? So, this is one of the big problems, the challenges that we face with Promises in Node Core. And here is an example. So, we had a session at the collab summit on this very topic and we kind of worked through some of these things as a group.

And this is pretty basic pseudo code. But the sum function call will  anything, you know, beyond that resolve will essentially get swallowed. And so, you could have code like this in and bunch of code after that, but if your promises resolved, all of that code is essentially  it's run. But if you have any errors, you have very little  I think no way of actually seeing what's going on there.

Your app may come down. You may have all sorts of problems. But you may never know that that's the issue.

And then this is another example. Essentially if you have a promise that never resolves and the event loop finishes all of the things that it needs to do, you just have this promise kind of hanging out out there, you don't really know that that's there because it never resolves and it's sort of invisible as well.

So, essentially this is all a debugging nightmare, right? But the great news is that there has been progress. And things are happening. There's a zero cost async stack traces were shipped not that long ago. And so, that can help us get some insight into some of these issues.

But here's another big one reason, right? So, how do you promise event emitters that don't resolve once? They may have multiple events coming back, right? So, what do we do there? Well, the  and, you know, as I said there are multiple responses. The tricky part too is that some of the key things that the core APIs use event emitter under the hood, right? So, this is a daunting problem.

Well, the good news is, events.once, that one's kind of easy to solve. And I can't remember if there's a PR. It may have already landed in Core. But events.on needs an async iterator.

This talk has been evolving minutebyminute, hourbyhour. We had the collab summit Thursday and Friday of last week and this Tweet came through yesterday morning.

So, I'm like continually updating these slides. And they're also my talk notes. So, if they come up and I haven't actually implemented a slide, don't blame me. I'm working on it.

But the great news is, there's a PR open by Matteo to implement async iterator for the event emitter. So, stuff is happening. It's really, really exciting.

Sometimes this is thrown up as a bit of a road block. And, you know, obviously we do need to be considering performance and the implementations that we're making. But I think we need to make sure we balance them between the developer experience and delivering on what is, you know, a modern interface for a modern platform, right?

So, like I'm kind of getting at, the good news is that things are getting better and progress is being made on a very ongoing basis. Some APIs are already done. As I mentioned, F S and DNS, events.once. There are a few others that are already implemented. Some PRs are open like the one we just looked at for a second there.

I have one I'm going to open over the next few days that's around crypto. So, things  things are definitely happening. And other development is in progress. And when I say that, I mean, this is from November of that Tweet about lol at Node Core to now, there has been a real swell of activity.

And especially with the Summit session that we had. Like the people are focused on this when we asked for volunteers. There were, you know, six or seven people that raised their hands. And I'm trying to think about this too from like a  something of a project management standpoint and helping list out the APIs, see what state they're in.

I'll be opening some issues soon to catalog what APIs still need to be done. And we can start to really track this effort in a holistic fashion.

So, we try to do everything in Node out in the open. You know, through issues and comments and PRs and such. And so, this is an issue in the Summit repo for the recent Berlin Summit. And so, as you can see here, one of the topics that we talked about was unhandled rejections. So, when  like I showed earlier, if you resolve your promise and you have a bunch of other stuff happen and you get exceptions, how can we handle that and surface these errors so that you know where they're coming from?

We have a survey that we're working through, if people are interested in. We're also  if I were able to scroll down on the screenshot, there's a link to user feedback survey that's I have been trying to get over the finish line that would be more for end users, what their experiences are, how they're managing promises and give us some more insight into what we may want to be considering in further development here.

And then that bottom link there is the list of all the APIs and some notes on the current state of things.

So, as mentioned, there's a survey, primarily focused for implementers as to what we think would be the best way to handle these rejections. This is that list of the APIs that I mentioned. There are only 50 of them. And a bunch of them are not relevant.

Some of them are already done. But, you know, we're trying to get a real handle on the overall effort.

But like I said, the ground swell within the project is real. People are focused on this. It's taken a little bit of time to get there. There are PRs open and there's work being done and it's actually kind of exciting to think we may have some more modern interfaces for those Core APIs. The kind of call to action here too is if you are at all interested, we always welcome people to get involved.

I know sometimes working on Node Core can be a little daunting. But it's not that bad. You should talk to the folks involved, Joyee, Fedor, any of the folks involved would be happy to share with you. It is a strategic initiative for the technical steering committee, the TSE.

Matteo is the champion. I'm trying to help along with Ruben to  Ruben Bridgewater to spearhead these things.

Myles is doing work. Everybody is doing work on these things. Benjamin is involved. A lot of people are focused on this now. This is exciting.

This is one of those slides that I should have updated, but I haven't filed that initial PR. Once I did, there would be next to Matteo's name a link to catalog where we are. And I promise you that we will resolve this problem. All right?

[ Applause ]