Popping Kernels: An Exploration of Kernel Development for Jupyter Notebooks

0 0

um so hello everybody my name is Safi um you can find me as Captain Sofia on Twitter and github and my website is Sofia rocks which is an amazing top-level domain although people think I'm a bit full of myself for having it I'm not trust me so this talk or this tutorial slot is gonna be a little bit interesting because it's a 45 minute tutorial slaw which is the shortest one of the day and so in a lot of ways it's going to be like a talk and like most talks it's going to be very focused on the slides and I've made the slides available to you at this bitly link if you go to bitly forward slash PDC Juby kernels you will find the google slides for this presentation you can download them read for them all that fun stuff and with that being said if you came here for the poppin kernels talk you are in the right place that I am the right person and stay in this room please for a little bit longer we're going to be talking about a couple of things today and I just want to give you a rundown of what you're getting yourself into over the next 45 minutes so we're gonna start off by talking about what kernels are and why they're important in the Jupiter ecosystem and then once we've established understanding of that we're gonna talk about the Jupiter messaging protocol which makes messaging possible and is really the crux of these kernels and once we've talked about that we're gonna talk about how we can build our own kernels by leveraging the Jupiter messaging protocol and then at the end I've prepared a quick exercise notebook that goes through how you build your own kernel arms that you can go through the process yourself practice a bit and then take what you learned and like do something awesome with it that's the goal of this so I'm gonna start off by posing a really simple question which is what is a kernel and there's a variety of definitions for what a kernel is depending on the context of computing that you're working in in this particular case we're going to keep a pretty um textbook style definition of what a kernel is we're gonna define a kernel as a program that runs an INT respects user code so in Jupiter kernels are in charge of keeping things like the execution state and you see this in that little number on the side of your cells and Jupiter that represents when you've most recently run a cell or what order you've run it in it keeps track of the output that cells should be returning or commands should be returning it tracks the state of the kernel itself so is this kernel alive is it dad has not been responding for a while and the kernel accepts processes and sends messages essentially it's really a communicator and more importantly a single kernel can be connected to multiple front tons so I can have my jupiter notebook and my jupiter console connected to the same kernel with the same context it's really a kernel is a messenger it manages state and manages program output and so what happens when you like go on your machine and you're like really excited to do data science or research or whatever it is that you do and you type in Jupiter notebook on your terminal well what happens is Jupiter is going to start a kernel and create what's known as a connection file and a connection file is really just a simple JSON document that contains some information about how a front-end in this case your web-based notebook can communicate with the kernel and once that connection has been established via the information found in the connection file all that happens is the front-end and the notebook or sorry the front-end and the kernel communicate with each other via JSON messages it's just JSON back and forth pop and back and forth requests responses it's like one really long chat now you're probably thinking the connection file seems to be like the start of this process like it's how front-end and kernel like you know how there's like When Harry Met Sally when frontend meant Colonel is like made possible by the connection file so what does a connection file look like what kind of information does it have that makes this connection possible so it looks like this Oh kind of scary well we'll dive through it together so you'll notice that one of the parameters that's provided is an IP in several different ports that compromise the different sockets of your kernel and we're gonna go through what those are and what they're responsible for in a little bit and you'll notice that you have a key and signature scheme field and those are designed to secure communication between a kernel that you started so that other users don't have access to that kernel and have the ability to execute code on it that you might not want them to and that's pretty much the crux of it so really what you want to focus on is those multiple ports so you've got the control port the shell port the standard input port the HP port and the i/o pub port and we're gonna dive into those in just a second so I'm gonna keep you on the edge of your seats until like I have like a big reveal where I go through it all all good so the next thing that's kind of really mysterious is how do you go about writing a kernel because remember I mentioned earlier that a kernel is a program and you write programs using programming languages and all that fancy stuff so there's two ways to write a kernel in a kernel for the JavaScript or the Jupiter ecosystem the first way is writing the kernel in Python itself so one of the Jupiter libraries that's included is called I pi kernel I PI kernel exposes a base class called kernel base that you can subclass that provides you all of the functionality that you need to override standard kernel functions this might seem great like whoa I can write a kernel for any language in Python like I don't have to like learn are to build a kernel for our because our syntax is gross because somebody decided that their assignment operator would be an arrow that was not know in my job with some people it doesn't jive with me so the only caveat there is that if you want to write your kernels in Python there has to be a repple some sort of rebel console system for the language that you want to write the kernel in and you're going to communicate with that rebel and send messages to it using P expect and P expect is a Python library that's used to launch child processes and communicate with them so for example if I was trying to build a Ruby kernel I would launch the IRB process which is Ruby's interactive console and send messages back and forth to that and have it doing the actual execution so that's how you write kernels in Python and that's what we're gonna be doing later on in our like quick little exercise so you guys are gonna get really comfortable and get your hands dirty with that and then there's another way to write kernels um that's a bit more involved so we're not going to be covering it in the later half of the tutorial and that's if you want to write the kernel in the native language so let's say that you're writing a kernel for Ruby again if you're trying to a kernel in the native language you'd be taking advantage of a networking library and communicating with all of the sockets at their appropriate ports that were mentioned earlier you would have to manage sending messages back and forth yourself you would have to do a lot of dirty work I think that building kernels in the native language is a great way to learn that language because it helps you get in touch with a lot of really advanced concepts in the language like networking or data structures so I've been programming for a while and I'm at that point where like intro to go or intro to Julia books are no longer appealing because they're so simplistic and they cover a lot of the basics this is a great exercise if you're an existing programmer and you want to gain knowledge in the new programming language but you know take the advanced route you can always have a stab at building a kernel for Jupiter in that language so that's a fun challenge for you if you're ever bored and curious on a Sunday night as many of us are I'm sure and so with that we've kind of covered what a kernel is which is essentially a program that messages through a front-end using port specified in a connection file now I want to talk about what kind of messaging happens what kind of messages are exchanged between the front-end and the kernel to make all the cool stuff that a stupider happen and do this protocol is known as the Jupiter messaging protocol it's really like a good name which you don't find a lot in computer science you usually find like not good names but so when I find a good one I get really excited about it and the Jupiter messaging protocol like any messaging protocol basically defines the specification for how messages are going to look when they are sent from the front end to the kernel through those ports and when they're sent from the kernel to the front end through those same ports so it's just a spec and it looks something like this this is the message structure of a traditional Jupiter messaging protocol message you're gonna start off it's a JSON payload it contains a header it contains a parent header which references data from the previously called command it contains some metadata so maybe perhaps the time the message was send the user it was sent by if you're building a kernel in the metadata you would have some kind of like kernel specific things that you might want to keep track of and it also contains the content of the message which is itself a nother dictionary or JSON payload and so there's the basic structure and there's different kinds of messages that are sent in this basic structure and what that means is that for a given message that looks like this you're gonna have different values in the content key and the value of those values is going to depend on which socket it's set through so remember when we looked back at that connection file and there were those different ports so had your standard in you had like a HP port you had like a shell pour all of that good stuff these ports are gonna come in handy when we talk about the different types of kernel sockets that exist so I'm gonna list them all out here and then we're gonna go through them one by one talk about what they do talk about how they're implemented a little bit and then talk about how they are useful to you as a jupiter user like how they're manifested in your daily use of the program so the first type of socket is the shell socket you remember we saw the shell port in our connection file the second type of socket is the i/o Publication socket which is not as named is named much better than the shell socket so you have a standard input socket oh you have a control socket and you have a heartbeat socket so all of these if you recall the connection file where referenced as ports in the connection file so we have a way to communicate with all of these when our connection file is created when we instantiate a Jupiter notebook instance all right let's start off by discussing the shell socket so the shell socket is really responsible for managing communication with all of the front ends the room just got really bright because of this like very bright meme it's flooded with light on all your faces so it's responsible with communicating with all of the front ends and it does this using a request reply pattern so what happens is we've got a front end and it sits there and it wants something done so it's going to package a message in that format that I showed you earlier and it's going to send it through that porch to the socket the sockets going to the current through the socket to the kernel the kernel is going to parse that message try and understand what the user is asking for and what they want and send a response back via a JSON payload that contains its response so it looks something like this so here's something really common that happens whenever you use the jupiter notebook tap complete if you want to check if a particular statement that the user has written is complete or not that's something that the kernel is responsible for the front-end can't do that independently so what you do is you might send a message that looks like this you've got your header which contains some information about the session the date that it was sent the version of the messaging protocol you've got your metadata and in your content you have the code that you want to check if it's complete in this case we've got an incomplete print statement say send this over through the socket port through the shell port to the curl the kernel looks at it says ok the user wants me to check if this message is complete let me go ahead and do that really quickly so it does that and it sends back to you a response the response looks pretty much the same except you'll notice that the content has changed instead now the content says that the status of the status is incomplete referencing that incomplete print statement that you sent earlier and this is really useful if you're doing something on the front end like say you want to have a bright red color flash whenever somebody's got an incomplete statement in their notebook or like you want to play you want to like rickroll them if they've got like an incomplete print statement you could use this specification to do that so again the shell socket is based on request and response and so you're probably wondering what can this help us do in our day to day Jupiter use in addition to checking if a statement is complete well there's a lot of other ways that it's used so tap complete is another way that you can use it if you want to get information about the kernel that you're connected to you know what's the name of my kernel what language is it in what version if you want to send out a request to the kernel to execute a chunk of code like so you've got a complete print statement and you want to tell the colonel all right take care of this for me send me back the response all of this is done by the shell socket it does a lot of the heavy lifting for those of you who might go on after this and start to develop your own Jupiter kernels the shell socket is probably one of the more complex wants to build in my opinion the neck socket that we're going to discover talk about is the i/o Publication socket you got a sneak peak of my jiff there because my hands were a bit quick but I like to think of the i/o Publication socket as the announcer in the family and the socket essentially publishes data streams to standard outputs and returns payloads that contain information about HTML or images that need to be rendered so for example if you've ever had to render like a matte plot lib graph or any kind of image when you're using the Jupiter notebook the i/o publication sockets helping you out with that so give it a thank you next time not but it doesn't have feelings it's like not a human so maybe don't so what else does it help us do well it helps us do things like keep track of the current Crowl state which is good to know you want to know that your kernel is alive and well it streams outputs that are like Consequences of execution like plots or renders and the next target socket that we're going to talk about is like really humble and self-explanatory and timeless which makes it great which is the standard input socket and you're gonna really going to need this when you have a situation where the kernel is going to want to request and put from the front end so for example perhaps you want to execute a Jupiter cell that contains Python code that uses raw input to ask for input from the user the standard input socket is going to be used in that case and then the second socket or the fourth socket getting ahead of myself there is the control socket and the control socket is like the muscle of the socket family it's the big gun and it can basically do everything that the shell socket can do with one additional caveat which is that it's a completely separate socket and it can Trump things in the queue so for example when you have like not that this has ever happened to me but say like you're playing around with so I can't learn and you start training a model and you realize one of your parameters is off and you want to like quickly like you know stop that cell interrupt the kernel and restart all over again all of that is handled by the control socket because you want your interrupt the kernel message to take priority over that ongoing you know command where you're running code that contains model training or model fitting and then the last socket is really simple to understand any meek in its own way but very critical to the kernel ecosystem and it's the heartbeat socket and so I've mentioned that all of the messages that we've been sending have been JSON messages so you're sending Python dictionaries the heartbeat socket is really special instead of sending JSON it sends a simple byte string back and forth between the front-end and the kernel and as the name might suggest it's used to keep track of the kernel to make sure that it's still open and alive so if you're using the notebook web app you might notice on the side there's that indicator that says you know kernel running there's a green button your kernels all well you like panic a little when it's red all that is being taken care of by the information that's sent from the heartbeat socket those are your five sockets these all make the magic happen um and they're all implemented on that standard sending back and forth of JSON messages in really an agreement on what messages look like and how responses to them should look like it's like specs are good or something Wow no one ever told me you're probably wondering like what makes the actual magic happen what are the socket things that I keep mentioning and where are these ports and how is that facilitated at a lower level and the answer to that is a lovely little library by the name of zero and Q is definitely not the topic or the concept of this talk but it's a messaging library and it provides a really excellent API for socket based communication raise your hand if you are not a data scientist but to write software to support data scientists okay nice raise your hand if you've ever done low-level programming yeah so the level network programming is like a form of torture in and of itself I think the CIA has classified it as that and 0 and Q really makes that easy it's got a simple API it's got tons of bindings across different programming languages so I mentioned earlier you might want to write a kernel in its native language you would use these 0 mq bindings available in that language to open up communication to the front-end and send your messages through that and a fun fact the Python bindings for 0 mq are actually maintained by a lot of the jupiter developers so give them a thank you and a cupcake next time you see them for doing that and so really the main point that I always try and like drive to people during this talk is not the low level how the message looks like or how it works because there's like documentation for that even though nobody reads it but it's really just a protocol it's an agreement on how messages should be sent and how responses should be sent and that's a really robust thing you don't have to limit yourself to thinking about just building a kernel for a language um if you've got a special kind of DSL or querying language that you use your company you could build a kernel for that you don't have to limit yourself to the standard um like I'm gonna build an our kernel or I'm gonna build a Giulia kernel you can get really creative because all it is is a messaging protocol it's like go crazy and disrupt as always how I like to say it so now you've got this knowledge you get what a kernel is we have understanding of the Jupiter messaging protocol we know how to pass messages back and forth and we know all about the different sockets the next thing that we're gonna try and go through is build a kernel and I know this is a tall order because it seems like a really intense thing to do and this is the part of the talk that I'm gonna try and make really interactive and fun so what I'm gonna be doing is going through a jupiter notebook that i've created already and explaining how you might go about building a jupiter kernel using the python subclassing technique I've created a version of the notebook that I'm going to go through that's fill in the blank style so it like makes you like run the gears in your brain a little bit which is fun to do occasionally it's located at this link if you want to check it out I'm gonna give everyone the chance to take a picture or open it up on their machine and I'm gonna go ahead and switch to mirroring my display right now so I can go through this notebook with you guys so you guys seeing my display yes you are so I'm gonna go ahead and open up is that good sighs that's not it's too small so I'm gonna go ahead and open up the existing notebook that I've written for this and you can find it online um this is basically what you have there but it's filled out completed and what I based this off is a Bosch colonel that was developed by Jupiter developer named Thomas and what I've done is kind of broken up that code and gone through it and we're just gonna go through each of the steps that are shown in this code and kind of like do a walk-through it'll be super fun who's feeling tired it's okay if you are I understand it's late afternoon it's like everyone's feeling dead can you guys engage in an experiment with me okay everyone stand up let's go yes now you're gonna put your hands up all the way to the top and try and reach all the way up yes come on try really hard both hands yes and then you're gonna gently bring them down slowly try not to slap the person next to you although that's okay they won't sue and then you're just gonna like shake your hips a little bit and loosen it up yeah do a little dance in your seat you know get it going all right now I could sit down do you feel like more energized now and ready for the second half of the talk this is gonna be a super quick walk through it will be linked in my Twitter after this talk is done the question was where can I find this notebook so I'm gonna tweet out the link once this talk is done so be sure to follow me on Twitter for the link and for like funny jokes yes and also just come up to me afterwards and I can show you the slide again so alright we're sitting here we've learned all this stuff from Sophia about like Jupiter kernels and like our brain is stuffed and full and we want to start to build our own oh we've got Python 3 install we've got Jupiter installed already because we've been using it in our day to day job or just playing around with it and we want to take it to the next level so we're gonna open up our text editor vim Emacs atom a stone tablet from the 21st 21st century that's just an iPad that you're about I said the wrong thing and then the joke just saw itself so to start off you're going to import the base kernel class which is stored in the I pie kernel module that kernel if you've ever had to install a Jupiter kernel before you like have to type in I PI kernel into the command line to install it so it's exposed as a shell command but also as a module that you can use which contains the kernel base class that's kernel right there and once you've imported that base class you can start to use it for all the fun stuff so the first thing we're gonna do is we're going to create a bash kernel class that's going to subclass the kernel that wage is created can everyone see the screen alright is that zoomed in well I forgot should have asked that earlier I was just so excited to make you guys stretch and shake your hips and stuff and then inside this bash kernel class you're going to need to specify a couple of different variables these include variables that describe the language that your kernel is in as well as the language version as well as the implementation of the kernel itself so this is my bash kernel and it's the first version I made last night at 11 p.m. because I definitely don't procrastinate promise there's this fun banner variable which when I first discovered it I was like what do you have to like look a lot in the documentation to find it out the banner is what is printed out when somebody starts your kernel in console mode so if they're not connecting to it via a notebook front end they're connecting to it via console front end the banner is gonna be printed the first thing so if you're like building something internally inside your company you might want to like print out you know license information or warnings about use or all that fun stuff so the banner is a good thing to know I keep a close eye as I go through these things because in the link that I shared with you previously the there's a lot of fill in the blank so if you want to take notes and like cheat to figure out what you need to fill in the blanks with that's fine too and then you're going to need to add another parameter or attribute to your new kernel class and that's something called language info language info I'm gonna ask a quick audience question to the audience to see how curious you guys are raise your hand if you've looked at the raw JSON for your notebook if you look you've opened a notebook file in a text editor or vim or something like that yeah and if you scroll through it and you see this language info key the language info key contains the exact information that you specify here in your kernel and this is really useful for a couple of things you'll notice here we've got the code mirror mode key inside our language info dictionary this is used by the front-end to determine what kind of syntax highlighting it should use so in this case we're telling it to use shell based syntax highlighting on the front end which is like a good thing to have if you want people to enjoy using your kernel it also specifies things like the file extension which links into when you export a file from a Jupiter notebook instance it'll add in that file extension at the end that d'audace sh this is a useful thing to know it might be blank in the notebook that I shared with you so this like is a good way to cheat on your homework I'm such a great teacher so once we've created sort of these basic attributes that need to be exposed in our kernel we're going jump in and instantiate the kernel itself there's a couple of things going on in our instantiation for the bash kernel here and it sounds overwhelming I'm gonna try and put it in a very simple way to start off we're going to call the initialization function in our parent class the kernel class and this is going to take care of instantiating things like the execution count the execution cow is that little number you see on the side of all your Jupiter cells that like changes as you run them or like clears and all that and then it's going to make a call to another function that's called start bash and this function instantiates a connection via P expect to abash repple and we store that connection in a variable called bash wrapper you can do this with other tools as well I mentioned earlier if you want to build a ruby kernel you might instantiate a connection to a IRB or an interactive Ruby session to start off just like back and forth this is how we initialize our kernel and then what's really great about the sub classing technique is that there's really only one function that you have to fill in or code up in order to get your kernel working and that's the do execute function and the do execute function has got a lot going on but it's responsible for handling believe it or not another thing in computer science that's named well what the kernel should do when it sent a request to execute a piece of code and it's a function that keeps takes in a couple of parameters these parameters include the code that it needs to execute there's a parameter called silent which describes whether or not it should suppress output there's a another parameter called store history which defines whether or not it should update the execution count there's an allow standard input which says you know is this execution request allowed to request input from the front-end and you can configure these to your liking based on what kind of functionality you want your kernel to have and then really in the bulk of the do execute function you're going to have a lot of your logic so the first thing you might think of is maybe you know I have a sneaky user and they like requests to execute an empty cell and you don't really want to do anything if they haven't sent you any code to execute so you know you might check you know have they sent me any code using this simple if statement in Python and if so you're going to return back a JSON payload because remember all of this is JSON um and that JSON payload is going to specify that it was an okay status like nothing broke no errors occurred Trump was not elected president the IceCaps did not completely melt it was mostly fine it's going to maintain the execution count and it's going to send back an empty payload and an empty user expression stick so this is kind of like the you asked me to do nothing I did nothing here's like just everything's okay and you're sending that back to the front end now let's say that they do have a valid command that they want you to interpret well one thing that you're going to want to do is send that command over to the bash wrapper that we created earlier and request that that command to be run and you're gonna store the result of that command in this output variable and you want to keep track of a couple of different things since this is a bash kernel let's say that like they decided to like do catch on a million line long file and they instantly regretted it and they're like pressing ctrl C to get out of the command you want to be able to detect that keyboard interrupt and appropriately send an interrupt signal to the bash to the bash ripple and then change the variable appropriately now similarly imagine like they truly panicked like they accidentally ran that shell script that they like built when they were 10 and just getting into programming that like overloads your disk with null data and just tears everything and they're like panicking and pressing ctrl D which sends in and of I'll signal and you just want to restart the entire kernel and all of that you can do that here this final accept so restart the bash session and Riaan Stan she ate the wrapper that you have around bash so this kind of takes care of executing a valid command or code and then all of the other things the user might send unexpectedly now we mentioned earlier that you can send back outputs using the i/o pub socket which is super handy and you'll notice that it's referenced here in the sender response function that we're making a call to sue essentially here you're saying if the user hasn't requested that output needs to be suppressed we're gonna go ahead and send back a simple text of whatever was cat it out so you know if it's like they're silly like ASCII art or what-have-you it's going to be sent back via the i/o pub socket and if they haven't actually executed anything if they've requested that the kernel be interrupted earlier we had sent the interrupt kernel and we've kind of set a boolean in addition to doing that we want to make sure that the front end knows that we recognized and accepted their request to interrupt the current session and to do that we're going to send back a simple payload and change the status so that the front-end is aware so notice it's pretty much standard syntax or standard payload for everything you're just kind of changing what message you're sending and another thing that you might want to check for is if something has aired out which is terrible for the user but also something they should know about so one way to do that is to request that the shell execute a simple command and see if it can do this okay like if it's fine to try and execute this echo command and no exceptions are raised then we know that it doesn't need to exit abruptly and we don't have to send back an error to the user once again the air is just a JSON message that you send back to the user and then finally if everything is okay you're basically going to send a simple payload that says you know the status is okay here's the execution count a payload and the results of user expressions and this is what do you execute looks like for our bash wrapper and I broke it down earlier and as you go through the notebook yourself and your spare time you'll be able to see it much better it seems like a lot right it's like a really long function like this is bad programming you should break it up into separate functions but this takes care of all of the edge cases and potential and potential uses for that kernel so you take care of when you're not getting a message you take care of when you're requesting it be interrupted you take care of errors all of that good stuff so when you're building kernels for other languages this is where a meat of the a bunch of the work happens and you can use a similar template if you'd like as you start to build your own kernels just kind of follow along with what I have but instead of bash wrapper and using like an echo command to test for errors you might use the language specific functionality that you're using so that's our do execute function but Jupiter and all of the other notebooks do so much more than just execute code cells so you've got like lots of cool functionality that I talked about earlier you've got tab complete you can check if something's complete like all sorts of cool stuff the kernel base class only requires that you override do you execute like you don't have to worry about anything else but if you want it to be like super fancy you can write a do complete function that handles tap completion out which I would do but then that would just take too long and also I did this at midnight last night so yeah definitely not a procrastinator but this is really kind of the most basic simplest kernel that you can create it specifies its implementation and the language it needs and specifies the kind of back end that's supporting it and it specifies how it executes code and once you've got this done um what you go on and do is install the kernel and this is usually done by specifying what's known as a kernel spec which is a file that explains how you would launch the kernel so what commands you would run it outlines what the name of the kernel is so in this case it's like my bash kernel and this is like the magic that connects what you just wrote to the way that I python or Jupiter new name branding everybody got it gotta stay on brand or Jupiter launches kernels and so all of these things are connected together so you've got a front-end you've got a messaging protocol that manages how messages are sent back and forth and you've got the kernel itself which could be either written as a Python wrapper or in the native language which is like super fun to do does everyone feel comfortable and understand that or feel like they can go back and read through this notebook and kind of get a grasp of it did I cheat did I enlighten you today so I'm gonna jump back to my presentation really quickly technical difficulties potentially No so once again if you want to go through and fill out the empty notebook for this lesson it's available at this link just go check it out download it play around with it once again I'm captain Sofia on Twitter and github if you'd like to see my open-source work or like what I do if you have questions but you're too nervous to ask in a public venue feel free to email me at Sofia at Sofia dot rocks and I will try and respond soon like once I'm done with the playlist of cat videos I have to get through I'll get to the emails I have to go through fast that was my talk I hope everybody enjoyed it I will be around today and the rest of the conference to answer any of your questions and any of your feedback and I'm gonna unplug and give it to the next speaker thank you everybody