How It's Made: opening jsconf.eu

0 0

[Music] [Applause] [Music] fearless put of ioannina please hop in especially Nadi's Bhutto janina's face everybody affinities loggias nemesis my lineage as Vietnamese like when action dispatches women sir they as complete them it's like when ox on this budget more than ever media to this doctor who enter in authorities are we school nima minimally likely one Wonderland Alice Moloch jihadi John Paris model Rasta man of Charlie's they a sponge ways for chair for the people there is burning the party business that whoo honking one of the beers everybody open audience no js' women since I'm in a phase everybody like when ox on dispatch it is honestly blows blood no Domino go Oh JavaScript run D show any browser any size rope on any device the biggest thing already still we go gusano c++ no Java no go JavaScript run D show all right ready to start [Applause] [Music] hello whoa so many people hey uh with the nested loops onion for you I'm Khalil Boris Martin yeah you might have seen us this morning or one-and-a-half years ago at this very place and we want to tell you how this entire thing was made and there's a story to it and I won't tell the story so the year was 2014 and I gave a talk at chess cons about Web Audio for music production and live performances and there was this part in this talk where was actually performing music and this is all look like [Music] [Music] I don't remember that load okay so what he saw was just me dabbling on an iPad and I had no idea how to make music I bootlegs jay-z Linkin Park they didn't sue me what's great and but at that time I had a much bigger dream and I was the free guy their drums and they taught me to use software so the problem is neither do I have three people capable of doing that including me nor do I have the MIDI instruments nor do I have the software so that was kind of a foreshadowing of what happened and so after that I had a plan the plan was find band members by MIDI instrument and write the software so just as I was just browsing Twitter a couple of weeks later and then I saw this one tweet by Khalil he's like Oh before I was the developer I was requisitioned check it out this is me so let's check it out [Music] whoa yeah now it's like I have to have to collaborate with this guy he knows how to make music and he has a voice that sounds good so meet the ill inspector okay so this is just a little story time so I was doing music before I was working at the developer so I was like like four years or so it's just like touring in Europe and doing like dancehall but living in Leipzig there was a label there and actually so this video was actually shot in Japan because there was at some point there was this Japanese guy he wrote as an email saying hey we would like to release an album or something into it okay great so we recorded an album for him and this track actually made for him as a request because he said in Japan they loved poopa son who was like hmm he's like a foundation dancehall reggae artist and so I made a song about him and we shot this video over they were in Japan for a week to do like promotions and stuff like that and that's how it kind of was traded and actually was my greatest hit I would say what so the cool thing was when I was talking to Khalil he actually told me well I have this brother Shiva in the background he's actually a music producer I was like that is great so let me count I need three people me Khalil and the guy who actually knows how to make music it's like yeah got the bad memories then I just had to buy media instruments and we just bought I just bought like one of these very sturdy very great has a lot of buttons that means you can do a lot of things I thought I had no idea like really so they call got the MIDI instrument just have to write the software writing software is easy right I mean we all know that it takes a week but should we really look like a huge music production app for this just to perform a task on knowing that it's like a couple of weeks before Jays Khan's probably not also what kind of music do we actually want to make so we reach out to the date comes organizers like hey what kind of performance were you thinking of and Martha was like be creative we trust you our excellent hands great so yeah we because I'm from Berlin they're from Stuttgart so we met up at Kelly's house and it's this is you okay yeah so that's where like we had our first meeting and we're basic city we're trying to figure out what we're going to do so we told Boris words we need music boards like okay I got music and was like cool so like because Boris has been doing music since forever and so he had like a bunch of beats lying around and we didn't have a lot of time so we had to there was not really time to do a whole new beat and so I guess so we chose one of his that was unreleased and we liked it so we we decided on the music first and then we were like okay what are we going to do like we have to do something in the browser like just music not enough so so we so the thought basically came up I don't know who had it or whatever but like we're thinking okay we're going to take talks I mean I think what you know like take some interesting snippets and then trigger them on the music in the browser like the video has to play in the browser the music has to be played in the browser and everything needs to be triggered by as the the pads basically right so so basically sorry so this was basically how it worked we had sorry we had the music running and we were triggering just like random snippets that we just cut out of the YouTube videos and we're just trying around stuff like this the music in the background triggering sniffing and so this was going on that weekend and we were just triangle for two days we're trying out stuff back and forth and and that weekend we kind of laid the groundwork for all of it and then I would track see just me performing over the beat so had some lyrics ready yeah okay what and this is basically I'm just going to go to this one okay so and then this video the last one was us just before the conference we had everything kind of ready the video the videos were cut to a certain so that you can see the speaker's really big and then we had Boris also put a filter over the video and so that was the first iteration and and then and actually this took like weeks and weeks of meeting up with each other arranging the snippet so that it kind of makes sense that it's funny and so like Dorothy and me were meeting once a week or so for a while and Boris was kind of yeah we were just trying to make sense out of the snippets we had and huh and we were trying to make sense we're trying to make it look good and actually Jana was programming the software the whole time like there was lots of little things that need to be figure out like how can we because because the videos needed to be instant like when you press the button it just needs to book happen right away it needs to be synced on the music and everything so there was a lot of crazy stuff that when I'm sure you're going to talk about this in a minute and so this is actually practicing the day before the conference 2015 in our hotel room you can see the venue right there just going through the whole thing okay that was us doing sound check the day before and yeah and now young with the software so yeah the software is really easy right so it just has to read MIDI data that this thing sends a thing to your browser and you like are cool these are events but this is what the data looks like that this thing is trying to communicate with you felt like the movie the arrival like you have to kind understand what does computer is talking to you it's like all of these numbers you have to make sense like what is the position what is like did it get pressed or not and is to figure out all of these you have to read the midi document for this thing and I was like oh I didn't know that why why did I god well that just takes like two weeks and then what you have like you come up with like an idea okay so for the first row I want to have this video played this and then that video played you come up with like an idea it's kind of happy but the thing is so you have a video and videos usually have like a video feed and an audio feed and you kind of want to have like the audio desync with everything so you have to kind of like split them up again to make put like your text on it and everything and then you have to kind of make like the video still be snappy because how do you play a video without a delay in the browser like using the video tag to you a just to video element dot play do you see preload and then play or do you see use an obscure heck and then call play on the thing a what be yet see so it kind of hacky so you have to load the video in an era buffer make a blog out of it with the correct mind type otherwise it doesn't know it's a video and then it's the greatest thing ever create an object URL out of this massive video which is really bad for RAM usage but we didn't care this is the only thing that makes like pushing a button and then being it isn't there possible all the other ones have like a lot of delay which I'm probably sure we would have never heard but the person actually pushing the button really tells it so we needed this and yes so the cool thing is you do the same for all the elements you have someone who's good at pushing buttons that guy and yeah yeah JJ's called opening performance software this war it's always more as we know the videos were pre-rendered so we just had videos with the effects on it out of like render it the beat was in life performed actually we just pushed the button and then the music was playing nobody noticed that was great but if we only had more time we would have done it all life so that was 2015 so we had a nice tune right so we had a pretty overwhelming performance like it's kind of like our little secret with multi for like a couple of weeks you couldn't talk about this like all giddy giddy nobody know we can know about this so the the response was great like we got so so much great feedback and I can tell you like before the talk or before the performance listen Foster syndrome thing that also applies for music performances especially if you don't know anything about music it's a very bad so yeah we donated in T story yeah so made the nested loop t-shirts so do you have the thing loaded up yeah and many people wanted the t-shirt we produce the Trisha for the yeah that's right and so we made the story fine thing I wanted to have selfies from everybody so anybody who has a teacher please you didn't send a selfie our way I want to have the selfie so I can put it into the story flower okay so there's a lots of lots of people who it's Allah and this yeah lots of cool people with the t-shirt and yeah so we so we saw those peace t-shirts on t spring and we we made a little bit of money so the margin on the t-shirt was very low the little money thing was like 300 dollars or something like that we all gave away for good causes and so they're like a hundred dollars for caught her cause or something like that so we didn't make any money from it but so yeah so we all felt really good and it's nice to see a lot of people with the t-shirt out here today it was really cool yeah they're all know that was a break a J's cones break and then we got up again to step it up a notch they didn't react to that but we felt we had to do that and this time I really wanted to build a real piece of software not this like six weeks heck that lived on this obscure video hack that is still in the production software now so this time we actually started a six months ago already like as you can see in October the first two commits and then we worked along the way on that and so 2015 what the thing did was push a button it it did a thing and in 2017 you push a button and it does 10x things so we kind of thought who who does know like which one is which there by the way this is this is our Tech's take an order to guess which one is redox react and electron so we thought like the one and a half year that has like everybody was talking about react Redux an extra ones we don't we have to make a software to make music with all of these tools so that we can dog feed or industry gullible so the app actually looks amazing you will see later on so you have like this is like a representation of the physical pad and then on the side you can see like what kind of clip you want to play and all these things whatever you want to attach to it and so what was really cool about this is you can import export project set up so the thing before was whenever Boris wanted to have like a button on a different thing there he is he was calling me and I was like okay I'm going to do this push that to github so that he can work on it an hour later this was horrible so now we can like exported zip and import and all these things it has perfect timing in there before that we had like a lot of human errors mainly mine and now there's a computer taking care of the timing as in if you push the button too early before the beat it can make this wait a bit and I click like it will be perfect to the beat so the problem is if you're too late we cannot take that because JavaScript doesn't have time travel yet let's see see see 39 was what's up this time we actually do have live music I mean you didn't notice before there was no like music but and also the we had a life video shoulders as you saw this morning but yeah this is like technical stuff let's let's hear how like a beat it's made so high so we this time we wanted to make an original beat or I want to do that and first up the guys listen to all the talks and they gathered snippets for me and then I'd about 150 snippets about stuff that I don't really understand and so the first thing we have to do we when we start from scratch we have to find something that we can build on so basically I was just trying out stuff like so it means that some of the code that's running now in angular 2 it and so it means that some of the code that's running now in angular 2 more food and so it means that the flow of doodle and played around with that I had to pitch function with this controller which is the Ableton push and I just played around and then at some point we had a chorus which is as you I think all know by now people got JavaScript this one the people got mad JavaScript and then I started to build a beat around it basically to have yeah pretty pretty simple yeah built built bent around it with the bassline drums first things pretty loud everything yes kick 8-bit kick as well and then just build on top of it make a bass line and some melody and then we just need some people talking JavaScript people got that people got bad you got JavaScript people got bad yeah that's basically what I did thank you ah yeah so 2017 brought a lot of changes and an event member yeah you might have noticed I mean we kind of said it as well Martin joined the band and was doing live lights and the tech behind that is pretty amazing and please tell us how he did it yeah technically and more roadie than proper band member but so can we can we switch to develop to see you know all right of course [Music] all right so um who was here last year's last time Jess cogs oh that's quite just few people but I gave a talk at just of you 2015 and there I kind of like left through how to control professional light equipment using javascript won't go until that much detail here but yeah that's basically what I did is a light show Jas but if you will there you already know that it was actually light shall see is there because everything you saw this morning considering the light show was entirely written in CSS with proper CSS killed was interpreted by my browser in real-time and everything I did was just toggling um classes on parent element using the same here so each of these buttons has a class name attached to it and if I prefer the class gets added if I release it the class gets removed and there are some buttons that act as switches where I can just press class gets edit press again class gets removed so um this was basically how the lecture was controlled so in order to do that we'll see us as we need some HTML to operate on and lose basically how that looks I have devices and I give them class names I have some special attributes like type and channel which is the technical stuff and it turns out browsers just don't care as I don't know the tag name they simply treat them just like HTML so all fine here I know I can use proper custom element if you sure but - neato so um the CSS I have a color property I have a brightest property and I define an animation over brightness on color the color is pretty simple there's something that CSS a way to provide the brightness is yeah something that I just made up and considering more complex devices like those here they can move around and stuff like that there's lots and lots of functions and they simply made up CSS property names for each of their features and all of them can be animated so I add some JavaScript errors in this case I would just pull the get computed style of all the devices notes and I get the computed style returns conveniently of the current animation state - including any animation and finish those can really running so I'm back there I had good Calavera I get the brightness value and I sent them over the DMX protocol side note and DMX is the serial protocol that is used by each and any piece of professional lighting hardware there is so I can be pretty sure if I come to any venue they will have a DMX system and my DMX interface will completely work with that in theory so I'm the only thing I will mention about DMX now is that each and every device has a unique and yeah very specific pre-configured address which is what you saw on the HTML earlier so the back to the spheres have code there's obviously two problems with us first of all we can't just make up CSS property names because they will simply get ignored by the browser get computed style won't return them so that's a no-go only now we have custom properties so if I add two dashes everything works fine right yeah except that custom properties together the transitions and animations currently do not work as intended in any browser so I have to find some way around this and this is this nice little hack instead of just using my properties like brightness can - whatever else I simply choose one of the properties that can be animated like left right top bottom margin padding whatever anything that can be animated I can use as a replacement so in this case I chose like left for brightness as a replacement and it can animate the left property and yeah everything worked like intended so the only thing I need to do in order to get that done a little hack which is I pass the CSS now using the rework path or then I do some ASC transforms introduce the custom properties introduced for animated properties that introduces fall backs and then serialize the CSS again injected into the page it gets applied to my device node then in order to read all that out I have to reverse all that so I have to use get computed I could get in both cases left property value and apply it to the DMX devices as brightness in this case this works pretty well only all those doesn't really help to prepare a light show simply because there's never enough time to prepare if I only have more time right so the last comfortable I do that as I have like at best two hours preparation time at the band you're being connected to the DMX devices and as it is I'm rarely connected to such huge setups of DMX divisor so most of the two hours are spent fixing bugs and then I have maybe like half an hour to program the light and this is not something I want to live with and especially for this conference I needed to come with samples come up with something else so I had from the awesome people that were planning a layout of this conference I guess it was like a month ago or something like that these 3d images of how the stage is supposed to look and it's pretty much like that now right so I started to think about what I could do if I threw together basically any software that I could find and the end result is this so I have to build a Mexican which is basically just coq10 for lecture and the nice thing about DMX channels that I have those runner which usual 3GS and a frame will provide me for the proper preview of what it might look like on stage turns out it doesn't really look like that because obviously my colors are much brighter than the real ones but yeah um to the left top um you see the HTML or devices panel those of the list of all the devices are found over in the main hall so we have like can a lot of these these spotlights here and this LP the short for light panels will be 2 by 2 meters panel there's 35 of them yeah I simply just got all the channel numbers and so on from the people at the venue when I arrived here two days ago and yes that and this here you'll see a CSS code that was actually able I'd show so am a lot of mixer and configuration basic settings things like that and then there are the a1 is simply on the top left bottom button and of oppressive it will just run the color animation that I defined there yeah and finally they also got there was this moving have spotlights which we can also toggle and trigger and yeah so nice to think about them just you didn't see that because in order to see this moving have spotlights you need a lot of haze on or smoke in the room and it's not good at the beginning of a conference so that's at it looks much better here reality bite if I only have enough time so um all of this and don't judge me about this twelve megabytes of JavaScript and it really uses everything barriers basically so I have the code Moreau editors Thanks for writing them I know he's here somewhere I used that Jes was actually actually an ASM compiled version of look at CSS or look that for something like that so I'm using a fmjs as well I'm using a frame from Mozilla what makes it really easy to build these virtual reality applications using three.js left here and all of the other Souls III think none of you also saw these libraries ever thought it would be used that way but thank you very much [Applause] and finally I left out all of the details so if you want to know more I will probably be giving slightly more technical introduction and the community area at large is due tomorrow so thank you very much oh and one thing we're going to do both now right we're going to do this now oh there's more that's always more that's a couple of announcements okay ah yeah yeah it's amazing yeah you can show the t-shirt so we have t-shirts again as you can see and also hoodies actually stuff so there's a new teespring campaign tool actually teespring campaign so there's this one and then this one so this one says people got mad JavaScript this one says people got JavaScript and has all the speakers that we used on and so these campaigns we need at least 20 people who want a t-shirt so that it gets shipped and it's just after three days it just restarts so if you want a t-shirt get twenty people together and get a t-shirt or hoodie it's not like that and so the URLs how you find it so quickly people got mad jx1 and people got mad js2 okay that's the one announcement the other one is you can stream and download the track for fries on soundcloud so that's just what with the URL again SoundCloud become slash nested - loops people got mad JavaScript alright so if you want to listen to it on the go and stuff cool stuff is there is there more oh yeah everything is open sourced you can find all the stuff like the code this snippet I don't have the snippet all I'll get but the artwork and everything on github you can play around with it you can do stuff with it you can remix it whatever it's all open source alright that's it shall we do it again so this is just to show you that it's an actual software that runs on electron because we didn't change the logo there we also kind of use a hack where that's like an express DEP inside this electron app just to get MIDI support or something I know it was late and it's six months ago I didn't want to change it so let's let's put up this thing listen everything's loaded don't some four screen mode also to the to the Ableton people you have to press tab to go to the session view and yeah let's go yeah [Applause] hi hi hahaha will will okay yes no no okay I can just feel the excitement we're all real okay yes no no okay I did you feel the excitement Oh over return to demonstrate what a merry hope over return to JavaScript what a nice pop over we turn to JavaScript what are you how do we turn to JavaScript what are you why did you rent Angela : well people got mad when I put it all in one file why'd you write modular code well people got mad when I put it all in one file [Music] people goddamn people got people goddamn you got people goddamn people that people goddamn people can't people got that you got people got an up so I'm gonna say things which are going to be shocking and hurtful which are not directed at you at all javascript is a hot mess all they can do is add forth and they're busy doing that's the worst API ever developed in history that's why the gold standard is on now the problem with now the problem with is everybody knows right with JSON that when you get a huge payload from the server what is in there what what what what is in there I don't know we gained something that not because free shaking what huh I didn't really get that reshapes free free shake because we need small turn but no one understand it's tradition it's just some generic information you know not scary at all higher-order question you know the smell scary at all which ends with a function in function etc you know scary at all flow of data employed when helpless is a program kind of like the concepts of time or return to the scoreboard JavaScript because of that will be the goodness has unexpectedly become the world's most popular functional programming you want to do everything with there's more there's totally more there's always more [Music] mm everybody happy not players fearless Brittani not me I mean our sales everybody of minorities loved us vanity tight enough a KL concrete I mean like wear socks on this budget monitor dummies like when Hawks on this budget more than the media you need anybody's harvest 11:00 in the morning lastly what $1 Kalevala mr. Collins murder russamano college they are unfaithful cheer about key person is sterling holidays nested loops multiple of the day everybody will be bodies Raja has vanity toy innovate am i becoming liquid option budget penalty Blatz Blatz know Java novo JavaScript run easily brother anytime I needed it existing already to we go whoa miss uh no C++ no Java no go JavaScript run D show any browser any sense run on any device the biggest thing already still we grow thank you [Applause]