React on Rails

0 0

[Music] all right I'm going to get started hi everyone I'm Joe I'm super excited to be here this is my first-ever railsconf so that's great and I come all the way from Melbourne in Australia I wanted to tell you a story today and the story starts in Melbourne it starts with a group of 4 engineers they started off a company called Colt tram and it's company that I work for now they built an online application using Ruby on Rails to run employee engagement surveys and it provided real-time analysis of the results now today we have around 50 people working in our product team we have 10 development teams working on the product and it's built with a whole bunch of technologies we still have that Ruby on Rails monolith we also have some Elms from elixir and a whole heap of JavaScript back at the beginning though it was a whole other world and I want to set the scene a little bit so that this story makes sense so I'm going to talk about the products in the market where we started out I'm going to talk about the front-end development scene and what was going on back then I'm going to talk about the evolution that happen to us that happen in the front-end development scene sorry and then the need to change where we ended up so the products we had was offering analysis of survey responses and it had two main areas of competition the first one was consultants they would run a survey they would go away and spend a long time for juicing results in the form of PowerPoint documents or keynotes and they looked beautiful but they took a very long time and then the other alternative was things like Survey Monkey or Google Forms and these would give you plenty of flexibility and real-time results but it was very difficult to do anything with the data you had to do a lot of the analysis yourself so in this market it wasn't very difficult for us to create a pretty groundbreaking front-end we put in a bit of real-time analysis and some basic interactivity so that you could drill down into the results and that was enough to beat off these generic tools and the consultants PowerPoint now back then the popular tools for front-end development were jQuery and bootstrap now it was 2011 we weren't about the dark ages here so you could kind of be forgiven for thinking why didn't they use something better I mean back backbone at that point was pretty mainstream angular was gaining popularity but when you think about jQuery it was a game changer when it was first introduced its motto was jQuery is designed to change the way that you write JavaScript it was designed to get rid of the JavaScript that people were writing then with opening in line inside their HTML so they gave you a way to move that JavaScript into separate files and its current staff line now is write less do more it's really unobtrusive it's really easy to introduce it into rails in fact it was included by default with rails 3.1 which is where we started it just makes things really easy and if you look at bootstrap at this time Twitter had just released their version 2 of bootstraps it was a whole library of layout and grid components so suddenly column layouts were available to the masses it was easy to add we could drop it in from the CDN or we could just put the file into the vendor folder in rails unlike many startups we really just wanted to have a reasonably nice UI and smooth and interactions for our users and we wanted to get something to market quickly we didn't want to waste a lot of time playing around with JavaScript and CSS these are our back-end Engineers primarily so what more do you want than these things that make life easy so years passed by we added a lot of new features we had a lot of back-end features permissions data analysis these kind of tools and the front-end really there was something of a means to an end it didn't get a lot of love and rails also makes it very easy to manipulate the vendor files which is one of those things where just because she can does not mean that you should because once you've changed one or two trying to upgrade as a nightmare so we just had a load of spaghetti and meanwhile the JavaScript world was moving on some of the early benefits to jQuery one of the things that had done really well with standardized across browsers so if you wanted something that behaves the same in Internet Explorer as it didn't I was going to say Netscape there well how old is that and Chrome and Firefox and so on then it would give you that it would let you have those behaviors but as a result the library had become really bloated and browsers were really standardizing so we didn't need this place anymore users meanwhile we're starting to use mobile and surf the web on their mobile devices which meant they were demanding faster downloads we had to build smaller JavaScript libraries that did more so javascript in this environment started to really become a first-class language we had to think about how we structured it we had to start adding in tests we had frameworks like Jasmine and Q unit that will cut becoming mainstream I mean while we were back in this world where we had all this spaghetti code and when we tried to fix a bug another one popped up somewhere else I saw this video when I was preparing my presentation I thought this is exactly what it feels like when you're trying to fix one thing and it just bursts somewhere else and this is how it n so we needed a change now what was happening at the time was a react was emerging so I'm going to talk a little bit about that I want to talk about why we decided to go with react how we evolved our asset pipeline to use it and how we started to build the first component now reacts with open source in 2013 and at first glance it really looks like another shiny JavaScript framework right there was a slide in the Elm talked earlier with a lot of different JavaScript frameworks I don't know if some of you saw that but people who were starting to learn JavaScript now are really overwhelmed by the number of choices that there are so Reax is actually a little bit different one of the things that really makes it stand out is the idea of one-way data flow properties in react only flow down from the initial states through the components and each individual component in caps encapsulate its own props can't modify those props can't modify the global state the only way it can do that is through callbacks react components are a representation of the UI given a specific state that it's in right now so we don't within a react component try and change the Dom the way jQuery does instead it just render the representation of the UI and it uses this language called JSX when you first come to reiax it really kind of makes you want to vomit a little bit because you're now mixing HTML in your JavaScript components and you've just been learning that that's really bad you should have them in templates have them separate but over time you start kind of realized that this is actually a much nicer way of working so JSX looks a bit like this it says it's an XML like syntax and it looks like HTML it supports HTML type components as well as custom component tag jsx allows us to pass properties down on the component tree into the other react components now react is also different because every time the state changes every time we change the data that we're using to display the UI on the page but it rear Enders the component it does this by maintaining a copy and memory of what it of the DOM and it calls this it's virtual Dom so every time state changes it can compare the current version of the Dom with the previous version and so it knows which of the components need to be rear-ended and then it will tell them to re-render themselves based on the new state so by avoiding rendering the whole Dom it means that users don't necessarily have to re scroll all the time and it maintains the position of your cursor and so on it also is a much more performant if you compare this to jQuery it turns out that trying to manipulate individual elements in the Dom is much slower than if you just redraw the thing so if you imagine implementing a to-do list you have a to-do the user entered some text and then hit the button tip to add a new one when you hit the button we would call a callback which would update the overall state of the page and add the new to-do to it and react would then know that that to-do list needs to re-render itself and includes that new data and one of the other benefits of react is that it's backed by Facebook it was open sourced by Facebook so it's got a big company backing it building the tools and investing in it unlike some of the other open-source tools all of which are amazing and built by amazing community members but having a big company behind it makes it a little bit more reliable and it's being maintained over the longer term one other thing I met to mention it's very easy to integrate one piece of reaction to the codebase there were things like angular you have to migrate an entire page over but with reacts what we were able to do is take one small piece and then just put that component on the page alongside all of the other spaghetti JavaScript and leave that alone so that we could just have this one new component so when we thought about what javascript framework we should use a node ehh talked yesterday about how really we should have gone and tried out seven and done all our due diligence and really done the proper technical investigation but surprise surprise we did not but we did consider a few factors first of all our app is a b2b app the users that we have are primarily people working in HR and they're usually accessing the app on a desktop computer so we were able we didn't have to worry about things like rendering on mobile phones rendering on small screens we could be quite targeted in the browsers that we had to build for and we can restrict our support so we actually support ie10 but we don't support anything less than that and we just support the latest versions of the other browsers so we were pretty lucky in that we didn't have to worry about these older browsers another consideration was that we had hired somebody there was a react expert so we had somebody in the team who could help us with this transition and I think this is a really important aspect of the decision for us we also didn't have to worry about SEO react is known for being able to do isometric JavaScript so server-side rendering and so on but for us we didn't that wasn't something we needed to do so given SEO wasn't concern it's actually quite difficult in rails as well to get reactor to work that way I don't know one who's done it so not having to worry about that kind of opened up our choices now one size doesn't fit all teams I don't want to try and promote the idea that reactors the framework you should all be using it was right for us at the time now before we jumped into building react components we needed to have a few things in place and JavaScript had really moved on from the moldy old code we were trying to work with we weren't even using any es5 features let alone es6 but out in the big wide world there were all these amazing things that have been introduced with es6 it was a really significant update and again it contributed to making javascript a much more first-class language to establishing it so it fixed some issues though block scoping and immutable constants were introduced with es6 we also have some took sugar so arrow functions were introduced we had string interpolation and we had the spread operator all of which made our lives of JavaScript developers much easier es6 introduced classes which some people love some people hate and it also introduced promises so that was something that had a long for a long time been part of jQuery but now it was part of the core JavaScript okay and although es6 would be a presentation in itself is not more so there's a really great repo by Luke Haven that you should check out if you want to see more information on the features now not all browsers support es6 features so we need to choose a transpiler and babel is one of the most popular ones we had to figure out how to get it to run in the asset pipeline and what the transpiler would do was to use appropriate polyfills when we started using es6 features you can tell it which browsers you want to target and it would introduce the polyfills when they were needed and back port the new es6 styles like the arrow functions to the older style JavaScript and Babel supports enabling different feature sets you can tell it what you want to use and what browsers you want to target now the default rails asset pipeline uses sprockets all of your Java scripts your images your CSS is processed by Crockett's before it gets on to the clients and sprockets users require trees so if you've written JavaScript files where you've required another file in it sprockets goes and gets the content from that other file puts it in as a bundle and then that's the one that gets served to the client in production when it creates these files when you run your assets compilation it puts a hash of the file contents into the file name so every time you update and you re release a new version it's making sure that because the file names changed it's preventing the browser from caching that old file for you now sprockets doesn't support transpilers like babel there are gems now to do this it was going back a while when we were trying to introduce it and so we looked at some of the tools that were being used in the JavaScript world so web pack and browserify were the two most popular and both of these are written in JavaScript unlike sprockets which is obviously written in Ruby now we're in a JavaScript environment processing javascript code so it kind of makes sense to use a tool written in JavaScript to do it so our first steps to integrate web pack into the rails asset pipeline looks a bit fifth we have a gem that would run we needed to run a node process in order to run webpack it needed to run inside mode so we wrote the gem that would run a node satellite process alongside the rails server and the gem was called Ruby node task we then had a separate gem called web pack rails and what that would do would run a pre build process within the node process so that we could incorporate these es6 features and the JSX templates would duel the transpiling within that step once web pack had finished its processing then sprockets would do it processing and bundle these new web pack bundled files in with the old JavaScript and then output them as normal now this was pretty gnarly I'm not necessarily recommending this is a good solution but it worked at the time as a first step it got us to the point where we could actually start using react but it was pretty soon when we found we needed to to update it one of the biggest problems we had was that source maps which is supported by web pack were not supported by sprocket so it was we weren't able to debug when we saw an error we would just get a long back trace and we couldn't figure out exactly where in our original files it was coming from so instead what we started to do was and we would separate I got it on the right slide well yeah we tweaked the fess up so the files that were processed and bundled by web pack where they're no longer being included into these legacy files that were being bundled by sprockets so they were treated as a completely separate asset and the impact was then we had two tags in the Hamill it wasn't too much of a big deal we tried to avoid that but ultimately this meant we could have source Maps we could have hot module reloading and development environments and so that made it much easier now we needed a way to introduce our react components onto the page and we were using Hamel for our template language within rails so we introduced this method into that we could use in our Hamel files it's a top level that goes in sorry it's a method that will add a top level component into the page and passes state from the Ruby side so we wanted to be able to pass props as a ruby hash into this component so we just output it as a content tag its output as a div and we use the data attribute we put the component name inside one called data reacts component and we put the hash inside reacts attributes we then had a JavaScript method that would register all of the components we intended to use as top-level components in the page this way and so we just registered them as the pages were loaded and this created a map with a component name and the actual component itself and what it meant was that we could then mount them on the page load and the mount components method looks at all the HTML in the page and it picks up any node in the HTML that have this data react component attribute it can then pick up the component name and the component props which have been loaded into these data attributes and these get passed into the mount component method so now it knows what the component is it knows what its props are and it knows the HTML node where we want it to appear on the page and we pass all of that to react from render and that does all the hard work of outputting it there for it so after the page is loaded then our react code loads and hey presto we had react working in rails so then we wanted to evolve a little bit move on and do some more interesting stuff with it we added some jest tests we added CSS modules in the end we lost sprocket we introduced reactor out and Redux for some more complex code we introduced animations and we also introduced immutable States so when we first started using gest we wanted to have with this beautiful clean code that we'd introduced we really wanted to have unit tests from the ground up just was created to work with react again it's been created by Facebook it's quite different to other frameworks like jasmine and Q unit were - we've been using before when we first introduced it it was designed to mark everything by default and this is really weird it's a really kind of difficult learning curve in our difficult minds that change when you first start using a framework where everything is marked ultimately I think we got to the point where we actually found that that made us write better code because you could only test the thing that was that was actually the thing you were trying to test the one unit so the test became much cleaner and just testing one individual thing they have actually changed it now so if you if you install just now the D what is that it doesn't mark everything they've written a blog post to explain their reasoning but largely it's because so many people just found it such a mind shift and so different and difficult to get to grips with jess is also introduced oh sorry I want to talk about shallow rendering and this is how we were able to test the output of react so given that it's rendering components we had to figure out how we can actually test the output because usually it would render into the Dom it would rend it into the browser dumb and shallow rendering as part of the reacts test utilities library what this does is rather than actually rendering the component into the browser Dom it renders it as a kind of an object so it just renders one level and you can then see you can test that the outputs of that rendering are what you would expect and we used it in compare in combination with a package called react shallow test utils and what that allowed us to do was to pick up nodes in the output and test whether the props were correct and whether they were appearing as we'd expect it and so this is what you would get from this shallow test utils that we'll go and find a node in the output and it would have something like this which is a normal JavaScript object if you output it to the console so it's quite easy to just inspect all the different props in it now as I was about say a minute go just more recently introduced snapshot testing and so what snapshot testing allows you to do is that you can save the output of your component and it saves it into a file so that every time you're updating your component you can just compare across versions you can decide if there's a change in the output is that something I meant to have there or is it a bug that I've now introduced what we've done with our components over time we've abstracted a lot of the logic into very small files very small JavaScript components so whether it's branches or case statements or actual business logic in the components we try and pull those out so we try and adhere to the single responsibility principle and just have a lot of these very small modules just doing one thing well and this is my testing of the logic of these much much more straightforward and if the component is just rendering a start a set of props there's really very little value in creating a lot of unit tests around it so the test can be focused just on the modules that need it now we also introduced CSS modules if this is something you haven't come across before it's a small implementation of names based class selectors for CSS so rather than having a CSS class that's just global across your application you can have classes within a CSS module that only apply to the one module where it's included and it was created by two prominent Australian front-end developers glen madden and mark valve leash so for us we were able to have for each of our javascript files a corresponding sass file or a CSS file so each JavaScript component had ownership over its own CSS styles and it lets us avoid the hell of CSS class name clashes when we were using these components in various places on the site our CSS modules uses a tool called post CSS and what that allows us to do is to process it with our JavaScript compiler compilers with JavaScript plug-in sorry so we could use webpack to process the CSS modules and output them as JavaScript and then within the JavaScript foe you can just use them like a JavaScript object the next thing that we wanted to do was to try and get rid of sprockets we were really starting to feel the pain of working with sprockets together with webpack in our development environment especially we wanted originally to try and do things the rails way but we wanted to kind of avoid a change that felt too big at the time but people getting used to web pack now and we're also finding that if it if it got stuck compiling something we had to restart the whole rail server because of this node satellite process that we'd introduced and the code base was growing in complexity so this was happening more and more often it was a real pain and we still couldn't sought support source maps in production because ultimately the code was still getting compiled by sprockets so the solution was to entirely replace sprockets with web pack the one problem that we had left to resolve here was that sprockets does this as I mentioned it it renames the files with this hash in the file name so we had to find a way of setting webpack up to do that same thing so that our production files weren't being cached by the browsers and the way sprockets does this is that it maintains a manifest file and it Maps the original name of the file that's going to be compiled against the name with the hash in it and then when you load your files in development it's going to get them from the local machine when you load them in production it's going to go and get the file name with the hash so we used an NPM package called web pack manifest plugin I think and that does the job of creating this manifest file for web pack and doing having the same functionality as sprockets is doing so then we didn't need two buckets anymore and web packs serve all of our files in production then we wanted to add single page transitions we had all this fancy code we had all these nice new components in react we wanted to use them to create a better user experience and this meant that we had to do introduced redux we needed something more complicated to manage the overall state so at a high level we also introduced react router at a high level Redux as a state management library and it was heavily influenced by flux and by Elm in its design so the state of the app is stored in a single global State a single global store and the only way that you can make changes to this store from a component is by emitting actions and actions will contain a payload so to go back to the to do list example if you add it if you omit an add action for it to do it will contain the details of the to do this interns call Corvo a method in the reducer which knows how to produce an updated version of the current state we don't change the state we don't actually edit the object itself we create a whole new version and that gets sent back to the store this then updates all the components reiax looks at its virtual Dom and decides which ones need to change and passes the updated State down to them and reacts router is a JavaScript ralphing library which allows the URLs to stay in sync with what's on the page there's a newer version of it version 4 and it's very very different to the point where there's a lot of stories on the internet about people who've tried to upgrade it matches the components in the URL to the components on the page that they can decide based on the URL components which pieces to display we're still using the old version we haven't managed to upgrade yet because it's such a big change and they are maintaining the old version indefinitely although I think if you're starting a new application now it's probably better to go with a new one now in our world we wanted to implement actions that could be called in response to a user interaction like a click so you click a link to view data for a specific question and you can have actions that trigger actions so the first thing we did was to trigger an action to say the URL has changed this triggers a second action which checks if you require new data to be loaded which if you're going to view a different question or a different page nine times out of ten you do unless it's the current one so this will change the URL and this would trigger a new action which would go and trigger an ajax request it uses fetch which returns a promise and then when the Ajax request returns it resolves the promise this triggers a new action again which updates the data or tells the tells us that the date has been updated and finally this goes to the reducer which processes the data the components of them re-rendered with the new version of the data so this is a very simplified overview of how we act router and redux are working in our application I really recommend if it's something you want to learn more about there's some awesome videos on egghead and a lot of them are free and they have a couple of series on the rear on redux and on react router so that's the best place to go we also introduced some animation with our single page transitions we needed some kind of feedback in the UI to show the user that the page was changing the usual way we do this is with animations and so we found this library called reacts motion and it did things like shifting the bars from the old value to the new value and the data was changing the NPM package and it supports a thing called Springs which is a way of animating animating things in the UI but unlike timed animations one of the nice things about Springs is that they're interrupted so you get these nice realistic kind of animation curves but if you have to stop halfway through it can then jump from wherever it is to the next place which is better than the usual timed animations that you would have in CSS some issues that we ran into you well when we tried to render PDFs we had to make sure that the animations were not running then and in our tests as well we had to find ways around it because we didn't want to check the Cavalia of the components with the animation half run now immune sorry Redux is based on the principle of a mutable state and we wanted to try and use that more across our application so this is one the libraries that we introduce very early on immutable jf which has helped us to think in terms of a mutual data and in terms of not changing something in place but returning a new version of it so it provides a number of different objects that we used map and list were the ones we use the most and the normal JavaScript if you edit an object or an array it just said it's the object in place but what immutable j/s does is that it will return you a new collection with the changes that you want without changing that original object so it's protecting you against a lot of subtle bugs that can come up in your JavaScript files and it's also highly optimized if you try to implement this on your own you probably end up in a state where you're running out of memory but it uses a lot of shared structures underneath so it can it's optimized for performance and you can use this a lot more safely in your code and this is an example of what it would look like if you changed a map and a mutable map you can see that the original map hasn't changed we also used record types from immutable jf and we use these to define specific types or data structures that we could pass between our components now there's a move towards more strict type checking in the JavaScript community in general and newer front-end languages like elm already have static types built in reacts allows us to define props for each of our components and so what we were able to do is define types with this rest of these records that would have specific properties that we knew should be present and we created this strong record class as a wrapper around our immutable j/s record so that it would present a small warnings and we could really check does it have the properties does it not have any that it shouldn't have and then when you pass the prop types to react we could do more checking to make sure whether something was required or not and what props it should it is expected to have and in the development environment it would give you warnings in the browser so you could see if you weren't passing the right thing so again this allows us to identify subtle bugs much more easily so what have we learned on this journey we've learned a lot for start it's helped us to introduce a lot of new patterns and better ways of approaching and structuring our code reactors inspired from functional programming principles it's something we also see happening in the Ruby and rails world there's more kind of functional programming ideas coming into that so we had a bit of a head start there and it really makes you think slightly differently about how you structure your code in any language for example state never being mutated creating a new object to store in state and passing things through functions together a new value back instead of modifying them and we've found that we end up creating cleaner code it's easier to test because you're just testing inputs and outputs instead of having to mock behaviors so does this story have a happy ever after well legacy javascript is still a thing we haven't got rid of it yet we now have a massive library of react components and we haven't given an awful lot of thought so far to the reusability of those components so that's starting to be an issue we have a number of different components that do very similar things and we do still have a lack of overall consistency in our CSS where we're using CSS modules it's awesome but we also have a lot of global Styles still and I never knew there were so many different ways you could make a button appear but I think we've got about 15 so so this story has an epilogue we are moving towards more of a shared component library something that a lot of companies are now doing is implementing a living style guide so people like ambato Lonely Planet Airbnb and Atlassian are all companies that have published their style guides in Lonely Planet's case they have a gem that they use and Atlassian they have a lasting UI which is a Java package that people can actually use to use their components and styles we're also experimenting with in alternatives like Elm we've got about a quarter of our application now which has an elm UI integrated into the react one and we are starting up a team that's going to own this front-end experience and try and replace all of those 15 different buttons with at least one or maybe two and I think since we started our journey we've really learned a lot our front-end code has moved on and the world around us continues to move on and evolve so it's still not perfect the biggest difference though is that we now treat our code much more the first-class citizen on the front end not just the spaghetti mess means to an end that it was before and we hear back from our users that they can really tell the difference because of that so I guess whatever you're doing wherever in your journey you are whatever library you choose to use I hope you may have got something out of our perspective and our experience with react for males thank you [Applause]