From Development to Production with a Free (and Fun) Cloud Stack

0 0

ok em hi guys I glad to see you all and let's try to do this fun as possible em it's a little bit of short presentation a long demo hopefully in all i'm going to talk about CloudStack so all the demo is on the cloud all the demo is in the internet and doing a internet demo in a conference it's kind of scary but i hope we're not on the main floor so hopefully the hot spots here we'll a will hold and everything will be good so my name is Barack I'm a developer advocate at j frog and explaining about Jeff rope in a second but developer advocate first of all this is my job description I hang out with the development with the operations Jay frog or users or customers and then I come to the conferences like this to tell you about it not an easy job trust me Jeff frog is a startup we're based in Israel and California all the development is in Israel I'm based in Israel to and as you might noticed we have a little bit obsession with frogs as you might see and if you still didn't get the Frog t-shirt a come to our booth to get one and we have two products one is more mature artifactory it's been in market for six years I think and the other is relatively new a bin tray in the distribution platform has been a little more than a year in public access in a public beta today I'm going to talk about this synergy between both of them and this is quite exciting because I'm doing talks about alternative factory for a four years now and talks about being tre about half a year now but I think this is one of the first times that i'm going to talk out the synergy between two of them and I'm excited and let's see a how to work so it's all about getting your creation to your users right this is what we're trying to help you to do how many of you develop software okay how many of you develop software as a pet projects open source project something that you do for fun or for your own development okay that's great this dog is for you because we're going to talk about how you the developer that have the day job and try to do some pet project maybe only weekend Mabel in the evenings may be honest they instead of sleeping mostly and you write your own brilliant code you have this idea you have some package that you want to deliver and on the other side there are masses that wants your creation right they want it so much and the question is how do you get it how do you care you're brilliant code and deliver it to everybody oh the steps are kind of trivia you need a place that you will host your sources on right and we start there and then you build your binaries beat java beach dotnet nougat packages dll's rubygems NPM packages if you create some a no J's library in maybe pi PI packages if you develop some python library whatever it will be you need the build of binaries then you need to store them somewhere right during the development you have some what we call snapshots or integration versions you need a place to store them in the meanwhile and then we have two options to deliver it first of all is deployed if you develop software as a service then you take your binaries and deploy them to maybe one of the platform services google apps Heroku in Amazon Elastic beans whatever right this is one option or the other option is distributing your binaries as a downloadable file it could be something like I don't know commons lang or Giada time or some kind of library that other users will consume by their dependency management through their maven by declaring it in pom.xml or whatever or it could be an executable like msi or executor or installer or maybe even some virtual machine or dr. box or whatever it will be it's downloadable so we are going to talk today about the second option about the distribution of the binaries as a downloadable software everything that I am going to show you is happening on the cloud and this is important for you because again as I'd say you do it instead of sleeping so nothing up that you are going to write this code if you are going to need to maintain two or three servers one for the sources one for the binder is one for the integration etc etc it's not fun you want to write the code or you want to sleep you definitely don't want to babysit any servers so taking something in the cloud that you need that you don't need to take care of it's important benefit and of course it should be free should be free because you probably do it now for non non profit reasons don't have the money to pay for all those services so those are the two requirements should be should answer all the list that I mentioned earlier should be in cloud it should be free so let's start with sources you write your code you commit your source code to the source control source control in the cloud is easy give me to github and google code is the third one bitbucket thank you so what you want to get from this server is managing the issues managing the documentation kind of wiki markup whatever it will be optionally a site or a blog which is kind of connected to the commendation so the options that you have is sourceforge if you remember that service guess what it's still alive maybe google code they had lately some changes or you cannot host binaries anymore there but we are talking about source code and they are ok for sources they have subversion service they have good service and of course the big two are a blessing big bucket and github naturally well who never worked with github that's exactly why I won't show it to you today you all know get up I also a big bucket most of you heard about it most of you even maybe tried and so it it's very solid competitor for github but a less popular so i will show you and i will show you the demo with bitbucket tori next step is the CI server right CI server as you all know babysit a watch the source control and once there is a new commit in the source control it will start the build it will build your software and of course again they are a lot all of you know Jenkins how many of you know that there is a Jenkins free Jenkins instance on the cloud called build hive from cloud with the company that bakes up Jenkins and there is a free Jenkins instance in the cloud that everybody can get any count on it can monitor github it can monitor bitbucket trigger build and build everything now what is the big strength of the biggest benefit of Jenkins the plugins ecosystem right when we are talking about shared instance in the cloud guess what you cannot deploy any plugins on it because it's not yours it's like everybody's so if you like Chuck Norris plugin for Jenkins because it shows Chuck Norris quotes or you prefer green balls instead of blue balls you cannot do it in the cloud this instance it's not your instance so the biggest benefit of Jenkins is not available when you're working with build hive although it's still nice still Jenkins that we all know in life now there are a whole bunch of new CI servers this is like a movement oci servers to the cloud I will name some of them code ship it's a very nice one does what they say I server should do monitor source control builds your software and optionally it could deploy if you're building SAS it could deploy to Heroku to a to a amazon etc my issue a little bit of issue with courtship the project is like private by default so only if you are on the team of this project you can see the bills you can see the log you can see the configuration and cetera et cetera only if you were invited by the owner of the project it's understandable but a little bit contradicts the idea of open source if I can see the sources on github bitbucket why can't I see how the build went so this is my a little bit of issue with Scott chip Travis CI I think this is the most popular of them all Travis CI I think it's the oldest in maybe two years now M it was one of the first it started all of them interesting enough started with Ruby support all this movement of CI servers in the cloud the first part they provided support for Ruby and then they go to other stacks like JVM etc etc so obviously i also started as a bi server but now it supports everything and it's good frankly i cannot even name one significant difference between travis CI and drone i 0 which is the next and I'm going to show I think it's just a matter of taste and they do have different this distribution target platforms the ones that you can deploy to which are not relevant in our case because we are not going to talk about deployment to the SAS platforms instead we're going to talk about distribution of the executables themselves so Travis line drawn aisle for me they are pretty much the same next in list is a record this is a new guy it looks very promising on better now a the interesting thing about Vega well how they are all work there are work the same they take the creative virtual machine for your build they just power up a virtual machine and run your bill which pre-installed with your build tool and pre-installed with the version control that you are trying to use so usually if you are let's say want to build Java may them on Travis CI you'll get virtual machine with maven installed and get installed and then get will fetch the sources and maven will build them this is the idea this is how it works then this is not about regular is that you can create your custom virtual machine you can create your custom image so you can decide on which java version do you want which may we want etc etc those are not available also shin are not available here you just get what they give you so they will give you the latest java they will give you may be the latest maven or Gradle but this is what is it and if your build won't work on May and three but only works on maven to you are you have a problem they won't support you but here in wrecker you can create your own virtual box your own image and upload the tools that you need in order to work and this is kind of a this is kind of nice it works with big bucket that works with github strongly in beta so I won't show you in denial em again for the same reasons I believe that if you are familiar with this family of CI server so probably familiar with Travis CI Drona is very good alternative but maybe less popular I will show it wrong anyhow okay this is the second piece and now we are going to the third one your CI server creates artifacts now if you are working on your own machine if you have Jenkins or teen CT or bamboo pre-installed you can select what they want to do with those artifacts there are build servers that give you more support of storing and managing them for example teamcity have their own like artifact management inside so you can select how you store those artifacts when you clean them up etc etc and there are that they consider the produced artifacts as not of their business right so for example with bamboo they don't care what you do with the artifacts they won't give you much support Jenkinson's in the middle you can discard all the bills with all the artifacts but anyhow you need to do something with their artifacts when we are talking about CI server and cloud because once the build is over your machine is destroyed with all the work space and the target directory inside the workspace and your artifact that will just build inside your working directory so if you don't take them out of this virtual machine during the build itself they are gone it might be okay with you but I hope it's not because you do care about those artifacts those are your snapshots you want to test them you want to check if they are okay etcetera etcetera so you need to save them before they are destroyed and you need to put them in binary repository and of course biorepository you probably a familiar with at least two of them I hope one of them is this guy what is this logo its Nexus sonotek Nexus which is the maven repository decent one and the other is artifactory wishes of course our guy well here my my selection was like kind of easy I will show you now just kidding you will see artifactory the interesting part about the factories you will see is that you will see a free instance on the cloud right so as I mentioned on the ground it's free we have this guy even have an open source project you are more than welcome to use our artifact or Easter instance so our factor it will be so far I pretty sure that nothing I told you nothing you maybe didn't hear all the names you didn't you it wasn't familiar with all the names of all the tools but the picture so far should be familiar and now the interesting part kicks in and this is the distribution platform what is distribution platform well the most known distribution platform specially for java developers is the maven central right you all know about it what was called rapper one what was called maven central what is called the central repository now this is like a huge warehouse of Java artifacts in which you should find most of the artifacts that you need X set of those that you cannot find them for reasons there are other repositories in the world so who ever downloaded artifacts from a bit central everybody if you use maven you used maven central it's baked in its call it hard coded inside maven and if you run a maven build you download your artifacts from there whoever submitted artifacts to maven central okay so I have one brave guy well let's see how to submit a lavarse su maven center you have the official guide and then you have another guide and then you have another guide and then you have another guide and another one and another one and one more and one more with the comics you didn't read that you didn't read that because they are very small because they are very long and they are very long and there are so many of them because of two reasons first of all maven central is popular everybody want to get their jars to maven central because other maven users and world are going to consume it that's one reason the other reason is it's very painful to get stuff to maven central those are just the summaries from the from the same guides that you saw earlier people suffer they like I will try to do it it will take me multiple days and probably I will give up in the end right okay and there are strange decisions and it's like quite hard to upload one of the guys when we try to upload artifacts to maven central created this if you have phone and have QR code scanner and you like good old adventure games like la request one should like this one you will have it on PDF later bottom line it's not fun so there is an alternative and one of the alternatives is bin tere bin tray is not what maven central is melons and roll is just bunch of jars right it's like I don't know take a huge apache server put all the jars in the world there you're more or less get maven central building is much more we will see it in a demo first of all it's not only for java and eternally for maven and it's an ongoing for jars it's much broader than that but it's also for jars it's also for maven and okay so I told you it's a new product so you probably want to know who use this anyway because if you're going to rely on your distribution platform you want to know that it's kind of popular otherwise no one will ever find your stuff so here's couple of examples Gradle which we truly believe is the bill to love the future and going to kick mavens s very very hard in very very short period of time if you are not familiar with greater that's the time to jump on the wagon Gregory's supports J center which is the Java repository inside bin tray out of the box groovy when you use grub which is run time dependency management in groovy you say I'm going to use some class from commons lang bring it to me I won't write any build scripts just give me commons lang it brings you commons lang from bin tray groovy itself as a language is hosted on a wintry so now when you download if you are one of the three million people the download groovy in the past year if your download groovy today you are going to download from ben tre SBT the scala build tool other pluggable nature if you need it plugin from SBT you will get it from bin tray Nettie project you know Nettie this is a new i/o like super fast socket server download it from vintry a vertex the javascript very popular JavaScript library download it from being tray if you need Ruby installer for windows if you understand the Ruby on Windows you will get it from bin train vagrant again etc etc the usage is as simple as maven central all you go is like repositories j center of this is example from Gradle this is how it works and soon enough much more to come grails you will get the plugins from Grails from j center into the four hopefully next month and Gradle all the plugins of Gradle will be served from being tre again very very soon hopefully in a month from now so all that who guys all the new guys are going there but we still have our maven right and again just a quick poll so you want a full asleep how many of you use maven how many of you use Gradle yeah so exactly you will all of you will eventually switch but until then maven have center built in and you when you want your library to be popular for all those maven users out there you need it in maven central and bean tree delivers so once your bin tray you can go to the maven central as well and you won't have pages long guides on how to do it once you're on Ben Tre you just click sync and boom it's in maven central and I think that's the demo time hopefully it would be like that I hope so okay so the only thing I will need for my demo is a browser okay and maybe mouse as well so what we're going to do we're going to do the whole cycle that I mentioned earlier commit the sources build the sources in a source control manage the snapshots in a factory and finally release it on wintry and seem to maven central this is the plan let's see what will work out of it so this is a very simple Gradle project and i will show you great i'll just be because it's more fun but it will be the same with maven basically let's see what the build look like build upgrade oh this is the Gradle script so as I mentioned you have the JSON the repository to get the plugins from and we're going to use the refectory published plugin as the factory publish plug-in does what the name implies it will publish to RT factory in the end of the build remember I said don't forget to take your artifacts out of your draw nigh on machine because the machine will going to be destroyed and here we have the regular staff we are going to build java and this is our version we build snapshots right and we're going to release a after we are going to have some snapshots inside and there is nothing fancy into it the angle the only interesting part is here so we are going to publish our Java components and the sources and the Javadoc we are going to publish sources in Java doc because first of all your publish open source software so you want your software your sources to be published and because this is a part of maven central requirements so if you want to get to maven central you have to have sources and Java Docs with Gradle it's that easy then you need to have proper pom.xml file to get to maven central again this is their requirements so in Gradle you need to go and tweak the pom.xml file you need to add packaging the name the description the URL the licenses the list of the developers who do it and the source control management where to get the sources again all those are truly optional if your target platform is bin tray if you want to go to maven central as well those are the requirements one of the pages long explanation how to get to maven central include these stuff that you must have so and a this is it now another very interesting part is here this part is telling Gradle how to deploy to artifactory remember it's the same in Nathan it's just less fun because it's all XML here it's groovy so that's the reason I'm showing you Grail and here we are going to ok we're going to push to publish 20 ssj4 work also a prologue this is the public instance of roti factory which is as I mentioned free to use and you are more than welcome to use it we are going to publish to snapshot repository and look at the username and password that you use you actually use your bin train credentials in order to publish to artifactory or ssj4 work and that's because we are letting you into the artifact or instance the free artifact or instance only if you are going to use bindry that's simple so as a understand you need AB into account in order to use it let's see what do you have here so this has been great and as I mentioned totally free to use an open source you register here and you can use your big your github account or euro your google account Twitter or just username and password and once you have your credentials you are going to need the API key you're going to edit your profile edit your profile and here you will find your API key the API key is the password is your password to open source a frog or next thing that you need to do you need to create a package package means whatever that sounds it's one of your product one of your could be a jar or a bunch of jars whatever you release under single version you need to place your package in one of the repositories repositories can be depending on the type of the software you create it could be maven repository for jars it could be a debian repository for deb files RPM repositories for young files etc etc we are going to use our maven repository and here you create a package Gradle demo will host the Gradle demo project from our bitbucket first thing you need to do after you created the package and provided some information about it is linkage to j center linkage to j centers says very very simple thing I want my package to be part of this big repository that everybody use here when we write our build we say bring me the artifacts from Jay center right this is what we said Jay center is exactly where i show you now you want your package to be part of Jason Bentley Jeff of entry other owners of Jay center so we will need to accept you too Jay center your package to j Zander if it qualifies it's valid open source project once you are there you will get the opportunity to stage your snapshot on ssj4 work ok so again you have your package you create app into account you select the report you create a package in it link it to Jay center and ask for snapshots for access to snapshots repository in audio you're done now once you are approved you can start a build so here's our build this is drawn in drone you connect it to your big bucket repository it will show you least once you logged in with big bucket it will show you a list of all your repositories you select which one they want to build and you need to give it the command so this is a gradual command that actually builds your software it goes like gradual w clean build artifactory publish this is something like nvm clean deploy right in we're talking in a in maven language and I also provide additional information from drone aisle to my artifactory i will show you in a second why i need it so i say the build number will be whatever drawn considers a valid build number and the build URL will be whatever drunken see there's the build URL and the build agent is drawn i'll provide additional information to my build this is done the same with maven maybe with minus D ok now let's trigger our build for that we need to change something in source so i will edit it and i'm going to change the version of my of my package it will be 1 the 2 snapshot commit okay not bad now I expect my drone aisle to build another build here we go right trigger the build from the changes and the sauce I can see what's going on it's a new virtual machine so downloads Gradle unzip seat and start the build now the artifact resolution all the all the artifacts are resolved from j center beam trade calm because i asked them to be resolved from j center this could be any maven repository ok it will take just a second more it's a very short build okay think we're towards the end already everything is coming from j center and eventually everything will be uploaded to our tea factory here we go deploying artifacts hope this is what our plug-in does the plugin that we used in Gradle also jefra gorg snapshot local and then the job deployed the Javadoc deploy the source is deployed and something that we call built info is deployed as well this building for is very important because this will build info is the bill of materials of your build it describes how the build went it describes all the dependencies that were used all the artifacts were produced of the meta data now let's go to artifactory OSS or SSJ formulas org and here we go let's go to the artifacts and I want to see my build so this is all the list of all the bills which are deployed to artifactory episodes 0 SS dot org there are like a lot of them eNOS you can see groovy here you can see in red peg here you can see a netflix plugins a lot of a very a cool interesting guys this is ours it was the last one Gradle demo build number for this is what we just built you can see the time matches so let's see what we have here this is the gradual demo build number four it was built by Gradle it was built on drawn dot IO how do I know that because this is what I passed the bill agent and this is the URL the URL you remember again I passed it from drawn so in any given moment of time I always can go back to the drone page of my build and see how it went see everything in here so except of that of course i have my publish modules and this is the only modular published it was single module built here you can see all the artifacts that were built and deployed to artifactory this is the same list this is the jar file each and every file that were built are now tagged with the properties of the build it does part of so this artifact always knows that it was built by Gradle demo and the build number four and from here of course i can go build to my build go back to my build here is the build tab and I can go to the build browser game okay so far again nothing that new of the factory knows how to do this build integration for I don't know like four years already now and this is not very useful for the outside world this information is for you you can look into this instance with your bin tere credentials username in da Peggy this is your stuff you don't distribute snapshots now once you have a snapshot that you believe that it's good to be released you want to release it to bin tere now the question is okay how do I do that first of all how do i convert my snapshot to release one option is i need to go to my prom xml or to my Gradle build and change the version from being one or two snapshot to be in gwanda to this means release and this is a good solution for very naive and toy projects like this if you have significant multi-module built you have to change them anyway anywhere including the dependencies between modules which should be now instead of being snapshot being a releases this is like not fun what we can do for you we can convert the snapshots to become releases and push them to Ben Tre all in one go all you need to do is to invoke a rest query so i will open comment line here should see it is it too small should i make it bigger right okay let's make it a little bit bigger I did it again eh maybe min 24 right that's better okay good okay so let's go to my project I have here the Gradle demo no okay Gradle demo and now I need to run a REST API that will go from artifactory on in the cloud convert the artifacts to be release and push them all the way to bin tray I have here a cheat sheet that I'm going to use okay here we go this is what I do i use curl post user jay from demo this is the user of been 30 ssj4 gorg api plugins build promote i want to promote my snapshots tube in Trey Radel demo build number for now I need the password which is my API key excuse me for not showing it to you please did I paste no it works okay so have been successfully promoted now will happen now I really expect it to be unpinned ray let's see I'm excited pop oh here we go and let's see what happened let's refresh this page you I have a new version 1 to 2 this was the release version of the snapshot that we built and it's here in bin tray ready for you to tweet about and celebrate first of all probably you are already done especially if you are not building java jar files that are targeted to be consumed by maven you are done already because probably Pinter will be the last stop now we are going to give those links to bin tray and here you have like you see you have the link to the version itself or you can give you can put a nice beige like I did here where is the source here this beige by the way just change to one or two because we've just released it was one that one this beige anyhow now you have a link to provide for the downloads and this is what you want to give now there is as I said there is one exception and this exception is your target audience is maven users maven users don't want to go to Binh Tuy because then they have to change their settings.xml and you don't want them to be to work so hard you just want your jobs in maven central and as I already mentioned there is a very simple way to get to maven central all you need to go is go to the dinosaur page in the venus out page you should read this line this line set from here you are on the mercy of sonatype which means you have to work out to be compliant for all their requirements you did most of the work you remember this ugly piece of tweaking the xml file the pom.xml this is most of the work there is one more requirement that we need to do and we need to sign the artifacts with a digital signature here you can say hey it sounds like a good idea anyhow and I would say well the problem with all those signatures is that you can take any name any user name and pretend generate key and pretend to be that name right so we'll feel like being Steve Jobs you can create a key pair for Steve Jobs name and email address and sign your jars with its with this keeper and it's like worthless right because no one will actually check your identity but that's a certain type requirement good thing is we have it building inside the entry you can add your key pair your public key and a private key to binh tay profile protected with the passphrase so nobody will be able to use it and then sign a version all the files inside the version all the files that we just a promoted to hear here you see all our files we need to sign them automatically we can do it with a simple arrest call again and again I have a nice cheat here so here we go curl musics post Minos Yuja from demo our user Minos d this is the JSON file that contains the key phrase that protects the keeper and then I go like ap I've endured com GP g Jeff rogue demo this is the main this is me this is the user maven this is the repo cradle demo this is the package versions wonder too I want to sign this version all the fans in this version and again I need the API key and it's done this was much faster because I didn't have to upload any files now let's see what happened in the file list look at those those are the signatures that sonatype wants you to have when you go to maven central those where those files are not yet published they are just uploaded to being tre and I want to publish them to make them public now all my requirements are fulfilled I have a good poem file I have my sources jar my javadoc jar and the signatures for everything so all I need to do is provide the key the username and the key did I have for maven central just need to register to get one and then you like go up I almost pushed demo jars to maven central but I didn't you will people want this checkbox to be checked when you do your actual real promotion but this is just a toy project I don't really want it on maven central and then it will go to the staging repositories maven central like there pre their Nexus instance which is from there they sink to maven central and it will take a long time because it's kind of slow successfully synced without closing durable now I can go to their interface take it from here oh it says sonatype org I will need to login yeah they're slow if I won't have time for questions is their fault I already don't have time for questions right yeah okay I'm sorry about that anyhow the files aren't there yeah just this is a good ending like ending point for my talk now you know what you should use right looking at that hey guys thank you very much I'm sorry for not having time for questions I'm in our booth all the time so you if you do have any questions please come and ask ok i will more than happy to help you with that thank you very much lunch