Meet MySam - An open AI experiment

0 0

thank you thank you and welcome and when I was five items I scientists in Switzerland British scientists put online the World Wide Web I don't think it made the headlines anymore but 25th birthday this one and it's pretty amazing what happened in last 25 years with it right we can do things like live in Vancouver and work for a company in Chicago we can you know meet in Iceland at a conference with people from all over the world and most important of all we can look at cute cat videos all times of the day so that was a pretty big change and that's kind of symbol of how I feel about machine learning I'd like to use the much cooler sounding intelligence or AI and yeah it's kind of like a like the Internet it's a general purpose technology we had two talks already that talked about using AI to generate art Google is using it to drive cars and I'm going to talk about using it to do natural language processing the cool thing about the web browser is that Google and Mozilla actually offering us an API maybe you've seen yesterday to do speech speech recognition but to make sense of that language is a little harder and just like parsing HTML not possible with regular expressions so I have this you probably can't see this but there's pile of wires over there it wasn't my gray decide you to bring my own audio setup there's some good helps to get it all working hopefully and the reason I said I brought I talked to it by pressing this button it's a little it's called a flick you can get them before pax 2 it's actually pretty cool you hook it up to your Android device or other Bluetooth enabled device we were about that yesterday and yeah it's literally just a button but you can program it to do stuff and I programmed it to get my friend to listen so let's see if that works hey can you hear us yeah so this already starting out that that's okay hey can you hear you might see that I don't know what you're talking about screen a little more often I'm gonna explain it a little more as well cool do you want to tell us your name actually came from me watching too many movies about hey I might know Samantha from the movie her and Samaritan from personal interest the you is actually inspired by it but besides that this thing has nothing to do with an AI companion that us run the world so maybe you can tell us what you do know do you want to tell us what you do maybe you can quickly go over what this is actually this screen is the texts that it got let's try one more time can you tell us what you do so it's an open source intelligent in quotation marks couse it is obviously still pretty dumb a system that you can train and extend yourself um you might know or use Siri or Alexa or M which is still in beta there's many many things like that come up in the last little while and they're all kind of still like black boxes and this thing you can teach your own language and also extent with plugins written in html5 and JavaScript it's pretty cool the whole thing actually is written in JavaScript and I'm gonna go into the tech in a little bit okay when you don't know what else to say you just talk about weather hey what's the weather like dramatic pause so this is actually an example for a plugin it's using plain gyah create simple weather now you can also ask for a city please don't need a city because yahoo doesn't allow to ask for anything else anymore and for I think the next little bit I might need a volunteer but let me first show you what you're getting into I brought another mic that you can use and we'll try is it stubborn for you this is the only evil robot joke in this talk I'm gonna not do any other ones cool what's the weather like cool and maybe the last thing is you can actually look up general things like cool so we can do three things if I have a volunteer that wants to try it out we can go through the process and see what happens this is you know this is like so let's let's see if it works so maybe start with you see this thing makes me way better jokes than me all right let's figure out the weather so the cool thing about this is that you get say hey this is not right I'm gonna teach you a new thing all right now the most the most interesting part is so what is the answer to the universe everything oh let's try what is the answer to life the universe and everything so as you see Thank You max all right this actually turned out way more entertaining than I thought it would be now before we look on under the hood on what's happening yes let's talk for a couple minutes about the technology so it's a natural language processing libraries per node it has like some basic things like tokenization of a sentence it also has basic classification and that was cool but the classification runs on statistical analysis and I thought it would be an interesting exercise because I've been watching too many movies yeah to hook it up to a neural network yesterday so you can use that to put sentences in to train it with certain sentences and then get label them and get probabilities back that I put something else so typical um yeah neural network machine learning thing now I know this isn't feathers is another thing I mean and I know this talk is not about feathers but I can't just and I think it also covers something that's that's still kind of important and not done that often which is how do we make machine learning data available publicly right I mean Siri and light box citizen living room but how do you how do you make it publicly available and not just within SDK but actually has an API so what feathers does it's kind of like a small library on top of Express that lets you create and consume rests and real-time API and that's it so you can pick whichever that database you want you can pick whichever front-end technology one it's just this connector in between and what this basically does in this case is gives us a database connection just to store stuff it's using any DB because the whole thing will it's running in a browser right now but it also can run an electron so that means that we don't want people to install MongoDB and stuff like that it's just a file based database and and then it also provides the connection because it also works in the Kline between the front-end app and the backend and then we have election react steel which is a module loaded throwing stuff dynamically without having to build it I needed that for the plug-in system which will hopefully have some time to look into cool um so there's two three hmm hey that's my man someone always changed so there's a couple things to it the first one and I don't even think I have to blow that up because this queen is huge maybe a little bit the first one is actions an action is basically something that you want it to do if you haven't met a sentence try and pull out something that's similar so in our case of the two tag goes from position 4 to 4 which is your so if somebody says can you hi there my name is it'll usually try and pull out the information around that sense this is actually a pretty dumb algorithm but it works so well because the classification of the neural network is so accurate that you don't have to worry about the sentences being so different so this is how it gets where what's the weather in Vancouver and things like that then we have something here reply with a couple of things and this is the search of course this has been done by people way smarter than me already and there's an API for it and a really cool one actually that I hope I have some time to show a little bit so this is actions there's only 11 of them and then the next part of the thing we have is trainings so you know people always think that millions and millions well it's somewhat accurate so this basically maps text to an action every time we make a new classification it'll save you when it's not sure it'll save and Retraining put that one so that every time master and the action is basically just the action oh sorry it's all good I'm gonna go back here then I thought this one this one did work sorry about that okay so this sounds better so now we have actions and trainings the last part of the API actually is the interesting one which is the classification and classification is basically you can just post to this classify endpoint with your actual text so if you say can you tell us the weather in Berlin and they send this it'll come back with something like this the texts the tokens tokens are just each individual word stems are interesting this is basically the base form of the word so that if you have singular plural it doesn't matter or if you have other languages that add a lot of things at the end of words you want to know what the actual word is about so that's pretty useful and by the way this should ideally also work with other languages I tried it briefly with German I unfortunately didn't have enough time but because it's all based on word stems and neural networks you can train it in your own language if it is possible too if no natural has a stammer for it so it can build the bit create the base forms over work cool this is the action that it created basically it creates a new action for us and this is the confidence so it's actually pretty confident already because we've done it many times there's 96 percent confident that this is what I want and this is the original action that it got trained with now if you look back into the browser you'll see that it's yeah there we go you'll see that it updated and browsers already now it still has to bend at humans land on the moon in it's pretty warm in Berlin alright so what happens in the background is let's make this a little bigger too because this is where the action machine learning part happens you'll see here it gives us a list of labels to values and those are the confidences of what action you want to perform so we'll see here there's a whole bunch of them that are like less than one lesson way less than 10% but this one is 92 percent and this is exactly the one that we wanted it to do so that was really cool to see under the hood and how it gets better the more you train it alright give me back my browser let's briefly look at how to build a plugin because I found it really important to make it almost as easy as even though jQuery is not used as much anymore it's still the creating a plugin for it was the easiest thing you could ever do and I really like that so a plugin basically has a client and a server file and on the server in our case for this one I'm gonna actually show you first what this plug-in is about this is the looking up information on the internet and it's using Wolfram Alpha which is by the people that also made Mathematica and it's a knowledge engine so instead of things like google it that looks up relevant links for you it actually tries to get the meaning from what you ask it so when did people land the moon it'll give us way more oops way more information then what we got back in the answer but you'll basically get the same the same things so this is pretty cool because I didn't have to do it you just used the API and then display it on the client so what we did here was use the node API plug-in create a new client with the app ID this is only a test API key if you want to use it and then it will create a new feather service it's kind of like a a middleware except for that it has a crud object that you pass and in this create so if you post or call create will create a query with grab the query from the data and then just get the result from the client and return it back so this is just an API endpoint just like the classified service that we posted to and on the client we have the same feathers app but two additional things one is learn so when you saw the I don't know what you're talking about screen you can basically add things to the drop-down and say okay now tell me the weather now do this and now do that you can create your own forms and submit your own data in here and then we have the actual actions so when the action is certain enough so when so more than 50% sure it'll just perform the action and what we do here is pretty much get the text make a query here to our service that we created on the server so the cool thing about this is that it's fine agnostic now so we could also put it on our mobile phones or mobile apps or electron which is what it's going to run in as the distributable and then we have the results and just print the main or primary primary result oh and then so that's pretty much it and then and then this is a plug tear down function so you just clear it out so you can use whatever technology you want you get all the information that it got trained with and then you can perform whichever actions you you would like so that was pretty fun and and this once I had the whole system in place was actually really quick to do and I spent like a whole afternoon asking it random questions all right so that this whole project started as I should have to remember to stay here this whole project started as a demo for the Calgary meetup that I hauled from the front moved to Vancouver and we wanted to just show hey here's some things that you can do with feathers and let's let's try and use some AI so we build a collaborative gesture teaching app which was really cool it was a clunky but it worked and the next step was to go into the nest language processing and I couldn't find anything related to it except for the no natural library for for JavaScript and there's only two personal assistant projects similar there is serious which is a research project that tries to do much more like make sense of photos and learn from everything not just language stuff and Jasper I think which is a Python app that's built to run on your Raspberry Pi or device like that and this one on the other hand I do really like to have as a desktop app and a web app and it worked really well as a demo and and then I just continue working on it to continue working on it and three weeks ago I was hacking away and I was teaching in new things and it recognized them and when it didn't then I showed it new things and all of a sudden went and once it learned a little more it was really weird cuz it got less and less sure of things that had learned before so if I said thank you it would be like I'm only 20 percent sure you want me to say you're welcome and that was really weird because you know as a programmer we used to for programming it in once and it'll like just work forever so I try to figure out what the problem is and I think sorry it's really helpful sometimes to have a non programmer sounding board because then you have to explain it in different terms and when I did that I said so I built this thing and it works but it's super confused with the old stuff the new stuff it knows really well but the old stuff it doesn't and we talked about it was like Dave you accidentally build Marvin the Paranoid Android and and and that was really interesting because the programmer right you always like okay a program at him once and it'll work and I get mad if it doesn't work even with Siri you're like cursing Apple because they didn't get what you wanted to do and that was the case here too the problem was no positive reinforcement it was I was only telling it when he was wrong what else it was supposed to do so that was really interesting for me to learn that you have to treat it less like programming and more like teaching a person or teaching someone which was really cool for me to see because I think in the near future we might be you know teaching our software as much as we program them and looking at where those data are stored how they're shared and where they're coming from is gonna be come more and more important and just like we meet better operating systems with at least making parts of it open-source I think there should be a way for training data for machine learning to be shareable as well there is not even a shareable format neural networks work various have a fairly similar architecture in most programming languages and there isn't a standardized format yet to to to exchange it so if I train it Google's tensorflow I can't drop it into brain js2 use it in the browser and I think that's gonna be going to be pretty important in the future and I find it pretty exciting because it's well it's way more unpredictable but that's how our work is every day anyway and working with the computer the same way we work with other people I find pretty exciting and one thing I want to work on a little more is turn this into a fancy generator that just scaffolds you app by asking you a couple questions it's a pretty basic step but it will make things a lot easier because you can actually interact more instead of going through a command line and yeah if you interest in this as well come chapter me this is it for me and thanks everybody for being here