Policy Search Methods and ML at Amazon

Jan 23, 2017 · Greater London, United Kingdom

Agenda is the same as usual:
- 18:30:  doors open, pizza, beer, networking
- 19:00: First talk
- 20:00: Break & networking
- 20:15: Second talk
- 21:30: Close
• An introduction to policy search methods - Tom Furmston Markov decision processes (MDPs) are the standard model for optimal control in a fully observable environment. It is well known that the global optimum of a MDP with finite state and action sets can be obtained through methods based on dynamic programming. Unfortunately, these techniques are known to suffer from the curse of dimensionality, which makes them infeasible for many real-world problems of interest. As a result, most research in the reinforcement learning and control theory literature has focused on obtaining approximate or locally optimal solutions. There exists a broad spectrum of such techniques, including approximate dynamic programming methods, tree search methods, local trajectory-optimization techniques, such as differential dynamic programming and iLQG, and policy search methods. In this talk I shall provide an introduction to policy search methods, which are a family of algorithms that have proven extremely popular in recent years, and which have numerous desirable properties that make them attractive in practice. In particular, I shall introduce some of the core concepts from the policy search literature, including the policy gradient theorem, actor-critic methods, compatible function approximation and natural actor-critic methods. Time permitting I shall also introduce some additional concepts, such as baseline methods and deterministic policy search methods. Bio: Under the supervision of Dr. David Barber, Tom obtained his Ph.D. at the computer science department of University College London. The subject of his Ph.D. was in reinforcement learning, which is an area of machine learning that focuses on optimal control in a possibly unknown environment. Having completed his Ph.D. Tom went on to undertake two post-doctoral research positions at University College London, before then moving into the private sector. Since joining the private sector Tom has held research positions at several ad-tech companies, first at Adform, an international display advertising company, and currently at Adthena, which provides competitive intelligence in search advertising.
* Amazon: A Playground for Machine Learning -  Cedric Archambeau Within Amazon, a company with over 200 million active consumers, over 2 million active seller accounts and over[masked] employees, there are hundreds of problems which can be tackled using machine learning. In the first part of this talk, I will discuss a number of problems we tackle with machine learning solutions. While machine learning is routinely used in recommendation, fraud detection and ad allocation, it plays a key role in devices such as the Kindle or the Echo, as well as the automation of Kiva enabled fulfilment centres, statistical machine translation or automated Fresh produce inspection. In the second part, I will discuss how we democratise machine learning at Amazon. While machine learning enables us to learn predictive models from data, it requires careful tuning of so-called hyperparameters (e.g., learning rates or the amount of regularisation) to ensure good generalisation capabilities. As of today, the tuning of hyperparameters is done in an ad hoc fashion or manually. We adopted a principled approach based on Bayesian optimisation which enables us to automate this process. In effect, Bayesian optimisation sits on top of machine learning, materialising machine intelligence by taking the human out of the loop when building machine learning applications. Bio: Cedric Archambeau is a Senior Machine Learning Scientist with Amazon, Berlin. He is the technical lead on the Machine Learning Platform & Applications team and served as a scientific advisor to Sebastian Gunningham, Amazon Senior Vice President Seller Services. Recently, his team delivered the learning algorithms offered in Amazon Machine Learning (https://aws.amazon.com/machine-learning), which is part of Amazon AI (https:// aws.amazon.com/amazon-ai). He is interested in large scale probabilistic inference, Bayesian optimization and machine reasoning. He holds a visiting position in the Centre for Computational Statistics and Machine Learning at University College London. Prior to joining Amazon, he was managing the Machine Learning and Mechanism Design area at Xerox Research Centre Europe, Grenoble.

Event organizers

Are you organizing Policy Search Methods and ML at Amazon?

Claim the event and start manage its content.

I am the organizer
Social
Rating

based on 0 reviews