IoT NY #54: Streaming Architectures in IoT

May 8, 2018 · New York, United States of America

In IoT, data is constantly in motion. It may move from edge devices, to gateways, to on-premise servers and into the cloud. As we discovered in our last Meetup, computation at the edge is a trend away from centralized to decentralized data processing. Similarly, we see a trend away from batch analytics (analyzing data at rest e.g., in Hadoop) to streaming analytics (analyzing data in motion e.g., in Kafka or Spark). In this month's Meetup, we'll learn about streaming architectures as applied to IoT.

\\ \\

Speaker 1: Pete Zybrick - Senior Lead Architect, Big Data Engineering @Medidata (
Bio: From Bell Labs to BMW to Big Data, Pete has architected, developed, managed, secured, tested and deployed large scale mission critical systems directly responsible for billions of dollars in annual transactions. Currently Pete is at Medidata, where he is a hands-on Sr Mgr of Big Data Engineering, leading a team of engineers building Big Data and IoT solutions to enhance current and future clinical trials.

Talk: iote2e - Internet of Things End to End.
Abstract: iote2e is an integrated set of Docker containers and Raspberry Pi based sensors and actuators that demonstrates how to combine Docker with external IoT devices for near real time processing. iote2e integrates 24 Docker containers implementing a Lambda architecture running Kafka, SparkStreaming, Ignite, Cassandra, Zeppelin, MySQL and a custom Java websockets application along with Raspberry Pi's and daughterboards running Python applications, using Apache Avro for the schema communication between the layers. All 24 containers are running on a single Ubuntu laptop, using a bridge network and compose files. Additionally a Windows laptop is used for Tableau based dashboard demonstrations. Four use cases will be demonstrated:, Dynamic Clinical Trial, and Big Data Black Box. For further information, please refer to the project in GitHub:

\\ \\

Speaker 2: Tim Spann - Senior Solutions Engineer @Hortonworks
Bio: Tim Spann was a Senior Solutions Architect at AirisData working with Apache Spark and Machine Learning. Previously he was a Senior Software Engineer at SecurityScorecard helping to build a reactive platform for monitoring real-time 3rd party vendor security risk in Java and Scala. Before that he was a Senior Field Engineer for Pivotal focusing on CloudFoundry, HAWQ and Big Data. He is an avid blogger and the Big Data Zone Leader for Dzone. He runs the the very successful Future of Data Princeton meetup with over 830 members at. He is currently a Solutions Engineer at Hortonworks in the Princeton New Jersey area. You can find all the source and material behind his talks at his Github and Community blog:

Talk: IoT with Apache MXNet and Apache NiFi and MiniFi
Abstract: A hands-on deep dive on using Apachee MiniFi with Apache MXNet on the edge device including Raspberry Pi with Movidius and NVidia Jetson TX1. We run deep learning models on the edge device and send images, sensor data and deep learning results if values exceed norms. Using S2S data is sent to NiFi for further processing, additional deep learning processing, data augmentation. A stream of data is landed as ORC files in HDFS with Hive tables on-top.

Processed data in AVRO format with a schema stored in Schema Registry. Visualization is shown in Zeppelin.

Use Cases: Security Camera Monitoring, Utility Asset Anomaly Detection, Temperature and Humdiity filtering for devices.

Event organizers

Are you organizing IoT NY #54: Streaming Architectures in IoT?

Claim the event and start manage its content.

I am the organizer

based on 0 reviews