College - Author 1

College of Engineering

Department - Author 1

Computer Engineering Department

College - Author 2

College of Engineering

Department - Author 2

Computer Science Department

Advisor

Joydeep Mukherjee, College of Engineering, Computer Science Department

Funding Source

Noyce School of Applied Computing

Acknowledgements

Maanav Patel

Date

10-2023

Abstract/Summary

In this project, we have developed a rather robust means of processing and displaying large sums of IoT data using several cutting-edge, industry-standard technologies. Our data pipeline integrates physical sensors that send various environmental data like temperature, humidity, and pressure. Once created, the data is then collected at an MQTT broker, streamed through a Kafka cluster, processed within a Spark Cluster, and stored in a Cassandra database.

In order to test the rigidity of the pipeline, we also created virtual sensors. This allowed us to send an immense amount of data, which wasn’t necessarily feasible with just the physical sensors. The web interface allows users to create as many of these virtual sensors as testing requires.

Once the data goes through the pipeline, it is made viewable on the same web interface. Users can search for key sensors, look through important data, and analyze as necessary.

Our IoT pipeline enables seamless data flow and near real-time analytics. Using industry-standard technologies allows for scalability and reliability, making it suitable for all sorts of data-intensive applications.

COinS
 

URL: https://digitalcommons.calpoly.edu/ceng_surp/13

 

To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.