Illustration Image

Cassandra.Link

The best knowledge base on Apache Cassandra®

Helping platform leaders, architects, engineers, and operators build scalable real time data platforms.

1/15/2018

Reading time:3 min

Kick-Start with SMACK Stack

by Knoldus Software LLP.

Kick-Start with SMACK Stack SlideShare Explore You Successfully reported this slideshow.Kick-Start with SMACK StackUpcoming SlideShareLoading in …5× 0 Comments 2 Likes Statistics Notes Paul Slattery , Technical Director at AKQA Streaming Analytics , Technology Manager at Trivadis at Trivadis No DownloadsNo notes for slide 1. Kick-Start with SMACK StackSandeep PurohitSoftware ConsultantKnoldus Software LLP 2. Agenda:● What is SMACK?● Why SMACK?● Brief introduction of technologies● How to Integrate all the technologies to create the data pipeline● Demo 3. What is SMACK?● Spark :Apache Spark is a fast and general-purpose clustercomputing system.● Mesos :Cluster resource management system that provideefficient resource allocation.● Akka :Akka is a toolkit and runtime for building highlyconcurrent, distributed, and resilient message-drivenapplications on the JVM.● Cassandra :The Apache Cassandra database is the rightchoice when you need scalability and high availability.● Kafka :distributed messaging system for handling realtime data. 4. Why SMACK?● Smack is used for pipelined data architecture which isrequired for the real time data analysis.● Smack is use to integrate all the technology at the rightplace to efficient data pipeline.● Smack is use to linearly scale your whole cluster withoutany hassle 5. SMACK Pipeline Architecture 6. Why Spark?● Its general purpose big data processing engine which have4 main components spark core, spark streaming, sparkml, spark graphx● So we can process our data which any of the componentat real time.● Its provide fault tolerant for real time application. 7. Why Cassandra?● Cassandra implements “no single points of failure● Cassandra Write-path is so fast so it can handle real-time data easily● It will support Datacenter architecture so we can easily use differentDC for different things.Ingestion DC Analysis DCCassandra Cluster 8. Why Mesos?Mesos MasterMesos MasterStandbyMesos MasterStandbyZookepeerMesos SlaveMesos SlaveMesos Slave 9. Models in SMACK● In SMACK models are Scala and AKKA.● We can use models to write highly concurrent and parallelapplications.● Example: We can use akka modules according to our usecase like akka-http, akka-scheduler, akka prioritymailboxes etc. 10. Models use in SMACKAkka-HttpAkka-Scheduler 11. Why Kafka● streams of data efficiently and in real time● Use Kafka for fault tolerance.● To create bridge between two applications.StreamingSourceKafkaBrokerSpark Receiver 12. Architecture of Spark and cassandraCassandra ClusterSpark WorkerSpark WorkerSpark WorkerSpark WorkerSpark worker nodeswill get the data onlocal node so it willavoid latency 13. Spark, Mesos, CassandraMesos Slaves and cassandra nodes are collocated to enforce the better datalocality for spark.DriverProgramMesosMasterMesos slaveCassandra nodeMesos slaveCassandra nodeMesos slaveCassandra node 14. Demo Application ArchitectureTweetsStore tweets inkafka topicRetrievehashtagsEvaluate Tophashtag inevery 10secondsStore tweets incassandra table 15. DemoSMACK_Tweets 16. Thank You!! Recommended PowerPoint 2016: Tips and TricksOnline Course - LinkedIn Learning Elearning Techniques: Visual DesignOnline Course - LinkedIn Learning Teaching Techniques: Creating Multimedia LearningOnline Course - LinkedIn Learning Akka Finite State MachineKnoldus Software LLP. Introduction to AWS IAMKnoldus Software LLP. Reactive Fast Data & the Data Lake with Akka, Kafka, SparkTodd Fritz Data processing platforms architectures with Spark, Mesos, Akka, Cassandra an...Anton Kirillov Real-Time Anomaly Detection with Spark MLlib, Akka and CassandraNatalino Busa Reactive dashboard’s using apache sparkRahul Kumar Data Science lifecycle with Apache Zeppelin and Spark by Moonsoo LeeSpark Summit About Blog Terms Privacy Copyright LinkedIn Corporation © 2018 Public clipboards featuring this slideNo public clipboards found for this slideSelect another clipboard ×Looks like you’ve clipped this slide to already.Create a clipboardYou just clipped your first slide! Clipping is a handy way to collect important slides you want to go back to later. Now customize the name of a clipboard to store your clips. Description Visibility Others can see my Clipboard

Illustration Image
Kick-Start with SMACK Stack

Successfully reported this slideshow.

Kick-Start with SMACK Stack
Kick-Start with SMACK Stack
Sandeep Purohit
Software Consultant
Knoldus Software LLP
Agenda:
● What is SMACK?
● Why SMACK?
● Brief introduction of technologies
● How to Integrate all the technologies to crea...
What is SMACK?
● Spark :Apache Spark is a fast and general-purpose cluster
computing system.
● Mesos :Cluster resource man...
Why SMACK?
● Smack is used for pipelined data architecture which is
required for the real time data analysis.
● Smack is u...
SMACK Pipeline Architecture
Why Spark?
● Its general purpose big data processing engine which have
4 main components spark core, spark streaming, spar...
Why Cassandra?
● Cassandra implements “no single points of failure
● Cassandra Write-path is so fast so it can handle real...
Why Mesos?
Mesos Master
Mesos Master
Standby
Mesos Master
Standby
Zookepeer
Mesos Slave
Mesos Slave
Mesos Slave
Models in SMACK
● In SMACK models are Scala and AKKA.
● We can use models to write highly concurrent and parallel
applicat...
Models use in SMACK
Akka-Http
Akka-Scheduler
Why Kafka
● streams of data efficiently and in real time
● Use Kafka for fault tolerance.
● To create bridge between two a...
Architecture of Spark and cassandra
Cassandra Cluster
Spark Worker
Spark Worker
Spark Worker
Spark Worker
Spark worker nod...
Spark, Mesos, Cassandra
Mesos Slaves and cassandra nodes are collocated to enforce the better data
locality for spark.
Dri...
Demo Application Architecture
Tweets
Store tweets in
kafka topic
Retrieve
hashtags
Evaluate Top
hashtag in
every 10
second...
Demo
SMACK_Tweets
Thank You!!

Upcoming SlideShare

Loading in …5

×

  1. 1. Kick-Start with SMACK Stack Sandeep Purohit Software Consultant Knoldus Software LLP
  2. 2. Agenda: ● What is SMACK? ● Why SMACK? ● Brief introduction of technologies ● How to Integrate all the technologies to create the data pipeline ● Demo
  3. 3. What is SMACK? ● Spark :Apache Spark is a fast and general-purpose cluster computing system. ● Mesos :Cluster resource management system that provide efficient resource allocation. ● Akka :Akka is a toolkit and runtime for building highly concurrent, distributed, and resilient message-driven applications on the JVM. ● Cassandra :The Apache Cassandra database is the right choice when you need scalability and high availability. ● Kafka :distributed messaging system for handling real time data.
  4. 4. Why SMACK? ● Smack is used for pipelined data architecture which is required for the real time data analysis. ● Smack is use to integrate all the technology at the right place to efficient data pipeline. ● Smack is use to linearly scale your whole cluster without any hassle
  5. 5. SMACK Pipeline Architecture
  6. 6. Why Spark? ● Its general purpose big data processing engine which have 4 main components spark core, spark streaming, spark ml, spark graphx ● So we can process our data which any of the component at real time. ● Its provide fault tolerant for real time application.
  7. 7. Why Cassandra? ● Cassandra implements “no single points of failure ● Cassandra Write-path is so fast so it can handle real-time data easily ● It will support Datacenter architecture so we can easily use different DC for different things. Ingestion DC Analysis DC Cassandra Cluster
  8. 8. Why Mesos? Mesos Master Mesos Master Standby Mesos Master Standby Zookepeer Mesos Slave Mesos Slave Mesos Slave
  9. 9. Models in SMACK ● In SMACK models are Scala and AKKA. ● We can use models to write highly concurrent and parallel applications. ● Example: We can use akka modules according to our use case like akka-http, akka-scheduler, akka priority mailboxes etc.
  10. 10. Models use in SMACK Akka-Http Akka-Scheduler
  11. 11. Why Kafka ● streams of data efficiently and in real time ● Use Kafka for fault tolerance. ● To create bridge between two applications. Streaming Source Kafka Broker Spark Receiver
  12. 12. Architecture of Spark and cassandra Cassandra Cluster Spark Worker Spark Worker Spark Worker Spark Worker Spark worker nodes will get the data on local node so it will avoid latency
  13. 13. Spark, Mesos, Cassandra Mesos Slaves and cassandra nodes are collocated to enforce the better data locality for spark. Driver Program Mesos Master Mesos slave Cassandra node Mesos slave Cassandra node Mesos slave Cassandra node
  14. 14. Demo Application Architecture Tweets Store tweets in kafka topic Retrieve hashtags Evaluate Top hashtag in every 10 seconds Store tweets in cassandra table
  15. 15. Demo SMACK_Tweets
  16. 16. Thank You!!

Related Articles

analytics
streaming
visualization

Keen - Event Streaming Platform

John Doe

2/3/2024

cassandra
spark

Checkout Planet Cassandra

Claim Your Free Planet Cassandra Contributor T-shirt!

Make your contribution and score a FREE Planet Cassandra Contributor T-Shirt! 
We value our incredible Cassandra community, and we want to express our gratitude by sending an exclusive Planet Cassandra Contributor T-Shirt you can wear with pride.

Join Our Newsletter!

Sign up below to receive email updates and see what's going on with our company

Explore Related Topics

AllKafkaSparkScyllaSStableKubernetesApiGithubGraphQl

Explore Further

mesos