Apache Spark with Scala / Python and Apache Storm Certification Training

Description

Apache Spark, a data processing engine is a well-known open-source cluster computing framework for fast and flexible large-scale data analysis. Scala, a scalable and multi-paradigm programming language which supports functional object-oriented programming and a very strong static type system implemented for developing applications like web services. Apache Storm is a well-developed, powerful, distributed, real-time computation system for enterprise-grade big data analysis. Python, a flexible and powerful language with simple syntax, readability and has powerful libraries for data analysis and manipulation.

Did you know?

1.  IBM announced its grand plans to dedicate and invest a large amount of research, education and development resources to Apache Spark projects which made its client companies to promote Spark.
2. Scala, the next wave of computation engines has taken over the world of fast data which rely on speed data processing and process event streams in real-time and used by companies like Apple, Twitter, and Coursera.
3.  Python is implemented for rapid prototyping of complex applications and also used as a glue language for connecting up the pieces of complex solutions such as web pages, databases, and Internet sockets.
4. Apache Storm, a fault-tolerant framework has a benchmark, which clocked it at over a million tuples processed per second per node that guarantees a well-processed data.

Why learn and get Certified?

Apache Spark with Scala/Python and Apache Storm training would equip with skill sets to become specialist in Spark and Scala along Storm with python since it will impact with the below-mentioned features:

1.  Apache Spark is not restricted to the two-stage MapReduce paradigm and enhances the performance up to 100 times faster than Hadoop MapReduce.
2.  In the last twelve months, demand for python programming expertise has increased by 96.9% in Big-Data realm.
3.  Apache Storm forms the backbone of the company’s real-time processing architecture by deploying in hundreds of organizations including Twitter, Yahoo!, Spotify, Cisco, Xerox PARC and WebMD.
4.  Apache Scala has matured and spawned solid support ecosystem that is successfully implemented critical business applications in most of the leading companies like LinkedIn, Foursquare, the Guardian, Morgan Stanley, Credit Suisse, UBS, HSBC, and Trafigura.

Course Objective

After the completion of this course, Trainee will:  

1.  Understand the need for Spark in the modern Data Analytical Architecture
2.  Improve knowledge on RDD features, transformations in Spark, Actions in Spark, Spark QL, Spark Streaming and its difference with Apache Storm
3.  Understand the need for Hadoop 2 and its installation application of Storm for real-time analytics
4.  Work with Jupiter and Zeppelin Notebooks
5.  Master the concepts of Traits and OOPS in Scala
6.  Learn on Storm Technology Stack and Groupings and implementing Spouts and Bolts
7.  Explain and master the process of installing Spark as a standalone cluster
8.  Demonstrate the use of the major Python libraries such as NumPy, Pandas, SciPy, and Matplotlib to carry out different aspects of the Data Analytics process

Pre-requisites

1.  Basic knowledge of any programming language and Working knowledge of Java
2.  Fundamental know-how of any database, SQL, and query language for databases
3.  Basic Knowledge of Data Processing
4.  Working knowledge of Linux- or Unix-based system which is desirable

Who should attend this Training?

This training is a foundation for aspiring professionals to embark in the field of Big Data by enhancing their skills with the latest developments around fast and efficient ever-growing data processing and ideal for:

1.  IT Developers and Testers
2.  Data Scientists
3.  Analytics Professionals
4.  Research Professionals
5. BI and Reporting Professionals
6. Students who wish to gain a thorough understanding of Apache Spark
7.  Professionals aspiring for a career in field of real-time Big Data Analytics

Prepare for Certification

ZaranTech is the first to offer a combination of Apache Spark with Scala / Python and Apache Storm to prepare Professionals for the Cloudera CCA175 certification and who want to stay on top of the market demand for Data Processing and Computation. ZaranTech’s best in-class blended learning approach of online training combined with instructor-led training will lead to higher retention and better results from the certification.

How will I perform the practical sessions in Online training?

For online training, ZaranTech provides the virtual environment that helps in accessing each other’s system. The detailed pdf files, reference material, course code are provided to the trainee. Online sessions can be conducted through any of the available requirements like Skype, WebEx, GoToMeeting, Webinar, etc.

Case Study

POC 1: Analyzing Book- Crossing Data Dataset URL: http://www2.informatik.uni-freiburg.de/~cziegler/BX/ The above dataset contains 3 sample csv file

Problem Statement: Based on Spark SQL

1.  Find out the frequency of books published each year
2.  Find out in which year maximum number of books were published
3.  Find out how many book were published based on ranking in the year 2002

POC 2: Crime Data Analysis

Dataset URL:

https://data.gov.in/catalog/crime-head-wise-cases-reported-under-indian-penal-code-ipc#web_catalog_tabs_block_10

Data Set:

crcIPC.csv , Contains 14 column where column1 = State Name , column2 = Crime Category , and rest other column are crime reported count from 2001 to 2012

Problem Statement: Based on Spark RDD

Idea is to compare crime reported for year 2011 and 2012 for each state and for crime category Murder and to find out whether crime reported has been increased or decreased or it is same between 2011 and 2012.

POC 3: Loan Analysis

Dataset URL:

https://www.lendingclub.com

Data Set:

Lending Club is an online financial community that brings together creditworthy borrowers and savvy investors to arrange loans. Since 2007, Lending Club has funded $3 Billion in loans.

Problem Statement:

1. Summarize loans by State, Credit Rating and Loan Title
2.  Identify top 10 cities with maximum number of loans
3.  Calculate total loan amount for each loan title in the state of New Jersey
4. Number of loans and loan amount in each month

Unit 1: Introduction to Data Analysis and Spark

1. What is Apache Spark
2. Understanding Lambda Architecture for Big Data Solutions
3. Role of Apache Spark in an ideal Lambda Architecture
4. Understanding Apache Spark StackSpark Versions
5. Storage Layers in Spark

Unit 2: Getting Started with Apache Spark

1. Downloading Apache Spark
2. Installing Spark in a Single Node
3. Understanding Spark Execution Modes
4. Batch Analytics
5. Real Time Analytics Options
6. Exploring Spark Shells
7. Introduction to Spark Core
8. Setting up Spark as a Standalone Cluster
9. Setting up Spark with Hadoop YARN Cluster

Unit 3: Spark Language Basics

1. Basics of Python
2. Basics of Scala

Unit 4: Spark Core Programming

1. Understanding the Basic component of Spark -RDD
2. Creating RDDs
3. Operations in RDD
4. Creating functions in Spark and passing parameters
5. Understanding RDD Transformations and Actions
6. Understanding RDD Persistence and Caching
7. Examples for RDDs

Unit 5: Understanding Notebooks

1. Installation of Anaconda Python
2. Installation of Jupiter Notebook
3. Working with Jupiter Notebook
4. Installation of Zeppelin
5. Working with Zeppelin notebooks

Unit 6: Hadoop2 & YARN Overview

1. Anatomy of Hadoop Cluster, Installing and Configuring Plain Hadoop
2. Batch v/s Real time
3. Limitations of Hadoop

Unit 7: Working with Key/Value Pairs

1. Understanding the Key/Value Pair Paradigm
2. Creating a Pair RDD
3. Understanding Transformations on Pair RDDs
4. Understanding Actions on Pair RDDs
5. Understanding Data Partitioning in RDDs

Unit 8: Loading and Saving Data in Spark

1. Understanding Default File Formats supported in Spark
2. Understanding File systems supported by Spark
3. Loading data from the local file system
4. Loading data from HDFS using default Mechanism
5. Spark Properties
6. Spark UI
7. Logging in Spark
8. Checkpoints in Spark

Unit 9: Working with Spark SQL

1. Creating a HiveContext
2. Inferring schema with case classes
3. Programmatically specifying the schema
4. Understanding how to load and save in Parquet, JSON, RDBMS and any arbitrary source ( JDBC/ODBC)
5. Understanding DataFrames
6. Working with DataFrames

Unit 10: Working with Spark Streaming

1. Understanding the role of Spark Streaming
2. Batch versus Real-time data processing
3. Architecture of Spark Streaming
4. First Spark Streaming program in Java with packaging and deploying

Unit 11: Spark MLLib and Installation of R in Jupiter notebook

1. Anatomy of Hadoop Cluster, Installing and Configuring Plain Hadoop
2. What is Big Data Analytics
3. Batch v/s Real time
4. Limitations of Hadoop
5. Storm for Real Time Analytics

Unit 12: What is new in Spark 2

Unit 13: YARN Overview

Unit 14: Storm Basics

1. Installation of Storm
2. Components of Storm
3. Properties of Storm

Unit 15: Storm Technology Stack and Groupings

1. Storm Running Modes
2. Creating First Storm Topology
3. Topologies in Storm

Unit 16: Spouts and Bolts

1. Getting Data
2. Bolt Lifecycle
3. Bolt Structure
4. Reliable vs Unreliable Bolts

About Apache Spark with Scala/Apache Storm with Python Certification

Apache Spark, a data processing engine is a well-known open source cluster computing framework for fast and flexible large-scale data analysis. Scala, a scalable and multi-paradigm programming language which supports functional object oriented programming and a very strong static type system implemented for developing applications like web services. Apache Storm is a well-developed, powerful, distributed, real-time computation system for enterprise grade big data analysis.

Apache Spark with Scala/Python and Apache Storm Certification Types

A well known certification authority for Apache Spark with Scala/Python and Apache Storm offers two important types of certification.

1. Cloudera Certified Administrator for Apache Hadoop (CCA500))
2.  Cloudera CCA Spark and Hadoop Developer Exam (CCA175)

Cloudera Certified Administrator for Apache Hadoop (CCA500)

A Cloudera Certified Administrator for Apache Hadoop (CCAH) certification proves that you have demonstrated your technical knowledge, skills, and ability to configure, deploy, maintain, and secure an Apache Hadoop cluster.

Pre-requisites

1.  Fundamental knowledge of any programming language and Linux environment
2.  Participants should know how to navigate and modify files within a Linux environment

Exam Details

1. Exam fees is $300
2. Exam type: Online Exam and Test centre
3. Questions: Based on Scala, Python, Java and SQL

Cloudera CCA Spark and Hadoop Developer Exam (CCA175)

A Cloudera CCA Spark and Hadoop Developer Exam (CCA175) certification requires you to write code in Scala and Python and run it on a cluster. You prove your skills where it matters most.

Pre-requisites

1. There are no prerequisites required to take any Cloudera certification exam. The CCA Spark and Hadoop Developer exam (CCA175) follows the same objectives as Cloudera Developer Training for Spark and Hadoop and the training course is an excellent preparation for the exam.

Exam Details

1.  Exam fees is $295
2.  Exam type: Online Exam and Test centre
3. Questions: Based on Scala, Python