Discover Classes. Earn 10% Rewards.

Big Data with Amazon Cloud, Hadoop/Spark and Docker is unfortunately unavailable

Thankfully we have 187 other IT Classes for you to choose from. Check our top choices below or see all classes for more options.

Python Data Visualization & Interactive Dashboards (Self-Paced)

Noble Desktop See all classes by this school - Self-paced Video Training

Transform raw data into compelling visuals and interactive dashboards using Python libraries including Pandas, Plotly, and Dash. Work through real-world projects at your own pace, deploy them online, and build a portfolio that demonstrates your data skills to potential employers.

(453) Intermediate 18 and older

Python for Data Science (Self-Paced)

Noble Desktop See all classes by this school - Self-paced Video Training

Kickstart your data science journey with our self-paced Python for Data Science course! Learn Python fundamentals, analyze data with Pandas and NumPy, and create stunning visualizations—all while building a strong foundation for machine learning.

(453) All levels 18 and older

AI & Data Literacy for the Federal Workforce

Noble Desktop See all classes by this school @ Zoom Conference

Through practical examples and hands-on activities, participants will learn core concepts, common terms, and best practices for working with AI tools and interpreting data.

(453) Beginner All ages

AI for Project Management

Noble Desktop See all classes by this school

Enhance your project management skills with AI by learning how to streamline planning, scheduling, and communication using cutting-edge tools. Discover how to improve decision-making while navigating the ethical and practical challenges of AI in real-world projects.

(453) Beginner 18 and older

AI for the Workplace with Claude (Self-Paced)

Noble Desktop See all classes by this school - Self-paced Video Training

Build powerful, personalized AI workflows at your own pace with Claude—from your first prompt to a fully automated system tailored to your role. Learn to connect tools, create Projects, and design end-to-end solutions that streamline your daily work.

(453) Beginner 18 and older
See all IT classes Online

Big Data with Amazon Cloud, Hadoop/Spark and Docker

  • Beginner
  • 18 and older
  • Earn 10% back in points
  • Price Lock

    If you find the same class on the same dates for less on another site within 48 hours of booking, send us the link. We match the price as credit or the difference.

  • 35 hours over 14 sessions

Start Dates (0) $2,990/person

No upcoming dates are currently available.

Book a Private Group Event

Booking this class for a group? Find great private group events

Or see all Professional Group Events

Explore group events and team building activities ranging from cooking, art, escape rooms, trivia, and more.

Class Description

Description

This is a 6-week evening program providing a hands-on introduction to the Hadoop and Spark ecosystem of Big Data technologies. The course will cover these key components of Apache Hadoop: HDFS, MapReduce with streaming, Hive, and Spark. Programming will be done in Python. The course will begin with a review of Python concepts needed for our examples. The course format is interactive. Students will need to bring laptops to class. We will do our work on AWS (Amazon Web Services); instructions will be provided ahead of time on how to connect to AWS and obtain an account.

What is Hadoop?

Hadoop is a set of open-source programs running in computer clusters that simplify the handling of large amounts of data. Originally, Hadoop consisted of a distributed file system tuned for large data sets and an implementation of the MapReduce parallelism paradigm, but has expanded in many ways. It now includes database systems, languages for parallelism, libraries for machine learning, its own job scheduler, and much more. Furthermore, MapReduce is no longer the only parallelism framework; Spark is an increasingly popular alternative. In summary, Hadoop is a very popular and rapidly growing set of cluster computing solutions, which is becoming an essential tool for data scientists.

Prerequisites

To get the most out of the class, you need to be familiar with Linux file systems, Linux command line interface (CLI) and the basic linux commands such as cd, ls, cp, etc. You also need to have basic programming skills in Python, and are comfortable with functional programming style, for example, how to use map() function to split a list of strings into a nested list. Object oriented programming (OOP) in python is not required.

Syllabus

Unit 1: Introduction to Hadoop

1. Data Engineering Toolkits

  • Running Linux using Docker containers
  • Linux CLI command and bash scripts
  • Python basics

2. Hadoop and MapReduce

  • Big Data Overview
  • HDFS
  • YARN
  • MapReduce

Unit 2 – MapReduce

3. MapReduce using MRJob 1

  • Protocols for Input & Output
  • Filtering

4. MapReduce using MRJob 2

  • Top n
  • Inverted Index
  • Multi-step Jobs

Unit 3 – Apache Hive

5. Apache Hive 1

  • Databases for Big Data
  • HiveQL and Querying Data
  • Windowing And Analytics Functions
  • MapReduce Scripts

6. Apache Hive 2

  • Tables in Hive
  • Managed Tables and External Tables
  • Storage Formats
  • Partitions and Buckets

Unit 4 – Apache Pig

7. Apache Pig 1

  • Overview
  • Pig Latin: Data Types
  • Pig Latin: Relational Operators

8. Apache Pig 2

  • More Pig Latin: Relational operators
  • More Pig Latin: Functions
  • Compiling Pig to MapReduce
  • The Parallel Clause
  • Join Optimizations

Unit 5 – Apache Spark and AWS

9. Apache Spark – Spark Core

  • Spark Overview
  • Running Spark using Databricks Notebooks
  • Working with PySpark: RDDs
  • Transformations and Actions

10. Apache Spark – Spark SQL

  • Spark DataFrame
  • SQL Operations using Spark SQL

11. Apache Spark – Spark ML

  • ML Pipeline using PySpark

12. Amazon Elastic MapReduce

  • Overview
  • Amazon Web Services: IAM, EC2, S3
  • Creating EMR Cluster
  • Submitting Jobs
  • Intro to AWS CLI

Project: Data Engineering Project

Remote Learning

This course is available for "remote" learning and will be available to anyone with access to an internet device with a microphone (this includes most models of computers, tablets). Classes will take place with a "Live" instructor at the date/times listed below.

Upon registration, the instructor will send along additional information about how to log-on and participate in the class.

School Notes: We offer a certification licensed by the NYS Board of Education.

Refund Policy

Any student wishing to withdraw from a program must notify CourseHorse in writing. The date of withdrawal for refund purposes is the last date of physical attendance. The failure of a student to notify us in writing of withdrawal may delay refund of tuition due pursuant to Section 5001 and 5002 of the Education Law.

Any student requesting cancellation within seven days after signing the Enrollment Agreement but before instruction begins will be refunded all money paid less 5% cancellation fee. Thereafter, in the event of cancellation or termination by the school, refunds will be prorated based on the student's last date of attendance.

Reviews of Classes at NYC Data Science Academy (31)

Similar Classes

Benefits of Booking Through CourseHorse

  • Booking is safe. When you book with us your details are protected by a secure connection.
  • No booking fees. You pay the school's price, with nothing added at checkout.
  • Price Lock. If you find the same class on the same dates for less on another site within 48 hours of booking, send us the link. We match the price as credit or the difference.
  • This class will earn you 29900 points. Earn 10% back in points. 100 points = $1 off a later class.

NYC Data Science Academy

NYC Data Science Academy offers hands-on projects and real-world applications to teach students the skills needed for a career in data science. By enrolling, students become part of a supportive network that includes the NYC Open Data Meetup.

View school profile
NYC Data Science Academy
Vetted by CourseHorse

A verified Online educator.

Loading...