Discover Classes. Earn 10% Rewards.

Big Data with Amazon Cloud, Hadoop/Spark and Docker is unfortunately unavailable

Thankfully we have 225 other IT Classes for you to choose from. Check our top choices below or see all classes for more options.

Excel Level I: Fundamentals

Noble Desktop See all classes by this school - Virtually Online

Build a confident foundation in Microsoft Excel through this practical workshop at Noble Desktop. You'll work with calculations, functions, charts, formatting, and print layouts to manage data and produce well-structured spreadsheets.

(452) Beginner 18 and older

Excel Bootcamp

Noble Desktop See all classes by this school - Virtually Online

Gain comprehensive Excel expertise in this intensive three-day course that moves from foundational calculations through to advanced automation with macros. Whether you're building on basic knowledge or aiming for expert-level proficiency, this well-paced program covers the full range of tools and techniques that make Excel an indispensable asset in any professional setting.

(452) Beginner 18 and older
$599

3 sessions

Gift it!

Excel Specialist Certification Program

Noble Desktop See all classes by this school - Virtually Online

Become an expert in Microsoft Excel and strengthen your resume through this comprehensive certification program. Get lifetime access to video courses, private training, and free proctored exams. Grow your skills and stand out in the competitive job market.

(452) All levels 18 and older
$959

3 sessions

Gift it!

Excel for Business Fundamentals

NYIM Training See all classes by this school - Virtually Online

Maximize your everyday productivity with this focused course on Excel's most essential tools and functions. Through hands-on exercises, you'll gain a confident working command of the formulas, features, and workflows that business professionals rely on most, helping you work smarter and more efficiently from day one.

(750) Beginner 18 and older

Python for Automation (Self-Paced)

Noble Desktop See all classes by this school - Self-paced Video Training

Automate repetitive tasks and extract data from websites using Python in this beginner-friendly, self-paced course. Build real-world automation scripts, learn large-scale data scraping techniques, and develop the skills to run your code efficiently and independently. A strong foundation for anyone looking to streamline workflows or move into web development.

(452) Intermediate 18 and older
See all IT classes Online

Big Data with Amazon Cloud, Hadoop/Spark and Docker

  • Beginner
  • 18 and older
  • Earn 10% Rewards
  • Price Lock

    Secure your booking now and we'll match any price drop within 48 hours across all booking platforms, ensuring you never miss out on savings!

  • 35 hours over 14 sessions

Start Dates (0) $2,990/person

No upcoming dates are currently available.

Book a Private Group Event

Booking this class for a group? Find great private group events

Or see all Professional Group Events

Explore group events and team building activities ranging from cooking, art, escape rooms, trivia, and more.

Class Description

Description

This is a 6-week evening program providing a hands-on introduction to the Hadoop and Spark ecosystem of Big Data technologies. The course will cover these key components of Apache Hadoop: HDFS, MapReduce with streaming, Hive, and Spark. Programming will be done in Python. The course will begin with a review of Python concepts needed for our examples. The course format is interactive. Students will need to bring laptops to class. We will do our work on AWS (Amazon Web Services); instructions will be provided ahead of time on how to connect to AWS and obtain an account.

What is Hadoop?

Hadoop is a set of open-source programs running in computer clusters that simplify the handling of large amounts of data. Originally, Hadoop consisted of a distributed file system tuned for large data sets and an implementation of the MapReduce parallelism paradigm, but has expanded in many ways. It now includes database systems, languages for parallelism, libraries for machine learning, its own job scheduler, and much more. Furthermore, MapReduce is no longer the only parallelism framework; Spark is an increasingly popular alternative. In summary, Hadoop is a very popular and rapidly growing set of cluster computing solutions, which is becoming an essential tool for data scientists.

Prerequisites

To get the most out of the class, you need to be familiar with Linux file systems, Linux command line interface (CLI) and the basic linux commands such as cd, ls, cp, etc. You also need to have basic programming skills in Python, and are comfortable with functional programming style, for example, how to use map() function to split a list of strings into a nested list. Object oriented programming (OOP) in python is not required.

Syllabus

Unit 1: Introduction to Hadoop

1. Data Engineering Toolkits

  • Running Linux using Docker containers
  • Linux CLI command and bash scripts
  • Python basics

2. Hadoop and MapReduce

  • Big Data Overview
  • HDFS
  • YARN
  • MapReduce

Unit 2 – MapReduce

3. MapReduce using MRJob 1

  • Protocols for Input & Output
  • Filtering

4. MapReduce using MRJob 2

  • Top n
  • Inverted Index
  • Multi-step Jobs

Unit 3 – Apache Hive

5. Apache Hive 1

  • Databases for Big Data
  • HiveQL and Querying Data
  • Windowing And Analytics Functions
  • MapReduce Scripts

6. Apache Hive 2

  • Tables in Hive
  • Managed Tables and External Tables
  • Storage Formats
  • Partitions and Buckets

Unit 4 – Apache Pig

7. Apache Pig 1

  • Overview
  • Pig Latin: Data Types
  • Pig Latin: Relational Operators

8. Apache Pig 2

  • More Pig Latin: Relational operators
  • More Pig Latin: Functions
  • Compiling Pig to MapReduce
  • The Parallel Clause
  • Join Optimizations

Unit 5 – Apache Spark and AWS

9. Apache Spark – Spark Core

  • Spark Overview
  • Running Spark using Databricks Notebooks
  • Working with PySpark: RDDs
  • Transformations and Actions

10. Apache Spark – Spark SQL

  • Spark DataFrame
  • SQL Operations using Spark SQL

11. Apache Spark – Spark ML

  • ML Pipeline using PySpark

12. Amazon Elastic MapReduce

  • Overview
  • Amazon Web Services: IAM, EC2, S3
  • Creating EMR Cluster
  • Submitting Jobs
  • Intro to AWS CLI

Project: Data Engineering Project

Remote Learning

This course is available for "remote" learning and will be available to anyone with access to an internet device with a microphone (this includes most models of computers, tablets). Classes will take place with a "Live" instructor at the date/times listed below.

Upon registration, the instructor will send along additional information about how to log-on and participate in the class.

School Notes: We offer a certification licensed by the NYS Board of Education.

Refund Policy

Any student wishing to withdraw from a program must notify CourseHorse in writing. The date of withdrawal for refund purposes is the last date of physical attendance. The failure of a student to notify us in writing of withdrawal may delay refund of tuition due pursuant to Section 5001 and 5002 of the Education Law.

Any student requesting cancellation within seven days after signing the Enrollment Agreement but before instruction begins will be refunded all money paid less 5% cancellation fee. Thereafter, in the event of cancellation or termination by the school, refunds will be prorated based on the student's last date of attendance.

Reviews of Classes at NYC Data Science Academy (31)

Similar Classes

Benefits of Booking Through CourseHorse

  • Booking is safe. When you book with us your details are protected by a secure connection.
  • Lowest price guaranteed. Classes on CourseHorse are never marked up.
  • This class will earn you 29900 points. Earn 10% back in reward points!

NYC Data Science Academy

NYC Data Science Academy offers hands-on projects and real-world applications to teach students the skills needed for a career in data science. By enrolling, students become part of a supportive network that includes the NYC Open Data Meetup.

View school profile
NYC Data Science Academy
Vetted by CourseHorse

A verified Online educator.

Loading...