This 4-day course provides a deep dive into Hadoop application development.
The Students will explore how to design and develop efficient and effective MapReduce applications for Hadoop 2.0 using the Hortonworks Data Platform. The course will focus on harnessing the power of Hadoop 2.0 to manipulate, analyze and perform computations on their Big Data. Students should be experienced Java software engineers who need to design and develop Java MapReduce applications for Hadoop 2.0.
Prerequisites:This course assumes students have experience developing Java applications and using a Java IDE. Labs are completed using the Eclipse IDE and Maven. No prior Hadoop knowledge is required.
At the completion of the course students should be able to:- Describe Hadoop 2.0 and the Hadoop Distributed File System
- Describe the new YARN framework in Hadoop 2.0
- Develop a Java MapReduce application
- Run a MapReduce application on YARN
- Use combiners and in-map aggregation to improve the performance of a MapReduce job
- Write a custom partitioner to avoid data skew on reducers
- Perform a secondary sort by writing custom key and group comparator classes
- Recognize use cases for the various built-in input and output formats
- Write a custom input and output format for a MapReduce job.
- Optimize a MapReduce job by following best practices
- Configure various aspects of a MapReduce job to optimize mappers and reducers
- Develop a custom RawComparator class
- Use the Distributed Cache
- Describe the various join techniques in Hadoop
- Perform a map-side join
- Use a Bloom filter to join two large datasets
- Perform unit tests using the UnitMR API
- Describe the basic architecture of HBase
- Write an HBase MapReduce application
- Describe use cases for Pig and Hive
- Write a simple Pig script to explore and transform big data
- Write a Pig UDF (User-Defined Function) in Java
- Execute a Hive query
- Write a Hive UDF in Java
- Use the JobControl class to create a workflow of MapReduce jobs
- Use Oozie to define and schedule workflows
School Notes:
Important:
All Sunset Learning Institute classes are taught in an instructor-led, live virtual environment. This price includes a facility fee of $300 (except Reston,VA or Denver, CO) to allow you to take it in a classroom-type environment. If you prefer to take the class from home, the $300 fee will be waived and refunded.