Week 1 : Setting Up Hadoop

Hi Everyone!

The first week of my internship IISc as a project trainee has ended. There was some exciting work this week. We completed introduction to Clay Codes and moved forward to work with setting up Hadoop.

Setting up Hadoop came with various hurdles. There weren’t many good tutorials online which talked about developmemt setup of hadoop. Build dependecies created serious problems on Ubuntu 18.04. Spent more than a day to figure out it’s the distribution’s issue. Still need to figure out what went wrong. Tried setting it up using out of the box docker setup, available in the Hadoop’s repo, but didn’t seem to work well.

Later resorted to setting it up on a vagrant Ubuntu Xenial machine, all was smooth then. Faced some common issues related to versions of various dependencies and found the solutions to most of them online.

Joined the Hadoop developer mailing list late into the week. Positive that it would help us a lot.

I will be writing a tutorial later on the setup to ease out the process for other new Hadoop developers.