Automate YARN Integration Tests

[lresende]

The existing integration tests require a running YARN server to execute. We could integrate with a YARN docker image to run these tests as part of the CI build.

[dimon222]

I've done some research after recent accidental findings of #75 since it does seem to feel that we suffer from not being able to run tests on actual real YARN environment.

One of implementations of docker that worked for me seems to be https://github.com/flokkr/docker-hadoop (after small tuning in flokkr/docker-hadoop#7)

It includes full-pledged stack of hadoop (so that means Hadoop & Yarn). I was able to run compose and bring up the working Yarn within 1 minute without much extra actions on Windows (perhaps that means that on Linux/OSX its even much easier).
Its full stack, so we thinking about quite plenty of RAM. I'm not sure how feasible its for usage in integration tests on free public CI platforms unless we're considering $$$.