Apache Ambari on ARM64
In this blog we try to explain about Ambari and its uses, Status of the Ambari on ARM64.
Apache Ambari is an open source administration tool deployed on top of Hadoop cluster and responsible for keeping track of running applications and their status. Apache Ambari can be referred to as an open source web-based management tool that manages, monitors and provisions the health of Hadoop clusters.
The Apache Ambari is currently one of the leading projects running under Apache Software Foundation. The reason is that Ambari eliminates the need for manual tasks used to watch over Hadoop operations. It gives a simple secure platform for provisioning, managing and monitoring Hortonworks Data Platform (HDP) deployments.
How Apache Ambari came into existence
The genesis of Apache Ambari traces the emergence of Hadoop when its distributed and scalable computing took the world by storm. More and more technologies were incorporated in the existing infrastructure. Gradually Hadoop matured and it became difficult for the cluster to maintain multiple nodes and applications simultaneously. That is when this technology came into picture to make distributed computing easier.
- It's a completely open source management platform for provisioning, managing, monitoring and securing Apache Hadoop clusters.
- Apache Ambari takes the guesswork out of operating Hadoop.
- Apache Ambari, as part of the Hortonworks Data Platform, allows enterprises to plan, install and securely configure HDP making it easier to provide ongoing cluster maintenance and management, no matter the size of the cluster.
Ambari ARM64 porting efforts
We (BigData team) in Linaro are trying to port the Ambari on ARM64 and upstream all the patches. But as you know the upstream process takes it's own time to complete. In first place we are trying to compile and generate the RPM/DEB packages on standard distribution. The compilation steps for Ambari can be found here - Ambari Compilation But you need couple of patches to get it compiled on ARM64 Ambari The patches basically does the following
- Replace needarch variable hard coding of x86_64.
- Replace npm/Node version in your distribution.
- The phantomjs versions which is having AArch64 support.
- Use the ember-handlebars-brunch version which has got ARM64 support.
- Replace hardcoded amd64 values for deb.architecture variable.
- The RPM/JDEB support patches for couple of missing packages.
The Ambari do not have any AArch64 bits, It has x86 bits hard coded in all places. We have fixed it from v2.5 and used bigtop to make the build then wrote mpack (Management pack) specs and used bigtop as mpack. We have created a collaborate page to Build and Install the v2.6.1 Ambari. If you want to build different version, It may be required to slightly tweak the patches depending on the version which you want to build. These patches you may find on our linaro git repositories.
Build/Install/Run Apache Ambari
In this section we try to explain example v2.6.1 Apache Ambari build from scratch on both CentOS and Debian/Ubuntu machines. The versions are very important The latest version build method will be almost similar with some minor changes. To build latest version of the Ambari use my git repository on git.linaro.org. The repository usually up to date with patches here are usually forward ported.
- Upstream: https://github.com/apache/ambari.git
- Release-2.6.1 is used in this Wiki
- Debian 9.0 64bit for AArch64, or CentOS-7.4 64bit for AArch64
email@example.com, firstname.lastname@example.org, email@example.com, firstname.lastname@example.org, email@example.com, python>=2.6, python-dev, rpm, yum, g++
Install Pre-requisitiesFor Debian 9.0:
Setup mavenTo setup maven 3.5.3
Make sure the version of Maven is 3.5.3 when the following command is issued.
Setup python tools
Setup nodejs/npmNodejs and npm come with different versions along with Ubuntu/Debian
Ubuntu/Debian, nodejs/npm can be installed by:
CentOS7, nodejs/npm need to be built from source.
The version of built out binaries are: firstname.lastname@example.org, email@example.com.
As long as they are installed, pom.xml in ambari-admin needs to be changed to reflect these versions. The target nodejs/npm version are defined in "configuration" field of "frontend-maven-plugin".
Build PhantomJSThe following steps explain about AArch64 supported phantomjs v2.1.1. Note that you have to install all the dependency packages before you proceed further. Refer collaborate page for PhantomJS
When the build is finished, create tar file for deployment
You can test phantomjs build by issuing:
phantomjs-2.1.1-linux-aarch64.tar.bz2to the system and add phantomjs to $PATH.
Check if phantomjs is properly installed by doing:
Replace frontend-maven-pluginAmbari uses firstname.lastname@example.org, which doesn't support AArch64. Do following to rebuild this plugin for AArch64.
Replace leveldbjnilevedbjni is used in Ambari-metrics. It only provides x86/x86_64 version in maven repo. So AArch64 version of leveldbjni needs to be built and installed.
To build Ambari, a certain version number should be provided. This version number IS 5-DIGITS, not "4-digits" mentioned on Ambari's Wiki Page. The last digit may vary but the first 3 digits should be same as Ambari source/release version. In our case this is 2.6.1. Patch is provided to make Ambari built on AArch64. Apply all the patches before you are going for the build. You can directly clone and build my AMBARI git repository - https://git.linaro.org/people/naresh.bhat/apache/ambari.gitAmbari Server will create following packages
- RPM will be created under
- RPM will be created under
- RPM will be created under
Ambari ServerFirst, install Pre-Requisities
Then install the Ambari Server RPM.
Initialize Ambari Server:
Start up Ambari Server:
To access Ambari, go to:
The initial username/password is admin/admin.
Ambari AgentInstall the Ambari Agent RPM.
Then edit the location of Ambari Server in /etc/ambari-agent/conf/ambari-agent.ini by editing the
Start Ambari Agent:
Jira issues on Ambari
Git repository to track upstream activity
The Ambari version 2.6.1 patches can be downloaded from http://people.linaro.org/~naresh.bhat/Ambari-2.6.1-patches/
Ambari Demo on ARM64 at Linaro connect
- Upstream Ambari ARM64 needarch support patches
- Management pack (mpack) implementations for BigTop project