Posts

Apache Drill on ARM64

Image
Apache Drill on ARM64
What is Drill ?
Apache Drill is a distributed MPP query layer that supports SQL and alternative query languages against NoSQL and Hadoop data storage systems. It was inspired in part by Google's Dremel.  Apache Drill is an Apache Foundation project.
Query any non-relational datastore
With the exponential growth of data in recent years, and the shift towards rapid application development, new data is increasingly being stored in non-relational datastores including Hadoop, NoSQL and cloud storage. Apache Drill enables analysts, business users, data scientists and developers to explore and analyze this data without sacrificing the flexibility and agility offered by these datastores.  
Drill supports a variety of NoSQL databases and file systems, including HBase, MongoDB, MapR-DB, HDFS, MapR-FS, Amazon S3, Azure Blob Storage, Google Cloud Storage, Swift, NAS and local files. A single query can join data from multiple datastores. For example, you can join a user pro…

usermod/groupmod tools - Rename username and usergroup in Ubuntu

The laptop come with default ubuntu installed.  In that case the username, usergroup they have created by default.  This blog explains you how you can rename the default username and group with your own username, group.

Unix-like operating systems decouple the user name from the user identity, so you may safely change the name without affecting the ID. All permissions, files, etc are tied to your identity (uid), not your username.

To manage every aspect of the user database, you use the usermod tool.  To change username (it is probably best to do this without being logged in):

STEP 1: Reboot your laptop with 1 as a command line parameter.
The laptop will be booted into a rescue mode with 1 as a parameter.  You can also boot your laptop in a single user mode.

STEP 2: Change your root password with the command "passwd"
This is just tobe secured in future,  because one can easily hack your laptop with your default user password.

STEP 3: Rename oldUsername with newUsername

# us…

Apache Ambari on ARM64

Overview:
In this blog we try to explain about Ambari and its uses, Status of the Ambari on ARM64.
Apache Ambari is an open source administration tool deployed on top of Hadoop cluster and responsible for keeping track of running applications and their status. Apache Ambari can be referred to as an open source web-based management tool that manages, monitors and provisions the health of Hadoop clusters.
The Apache Ambari is currently one of the leading projects running under Apache Software Foundation.  The reason is that Ambari eliminates the need for manual tasks used to watch over Hadoop operations. It gives a simple secure platform for provisioning, managing and monitoring Hortonworks Data Platform (HDP) deployments. 
How Apache Ambari came into existence
The genesis of Apache Ambari traces the emergence of Hadoop when its distributed and scalable computing took the world by storm. More and more technologies were incorporated in the existing infrastructure. Gradually Hadoop matured …

Benchmarking BigData

Image
Purpose:
The purpose of this blog is try to explain about different types of benchmark tools available for BigData components.  We did a talk on BigData benchmark Linaro Connect @LasVegas in 2016. This is one of my effort to collectively put into a one place with more information.
We have to remember that all the BigData/components/benchmarks are developed  Keeping in mind x86 architecture.  So in first place we should make sure that all the relevant benchmark tools compile and run it on AArch64.  Then we should go ahead and try to optimize the same for AArch64.Different types of benchmarks and standards Micro benchmarks: To evaluate specific lower-level, system operationsE.g. HiBench, HDFS DFSIO, AMP Lab Big Data Benchmark, CALDA, Hadoop Workload Examples (sort, grep, wordcount and Terasort, Gridmix, Pigmix)Functional/Component benchmarks: Specific to low level functionE.g. Basic SQL: Individual SQL operations like select, project, join, Order-by..Application levelBigbenchSpark bench The …