Benchmarking BigData
Purpose: The purpose of this blog is try to explain about different types of benchmark tools available for BigData components. We did a talk on BigData benchmark Linaro Connect @LasVegas in 2016. This is one of my effort to collectively put into a one place with more information. We have to remember that all the BigData/components/benchmarks are developed Keeping in mind x86 architecture. So in first place we should make sure that all the relevant benchmark tools compile and run it on AArch64. Then we should go ahead and try to optimize the same for AArch64. Different types of benchmarks and standards Micro benchmarks: To evaluate specific lower-level, system operations E.g. HiBench, HDFS DFSIO, AMP Lab Big Data Benchmark, CALDA, Hadoop Workload Examples (sort, grep, wordcount and Terasort, Gridmix, Pigmix) Functional/Component benchmarks: Specific to low level function E.g. Basic SQL: Individual SQL operations like select, proje...