What is IMPALA ?
Impala is a High-performance SQL Engine for vast amounts of data.
Impala is a High-performance SQL Engine for vast amounts of data.
- Massively parallel procesing
- Inspired by google's Dremel project
- Query latency measured in milliseconds
Impala runs on hadoop clusters
- Can query data store in HDFS or HBase tables
- Read and Write data in common hadoop file formats
Developed by cloudera
- 100% open source under apache server licence
Impala supports a subset of SQL-92
Why to use IMPALA ?
Many benefits are the same as with Hive or PIG
- More productive than writing Map-Reduce code
- Leverage existing knowledge of SQL
One benefit exclusive to impala is Speed
- Highly optimized for queries
- Almost 5times faster than Hive or Pig. Often 20 times faster or more
IMPORTANT:
Pig & Hive functions from your laptop where as the impala sits on Hadoop cluster on all data nodes.
Comparing Impala to Hive & Pig
Similarities:
- Queries expressed in high-level languages
- Alternatives to writting mapreduce code
- Used to analyze data stored on Hadoop clusters
- Impala shares the meta store with Hive (Tables created in Hive as visible in Impala (viceversa))
Contrasting Impala to Hive & Pig
- Hive & Pig answers queries by running Mapreduce jobs.Map reduce over heads results in high latency.(even a trivial query takes 10sec or more)
- Impala does not use mapreduce.It uses a custom execution engine build specifically for Impala.
- Queries can complete in a fraction of sec.
- Hive & Pig are best suited long-running batch processes (Data Transformation Tasks).Impala best for interactive/Adhoc queries.
- Impala can't handle complex data types(Array,Map or Struct)
- No support for binary data type.
If a node fails in Hive or Pig, They answers queries by running mapreduce jobs in other nodes. But,incase of impala if the node fails during a query,the query will fails and it has to be re-run
Relation Database vs Impala
