Tuesday, May 19, 2015

How to install Apache Spark on windows machine

1) Download Apache spark from http://spark.apache.org/downloads.html
2) Extract files and open Readme.md documentation.
3) Building Spark using Maven requires Maven 3.0.4 or newer,Java 6+ and scala.
4) Install JDK and Maven and setup the paths in system variables.(http://maven.apache.org/download.cgi)
5) Run command mvn -DskipTests clean package for building the package
    
6) Install scala http://www.scala-lang.org/
7) Update scala system path in system variables.


8) After building completes invoke spark shell using ./bin/spark-shell




Saturday, May 16, 2015

How to use power shell in console2

Power shell can use speed of command line and power of scripting language. To use power shell in console2.


In console2 setting set shell to
%SystemRoot%\syswow64\WindowsPowerShell\v1.0\powershell.exe