Apache Flink Local Mode Installation on Ubuntu 14.04

posted on Aug 02nd, 2017

Apache Flink

Apache Flink is an open source stream processing framework developed by the Apache Software Foundation. The core of Apache Flink is a distributed streaming dataflow engine written in Java and Scala.

Pre Requirements

1) A machine with Ubuntu 14.04 LTS operating system and

2) Apache Hadoop 2.6 installed (How to install Hadoop on Ubuntu 14.04)

3) Scala 2.10.5 Software (Download Here)

4) Apache Flink 1.2.1 Software (Download Here)

NOTE

If you want to interact with Hadoop (e.g. HDFS or HBase), make sure to pick the Flink package matching your Hadoop version.

Flink Local Mode Installation

This post is intended to provide instructions on how to install Flink locally on a single machine.

Installation Steps

Step 1 - Update. Open a terminal (CTRL + ALT + T) and type the following sudo command. It is advisable to run this before installing any package, and necessary to run it to install the latest updates, even if you have not added or removed any Software Sources.

$ sudo apt-get update

Step 2 - Installing Java 7.

$ sudo apt-get install openjdk-7-jdk

Step 3 - Install open-ssh server. It is a cryptographic network protocol for operating network services securely over an unsecured network. The best known example application is for remote login to computer systems by users.

$ sudo apt-get install openssh-server

Step 4 - Creating scala directory.

$ sudo mkdir /usr/local/scala

Step 5 - Change the ownership and permissions of the directory /usr/local/scala. Here 'hduser' is an Ubuntu username.

$ sudo chown -R hduser /usr/local/scala
$ mv spark-1.6.1-bin-hadoop2.6/* /usr/local/spark

Step 6 - Untar the scala-2.10.5.tgz file. In my case the downloaded scala-2.10.5.tgz file is in /home/hduser/Desktop folder. For you it might be in /downloads folder check it.

$ tar xzf scala-2.10.5.tgz

Step 7 - Move the contents of scala-2.10.5 folder to /usr/local/scala

$ mv scala-2.10.5/* /usr/local/scala

Step 8 - Creating flink directory.

$ sudo mkdir /usr/local/flink

Step 9 - Change the ownership and permissions of the directory /usr/local/flink. Here 'hduser' is an Ubuntu username.

$ sudo chown -R hduser /usr/local/flink

Step 10 - Untar the flink-1.2.1-bin-hadoop26-scala_2.10.tgz. In my case the downloaded flink-1.2.1-bin-hadoop26-scala_2.10.tgz file is in /home/hduser/Desktop folder. For you it might be in /downloads folder check it.

tar xzf flink-1.2.1-bin-hadoop26-scala_2.10.tgz

Step 11 - Move the contents of flink-1.2.1 folder to /usr/local/flink

mv flink-1.2.1/* /usr/local/flink

Step 12 - Edit $HOME/.bashrc file by adding the java, flink and scala path.

$ sudo gedit $HOME/.bashrc

$HOME/.bashrc file. Add the following lines. Save and Close.

export SCALA_HOME=/usr/local/scala
export FLINK_HOME=/usr/local/flink
export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64
PATH=$SCALA_HOME/bin:$JAVA_HOME/bin:$FLINK_HOME/bin:$PATH

Step 13 - Reload your changed $HOME/.bashrc settings

$ source $HOME/.bashrc

Step 14 - Change the directory to /usr/local/flink/bin

cd /usr/local/flink/bin

Step 15 - Start flink

./start-local.sh

Step 16 - The JPS (Java Virtual Machine Process Status Tool) tool is limited to reporting information on JVMs for which it has the access permissions.

$ jps

flink Local Mode Installation on Ubuntu 14.04

Step 17 - Stop flink

./stop-local.sh

flink Local Mode Installation on Ubuntu 14.04

Please share this blog post and follow me for latest updates on

facebook             google+             twitter             feedburner