Summary


Gimel Standalone

Overview

The Gimel Standalone feature will provide capability for developers / users alike to


Install Docker


Download the Gimel Jar

cd gimel

Run Gimel Quickstart Script

cd $GIMEL_HOME
quickstart/start-gimel {STORAGE_SYSTEM}
quickstart/start-gimel kafka,elasticsearch,hbase-master,hbase-regionserver

Note: This script will do the following * Start docker containers for each storage * Bootstrap the physical storages (Create Kafka Topic and HBase tables)

docker exec -it spark-master bash -c \
"export USER=an;export SPARK_HOME=/spark/;export PATH=$PATH:$SPARK_HOME/bin:$SPARK_HOME/sbin; \
/spark/bin/spark-shell --jars /root/gimel-sql-1.2.0-SNAPSHOT-uber.jar"

Note: You can view the Spark UI here


Common Imports and Initializations

import org.apache.spark.sql.{DataFrame, SQLContext};
import org.apache.spark.sql.hive.HiveContext;
import com.paypal.gimel.sql.GimelQueryProcessor

val gsql = GimelQueryProcessor.executeBatch(_:String,spark)