Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .env → .env.example
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@

DRILL_VERSION=1.16.0
PARQUET_DATA_DIR=

1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@ data/*
*.parquet
*.jar
*.csv
.env
26 changes: 13 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,11 @@ Server: Docker Engine - Community

The docker image contains the following files:

* The `docker-compose.yml` file includes the description of how to build and configure the container that will run Apache Drill.
* The `.env` file contains the parameter `DRILL_VERSION` that determines which version of Apache Drill is being built and run. Currently, the `DRILL_VERSION` is set to `1.16.0`. If you need to change this, adapt the `.env` file.
* The `docker-compose.yml` file includes the description of how to build and configure the container that will run Apache Drill. The local data folder is parameterizable through the `PARQUET_DATA_DIR` environment variable, which is mounted as `/data` inside the container.
* The `.env` file contains the parameter `DRILL_VERSION` that determines which version of Apache Drill is being built and run. Currently, the `DRILL_VERSION` is set to `1.16.0`. It also contains the parameter `PARQUET_DATA_DIR`, which points to the local folder with the data files (e.g., parquet files) that will be mounted as `/data` inside the container. If you need to change either, adapt the `.env` file.
* The `/build/Dockerfile` contains the build descriptions for the container.
* The `run_drill.sh` contains the startup script for Apache Drill.
* The `.gitignore` prevents that any file in the data folder or any parquet/csv files will be added to the repository.
* The `.gitignore` prevents that any file in the data folder or any parquet/csv/jar files will be added to the repository.

## Get the docker image

Expand All @@ -63,7 +63,7 @@ git version 2.20.1 (Apple Git-117)

2. Check out the repo into a directory of your choice:
```
git clone https://github.com/mschermann/docker_apache_drill_datagrip.git
git clone git@github.com:multialejo/drill_datagrip.git
```

## Build and start the container
Expand Down Expand Up @@ -92,7 +92,7 @@ services:
restart: on-failure
tty: true
volumes:
- /<YOUR PATH>/data:/data:rw
- /home/ariel/Escritorio/sri-scrapper/.staging:/data:rw
```

2. Build the container
Expand All @@ -103,7 +103,7 @@ docker-compose build
You should see that docker starts to build the container. This will take a while depending on your internet speed and machine configuration.
```
Building drill
Step 1/13 : FROM centos:latest
Step 1/14 : FROM centos:7
...
```

Expand All @@ -130,19 +130,19 @@ Stopping e382a3c16e10_drill ... done

Drill is starting a web GUI at [http://localhost:8047](http://localhost:8047).

![Drill Overview](https://github.com/mschermann/docker_apache_drill_datagrip/blob/master/assets/drill_overview.png)
![Drill Overview](assets/drill_overview.png)

If you click on `Query`, you can run SQL queries directly from the browser (Do not use this for any heavy-load querying).

![Drill Overview Query](https://github.com/mschermann/docker_apache_drill_datagrip/blob/master/assets/drill_overview_query.png)
![Drill Overview Query](assets/drill_overview_query.png)

Make sure that everything works fine by entering the example query `SELECT * FROM cp.`employee.json` LIMIT 20`.

![Drill Example Query](https://github.com/mschermann/docker_apache_drill_datagrip/blob/master/assets/drill_query_example.png)
![Drill Example Query](assets/drill_query_example.png)

It will show you a waiting screen and, if everything works fine, the results.

![Drill Example Query Results](https://github.com/mschermann/docker_apache_drill_datagrip/blob/master/assets/drill_query_example_results.png)
![Drill Example Query Results](assets/drill_query_example_results.png)

Now, head over to the [Drill Documentation](https://drill.apache.org/docs/query-data/) and start learning how to use Drill.

Expand All @@ -151,17 +151,17 @@ Let's connect [Data Grip](http://www.jetbrains.com/datagrip) to the Drill contai

1. Create a new Driver in Data Grip by pointing towards the JDBC driver for Apache Drill in the `/build` folder.

![Data Grip Driver](https://github.com/mschermann/docker_apache_drill_datagrip/blob/master/assets/data_grip_drill_driver.png)
![Data Grip Driver](assets/data_grip_drill_driver.png)

2. Create a data source using the Drill driver. Test the connection and make sure you get the green checkmark.

![Data Grip Datasource](https://github.com/mschermann/docker_apache_drill_datagrip/blob/master/assets/data_grip_drill_connection.png)
![Data Grip Datasource](assets/data_grip_drill_connection.png)

3. Run a Sample Query

Using the same query as above (`SELECT * FROM cp.`employee.json` LIMIT 20`), you should see the following output.

![Data Grip Sample Query](https://github.com/mschermann/docker_apache_drill_datagrip/blob/master/assets/data_grip_sample_query.png)
![Data Grip Sample Query](assets/data_grip_sample_query.png)

At this point, you are all set. Add your data files to the `/data` folder, and you should be able to query them.

Expand Down
9 changes: 6 additions & 3 deletions build/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
# THis is the build file for Apache Drill
# Adapted from https://github.com/apache/drill/blob/master/distribution/Dockerfile

FROM centos:latest
FROM centos:7

# Point yum to the CentOS Vault since the mirror.centos.org servers are offline (EOL).
RUN sed -i 's/^mirrorlist=/#mirrorlist=/g; s|^#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*.repo

# Make sure that everything is up to date.
RUN yum -y upgrade
Expand All @@ -19,8 +22,8 @@ RUN rm -rf /var/cache/yum
# Make the current version of drill available
ARG DRILL_VERSION

# Get drill
RUN wget http://apache.mirrors.hoobly.com/drill/drill-$DRILL_VERSION/apache-drill-$DRILL_VERSION.tar.gz
# Get drill (the old mirror is dead, use the Apache archive)
RUN wget https://archive.apache.org/dist/drill/drill-$DRILL_VERSION/apache-drill-$DRILL_VERSION.tar.gz

# Create the install directory for drill
RUN mkdir /opt/drill
Expand Down
6 changes: 2 additions & 4 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
version: '3'

services:
drill:
drill:
build:
# We build the file from the current folder
context: ./build
Expand All @@ -21,7 +19,7 @@ services:
# Expose the build directory to get the JDBC driver
- ./build:/drill
# The data directory for the data files.
- ./data:/data
- ${PARQUET_DATA_DIR}:/data
# This is the script that is executed when the container is started.
command: ./run_drill.sh
# We make the standard drill ports available
Expand Down