Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 23 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,37 @@
# dockerfiles
# Kafka docker builds

Automated [Kafka](http://kafka.apache.org/) builds for [solsson/kafka](https://hub.docker.com/r/solsson/kafka/)
and related `kafka-` images under https://hub.docker.com/u/solsson/.

Nowadays we're using separate repositories for dockerization projects.
---

This repo used to contain misc dockerfiles, but they've moved to separate repositories for dockerization projects.
We've kept the repository name to avoid breaking the automated build of solsson/kafka in Docker Hub.

For legacy Dockerfiles from this repo (if you navigated to here from a Docker Hub [solsson](https://hub.docker.com/u/solsson/) image),
see https://github.com/solsson/dockerfiles/tree/misc-dockerfiles.

# Kafka docker builds
---

This repository maintains automated [Kafka](http://kafka.apache.org/) builds for https://hub.docker.com/r/solsson/kafka/
and related `kafka-` images under https://hub.docker.com/u/solsson/.
Our kafka images are tested in production with https://github.com/Yolean/kubernetes-kafka/.

These images are tested in production with https://github.com/Yolean/kubernetes-kafka/.
You most likely need to mount your own config files, or for `./bin/kafka-server-start.sh` use overrides like:
```
--override zookeeper.connect=zookeeper:2181
--override log.dirs=/var/lib/kafka/data/topics
--override log.retention.hours=-1
--override broker.id=0
--override advertised.listener=PLAINTEXT://kafka-0:9092
```

## One image to rule them all

Official [Kafka distributions](http://kafka.apache.org/downloads) contain startup scripts and config for various services and clients. Thus `./kafka` produces a multi-purpose image for direct use and specialized docker builds.

We could build specialized images like `kafka-server` but we have two reasons not to:
* Won't be as transparent in Docker Hub because you can't use Automated Build without scripting.
* In reality you'll need to control your own config anyway.

### Example of downstream image: Kafka Connect

See ./connect-jmx
Expand All @@ -28,11 +44,7 @@ TODO

Rudimentary compliance with kubernetes-kafka is tested using a [build-contract](https://github.com/Yolean/build-contract/).

Build and test using: `docker run -v /var/run/docker.sock:/var/run/docker.sock -v $(pwd)/:/source solsson/build-contract test`

To keep kafka running for local use, uncomment `ports` 9092 and run: `docker-compose -f build-contracts/docker-compose.yml up --force-recreate`.

While timing issues remain, start services individually...
Build and test using: `docker run -v /var/run/docker.sock:/var/run/docker.sock -v $(pwd)/:/source solsson/build-contract test`. However... while timing issues remain you need some manual intervention:

```bash
compose='docker-compose -f build-contracts/docker-compose.yml'
Expand Down
3 changes: 3 additions & 0 deletions build-contracts/docker-compose.files-aggregation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,12 @@ services:
build: ../kafka
links:
- zookeeper
entrypoint: ./bin/bin/kafka-server-start.sh
command:
- config/server.properties
- --override
- zookeeper.connect=zookeeper:2181
- --override
- broker.id=0
- --override
- advertised.listener=PLAINTEXT://kafka-0:9092
Expand Down
3 changes: 3 additions & 0 deletions build-contracts/docker-compose.monitoring.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,12 @@ services:
- JMX_PORT=5555
expose:
- '5555'
entrypoint: ./bin/bin/kafka-server-start.sh
command:
- config/server.properties
- --override
- zookeeper.connect=zookeeper:2181
- --override
- broker.id=0
- --override
- advertised.listener=PLAINTEXT://kafka-0:9092
Expand Down
3 changes: 3 additions & 0 deletions build-contracts/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,12 @@ services:
com.yolean.build-target: ""
links:
- zookeeper
entrypoint: ./bin/kafka-server-start.sh
command:
- config/server.properties
- --override
- zookeeper.connect=zookeeper:2181
- --override
- broker.id=0
# unlike Kubernetes StatefulSet, compose gives containers a random hostname (leading to redirects to a hex name)
- --override
Expand Down
5 changes: 2 additions & 3 deletions kafka/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ RUN set -ex; \
rm /var/log/dpkg.log /var/log/apt/*.log

WORKDIR /opt/kafka
ENTRYPOINT ["bin/kafka-server-start.sh"]

RUN sed -i 's/zookeeper.connect=localhost:2181/zookeeper.connect=zookeeper:2181/' config/server.properties
CMD ["config/server.properties"]
COPY docker-help.sh /usr/local/bin/docker-help
ENTRYPOINT ["docker-help"]
15 changes: 15 additions & 0 deletions kafka/docker-help.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/bin/bash
echo "Hi,"
echo ""
echo "This image is basically just the official Kafka distribution,"
echo "containing both servers and utils, each with its own help output."
echo ""
echo "Select as entrypoint one of these scripts:"
find ./bin/ -name *.sh
echo ""
echo "You might find one of the sample config files useful:"
find ./config/ -name *.properties
echo ""
echo "Add more using volumes, or downstream images."
echo "Enjoy Kafka!"
echo ""