Alpakka sample

Read from a Kafka topic and publish to Elasticsearch

This example uses Alpakka Kafka to subscribe to a Kafka topic, parses JSON into a data class and stores the object in Elasticsearch. After storing the Kafka offset is committed back to Kafka. This gives at-least-once semantics.

Browse the sources at Github.

To try out this project clone the Alpakka Samples repository and find it in the alpakka-sample-kafka-to-elasticsearch directory.

Dependencies

Dependencies (sbt notation)

copysourceval AkkaVersion = "2.9.0"
val AlpakkaVersion = "7.0.2"
val AlpakkaKafkaVersion = "5.0.0"
val AkkaDiagnosticsVersion = "2.1.0"

  "com.lightbend.akka" %% "akka-stream-alpakka-elasticsearch" % AlpakkaVersion,
  "com.typesafe.akka" %% "akka-stream-kafka" % AlpakkaKafkaVersion,
  "com.typesafe.akka" %% "akka-stream" % AkkaVersion,
  "com.typesafe.akka" %% "akka-actor-typed" % AkkaVersion,
  "com.typesafe.akka" %% "akka-actor" % AkkaVersion,
  "com.lightbend.akka" %% "akka-diagnostics" % AkkaDiagnosticsVersion,
  // for JSON in Scala
  "io.spray" %% "spray-json" % "1.3.6",
  // for JSON in Java
  "com.fasterxml.jackson.datatype" % "jackson-datatype-jdk8" % "2.13.3",
  "com.fasterxml.jackson.datatype" % "jackson-datatype-jsr310" % "2.13.3",
  // Logging
  "com.typesafe.akka" %% "akka-slf4j" % AkkaVersion,
  "ch.qos.logback" % "logback-classic" % "1.2.11",

All Alpakka samples

Show Alpakka samples listing.