Skip to content

This is a collection of simple runnable self contained examples from various akka streams docs, tutorials and blogs

License

Notifications You must be signed in to change notification settings

scala-steward/akka_streams_tutorial

 
 

Repository files navigation

Build Status Scala Steward badge

Akka streams tutorial

Akka vs Pekko
As of umbrella release 22.10 Lightbend has changed the licensing model. Apache Pekko is the open source alternative. A BIG Thank you to the committed Pekko committers.

For now the branch migrate_pekko contains a 1st basic migration (with a few losses). The plan is to move the content of this branch to a new pekko_tutorial repo.

"It's working!" a colleague used to shout across the office when yet another proof of concept was running it's first few hundred meters along the happy path, aware that the real work started right there. This repo contains a collection of runnable and self-contained examples from various Akka Streams and Alpakka tutorials, blogs and postings to provide you with exactly this feeling. See the class comment on how to run each example. These more complex examples are described below:

Many of the examples deal with some kind of (shared) state. While most Akka Streams operators are stateless, the samples in package sample.stream_shared_state also show some trickier stateful operators in action.

Other noteworthy examples:

Remarks:

Other resources:

Element deduplication

Dropping identical (consecutive or non-consecutive) elements in an unbounded stream:

The following use case uses a local caffeine cache to avoid duplicate HTTP file downloads:

  • Process a stream of incoming messages with reoccurring TRACE_ID
  • For the first message: download a .zip file from a FileServer and add TRACE_ID→Path to the local cache
  • For subsequent messages with the same TRACE_ID: fetch file from cache to avoid duplicate downloads per TRACE_ID
  • Use time based cache eviction to get rid of old downloads
Class Description
FileServer Local HTTP FileServer for non-idempotent file download simulation
LocalFileCacheCaffeine Akka streams client flow, with cache implemented with caffeine

Windturbine example

Working sample from the blog series 1-4 from Colin Breck where classic Actors are used to model shared state, life-cycle management and fault-tolerance in combination with Akka Streams. Colin Breck explains these concepts and more in the 2017 Reactive Summit talk Islands in the Stream: Integrating Akka Streams and Akka Actors

Class Description
SimulateWindTurbines Starts n clients which feed measurements to the server
WindTurbineServer Start server which a accumulates measurements

The clients communicate via websockets with the WindTurbineServer. After a restart of SimulateWindTurbines the clients are able to resume. Shutting down the WindTurbineServer results in reporting to the clients that the server is not reachable. After restarting WindTurbineServer the clients are able to resume. Since there is no persistence, the processing just continuous.

Apache Kafka WordCount

The ubiquitous word count with an additional message count. A message is a sequence of words. Start the classes in the order below and watch the console output.

Class Description
KafkaServerEmbedded Uses Embedded Kafka (= an in-memory Kafka instance). No persistence on restart
WordCountProducer akka-streams-kafka client which feeds random words to topic wordcount-input
WordCountKStreams.java Kafka Streams DSL client to count words and messages and feed the results to wordcount-output and messagecount-output topics. Contains additional interactive queries which should yield the same results WordCountConsumer
WordCountConsumer akka-streams-kafka client which consumes aggregated results from topic wordcount-output and messagecount-output
DeleteTopicUtil Utility to reset the offset

HL7 V2 over TCP via Kafka to Websockets

The PoC in package alpakka.tcp_to_websockets is from the E-Health domain, relaying HL7 V2 text messages in some kind of "Alpakka-Trophy" across these stages:

Hl7TcpClientHl7Tcp2KafkaKafkaServerKafka2WebsocketWebsocketServer

The focus is on resilience (= try not to lose messages during the restart of the stages). However, currently messages may reach the WebsocketServer unordered (due to retry in Hl7TcpClient) and in-flight messages may get lost (upon re-start of WebsocketServer).

Start each stage separately in the IDE, or together via the integration test AlpakkaTrophySpec

Analyse Wikipedia edits live stream

Find out whose Wikipedia articles were changed in (near) real time by tapping into the Wikipedia Edits stream provided via SSE. The class SSEtoElasticsearch implements a workflow, using the title attribute as identifier from the SSE entity to fetch the extract from the Wikipedia API, eg for Douglas Adams. Text processing on this content using opennlp yields personsFound, which are added to the wikipediaedits Elasticsearch index. The index is queried periodically and the content may also be viewed with a Browser, eg

http://localhost:{mappedPort}/wikipediaedits/_search?q=personsFound:*

Movie subtitle translation via OpenAI API

SubtitleTranslator translates all blocks of an English source .srt file to a target language using the OpenAI API endpoints:

  • /chat/completions (gpt-3.5-turbo) used by default, see Doc
  • /completions (text-davinci-003) used as fallback, see Doc

Akka streams helps in these areas:

  • Easy workflow modelling
  • Scene splitting with session windows. All blocks of a scene are grouped in one session and thus translated in one API call
  • Throttling to not exceed the API rate-limits
  • Continuous writing of translated blocks to the target file

About

This is a collection of simple runnable self contained examples from various akka streams docs, tutorials and blogs

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Scala 75.3%
  • Java 21.1%
  • HTML 3.4%
  • Shell 0.2%