Tejas Garde

Akka is powerful open source library that helps you to develop distributed application which are fault tolerant and highly concurrent. Akka Streams is implementation of Reactive Streams specification which helps you to develop application that requires stream consumption.

Why Application need Stream Processing

In todays modern application development , stream consumption is one of the most important way to consume data , whether its stream of video coming from OTT platform to your device or in some cases your mobile scroll events going from end-user to server. All these are examples of stream of data flowing from one end to another. In such cases Stream processing becomes highly important. Moreover in todays micro-service based cloud application development micro-services not just rely on REST based communication but also events streams and data streams are common communication medium.

Akka Streams

Akka Streams which are built on top of popular Akka Actors library is one the most popular Stream processing library. A Stream can be visualised as a process that includes transforming and operating on continuous flow of data . Now the Source of this data can be anything a simple array , or File System or in most cases the Stream is coming from the streaming service like Kinesis or Kafka.

Akka Stream Concepts

In this post we will look into only linear processing in akka streams . Later in this series we will explore graph based stream topology.To Start with basic elements of the Akka Streams are Source, Sink and Flow . You can imagine Source as name suggest from where the stream initialise. Akka Stream API provides number of way to initialise source something like this.

Source<Integer, NotUsed> source = Source.range(1, 10);

This represents a Source of Integer stream. This stream can be run using method runForEach which will call a Producer lambda for each value of the stream data.

source.runForeach(i -> System.out.println(i),  actorSystem)

In above we have to also provide the actorSystem object in order to run this stream. Putting it all to gather we have

public void runBasicAkkaStream(){
        runningForEachElement(getIntegerRangeSource())
                .thenRun(() -> System.out.print("Stream Consumed"));
}

public CompletionStage<Void> runningForEachElement(Source<Integer,NotUsed> source){
  return source.runForeach(System.out::println,  actorSystem)                .thenRun(() -> actorSystem.terminate());}

public Source<Integer, NotUsed> getIntegerRangeSource(){
    return Source.range(1, 10);
}

Executing this code will will print the values in the stream from 1 to 10. There are number of operations that you can do as well. The important thing to notice here is the Source can be reusable , you can have another operations on this source and consume it .

These was just an introduction to Akka Streams we will explore more in upcoming post . You can check the code on github

Docker makes all development things really easy. Just one line command and you are up with whatever you want (of course if it’s available as docker Image 😉).

Gone are those days when you have to spend hour or so to setup just a simple database on you local development environment , worrying about process , ports etc. Docker makes it really easy for you . In this article we will use a simple docker compose file to setup the PostgreSQL data base as well as PG Admin.

Here is the docker compose file to set it up. Save this code as docker-compose.yaml

version: "3"
services:
  db:
    container_name: my-pg-db
    image: postgres:13.4-buster
    restart: always
    environment:
      POSTGRES_USER: pguser
      POSTGRES_PASSWORD: pgpassword
      POSTGRES_DB: mydatabase
    ports:
      - "5432:5432"
    volumes:  # Persist the db data
      - database-data:/var/lib/postgresql/data
  pgadmin:
    container_name: my-pgadmin
    image: dpage/pgadmin4
    restart: always
    environment:
      PGADMIN_DEFAULT_EMAIL: mypg@example.com
      PGADMIN_DEFAULT_PASSWORD: Hello123
    ports:
      - "9080:80"
volumes:
  database-data:

To run this docker container go to the folder in which it is saved and type the following command

$ docker-compose up

The above command will run 2 docker containers , one is postgres db container my-pg-db and other one is PG Admin container my-pgadmin. You can check these with docker ps command . Now as the PG Admin container is running at port 9080 got to http://localhost:9080 , you should see page like this.

Here you have to login with credentials given in my-pgadmin container . One login you will have to add server from left side

In create Server page , in general section provide any name

In Connection Tab as shown you will have to provide IP address of the my-pg-db container.

to get IP address of the container , type below command

$ docker inspect <CONTAINER_ID_OF_my-pg-db> | grep IPAddress

the IP address returned by this command should be used at Host Name/address field .
Other fileds are similar to environment variables provided to my-pg-db in you docker-compose.yaml file. Click on save and you will see the database listed in you server.

To access this database from your development environment just use IP as localhost and port as mention in docker file (int this case 53432)

for example your JDBC url should look like for postgres database

jdbc:postgresql://localhost:5432/postgres

That’s it , it is as simple as that .

नमस्कार 🙏

2021 marks the most successful year for the startups in India. In one year alone India witnessed exponential rise in the number of startups reaching Unicorn milestone, to be precise 33 new unicorns .

What does this mean for Indian Startups and India ?

This number pushed Indian startup unicorn ranking from 4th to 3rd place in global unicorn race. India is now 3rd in place leaving behind UK and just below USA and China. However if you see the numbers, India is still behind these two nations, USA 254 unicorns and China with 74^[1] adding in year 2021.

Even though Indian unicorn number is not large, when compared to the global leaders in unicorns, it is quite significant for the Indian startup ecosystem.

The new unicorns in this year, ranging from variety of sectors including first ever unicorn in E-pharma (Pharm Easy) , first Health Tech unicorn (Innovaccer) and first social commerce unicorn (Meesho) not to forget multiple startups and unicorns in FinTech, On Demand Delivery and E-commerce sectors. Not only this Indians are now leading 119 Unicorns all over the world with 54 of them in India.

India has always been centre for number of multinational companies to open up their offshore development centre , Research & Development centre , outsourcing etc. However homegrown startup ecosystem has been a big leap ahead, young graduates can now not only look for big brand to work with but working on a startup is also new and exiting opportunity & with success stories like unicorn OYO Rooms founded by 24 years old Ritesh Agarwal and story of Zerodha founded by Nithin Kamath a Leading stock brokerage firm in India, Entrepreneurship is a viable option and no longer a distant dream at 50 something.

Indian television programs these days are filled with advertisements of successful startups be it digital payment app Like PhonePe, PayTM or On Demand delivery apps like Swiggy, Zomato. Not only these but there are multiple startups heading to become unicorn and whats the most exiting about them that they are being quickly adapted by consumers and you can see those not just in Television advertisements but also in IPO’s. Zomato being one of the most overbought IPO of recent time. Of course not all business models are Investor/Traders’s favourite (they see it through 😉) and stock market is different creature all together however the IPO listing can be considered as first step for any startup heading towards an enterprise.

The leading factor in this growth

There are multiple factors that triggered this growth. One of the most important factor is adaption of smart phones by majority of the population leading to larger user base for any App startup. Other factor is digital first business model, be it On Demand delivery of the food, medicines or Shopping products online.

One of the trigger point, I think, that boosts to digital payments is Demonetization decision taken by Government of India . Yes, its economical benefits are still debatable , yes it is still being criticised or praised depending upon the view, however knowingly or unknowingly it has given a much needed push for digital payments as it was one of the easiest way for the payments during demonetization period. This has increased the user base for many digital payment apps. Five years, and now in 2021 India is witnessing worlds largest digital payment with over 3.65 billion transaction^[2] this year alone.

The Road Ahead

Right from startup till being an unicorn, the journey is tough. I usually compare it with climbing 🧗🏻‍♀️ a mountain and that too not any mountain but climbing Mount Everest 🏔. Many dream of it, many train for it, few has courage to kickstart this journey (as full time or part time), some quickly give up, some push ahead of base camp 🙂(timely execution & get up to Series C funding) and finally only courageous one reaches the top 🦄.

As of now, we are just counting the private startups becoming an unicorn however there is a much bigger picture when it comes to technology or digitalization to look at. Each and every sector/business is now becoming digital and going online.

How can a government be behind? It is not, rather it is leading this sector. One of the best example is National Payments Corporation of India who developed the Unified Payments Interface  and now its playing major role in instant digital payments. Many government schemes/facilities are available online be it Railway Reservation or Passport application or filing your Income Tax return improving every year in its features and ease of use. Bringing all such facilities provided right from central government till the gram panchayat could be the largest ever digital transformation and would defiantly need lot of talent pool and therefore creating multiple opportunities.

No one can predict a definite growth however one thing is for sure, India has the right ingredients (talent pool , needed education system & encouraging/success stories) to move ahead, grow at much faster rate in terms of not only creating Unicorns but large enterprise that could shape the future, who knows the next Google, Apple or Netflix could be from India as well .

Reference

[1] Article-1

[2] Article-2

In this section we will create a POST request to user controller to create new User in our Data Base. We will now use new Connection Factory method to crate ConnectionFactory instance. Like below

@Bean
public ConnectionFactory connectionFactory() {
  return MySqlConnectionFactory.from(MySqlConnectionConfiguration.builder()
      .host("localhost")
      .port(3306)
      .username("root")
      .password("")
      .database("ExampleDB")
      .build());
}

In above code snippet we have created instance of ConnectionFactory from MySqlConnectionFactory class .

Creating Request Object

We will create a UserSaveRequest class which will match our JSON POST Request Object so that the incoming JOSN data we get as an object in our controller.

@Data
@AllArgsConstructor
@NoArgsConstructor
@Builder
public class UserSaveRequest {
  private String firstName;
  private String lastName;

}

Controller Changes for new POST Method

Add a @PostMapping to UserController class as below.

@PostMapping
@ResponseStatus(HttpStatus.CREATED)
public Mono<UserSaveRequest> add(@RequestBody UserSaveRequest userSaveRequest) {
  return userService.saveUser(userSaveRequest);
}

Using Repository Default Method

We will use inbuilt save method of ReactiveCrudRepository to save our User Object. This save method has signature as below

<S extends T> Mono<S> save(S var1);

We don’t have to do anything other than passing an object of User class created from incoming request to this method. It will save that object into the data base.

Modifying UserService

Add the save method in UserService interface and implementation in UserServiceImpl class . This method is simple it will take the object of UserSaverRquest and create new Object of User from it and then call save method of Repository.

public interface UserService {

  Flux<UserResponse> getAllUsers();

  Flux<User> findUserByFirstName(String firstName);

  Mono<UserSaveRequest> saveUser(UserSaveRequest userSaveRequest);

}

@Override
public Mono<UserSaveRequest> saveUser(UserSaveRequest userSaveRequest) {
  User user = new User();
  user.setFirstName(userSaveRequest.getFirstName());
  user.setLastName(userSaveRequest.getLastName());
  return userRepository.save(user)
      .thenReturn(userSaveRequest)
      .doOnError(e -> {
        if(e != null) {
          throw new RuntimeException("Exception occurred during data base operation");
        }
      });
}

userRepository() method will save the object to Data base. the .thenReturn() method will get called when Mono is completed successfully and return the same UserSaveRequest object as we are not exposing any of the primary keys of data base to the client of the api. The .doOnError() will throw a RuntimeException if there some error while saving it to data base.

Response From API Call

You can check the code commit for this changes here

In order to understand Spring WebFlux more deeply we will implement a simple REST end-point for User. This User end-point as of now will have following REST action.

Creating Spring Boot WebFlux Application

So first we will have to create Spring WebFlux application. Its simple to stat with , just go here and fill up Project metadata and in dependencies use Spring Reactive Web. This will generate a folder with requires pom.xml and other Spring WebFlux dependencies . Once you download it , extract and import it in your favourite IDE . Make sure the pom.xml file has following dependency .

<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-webflux</artifactId>
</dependency>

Creating REST Controller

All is set up for us to create Controller . But before writing controller code directly , let’s just first take a moment and understand what are the information we are going to need to implement the controller . So we already know few things , and these are

• We need REST End Point for Users
• We have to use HTTP GET Method
• We Know Path/URI for this HTTP GET Method
• We Know what to send in Response.

Now we need to implement above functionality in Spring WebFlux application, Spring provides annotation for all of the above requirement , we just have to implement it in correct way. So here are those annotation in Spring Reactive Web Provides

• @RestController for REST End Point for Users
• @GetMapping for HTTP GET Method
• @RequestMapping for Path/URI for this HTTP GET Method
• @ResponseBody to send in Response.

Lest understand what each of this annotation do.

@RestController

@RestController is composed annotation. It is composed of @Controller and @ResponseBody annotations. @Controller defines bean as controller whereas @ResponseBody indicates that every method this bean writes directly to the response body . The @RestController annotation is applied at the top of the class .

@GetMapping

@GetMapping annotation is type of request mapping annotation that gets invoked when GET HTTP method is called . We can also use @RequestMapping annotation here like this

@RequestMapping(method = RequestMethod.GET)

@RequestMapping

@RequestMapping annotation maps the incoming request for the given URI to the particular endpoint . In our case we need to point all the request (irrespective of HTTP Method) to our controller , so we will declare this annotation at top of the Controller class mentioning the URI.

@RequestMapping(USER_API_V1+SLASH+USERS)

where values in parenthesis is the URI in our case it is “/v1/users” indicating its version-1 of the API with endpoint “users”. The same @RequestMapping annotation can also be used along with HTTP method above functions to define the routing as mentioned above.

@ResposneBody

@ResposneBody annotation adds the return type of the method where it is declared to the repose body of the request. It is defined at the top of controller methods. In Simple word if the method is returning String , then response of that endpoint will be String.

With above annotations knowledge lets create a controller with String as @ResposneBody.

@RestController
@RequestMapping(USER_API_V1+SLASH+USERS)
public class UserController {
  @GetMapping
  public String getValue(){
    return "Hello World";
  }
}

If you run the Spring application with above controller we will get response in browser for GET HTTP call on http://localhost:8080/v1/users as “Hello World” .

Till now its exactly similar to how we do it in Spring MVC/Boot and we have not yet used any of the WebFlux feature, in order to use it , lets enhance the application and add Service layer for User and return UserResponse object.

@Service
public class UserServiceImpl implements UserService {
  @Override
  public Flux<UserResponse> getAllUsers() {
    return Flux.fromIterable(generateUsers());
  }

  private List<UserResponse> generateUsers() {
    return Arrays.asList(UserResponse.builder()
            .userId(1)
            .userName("User-1")
            .build(),
        UserResponse.builder()
            .userId(2)
            .userName("User-3")
            .build(),
        UserResponse.builder()
            .userId(3)
            .userName("User-3")
            .build(),
        UserResponse.builder()
            .userId(4)
            .userName("User-4")
            .build());
  }
}

Flux and Mono in Reactive Spring

As you can see we have UserServiceImpl class which has getAllUsers() method that Returns Flux<UserResposne>. Since we are returning the more than 1 entity in response we will have to use the Flux of that entity.

There are 2 Types of Publisher in Spring , Mono and Flux. Mono is a Publisher that produces 0 to 1 value . Flux is publisher that produces 0 to N values. As of now just keep in mind that we want to return more than 1 UserResposne entity we are using Flux. When we will use endpoint to get single entity of UserResposne that time we will use Mono. More on Flux and Mono in upcoming post.

Final Controller Code

@RestController
@RequestMapping(USER_API_V1+SLASH+USERS)
public class UserController {
  @Autowired
  private UserService userService;

  @GetMapping
  public Flux<UserResponse> getValue(){
    return userService.getAllUsers();
  }
}

When you run this application using postman with url http://localhost:8080/v1/users the you should see the following response

Now we are done with first part of Spring WebFlux tutorial. There are lot more things to learn and things are getting super exited, we have just started.

The Code for above you can find it here at GitHub.

Thank You 🙏