How we moved from monolith to microservices

The story about how we moved from monolith to microservices.

Contents

Introduction: once upon a time

Introduction

At the end of 2013, when I started the prima.it project, I was an expert PHP programmer. I had been working for years with Symfony framework and I deployed a lot of PHP projects.

Using PHP with Symfony framework and mysql as database was natural for me and Matteo, the colleague that started the project with me. We thought that having all the features in one project could help us being more efficient and in control of our development.

Another important point was that neither of us wanted to manage the server. We are on AWS, but we want to do the minimum needed, hoping for the arrival of a devops as soon as possible :)

In the first months we developed a lot of features and we were able to present a working demo with the functionalities requested.

But the more features arrived, the more it was clear that the monolith started to be too heavy for our situation.

All the Symfony tools and plugins that helped us to develop a working website (form, doctrine, routing, jms serializer, the rest bundle) turned out to be a limitation. PHP too started to show his limits. We need to elaborate big files or perform async tasks. We also had the bad idea to write worker processes with Symfony and PHP.

Number zero

Number zero

As well as the main website, we needed an internal one for our backoffice operators, so that they can manage all the customers requests. We decided to write our backoffice application as a SPA because we had already written a lot of APIs. The ability to deploy a single project, the separate development context and the decoupling of business logic were instantly seen as strengths.

Almost by accident we built the first microservice.

Go live!

Go live

After a lot of code and days we finally went live. It was clear very early that some worker processes wrote to perform specific task were not suitable: the long duration of processes, the memory usage, and the resources needed were problems.

From day zero our app was dockerized and we already used RabbitMQ messages. These two technologies helped us taking the big next step: split the monolith.

Ch-ch-ch-ch-Changes

Changes

In the middle of 2015 we needed to implement a SOAP client to talk with a third-party API. The task must be async and the time of response was unknown. We also knew that over the months the requests would grow quickly. We needed to perform parallel requests, and it was unthinkable to develop this request in PHP.

We wanted to try a “new” programming language that had an high hype, elixir: I read some articles and Matteo, the company innovator, already tried it out.

We exposed the idea to the rest of developers team and illustrated the advantages of this choice to the management. Matteo took on the responsibility of the project and in a couple of weeks the microservice was live.

It wasn’t easy: the usual issues one meets when starting with something new (best practices, finding useful libraries, “something different from the examples in the docs?”), the infrastructure and the communication between services were worries that kept us awake more then a few nights. But at the same time it was an exciting challenge.

Reading a lot of articles about success stories of people and companies that adopted elixir helped us to do the big jump.

Elixir is awesome, so every developer in the team wanted to write some piece of code in that language. The business as well understood and appreciated the flexibility of isolating business logic in different components / projects.

To infinity and beyond

To infinity and beyond

Now, in early 2017, we have 23 microservices in production. We are able to develop new features with the infrastructure, technology, and language that better fit our necessity.

With context isolation we can quickly fix, test, and deploy single features.

Lesson learned

Lesson learned

I think, and I’m not alone, that we took the right way: starting simple and implementing complex things only when it’s really needed.

It’s not simple to have so many services communicating with one another, and you need good devops skills to manage, deploy, and have every microservice under control. The metrics and the logger are essential.

Every microservice shares similar patterns and the development has sped up.

Pros and cons

Pros and cons

Pros

  • isolate business logic
  • development of features in a specific context
  • independent and parallel development
  • a microsevice’s codebase is small: more maintainable and testable
  • independent technology stacks
  • indipendent scalability

Cons

  • complex infrastructure
  • more effort to have high availability
  • proliferate of microservices proliferation information
  • you need a strong infrastructure for QA and testing environment
  • difficult deploy when more microservices are involved
  • thoroughly log and metrics to check every services

Alessandro - @aleinside

«To live is the rarest thing in the world; most people exist, that's all.»

Passionate programmer, basketball fan, whisky lover, full stack developer.
#php (always less), #elixir (awesome), #react and #redux (present), #elm (future)