Context
I work on a big 10 year old application (500k blocs of code) with some really bad choices:
- No unit tests
- an Anemic architecture But in a great company:
- A QA team who test regressions
- A small team (5 dev only)
- Times to make clean code
- A project of rewriting the application with new technologies As you imagine, the QA teams are a big bottleneck and everything that can reduce test regression is good.
The rewriting
With rewriting the CTO wants :
- unit tests :)
- A Microservices architecture with :
- One service by feature with duplications between services
- A single database
- A single repository, but one makefile by service
What's I think
I agree that Microservice can avoid regression, but I think it's too complicated. I don't want to have to manage database conflicts or update 200 services each time I change the database...
My guess is, with:
- Unit tests
- Open/Close principle
- DDD
- And maybe SOA (1 service by bound context) We can almost avoid regressions and it will be a better start if we really need microservices. What do you think?
Top comments (18)
Not a good idea to implement microservices. Your dev team is too small. It would make more sense if your team is composed of squads (each squad being a team of people, not a single person) to own certain services. Otherwise it would be a big burden to own and maintain them
I agree with Oscar, and I'd like to add a point: one could argue that spliting a large codebase into several smaller ones could actually benefit the team, and for that I'd say: be aware that you will end up with more "moving pieces", so it might be a bit of a trap: trading coding complexity for operational complexity.
One other thing: presuming this code has little to no coverage, I would first aim at integration tests rather than unit tests - test bigger chunks first, making sure that bigger flows are actually understood (and at the time building a sort of documentation for this code), and then moving in to unit testing smaller pieces.
Agree with Oscar. Microservices in your case has more downsides than upsides
You even not mentioned operations team. But infrastructure is integral part of the any microservices-based design.
Here you can find more information.
Multiple services with 1 database is the Integration Database pattern.
The friction to make changes is going to be amplified with this approach. It would be moving from spaghetti code to spaghetti architecture. You'd be better off modularizing the code into different libraries if sticking to 1 database.
I don't know your situation but generally speaking, rewriting a large application is going to take multiple times longer than probably will be estimated and it will be a bad time for devs and users. Because users depend on so many behaviors of the current system that they don't realize, so they can't tell you. So when they try to switch to the new system they will hate it because it doesn't do all the things they depend on. You can mostly prevent this but you need to have access to user experts. Interview them to find out their actual business workflows, not just the data they save, and not just reworking the legacy code you have. When something doesn't make sense, rather than forging ahead with your best guess (which we devs seem to do without realizing it), go back and ask your experts more questions. You often find you had some assumptions wrong and need to do some redesign. You will be doing this and discovering new requirements all the way to release and beyond. This is basically applying the non-coding parts of DDD. And of course, have users test out their workflows as you complete them so you get feedback early.
It's not enough details for a good answer so only based on my experience with microservices.
One of the keys for microservices success is automation if you have DevOps and QA that can write automated tests then even team of 5 devs can handle this approach.
Devs should be able to write high quality unit tests that validate positive, negative and edge cases it will help to the devs to re-run their code a lot of times and reduce amount of bugs + prevent regressions but slows down the development process. Devs have to run at least positive tests of their code. All the above will reduce a huge amount of work from the QAs that can focus on automated tests that have to be integrated into CI/CD process and again prevent the regressions. In it's turn automated tests will help to devs to be sure that their code don't destroy existing functionality and increase their productivity. It leads me to the staging environment where you can deploy your "clean" code and run all the possible tests on iin, in case of failure everything should be rolled back to the previous state.
If you can automate your processes to this level nothing should prevent you from using microservices.
From the experience with unit tests, design your code to support them it will reduce amount of time that you invest into unit tests development and maintenance.
Edit:
If you select distributed architecture it will required good level of observability. I.e. logs collected into centralized place and all the elements are monitored. So the main overhead is in management of everything and automation of everything.
Microservices in terms of code architecture are IMHO a bad choice. You should base your decisions based on what makes sense for you and what problems you have to solve.
Upsides of microservices usually are the standalone nature of said service, the independence from other teams and the independent deployment cycle. (teams are not blocked by each other)
If you don't need those or you don't have these problems, sticking with a traditional approach is probably the better choice.
Monolyths (with a modular code structure) are usually easier to debug and for lower throughout faster since you don't need to have the additional http overhead.
Also one nogo for me: sharing database between services. That's a big big red flag! Don't do that, it will cause so much pain
A skilled dev team can utilize packages/modules will have a way more consistent and efficient Programm that is easier to debug and operate, micro-service or even micro-frontend architectures solve an organisational problem, rarely a technical one
Only use microservices when you need to deploy a service independently and it must scale out individually. Otherwise use modules inside a well structured code base. Microservices only makes sense if you can separate the business logic into further independent domains and benefit of separated deployments. Mostly teams fail applying mircroservices since they think its cool or new and end up in a distributed monolith
I guess using microservices is doable even on smaller teams. It's way more organized than the legacy one. Both approach has its downside but it's more important to aim for a much maintainable codebase, that way it will benefit your small team.
The given context is too few. I can give my opinion in general context.
Microservice if can add more resources (people, money, time). Microservice is not a silver bullet, but really helpful for colossal application in term of maintaining the codes for long term.
SOA if the resources are very limited.
The unit tests itself will consume a lot of people and time resources.
What's the missing informations you need ?
Many.
First example, I can't tell how large the application is. "10 year old application (500k blocs of code)." 500k blocks of C++ code is very different with 500K blocks of python code. Not to mention if there are any deprecated codes that haven't been deleted. 5 people maintaining 500k blocks also make it harder to assume the size of the apps.
Second example, how many "feature" it has, and how it is dependent to each other. Because microservice and SOA usually separated per features/service. The more features it has, the more service that have to be developed, the more complex it will be. Especially if the features/services are dependent to each other. The decoupling will be very tricky to do.
Third example, how good and complex the datamodel is. the worse and the more complex the current datamodel, the more I'll go with microservice.
etc.
But the basic problem of developing the microservice is it will consume more resources: people, time, and money. So if the resources is pretty limited and the resource expansion is also limited, just go with SOA.
You need a capable consultant to answer this question anw. This kind of question won't get a proper answers in internet forums.
Having run a project with a very small team and microservices, from scratch, for ten years: nope. Unless you are going to invest heavily in training and education for your devs, unless they are used to distributed architecture: nope.
I get your cto's points. It's why I made the choice to go microservices in the first place. However, unless you have an excellent ops team, an excellent toolsmith or six, I highly suggest not to do it. If you have a very very small amount of services, maybe, but by definition the amount of them grow. You need so many tools and such a buttoned up ci/cd process. Deployments and rollbacks need to be very automated. Qa needs to be very automated. You need to automate ALL the things with that few people.
We started with about 8 services. We are now up to around 35. It's a LOT to manage with very few heads.
I don't fully agree with a lot of concerns others posted here, the simple reality is this, micro-service architecture is great if you know how to do it and horrible if you don't. Be honest with yourself do you really know what problems you will encounter and will you know how to deal with those problems? I would suggest not to do micro service architecture unless you've read a comprehensive book on it e.g. Microservice Patterns because the changes are you don't know what you are getting yourself into.
To be able to refactor a large legacy code base I would recommend looking into Vertical Slice Architecture and before you fall into the trap of unit tests watch this talk TDD, Where Did It All Go Wrong
It's been long since this post got out, but i hope you managed to find a suitable trade off for your needs. I think, this is how post about any kind of technology should start. By first presenting a real world problem, and then exposing an actual solution that works, as there is no such thing as a universal best way to do something. So thanks
IMHO microservices can only work well on a single team if they have really solid DevOps skills and tooling already founded within the team. Otherwise, they are going to amplify all the inefficiencies in your team that are part of your value stream. The velocity of a team with immature delivery skills, practices, and tooling after moving to microservices can even be much, much worse than before.
On the flip side, it can create a much faster velocity and a much more stable system as a whole.
I guess my point is start off on the right foot by making sure your team has a DevOps mindset and builds a strong and lean delivery muscle and then refactor toward microservices.
Thinking of QA as a bottleneck is a bad development vision. The best developers partner with QA to develop reusable test automation in continuous integration pipelines. Without this automation you'll never be able to develop microservices.
There is this youtube video from CodeOpinion where he says he prefers to start with a monolith but each bounded context would only communicate through public API and events as in microservices style. Everything is in a single codebase so quite easy to modify and since each bounded context is loosely coupled it should be easy to extract them if necessary.
youtu.be/Z_pj1mUDKdw
And actually if microservices is not a good choice in you case, SOA would be worst. You will get most of the distributed architecture issues but in a more tightly coupled style do... More issues...