Currently, I have 20 microservices for one project. And every microservice stored in separate GIT reposotiry. Subsequently, the number of services will increase to 200 (or more).
Every service has unit tests and integration tests. Every service has build in TeamCity (Continuous integration server).
Question: How to store source code of 200 microservices for one project? In one repository or in separate repositories?
Unless those micro-services are tightly coupled (meaning it wouldn't make sense to download only some of them, and you would only work with all of them), keeping them each in a separate Git repo is recommended.
But you can still reference them as submodule in a parent repo in order to keep track of their state at any given time.
git submodules and subtrees have their own problems too. Facebook and google? have a single repository for all their microservices. There is no clear answer to this problem but things such as refactoring, dependency management and project set up get benefits from a mono repository. Again, the coupling of the services and how different teams interact with the repo is key to choose one or another.
We have not used submodules; What we have done is branching strategy
Every micro-service has its own folder under base_folder folder.
There is a release branch --> currently one master ( this has everything) There is an interface branch --> interfaces ( this has only interfaces like for example protobuffer/ grpc files for all services) . This branch is always merged to master
Each service has a branch --> sprint_service_name where code is pushed (for code review ) and a merge request created to master branch
Code flow
For new component
git checkout interfaces
git checkout -b sprint_service_name
(create branch from interface branch)
For existing component
git checkout sprint_service_name
git fetch origin interfaces
git merge origin/interfaces (don't use git merge origin interfaces !!)
(or for above two steps git pull origin interfaces)
Here is a Developer flow
Push to sprint_service_name branch and create merge request to master branch
git push origin sprint_service_name
Branch flow
sprint_service_namexxx --> Merge Request --> master
sprint_interfaces --> Merge Request --> Interfaces -->master
interfaces --> sprint_service_namexxx (by all, every day)
All common parts will be in interfaces branch
(you can have any number of private branches; But be careful not to merge of master into sprint_service_name or master into interfaces ; else unwanted files will be in your branch)
Pros Each micro service will have only its code and interfaces folder
Cons
I have seen that the below flow does not always happen ideally, like
sprint_interfaces --> Merge Request --> Interfaces -->master
It is more like
sprint_interfaces --> Merge Request --> master
Which means that someone has to manually take Interfaces folder from master and merge to Interfaces branch. But this is just a discipline thing and has not broken anything
My choice would be to store in different repositories and I am doing it for years and think its make it easier to manage then making it complex. If you refer to 12 factor app you should know the benefits of having separate repos. You will have different test suite, different release and life cycle for each microservice. It will end up increasing the velocity of the deliverables.
A single repository for all may work but its better to separate them out into different repos and have a complete set of pipeline tools for each of the service. As the micro services principle says services can be managed by different teams and can be build in different technologies and should have their own releases which can only be achieved with different repos.
If above mentioned factors are not enough then you can think of a scenario where only one service needs to be changed and tested separately through pipeline then separate repos will make it easier to release.
You should have only one repo for the entire project whether it's big or small.
You can refere 12 Factor Application for building such project which handles most of the things from here.
Concept of the 12 Factor application is for large project which have many microservices.
I had worked in a company where we used to work with 130+ services. Some services were connecting to external API, i.e. out of our ecosystem. But we just used keep each service into their own GIT repository. We had commons library for encryption, logging etc under one BOM. Creating individual repository will stop you accidentally making interlinked microservices, and that is the goal of whole idea. In future, if needed, you can further cut down individual service. With 130+ microservices we never had issue of inter dependent code and also deployments were smooth without being concerned about the other service.
Having separate repositories will help at the startup but when the project grows it becomes very difficult to manage. 12-factor app guideline also suggests maintaining a single monolithic repository. Those shouldn't be submodules, except scenarios like you manage configurations in a separate repository. Having a monolithic codebase with subprojects for each microservices allows you to build your CI/CD pipeline. That's another big advantage. Ex: let's assume you need to work on two or more microservices to get done a particular feature or an issue. You can track all the services with one commit for that issue.
