If you do not live in a cave, you may know that microservices are the architecture of today. With the development of this trend, in the Segment product
, it was adopted at an early stage
as the best practice that served well in some cases, and, as you will soon see, not so well in others.
In short, microservices are a service-oriented software architecture in which server-side applications are built by combining multiple single-task, minimal network services. Benefits include improved modularity, simplified testing, better functional composition, isolation of the environment, and autonomy of development teams. The opposite is a monolithic architecture, where a large amount of functionality is located in one service, in which testing, deployment and scaling occur as a whole.
In early 2017, we reached a turning point with the main part of our
Segment product . It looked as if we were falling from the microservice tree, hitting each branch on the way down. Instead of developing faster, the small team is immersed in increasing complexity. The significant advantages of this architecture have become a burden. As our speed dropped, the number of defects increased.
As a result, the team was unable to succeed with three full-time engineers who spend most of their time simply maintaining the system. Something had to change. This post is a story about how we took a step back and adopted an approach that fit our requirements and needs of the team well.
Why microservices worked worked
Segment's client data infrastructure takes hundreds of thousands of events per second and redirects them to a partner API, what we call server-side destinations. There are more than one hundred types of these directions, such as Google Analytics, Optimizely, or custom web hooks.
Years ago, when the product was originally launched, the architecture was simple. There was an API that received events and sent them to a queue of distributed messages. The event in this case was a JSON object generated by a web or mobile application containing information about users and their actions. An example of the payload was as follows:
{
"type": "identify",
"traits": {
"name": "Alex Noonan",
"email": "anoonan@segment.com",
"company": "Segment",
"title": "Software Engineer"
},
"userId": "97980cfea0067"
}
, , . API , , , – Segment API, , . Segment .
, . , . , , , . , HTTP 500, -. , , , , . , .
, , , ,
. , , ,
.
, X -. , , X, . , , . , X . , .
, . -, . , , , . , , , .
API , . – X
traits.dob, API
traits.birthday. X :
const traits = {}
traits.dob = segmentEvent.birthday
Segment, . , API . , , XML .
, , . , , . , . . , .
. .
50 50 . , , HTTP , .
, ,
event.name() .
name Name. ,
firstName,
first_name,
FirstName. , .
Identify.prototype.name = function() {
var name = this.proxy('traits.name');
if (typeof name === 'string') {
return trim(name)
}
var firstName = this.firstName();
var lastName = this.lastName();
if (firstName && lastName) {
return trim(firstName + ' ' + lastName)
}
}
. .
. . . , , . .
, . , . . , , .
, . , – . , .
, , - , .
, , , . , . .
140 . . , .
, . , , .
Centrifuge. Centrifuge .
, , , . , .
120 . . , .
, . , . .
, . , .
, . HTTP , , , .
, . , . , HTTP , . , . . , , .
HTTP . , . 5 . 140 .
Traffic Recorder. Traffic Recorder
yakbak, . , , . . , . , HTTP , .
, , Traffic Recorder. , 140+ . .
., . , . 140+ . .
. 2016, , 32 . 46. 6 , 2016 .
. , . , , .
, :
- . , , , . , . , , .
- . , , , . 3000+ , . - Redis , . , .
, . , . , . , .
, . , , , .
- . , , . , , .
- , , , . , .
. , , . .