How it all beganIdeological tormentTechnologies and how they are not uniqueHow to store and where?Not only store, but also searchThis mysterious SEOCDN is our everythingSummarizeHow it all began
I want to share our semi-annual history of creating an online search service and booking copyright travel around the world. When creating this article, the goal was to tell about the experience of creating your own solution from scratch and those problems and challenges that we encountered on this path, so do not judge strictly.
To begin with, what is this service, and why it was necessary to create it. For a long time, our team has been working on various large projects for business clients. These are portals for banks, enterprises, large holdings, as well as document management systems. Naturally, working with such a segment implies the satisfaction of a narrow circle of interested employees and does not always end with a long and successful use.
There are several reasons for this: lack of understanding of the project at the initial stage, the desire to save and get the maximum configuration, changing the requirements at the final stage without any revisions of deadlines and budgets. As you can understand, satisfaction from the project on both sides, after such conditions, is very difficult to achieve.
So, after all this experience, we decided to try our hand at the masmarket, where feedback from the client is lightning, you understand the interest to the product as quickly as possible and see the use of your service every day.
As it turned out, this is where our calm days are over!
Ideological torment
The first thing to do was to come up with an idea, what do we want to provide so useful, what is not on the market and what will be perceived and expected. We had several such tests and during the first two months we were engaged in experiments to find that very future. As you can understand with technologies, there was also a problem, as it is necessary to make functioning layouts in the shortest possible time, test them for operability and constantly improve or redo them anew.
At the start, the luggage technology we had was not very jQuery and C # (which we used in our previous projects). These tools did not allow us to quickly and flexibly respond to the challenges posed. Fortunately, we have already considered new approaches for both client and server development. Our choice fell on Angular 2 and Node.js. This solution allowed us to quickly form components and modules that we could modify in the shortest possible time (we could redo them two or three times a day).
It came to the conclusion that each component should be as small as possible and integrable into other components and modules. Thus, in the process of experiments, we have accumulated enough templates from which it was possible to assemble various working layouts.
But if we figured out quite quickly with the toolkit and pumped up our skills more and more in the battle (I’ll immediately notice that Google will help you, but do not forget about the video lessons), then another question arose where to store the data. But I will tell about it a little later, and now we will return to the idea itself.
The main vector of our service, we have focused on the organization of travel, you will immediately say that in our time there are a lot of travel websites and services, plus the same bunch of travel agencies and I agree with you. But the question is how to change the attitude of the average tourist to rest and give him the opportunity to get the maximum set of impressions and feel an individual approach to his beloved.
We understood that such ambitious plans could not be solved with one service or with one solution, but we always have to start somewhere. And now, having established ourselves with the main goal, we began to look for that first brick, which will serve as the foundation of our future solution. After a long search, we had the idea of a service, which now I want to talk about.
What is this?
In brief, this is UBER for travelers. Imagine you are an experienced traveler who has traveled to many interesting places around the world and really wants to share his experience, and earn some money. On the other hand, a tourist who is tired of the Turkish holiday with lying on the beach and eating and drinking. And these people need to somehow combine. So we decided to help these people find each other.
The service is a market where experienced tourists can register to design their tours and provide them for the masses of future travelers.
Technologies and how they are not unique
How to store and where?
As I already wrote, after selecting the tools for creating the service, the question arose of choosing a data warehouse. Of course, over the years we have become accustomed to relational databases and to SQL Server, in particular. But, there was a need for a platform for minimum configuration at the start and a minimum of effort with further support, as well as the possibility of scaling in the development of our service and the growth of visitors.
Thus, during the search, a solution was found that covered all our requirements, it was the Firebase Realtime Database. This service helped us solve the issues of data storage, hosting, authentication service, storage for storing static data. Plus, this service is under the wing of Google, which in turn gave us nice bonuses, as it provided separate libraries for working with Angular 2 and it was convenient and fast. But the coolest thing we got is the RealTime Database out of the box. Our first sensations were just fantastic.
We now did not have to constantly make requests to the service, monitor the relevance of the data on the client, wait until the data comes to the client (well, I note, here we were helped by Angular). We configured all this for about a day after which we began to create our service directly, without being distracted by infrastructure issues. Immediately, in the development process, we did not spend a penny on this service, since the basic version is free and quite enough for development, testing and experimentation.
Not only store, but also search
And as soon as the first beta was ready, a question arose in filtering the data, and we realized the first cons in Firebase. As it turned out, the process of sampling data for a large number of conditions is simply not supported. It is possible to select data with only one condition, or collect several values in one field and then filter on it. This approach did not suit us, and we again set about finding a solution.
Of course, we did not turn away from Firebase, since this one minus did not overlap that mass of advantages provided by this service. Fortunately, we had the experience of using Google Big Query, which we got in the process of our research, and we decided to use it. The first plus is that it is Google, the second is the native and favorite SQL queries and the low cost of owning 1TB of data per month for free.
Again, everything became clear and understandable, we wrote a data sending service and successfully hooked it to the Cloud Function. I forgot to say, about Bek in Firebase was also taken care of, you can write your own triggers on Nodejs and hang them on any event in the database, as well as on the http request.
As you can see, the process of finding solutions and approaches has become a daily task for us, since almost every day we faced new challenges that needed to be addressed promptly.
We didn’t have time to relax and calmly continue to create our service, as the testing of the focus group began, and we realized that users were used to switching filters very quickly and want to immediately see the result of their work. Such demands made us abandon Big Query and look for something new. Since Big Query gave us a response speed of a maximum of 2 seconds, and this with a minimum amount of data. There’s certainly nothing to scold them for, since this service is intended for processing large volumes of information, and not for a quick response to a bunch of consecutive requests.
As a result, we opted for Elastic Search. This product allowed us to build a fast search engine, our filtering began to meet the requirements of our users. I will not tell much here, since this product has begun to gain popularity and there is enough information on it. The only thing I want to note is that this product requires a virtual machine that needs to be hosted somewhere. This feature provides both Google and Microsoft, so the choice is yours. Configuring it is just as easy using the bitnami resource. We chose Azure for ourselves, this choice was not so much due to technological features, but more due to third-party factors.
This mysterious SEO
And now the service is running around, the platforms are being mastered and everything seems to be good, we are striving for a brighter future. But, here comes the question of promoting our service and the key issue of SEO. I never thought that this question might not be so worked out by the creators of Angular. As it turned out, Single Page Application provides this opportunity very poorly, and if you are honest, it does not provide it at all. Yes, you can sew static data into meta tags and produce the same information when going around a kral or when sharing it, well, it will be somehow wrong and ineffective. Having gone through the Internet, we stumbled upon such a wonderful service, which is part of Angular 4, Angular Universal. After reading the description, we realized that this is what is needed and that in vain we scolded the developers of the framework.
The epic began with screwing this happiness to our project, I note, at this point about 10 large modules were already written and about 12 third-party npm packages were used. The first configuration of the clean project went great, thanks to the manuals on the Internet, and everything seemed to start. And we twisted the server part on the same Cloud Function from Firebase. Well, now we need to try and try the combat code, and then there were problems. The first, as it turned out, all third-party npm packages must support Angular 4, in the code on the server side you cannot use window, document variables, etc., well, debugging all this happiness is simply not realistic.
I will not describe all our torment, with this service, as it was a lot and painful. I will say one thing: we never overcame it, I don’t know or didn’t fully understand it, or it’s just raw and not ready for productive use. On the whole, it’s up to you to decide who can succeed, but we decided to stop there. As a result, a service was written that listens to all http requests and returns it when index.html is requested, but before that, it throws necessary meta tags into it. As a result, it turned out well and for 3 months the flight is normal. By the way, we host all this too on Azure for all the same third-party factors.
CDN is our everything
Here and again some time stability, and relatively quiet work. And no one thought that a surprise awaits us from Facebook. One day it turned out that sharing in FB does not work. At first, we thought that this was due to the tightening of security policies in the FB, then with IP blocking, but we did not find the reason. Appealed to support both FB and Firebase, but each kicked at the other.
The only thing is that we determined that the problem is in urls for pictures that were stored in our Firebase store, and url, I will tell you right away, there is a very specific one. We decided to transfer all static content to Azure as well, and I’ll tell you this solution was correct, since the photo upload speed has increased, and you can manage it all more flexibly and transparently.
Summarize
At the moment we are already in the 3rd month of production, constantly improving and expanding the functionality. For version control, of course, we use Git, little by little we automate assemblies and deploy. On the day, we get about 450 new unique hits, there are jumps up to 1000 users. And it all works.
What would you like to sum up from all of the above:
- No need to try at the expense of one service or one technology to solve all the tasks that you have in projects.
- Try to develop universal modules for more flexibility in the future.
- Choosing a cloud service provider is a very personal matter, since in general they all provide the same set of services. The question remains in the price and your preferences.
- Develop your solution so that you can migrate between different service providers and technologies, or at least think over the strategy of a possible move.
- Well, do not be afraid to experiment.