Author: Mikhail Aksenov, .NET Developer, DataArtHow it all began
In the fall of 2014, there was joyful news that Yandex.Transport was launched in Voronezh, which shows the movement of buses and trolleybuses. It was very cool right at the bus stop to open the schedule and find out what your bus will do in 5 minutes.
But after a couple of months everything went according to a quite predictable scenario. Since all the transport in Voronezh from the trolley bus to the bus works "in cash", each driver is interested in carrying as many passengers as possible. As a result, they arrange a race between themselves.
New service drivers used to follow each other on the map, overtaking and cutting competitors on the road. Some began to fall revenues, they began to complain to the owners of the routes, and they went to the people selling equipment with GLONASS and GPS and sending data about the movements of cars in Yandex. At the request of the owners of fleets, they stopped sending information, and the buses immediately began to disappear from the map. Now there are a little more of them, but this is still much less than it actually is.
For those who are too lazy to read further, I offer a video report on this topic:
For the rest - I will describe the project in the form of text.
Where to get the data?
At the same time, in Voronezh there is a place where all the data is collected - the
Center for the Organization of Road Traffic (CTO). They have a large and heavy web application with non-debugged JavaScript and other artifacts, while everything in it goes by POST requests and is not cached. Using the site itself, you can easily put a service that collects data on the movement of buses.
There was an option that allows you to poke on the map and find out when the bus will come and what it will be like. Provided that you approximately get to a stop (± 50 meters). In the menu, you could open a list of routes and see exactly where the buses go, but there was a problem - they drove strangely. For example, in Koltsovskaya, a street in the center, where you can’t especially accelerate, the bus moved forward and backward at a speed of about 200 km / h.
Why python?
I chose Python because it's cool, the batteries are bundled, and so on. And 3.6 because there is formatting of string literals, typing, that's all. Let's look at what I basically used:
- Many complained that it was very difficult to drive the exact name of the stop with all punctuation marks. Then I realized the possibility of a fuzzy search and created the first unit test for it.
- The whole system is hosted on Heroku. Free of charge, since I do not have a database - I hope to get by with the TSODD database. I agreed on its use with the Center when I realized that the data that I collect through the web interface is not enough.
- Actually, the hosting goes through the Tornado web server. I think it is familiar to anyone who has encountered web queries in Python.
- I needed the Pytz package because the Heroku server is in a different time zone, and the data comes to me without specifying the time zone. Therefore, I myself took care of localization.
- To create a bot, I used the recommended library Python-telegram-bot . The documentation here is quite adequate, and in general the library meets the requirements of the minimum viable product. Initially, the entire project was a Telegram bot, which in response to sending your location sent a bus schedule.
- Firebird, the former Interbase - Open source database, which, I believe, many of you have worked with. Of course, it is not as cool as the same PostgreSQL, but for a very large number of queries it is enough. In our case, no more is needed.
- Cahetools is a very simple module that allows you to cache calculations. This is, of course, not about memecached or caching web pages, but about cases when you need memoization for long queries. Just take, add the appropriate decorator - and it works. The cache has different options, I use the TTL version, which saves data for a specified time, because I know that the data will not be updated more often than at certain intervals (in my case - 30 seconds).
Bot functionality
- / nextbus stop name — expected arrival time;
- / last route numbers separated by a space - the last stops;
- Sending location — estimated time of arrival for the next three stops;
- Free entry - route numbers and distance to buses (when sending location).
I tried to describe the teams in full enough so that people would not ask how to use them. There are basically two basic commands: nextbus, when you write the name of the stop, and sending the location.
Site functionality
I made the
website as minimalistic as possible. I used only two of the third-party libraries to support fetch and promise, because the old versions of the pre-installed browsers in the phones cannot do without them. If you do not support these methods, you get quite heavy noodles from callbacks for the queries themselves. With fetch, everything looks sleeker.
Actually, the functionality here consists of 4 items:
- Arrival. You can just see the stop. By clicking on the button (by the way, it may be worth doing more - I’ll think about it!), You send your location, the system searches for the three stops nearest to you and shows information on them. You can enter the name of the stop and get information on it. Since you hardly need all the 20–30 buses that can come to a bus stop, you can filter them by routes.
- Buses. The second page contains information on buses. This is connected with another story that pushed me towards this project. One day I forgot my hat on the bus, called the control room, explained where it happened. The dispatcher offered me to catch my bus in the center, where he would return, turning on the final one. I remembered that I had a login and password from the Traffic Management Center system, but it turned out that I still could not track the location of the car I needed through the web interface. I returned the cap, but with great difficulty.
I thought that the search for the bus could be much easier. I wrote the route number, and the system shows you the cars on the line, their current location and the time when they passed the stops along the way. And even if you know the route number, but you know the bus number (this also happens), you can still find it. - Map. Map on the site without animation. Here you can choose buses from the list or enter numbers manually, you can filter them. Up to 600 buses can be shown on the map, although after 20 it is difficult to find something in the center, simply because the routes are all concentrated here. Therefore, it is better to limit the search parameters more rigidly.
- About the project. Now the project has a website, there is a Telegram-bot, groups in social networks.
***
Fortunately, Python is a language with batteries out of the box. For simple things, you don't even need to download anything, just read the documentation carefully.
Of course, such services can be done for other cities, moreover, in some they already work. Much depends on the local data processing center, in some places, for example, in St. Petersburg there is even an API for developers. But the most important thing is the citizens who want to change their city for the better.