Rise of the robots. It seems that this is something incredible. And even if probable, it will not happen soon. However, it is worth considering - the staff of robots from our partners for the year grew by 22 cars. And they are clearly not going to stop. What if the next upgrade does not go according to plan and the robot refuses it. Or say that he was tired of working in accounting. More under the cut!
Button - back office for entrepreneurs. We keep accounts, solve legal issues and take away the routine.This article is a continuation of this:
Rise of the Machines: How robots captured accounting .
I give the word to the author.How many robots and why do we need them
Now 64 robots are launched regularly. We need robots not only for intellectual work, but also for everyday roasting of pots. For example, for collecting statistics, for monitoring information from 1C for our application. Robots help bill and even scavenge for other robots.
To automate the work of an accountant, we closely interact with the databases in 1C. We try different options for this. Tell you more.
Interaction through COM. Historically - this is the first way that managed to establish interaction. We got a reliable job of writing to the 1C database, but we met minuses:
- Connections are established for a long time, sometimes they hang or suddenly break.
- In each base 1C you need to connect separately, bypassing all bases can take hours.
Interaction through the postgreSQL database using a translator. This approach gave us a general picture of the status of 1C databases for each client. For example, are there counterparts or duplicate documents?
Pros:
- Connecting to the database is very fast.
- You can work immediately with a whole cluster of 1C databases, which increases the processing speed by orders of magnitude.
Minuses:
- Works only on reading from the database. Theoretically, the work on the record can also be done, but for this you need to repeat all the logic of 1C - it is inappropriate.
- Development is more time consuming and process support is more complicated than in the COM method.
Interaction through OData. Tried it recently.
Pros:
- No problems with freezes or broken connections.
Minuses:
- The first operation takes much longer than subsequent ones, although the connection is established quickly.
- Investments in the interaction infrastructure are needed in order to use it to become as convenient as in the case of COM.
- Additional efforts are needed to administer 1C, especially when updating the configuration.
Interaction through web services. This approach was tested relatively recently.
Pros:
- It works quickly and without delay.
Minuses:
- We need development within the 1C platform, the environment is nonstandard for us, so there is still not enough competences and additional infrastructure is required to fully implement CI.
- As in the case of COM and OData, tasks can be solved only within the same 1C database.
Robots need to be controlled
Robots are imperfect. Like their creators, they stumble, fall and break. The reasons can be very different: incorrect configuration of services, errors in the code, problems of the hardware level - equipment failures. It is important for us to know this as early as possible. Because of the robots tied many business processes of the company. We created monitoring robots to keep track of everyone.
We are watching when the robot was last launched. If we know that it should be launched once an hour, but it has been resting for 3 hours already, this is a reason to encourage it.
Sometimes, the robot pretends that it works, but in fact it is frozen. To catch this, we look at the journal that the robot writes about its work. If the records do not appear long enough, then you need to restart it.
There are robots whose consequences and mistakes are not immediately noticeable. In order to notice something amiss in time, we created metrics that focus on the results of work. So we control the most important robots.
For example, the robot guide, which is important for us. He ensures that incoming documents are correctly taken into account in 1C. The robot makes one detour of all 1C bases in 3-4 hours - half a working day. Recall that we serve 1300 entrepreneurs. The robot does not enter any bases in each bypass. Most likely, only the next day, the accountant will notice that the robot has not looked into a specific database.
Rate this robot worked in a specific database - a separate serious task for the accountant, who in addition to carrying out documents has other tasks for accounting and advising clients. Therefore, in reality, this error can only be noticed after a week, or even a couple. And in time, unsent documents are incorrectly calculated tax.
Therefore, we set up the robot so that it records which bases it has bypassed and when, and in this case we see how many, which bases the robot has not circumvented, and for how long. As a result, we draw the accountant’s attention to where it is needed and at the right moment.
Robots need to be changed
In dreams of a robotic future, we can hope that a brilliant robot who just came off the assembly line will do his job forever, and we will be able to do something more interesting. In real life, it is not. Time passes and the fact that yesterday seemed to be quite adequate work, today is no longer satisfied. The quality of the result is not that either works too slowly, or even does what in a changed situation is meaningless. Robots are aging, and they need to be upgraded.
Robot consolidation
Some robots are engaged in related tasks, but their actions are not coordinated or they have been waiting for each other for a long time. We sort such robots and assemble one that deals with all related tasks.
We always keep “robots” that work on a task with a person, “on duty”, in anticipation of the work they receive from a person.
Take, for example, the process of turning scans of paper documents into documents recorded in 1C. Previously, several robots were doing this together:
- The retrieval robot unpacks the archives into separate files, and a pdf with several pages turns into several pictures.
- The robot sorter examines images, categorizes and arranges them into folders.
- Next, the robot recognizer does its job.
After their work, a primary accountant is connected to check and correct for the robot, if necessary. After the person is connected, a robot that saves documents in 1C, and then a robot guide, which conducts them. All of these robots were configured in polling mode. They ran on their own schedule once in a while and looked to see if there was work. If there is, they do it, and then they fall asleep again.
There are a lot of robots, so time delays were added from the schedule of each robot. It turned out that from the moment when we saw the document, we realized it as such, we began to work with it before the conducted document appears in the 1C database, it can take half an hour, or even an hour. In order to inform the client of the current tax rate on the available documents, the accountant had to wait a long time. Then I want to answer the client online.
For a new solution, we disassembled robots for parts and collected two new ones. The first turns archives and pdf files into documents that can already be sent to 1C. Further work is checked by the person, if necessary - something rules. The second robot immediately picks up the document after verification, saves and holds it in 1C.
Now the work is set up in the event model: the robots are always in standby mode and react immediately as the person has completed his part.
Improving the robot
After reducing the time of the document in 1C, we are faced with another problem. We recorded cases where documents were lost on our side. I had to request them from the client again, which is extremely unpleasant. Just solve this problem now. And it seems again we will analyze and assemble new robots.
We also retrained neural networks to classify and rotate documents into more neural and reticular. This gave an increase in accuracy of 1%. A few more hundred pages per week no longer require human attention.
Upgrading robots due to platform change
Last time we told you why we decided to switch from Java to Dotnet and how it was. A year ago, there was still a lot of code on Java. We sawed this golden weight incessantly. And only in March, they were able to completely turn off the servers on which Java was working. Together with her, another part of the robots went into the world.
New robots
Tegun
Customers write to us on different issues. In order for these questions to find their performer, we categorize them by tags. At first, people did this, but we decided to ease their work and wrote a robot. And they called him - テ 軍 (Tegun).
The first version was very simple. Work in our application is arranged in the form of correspondence with branches. In one branch there can be several tasks. The robot categorized - put the tag, based on what tag was in this correspondence. It is obvious that in the new branches, where there is no tag yet, the robot did nothing. In this implementation, the robot handled 40% of the work of people, while making a mistake in 10% of cases.
So that the robot could identify the artist in the new branches, we taught him to read. For this, logistic regression and gradient boosting were used for prediction, in combination with tf-idf and WordToVec for reading. The latest version of the robot allowed to take 60% of the work of people. Knowledge is power! Although the robot began to make mistakes in 15% of cases, tasks still began to come to the performers faster.
Sending reports
It does not always require intelligence to perform the necessary and useful work. Programmers and accountants know that routine can take up to 80% of the time. For example, to send a SZV-M report, a payroll accountant should open the 1C base, press about ten buttons, enter data in between, wait for 1C to say “ok” and close the base.
Each such operation takes 5 minutes. Little. But to do this for all customers of the button, which need a report (there are about 600), you will have to spend 5 * 600/60/8 = 6 working days of continuous mechanical work. A few clone robots that do about the same thing do the job for the day. Recall that such a report should be submitted every month, and there should be 20 working days in a month. A third of the working time of one accountant would be spent only on the continuous sending of the report.
Synchronization with 1C
Another important class of robots that have appeared over the past year is synchronizers.
In our application, there are documents and bank statements that are closely related to the data in 1C objects. For us, it is important that the state of the application most closely matches the state of affairs in 1C. We use this to search for duplicates, to easily search for objects and to display correct data to clients. Using the interaction through the postgreSQL database, we were able to track the changes that occur in 1C and quickly reflect them in the application database. To date, the discrepancy of data in time does not exceed 5 minutes.
Do robots benefit
Abandon the robots, we can not. They automate a significant part of the routine, so much so that it has allowed us to grow in the client base, without increasing proportionally the accounting staff. If we turn off our robots, then somewhere we will lose 2 hours at a time in response time, and somewhere we will start to make a big mistake or miss mistakes.
Our accountants are already largely focused on metrics that show the result of a particular robot. This allows us to “rehearse reporting” one month before the real reporting. Plan in real days, how much time we need to have time to generate statements and pass it on time. This is a huge step towards transparency and reliability of the accounting process for the client.
Some robots are still not enough, you need a man. Because robots break and need to be repaired. Robots can be wrong, they need to be checked and trained. Another difficulty is the creation of effective interaction between robots and humans. On the one hand, you need to create robots that are convenient for humans, and on the other, be able to teach people to use them.
Thank you for the article Vasily Chernoyvan.