annotation

The article talks about an unconventional, but useful form of tests, and also summarizes seven years of work in the development of tests.

Why do we need component tests?

After all, there are, say, unit tests, which test in detail the offal of the components. They thoroughly verify that the component is working in accordance with the developer's intention. But often it is a check of "buttons", and not how the suit sits in general. And not always the behavior conceived by the programmer coincides with what the customer wanted.

And there are, for example, acceptance tests. And they eliminate all the indicated disadvantages. But, unfortunately, make new ones. They are slow, often unstable, and usually manual. However, they only indicate the problem, but do not localize it.

Obviously, the need arises for intermediate tests that will become the golden mean between modular and acceptance tests. This middle can be component tests.

What are component tests?

These are tests for the public API of the component. Accordingly, they are written in the same language as the component. Task tests:

• check the compliance of the component with its contracts
• check adherence
  The latter is especially important, since Unit tests are usually written based on the expectations of the developer, and here it is necessary to check the expectations of customers.

Obviously, component tests make sense when you have dedicated components with an extensive interface. For example, a dynamic library or a COM object. Then the component tests will give the maximum effect.

Pros to-tests:

• Give stability to the development. Obviously, a comprehensive check of public interfaces allows you to keep the component in a more or less efficient form.
• Accurately localize problems. Even if the component test itself is fairly general, it can always be debugged deep into, quickly reaching the combat code. In this case, the test environment will be minimal and automatically configured.
• Accelerate development with a critical number of developers. It is known that programmers are like horses. If one horse pulls a cart with a capacity of 1 l. with., then eight horses pull with a capacity of only about 4 liters. with. So adding another developer to the team (especially at the end of the project) often not only does not speed up, but also slows it down. While the addition of a component test developer always goes to plus, since it acts relatively independently of the command: it does (essentially external) tests, speeds up the build of the build, optimizes the build files, etc.
• Clarify (and then check) customer requirements. Since the developer of component tests is not tied to the implementation, he is less susceptible to the effect of "I myself know how to." In the case of an ambiguous requirement, the usual developer tends to do as more convenient (more interesting, faster, easier). Whereas a ct developer in this case is inclined to specify what exactly is expected by the customer.
• Relatively stable and relatively fast (compared to manual tests and automated tests through the user interface).

Cons to-tests:

• Time to develop and support. Obviously, the alpha version of the product, if part of the time spent on writing component tests, will appear later. Will there be a win in general? Will the release come sooner? This is a good question. My opinion: when developing with component tests, the release will be released approximately at the same time. But - with less hard work and more predictable in terms. The development time will naturally increase, the bugfix and stabilization time will decrease. Since the second part is much less predictable, its reduction will have a beneficial effect on the amount of processing and hassle. However, this is only my experience, and the reality may be different. It is possible that the time allocated for component tests will be spent ineptly on duplicating existing unit tests.
• Less interchangeability. Separating roles increases efficiency, but reduces interchangeability. Developers rarely want to crawl deep into component tests that can have (or imitate) a rather complex environment. Component test developers are not so well aware of the combat code to easily edit it.
• Irritant duplication. With good unit tests, component tests are often largely redundant. This is both annoying and questioning their necessity. Coordination of plans for unit and component testing helps, but duplication is usually not completely eliminated.
• The need to follow the right workflow. Upon receipt of the requirements, the task should be placed simultaneously on the developer and developer of tests. Then they finish the job at about the same time. The component is run through tests, errors are promptly caught and corrected, and a more or less finished product goes to the output. In this case, the benefit from the component tests is maximum. But it often happens that the deadlines are long-lasting, all are abandoned to write only the code, and the component is sent to manual testing without tests. And then the developer of tests is invited - it’s not good that the code without tests has been released, it would be necessary to add them. In this case, most of the bugs are hand-held testers, the developers make corrections blindly (or they themselves test hand edits), and the after-the-fact tests find only a small number of errors (which negatively affects the morality of their writer). Such use of component tests is useless at best, and it is difficult to insure against it.
• Few suitable people. The developer of component tests should, on the one hand, be able to write code in the component language (and this, for example, C ++). Moreover, if the environment for launching the component is extensive, the code can be quite complicated. On the other hand, to be able to scrupulously test someone else's work. Such people are not very many, and they usually go immediately to the developers. Still, there are such people, and the next part is about them.

Summary 1

Component tests are good, but only if you have all the conditions for them: a wide public API, the right workflow, and the right people in the team.

What is it like to be SDET?

Obviously, SDET - Software Development Engineer in Test is an ideal candidate for writing component tests. He knows how to write code, and he knows how to think with tests. He also provides a second opinion, which also improves the quality of tests and code. It all sounds interesting and tempting - perhaps you already want to be one. Here I will tell you what the work of SDET differs from the work of a pure developer.

Advantages of SDET:

• New code. Almost always SDET writes tests from scratch. And quite often the environment is written from scratch. It is very nice and gives a lot of room for creativity.
• Low dependency on legacy code. How terrible the combat code was, the tests for it could be done correctly and beautifully. Of course, poorly designed code creates ugly tests, but they can still be made an order of magnitude better than the code itself.
• More frequent refactoring. Changes in the tests are much less dangerous, so they agree more often. This is a good opportunity to work on bugs and practice writing clean code by refactoring.
• The development of critical thinking. Autotest - an interesting search for how to break someone else's code. Moreover, the search is not stupid, not sitting down and poking, but with the help of logic, combinatorics and the ability to see vulnerabilities. Plus, once created, the verification will continue to work for you all the time.
• Development of skill test code. In hand-to-hand combat training, they often give introductory ones: “now we work with our feet only; now with our heads only”. Using only one mechanism (in our case - autotests) allows you to hone it to mastery.
• Smaller number of promptings. SDET'a much less tugging at the weekend. They do not have to urgently go to work to fix critical bugs. Well, the chance to make a serious mistake is much lower.

Disadvantages of SDET:

• Low coding complexity. Test code is usually still easier than the combat code. Preconditions, the call of the combat code, postconditions - and so repeat for each test. The code for creating the environment is more complicated, but still it does not hold out to the combat. Selection of optimal algorithms, designing complex data structures, building class hierarchies - usually all this passes by the test developer.
• Slower gaining experience . The variety and complexity of situations in which the test developer falls is much less. Failure of the assembly line, red tests, sometimes a dump - this is the main set that you usually have to work with. The developer has a much higher level of problems: from variations of linking and assembly, continuing with glitches from a specific customer and cunning dumps, ending with searching for bugs in the compiler and third-party libraries. And not only...
• A serious difference in the style of tests with the developers. Typically, SDETs prefer compact expressive tests that allow you to create complex environments in literally several lines, and atomic checks in the style is equal / not equal (that is, the requirement is fulfilled or not). Sometimes it comes to your DSL. Developers simply prefer tests with fine-tuning of the environment and numerous checks of various aspects of the program's behavior, which leads to fairly multi-line tests. Sometimes it comes to copy-paste (that even the best developers do not consider in this case a sin). Here you can discuss for a long time how best it is, or even write down a separate article, but the fact is that when a developer tries to modify SDET tests (or vice versa), this often leads to long and poorly productive discussions.
• Below the "grade". Perhaps due to simpler code and lesser responsibility, but ultimately it does not matter. Just usually it is.
• Difficult transition to a new position. SDET may well get a question in the forehead: you wrote tests for so long, that is, in fact, you simply called functions and compared the results - can you write real code? Do you know all the pitfalls of the tongue? Did you solve difficult problems? Have to disassemble ornate bugs or dumps? Is there experience with multithreading? Do you have ambitions, after all?

Summary 2

As a person who worked for many years as a developer, then went to SDETs for several years and then returned to development, I can say the following.

I highly recommend SDET for at least a year or two. This is a very useful experience for any developer. But linger there, in my opinion, not worth it.