Software testing and debugging. Debugging programs. Software Debugging Techniques

Debugging the program

Debugging- stage of computer program development, at which errors are detected, localized and eliminated. To understand where the error occurred, you have to:

• find out the current values ​​of variables;
• find out which way the program was executed.

There are two complementary debugging technologies.

• Using debuggers - programs that include a user interface for executing a program step by step, statement by statement, function by function, stopping at certain lines of source code or when a certain condition is reached.
• Outputting the current state of the program using output statements located at critical points in the program - to the screen, printer, loudspeaker or to a file. Outputting debugging information to a file is called logging.

Place of debugging in the program development cycle

A typical development cycle, repeated many times over the life of a program, looks something like this:

1. Programming - introducing new functionality into a program, correcting errors in existing ones.
2. Testing (manual or automated; by a programmer, tester or user; “smoke”, in black box mode or modular...) - detection of an error.
3. Reproducing the error - finding out the conditions under which the error occurs. This can be challenging when programming parallel processes and with some unusual errors known as Heisenbugs.
4. Debugging- detection of the cause of the error.

Tools

A programmer's debugging ability is likely the most important factor in discovering the source of a problem, but the difficulty of debugging depends greatly on the programming language and tools used, particularly debuggers.

Debugging tools

A debugger is a software tool that allows the programmer to observe the execution of the program under study, stop and restart it, run it in slow motion, change values ​​in memory, and even, in some cases, go back in time.

The following may also be useful tools in the hands of a programmer:

• Profilers. They will allow you to determine how long a particular section of code takes to execute, and coverage analysis will allow you to identify non-executable sections of code.
• API loggers allow the programmer to track the interaction of the program and the Windows API by recording Windows messages in the log.
• Disassemblers will allow the programmer to view the assembly code of the executable file
• Sniffers will help the programmer to monitor the network traffic generated by the program
• Hardware interface sniffers will allow you to see the data exchanged between the system and the device.
• System logs.

Tools that reduce the need for debugging

Another direction is to make sure that debugging is needed as little as possible. For this purpose:

• Contract programming - so that the programmer confirms in another way that he needs exactly this program behavior at the output. In languages ​​that do not have contract programming, the program uses self-checking at key points.
• Unit testing is testing the behavior of a program in parts.
• Static code analysis - checking code for standard errors “due to oversight”.
• High programming culture, in particular, design patterns, naming conventions and transparent behavior of individual blocks of code - to declare to yourself and others how this or that function should behave.
• Extensive use of proven external libraries.

Code Security and Debugging

The program code may contain so-called undocumented behavior - serious errors that do not appear during normal program execution, but are very dangerous for the security of the entire system in the event of a targeted attack. Most often this is the result of programmer errors. The best known examples are SQL injection and buffer overflow. In this case, the debugging task is:

• Identifying undocumented system behavior
• Eliminating unsafe code

The following methods are distinguished:

• static code analysis. At this phase, the scanner program searches for sequences in the source text that correspond to unsafe function calls, etc. In fact, the source text of the program is scanned based on a special rule base that contains a description of unsafe code samples.
• fuzzing. This is the process of feeding random or incorrect data to the program input and analyzing the program's response.
• Reverse engineering. This case occurs when independent researchers look for vulnerabilities and undocumented features of a program.

Literature

• Steve Maguire, "Creating Reliable Code" (Steve Maguire. Writing Solid Code. Microsoft Press, 1993)
• Steve McConnell, "Perfect Code" (Steve McConnel. Code Complete. Microsoft Press, 1993)

See also

Links

• Debugging AMD64 at the machine code level using the dbx debugger (Russian)

Wikimedia Foundation. 2010.

See what “Program Debugging” is in other dictionaries:

The stage of computer program development, during which obvious errors in the program are detected, localized, and eliminated. Typically, debugging is performed on test cases with known results. In English: Program debugging… … Financial Dictionary

program debugging- program control program control program control - [L.G. Sumenko. English-Russian dictionary on information technology. M.: State Enterprise TsNIIS, 2003.] Topics information technology in general Synonyms program controlcheck... ...

program debugging- program debugging; debugging The process of detecting and correcting errors in a program, as well as establishing the fact of its correct functioning in the machine...

debugging (programs)- - [A.S. Goldberg. English-Russian energy dictionary. 2006] Topics of energy in general EN debugging ... Technical Translator's Guide

On a computer, detecting and correcting errors in a computer program using the machine itself. It is carried out in 2 stages: 1) autonomous check of the department. program areas; 2) comprehensive verification by solving several programs being debugged. examples, results... Big Encyclopedic Polytechnic Dictionary

debugging- programs; debugging The process of detecting and correcting errors in a program, as well as establishing the fact of its correct functioning in the machine... Polytechnic terminological explanatory dictionary

debugging a machine program- - [A.S. Goldberg. English-Russian energy dictionary. 2006] Energy topics in general EN code refinement ... Technical Translator's Guide

DEBUGGING- (1) a program, a systematic process of searching and eliminating (see) in (see), carried out based on the results of its run on a computer or using a special auxiliary program (debugger), providing a given mode for its debugging and issuing ... ... Big Polytechnic Encyclopedia

Debugging is the stage of computer program development in which errors are detected, localized, and eliminated. To understand where the error occurred, you have to: find out the current values ​​of the variables; and find out which path the program was executed in.... ... Wikipedia

Books

• Android for the user. Useful programs and tips, Denis Nikolaevich Kolisnichenko, The most interesting, useful and most popular Android applications among domestic users are considered. Tips are given on how to turn your Android device into a remote control... Category: Operating systems Publisher:

Definition and testing principles

Testing a software tool (TS) is the process of executing programs on a certain set of data for which the result of application is known in advance or the rules of behavior of these programs are known. The specified data set is called test or just test. Program testing is one of the components of a more general concept - “program debugging”. Debugging is understood as a process that makes it possible to obtain a program that functions with the required characteristics in a given area of ​​change in input data.

The debugging process includes:

actions aimed at identifying errors (testing);

diagnostics and localization of errors (determining the nature of errors and location);

 making corrections to the program to eliminate errors.

Of the three listed types of work, the most labor-intensive and expensive is testing, the costs of which are close to 45% of the total costs of software development.

It is not possible to guarantee that the program will be free of errors. At best, you can try to show the presence of errors. If a program behaves correctly for a large set of tests, there is no reason to say that it is bug-free. If we assume that a set of tests is capable of detecting possible errors with a high probability, then we can speak of a certain level of confidence (reliability) in the correct operation of the program established by these tests. Let us formulate the following statement: if your goal- show the absence of errors, you will not find too many of them. If your goal- show the presence of errors, you will find a significant part of them.

Reliability cannot be added to a program as a result of testing; it is determined by the correctness of the design stages. The best solution to the reliability problem is to prevent errors in the program from the very beginning. However, the likelihood of being able to design a large program flawlessly is low. The role of testing is to locate the few errors remaining in a well-designed program.. Trying to test the reliability of a poorly designed program is hopeless.

Testing turns out to be a rather unusual process (hence considered difficult) since the process is destructive. After all, the goal of the reviewer (tester) is to make the program go astray.

Programs, as objects of testing, have a number of features that distinguish their testing process from the generally accepted one used in the development of equipment and other technical products. Features of PS testing are:

lack of a standard (program) to which the program under test must correspond;

high complexity of programs and fundamental impossibility of exhaustive testing;

 the practical impossibility of creating a unified testing methodology (formalization of the testing process) due to the wide variety of software products (SP) in terms of their complexity, functional purpose, area of ​​use, etc.

Testing - It is the process of repeatedly executing a program to identify errors. The goal of testing is to detect as many errors as possible. Therefore, a test run that does not reveal any errors is considered unsuccessful (ineffective).

There are several empirical rules for testing programs that summarize the experience of testers.

1. The testing process is more effective if it is not carried out by the program author. At its core, testing is a destructive process. This is precisely why many people find it difficult. It is especially difficult and ineffective for the author of the program, since after completing the constructive part of designing and writing the program, it is difficult for him to switch to a destructive way of thinking and, having created the program, immediately begin to impartially identify errors in it. Therefore, special testing groups are created to conduct testing. This does not mean that a programmer cannot test his program. We are talking about increasing the efficiency of testing.

2. A necessary part of the test data set should be a description of the expected values ​​of the results of the test runs. Testing as a process of repeated execution of a program is carried out on numerous input data sets. To determine the correctness of the data obtained as a result of the next test run, you need to know the expected result. Thus, the test data set must include two components: a description of the input data, a description of the exact and correct result corresponding to the input data set. This principle is difficult, and in some cases impossible, to implement in practice. Its difficulty lies in the fact that when testing a program (module), it is necessary to manually calculate the expected result for each input data set or find an acceptable interval for changing the output data. This process is labor-intensive even for small programs, since it requires manual calculations, following the logic of the program algorithm. From the principle considered, which is difficult to implement, but which should be followed logically, the following follows.

3. It is necessary to study the results of each test. From practice it follows that a significant part of the detected errors could have been identified as a result of the first test runs, but they were missed due to an insufficiently thorough analysis of their results.

4.Tests for incorrect and unintended input data should be developed as carefully as for correct and intended ones. According to this principle, when processing data outside the range of acceptable values, the program under test must provide diagnostics in the form of messages. If there is no message about the reason for the impossibility of processing according to the proposed algorithm, and the program terminates abnormally or behaves unpredictably, then such a program cannot be considered workable and requires significant modification. Test data sets from the region of invalid input values ​​have greater detection power than tests corresponding to valid input data.

5. It is necessary to check not only whether the program does what it is intended to do, but also whether it does what it should not do. This statement logically follows from the previous one. It is necessary to check any program for unwanted side effects.

6.You should more carefully check those areas of programs where more errors are found. It is stated that the probability of having undetected errors in any part of a program is proportional to the number of errors already discovered in that part. It is possible that those parts of the program where a larger number of errors were discovered during testing were either poorly developed from the point of view of system analysis, or were developed by programmers of lower qualifications.

Basic definitions

Testing is the process of executing a program or part of it in order to find errors.

Proof is an attempt to find errors in a program regardless of the environment external to the program. Most proof methods involve formulating statements about the program's behavior and proving mathematical theorems about the correctness of the program. Proofs can be considered a form of testing, although they do not involve direct execution of the program.

Verification is an attempt to find errors by executing a program in a test, or simulated, environment.

Validation is an attempt to find errors by executing a program in a given real environment.

Debugging is not a type of testing. Although the words “debugging” and “testing” are often used interchangeably, they refer to different types of activities.

Testing is an activity aimed at detecting errors.

Debugging aims to determine the exact nature of a known error and then correct that error. These two types of activities are related, because testing results are the initial data for debugging.

Module testing, or autonomous testing (module testing, unit testing) - control of a separate software module, usually in an isolated environment (isolated from all other modules).

Unit testing sometimes includes a mathematical proof.

Testing of interfaces (integration testing) - control of interfaces between parts of the system (modules, components, subsystems).

External function testing is the control of external behavior defined by external specifications.

Comprehensive testing (system testing) - control and/or testing of a system in relation to initial goals.

Comprehensive testing is a control process if performed in a simulated environment, and a testing process if performed in a real environment.

Acceptance testing - checking whether a program meets user requirements.

Installation testing - checking the compliance of each specific system installation option in order to identify any errors that arose during the system setup process.

The relationship between these types of tests and design processes is shown in Fig.  15.

Test Design Strategy

Software testing includes setting up a test problem, designing, writing tests, testing tests, executing tests, and studying test results. Test design plays an important role. The following approaches to test design strategy are possible:

1. Testing against specifications (without worrying about the program text).

2. Testing in relation to the program text (without worrying about specifications).

To help guide test design strategies, it's worth considering two extreme approaches on the ends of the spectrum. It should also be noted that many of those who work in this field often go to water or the other extreme.

Proponents of the first approach design tests by examining the external specifications or interface specifications of the program or module that they are testing. The program is treated as a black box. Their logic is as follows: “I don’t care what this program looks like or whether I followed all the commands. I am satisfied if the program behaves as specified in the specifications." That is, the video will check all possible combinations of input values.

Proponents of the second approach design their tests by studying the logic of the program. They start by trying to prepare enough tests so that each command is executed at least once. So that each conditional branch instruction is executed in each direction at least once. Their ideal is to test every path, every branch of the algorithm. At the same time, they are not interested in specifications.

Neither of these extremes is a good strategy.

This leads to the following testing principle: testing- This the problem is largely economic. Since exhaustive testing is not possible, it is necessary to limit yourself to something less. Each test must provide the maximum return relative to its cost. This return is measured by the likelihood that the test will reveal a previously undetected bug. Costs are measured by the time and cost of preparing, administering, and validating the test results. Assuming that costs are limited by budget and schedule, it can be argued that the art of testing is essentially the art of selecting tests for maximum impact. Moreover, each test must be representative of some class of input values ​​so that its correct execution creates confidence that the program will execute correctly for a certain class of input data.

Module integration

The second most important aspect of testing (after test design) is the sequence of merging all modules into a system or program. The choice of this sequence (to be decided at the project level and at an early stage) determines the form in which tests are written, the types of test tools required, the sequence in which the modules are programmed, and the thoroughness and economy of the entire testing phase.

There are several approaches that can be used to merge modules into larger units. For the most part, they can be considered variants of the six main approaches described below. Test methods

The high labor intensity of testing and limited resources lead to the need to systematize the process and testing methods. IN  In testing practice, the following sequentially applied methods are used: static, deterministic, stochastic and real-time.

Static testing is carried out without the use of a computer by viewing the program text after translation, checking the rules for the structural construction of programs and data processing. Firstly, internal specifications and, secondly, the collective experience of testing specialists are used as standards. The use of static testing is quite effective. For typical programs, according to IBM, you can find from 30% to 80% of logical design and coding errors. This method helps to significantly increase the productivity and reliability of programs, allows errors to be detected earlier, and therefore reduces the cost of correction.

Deterministic testing is the repeated execution of a program on a computer using certain, specially selected test data sets. In deterministic testing, every combination of inputs and corresponding results, as well as every statement in the specification of the program under test, are controlled. This method is the most labor-intensive, so deterministic testing is used for individual modules during the program assembly process or for small and uncomplicated software complexes.

Stochastic testing involves using a set of random variables with corresponding distributions as initial data. To compare the results obtained, distributions of random variables are also used. Stochastic testing is used mainly to detect errors, and to diagnose and localize errors it is necessary to move on to deterministic testing using specific values ​​of the initial data, from the area of ​​​​changes in previously used random variables. Stochastic testing is best automated through the use of random number sensors (random variable generators) and is used for complex testing of PPP.

Real-time testing is carried out for PPPs intended to operate in real-time systems. During such testing, the results of processing source data are checked taking into account the time of their receipt, the duration and priority of processing, the dynamics of memory use and interaction with other programs. If a deviation of program execution results from those expected for error localization is detected, it is necessary to record the time and move on to deterministic testing.

Each of the considered testing methods does not exclude the use of another method; rather, on the contrary, the requirement to improve the quality of the PPP presupposes the need to subject them to various testing methods and their combinations, depending on the complexity of the PPP and the area of ​​its application.

Bottom-up testing

In a bottom-up approach, the program is assembled and tested from the bottom up. Only the lowest level modules (“terminal” modules; modules that do not call other modules) are tested in isolation, autonomously. Once testing of these modules is complete, calling them should be as reliable as calling a built-in language function or assignment operator. Then the modules that directly call the already tested ones are tested. These higher-level modules are not tested independently, but together with tested lower-level modules. The process is repeated until the top is reached. Here the testing of modules and testing of program interfaces are completed.

For bottom-up testing, each module requires driver: you need to submit tests in accordance with the interface of the module under test. One possible solution is to write a small host program for each module. Test data is represented as “built-in” variables and data structures in the program, and it repeatedly calls the module under test, passing it new test data with each call. There is a better solution: use a unit testing program - this is a testing tool that allows you to describe tests in a special language and eliminates the need to write drivers.

Top-down testing

In a top-down approach, the program is assembled and tested from top to bottom. Only the head module is tested in isolation. After testing of this module is completed, the modules directly called by it are connected one by one (for example, using the link editor), and the resulting combination is tested. The process is repeated until all modules have been assembled and tested.

This approach raises two questions: what to do when the module under test calls a lower-level module (which does not yet exist), and how the test data is supplied. The answer to the first question is that to simulate the functions of the missing modules, they are programmed stub modules, which model the functions of the missing modules. The phrase "write a stub" is often used when describing this approach, but it can be misleading because the task of writing a stub can be difficult. After all, a stub is rarely reduced to just a RETURN operator, since the calling module usually expects output parameters from it. In such cases, the stub is embedded with a fixed output data, which it always returns. Sometimes this is not acceptable because the caller may expect the outcome of the call to depend on the input. Therefore, in some cases the stub must be quite sophisticated, approaching the complexity of the module it is trying to model.

The second question is interesting: in what form is the test data prepared and how is it transmitted to the program? If the head module contained all the necessary input and output operations, the answer would be simple: tests are written in the form of external data common to users and transmitted to the program through the input devices allocated to it. But this rarely happens. In a well-designed program, physical I/O operations are performed at lower levels of the structure, since physical I/O is a fairly low-level abstraction. Therefore, in order to solve the problem cost-effectively, modules are added not in a strictly descending sequence (all modules of one horizontal level, then modules of the next level), but in such a way as to ensure the functioning of physical I/O operations as quickly as possible . Once this goal is achieved, top-down testing gains a significant advantage: all further tests are prepared in the same form, which is designed for the user.

The top-down method has both advantages and disadvantages compared to the top-down method. Its advantage is that this method combines module testing, interface testing and partially testing external functions. This is also related to its other advantage - when the I/O modules are already connected, tests can be prepared in a convenient form.

A top-down approach is also beneficial when there are doubts about feasibility the program as a whole or if the program design may contain serious defects.

The advantage of the top-down approach is often considered to be that there is no need for drivers; Instead of drivers, you should write “stubs”. However, this advantage is debatable.

The top-down testing method has its disadvantages. The main one is that the module is rarely tested thoroughly immediately after it is connected. The fact is that thorough testing of some modules may require extremely sophisticated stubs. The programmer often decides not to spend a lot of time programming them, but instead writes simple stubs and checks only part of the conditions in the module. He, of course, is going to come back and finish testing the module in question later, when he removes the stubs. This test plan is not the best solution because deferred conditions are often forgotten.

The second disadvantage of the top-down approach is that it can create a belief that it is possible to start programming and testing the top level of a program before the entire program has been fully designed. This idea seems economical at first glance, but usually the opposite is true. Most experienced designers recognize that program design is an iterative process. Rarely is the first project perfect. The normal style of designing a program's structure is to go back after designing the lower levels and tweak the upper level, making some improvements or fixing errors, or sometimes even finishing the project and starting all over again because the designer saw a better approach. If the head part of the program has already been programmed and tested, then serious resistance arises to any improvements in its structure. In the end, these improvements can usually save more than the few days or weeks that the designer expects to gain by starting programming too early.

Modified top-down method

Using top-down testing exactly as described above, it is often impossible to test certain logical conditions, such as error conditions or security checks. The top-down method also makes it difficult or impossible to check for exceptions in a module if the program is running only in a limited context (meaning that the module will never receive a sufficiently complete set of input values). Even if testing such a situation is feasible in principle, it is often difficult to determine exactly what tests are needed if they are introduced at a point in the program that is remote from the place where the corresponding condition is tested.

An approach called the modified top-down method solves these problems by requiring each module to undergo standalone testing before being included in the program. This solves the listed problems, but it requires drivers and stubs for each module.

Big Leap Method

One approach to integrating modules is the big leap method. According to this method, each module is tested autonomously. Upon completion of testing the modules, they are integrated into the system all at once.

The Big Leap Method has many disadvantages and advantages compared to other approaches.

Stubs and drivers are required for each module. Modules are not integrated until the very last moment, meaning that serious interface errors may remain undetected for a long time.

The Big Leap method makes debugging much more difficult.

If the program is small and well designed, the big leap method may be appropriate. However, for large programs it is usually unacceptable.

Sandwich method

Sandwich testing is a compromise between bottom-up and top-down approaches; an attempt to take advantage of the advantages of both methods while avoiding their disadvantages.

When using this method, upward and downward testing begins simultaneously, assembling the program both from below and from above and meeting somewhere in the middle. The meeting point depends on the specific program being tested and must be determined in advance by studying its structure. For example, if a developer can think of his system as an application module level, then a query processing module level, then a primitive function level, then he may decide to apply a top-down approach at the application module level (programming stubs instead of query processing modules) and apply the rest of the layers bottom-up method.

The sandwich method is a smart approach to integrating large programs, such as an operating system or application package.

The sandwich method retains the advantage of top-down and bottom-up approaches in starting system integration at a very early stage. Since the top of the program is built in early, like the top-down method, a working framework of the program is obtained at an early stage. Since the lower levels of the program are created using a bottom-up method, those problems of the top-down method that were associated with the inability to test certain conditions deep in the program are eliminated.

Modified sandwich method

Sandwich testing suffers from the same problem as the top-down approach. The problem is that it is impossible to thoroughly test individual modules. Bottom-up sandwich testing solves this problem for lower-level modules, but it may still be open to the bottom half of the top half of the program. In the modified sandwich method, the lower levels are also tested strictly from the bottom up. Upper-level modules are first tested in isolation and then assembled using a top-down method.

Thus, the modified sandwich method also represents a compromise between bottom-up and top-down approaches.

Comparison of testing methods

From the point of view of software reliability, testing strategies can be assessed according to seven criteria (Table 9).

The first criterion is the time until the modules are assembled, since this is important for detecting errors in the connections and assumptions of the modules about each other’s properties.

The second criterion is the time until the first working “skeleton” versions of the program are created, since this is where the main design defects may appear.

The third and fourth criteria concern whether stubs, drivers, and other testing tools are necessary.

The fifth criterion is a measure of parallelism, which is possible at the beginning or early stages of testing (but not at the end of the testing cycle).

The sixth criterion is related to the answer to the question: is it possible to check any specific path or any condition in the program?

The seventh criterion characterizes the complexity of planning, supervision and management during the testing process.

Let's evaluate six testing approaches using the listed criteria. A version of a rough estimate is given as an initial approximation for performing estimates. The first step is to weigh the relative impact of each criterion on software reliability.

Early builds, getting a working skeleton of the program, and being able to test any specific conditions, seem to be the most important and are given weight3.

The complexity of stub preparation, scheduling, and test sequence management are also important and receive a weight of 2.

The third criterion, the need for drivers, is weight1 due to the availability of common testing tools.

The criterion associated with parallelism of work has a weight of 1 (it may be important for other reasons, but does not greatly affect reliability).

The sixth criterion is weight3.

The seventh criterion receives weight2.

In table  10 shows the results of this assessment. In each column of the table, the weight is taken with a plus or minus sign or is not taken into account, depending on whether the corresponding factor manifests itself favorably, unfavorably or indifferently in the approach under consideration. The modified sandwich method and the bottom-up method appear to be the best approaches, while the big leap method appears to be the worst. If the evaluation method is close to the specific situation, the modified sandwich method should be recommended for testing large systems or programs and the bottom-up approach for testing small and medium-sized programs.

Testing stages

The process of testing the PPP begins with checking the correct operation of individual modules and ends with acceptance after testing the PPP upon delivery to the customer or the beginning of commercial sales of the PPP. Let's look at the typical stages of testers' work.

Testing of software modules is the most formalized and automated testing process.

The main task of testing is to check the processing of incoming information by software modules and the correctness of the output data in accordance with the functions reflected in the specifications.

The correctness of the structure of modules and their structural main components is checked: procedures, cycles, blocks, conditions, etc.

Testing is planned taking into account the structure of modules and information processing features and is carried out primarily deterministically.

Testing of functional groups of modules is intended to verify the correctness of solving large autonomous Functional problems of the software. The correctness of control and information connections between modules is checked, as well as the correctness of calculations during information processing. The complexity of the objects being tested and, accordingly, the volume of tests increase significantly. As a result, the requirements for automation of testing and the costs of its implementation are increasing.

Deterministic testing checks the structure of program groups and the main information processing routes. In some cases, the results are obtained using stochastic testing methods. These methods are still poorly formalized, and their application largely depends on the specific functions of the group of programs being tested.

Comprehensive testing is a complex process in which the verification of the correct functioning of programs with correct input data is completed, and basic checks are carried out in case of distortions at the input.

The reliability of the functioning of the entire PPP in real conditions and the effectiveness of the means by the protection and restoration program are checked. The correct use of computer resources by programs and the functioning of programs under critical conditions are determined. Formalization of the testing process at this stage is the most difficult, and the assessment of the completeness of testing is carried out mainly by the degree of performance of functions and by the reliability characteristics of the operation of the control panel. For this purpose, predominantly stochastic testing and real-time testing are used.

Before you begin testing the software complex as a whole, it is necessary that its constituent parts (individual modules or functional groups of modules) be thoroughly tested.

Assembly of modules into a software package can be carried out using two methods: monolithic and step-by-step.

Step-by-step assembly can, in turn, be ascending (bottom-up) and descending (top-down).

As an example, consider a software package consisting of nine modules (Fig. 16).

The monolithic assembly method involves testing each of the nine modules separately, and then simultaneous assembly and testing of them as a whole.

For autonomous testing of each module, a driver module is needed to call and transfer to the module under test the necessary input data and process the results, and one or more stub modules that imitate the functions of the modules called by the module under test.

Rice. 16. Structure of a package of 9 modules

For the example under consideration, driver modules are needed for all modules except the M1 module, and stub modules are needed for all modules except M5, M6, M7, M8, M9 (i.e., the lowest level modules).

Thus, with a monolithic assembly, it is necessary to develop eight driver modules and at least nine stub modules.

The step-by-step method assumes that modules are not tested autonomously, but are sequentially connected to a set of previously tested modules.

Let's assume that we are testing from top to bottom. Then for module M1 you need to develop three stubs. Next, module M2 is connected, for which two plugs must first be developed, modules M1-M2 are tested. Then the M5 plug is replaced by the M5 module and the M1-M2-M5 chain is tested.

The process continues until the entire complex is assembled. There is the possibility of some parallelization of work and autonomous testing of chains M1-M2-M5 (M6), M1-M3-M7, M1-M4-M8 (M9).

It turns out that with step-by-step top-down assembly, you need to develop nine stubs, but no drivers are needed.

When testing bottom-up, the process is organized as follows: low-level modules are tested - M5, M6, M7, M8, M9. Each of them requires a driver.

Thus, for bottom-up testing, a maximum of eight drivers will need to be developed, but no stubs will be required.

Comparing monolithic and step-by-step assemblies of programs, one can note a number of advantages and disadvantages of each of them.

Monolithic assembly is costly, since it involves additional development of drivers and stubs, while with step-by-step assembly, either only stubs or only drivers are developed.

During step-by-step testing, errors in interfaces between modules are detected earlier, since program assembly begins earlier. With the monolithic method, modules “do not see each other” until the last phase. But its undoubted advantage is the greater possibility of parallelizing work.

System testing (or testing of a software package) is intended mainly to verify the compliance of the application software package with the technical specifications and to assess its suitability for regular use and maintenance.

To do this, the completeness and accuracy of technical documentation and the quality of functioning of the application software package are checked in accordance with all the requirements of the technical specifications.

Checking suitability for maintenance includes testing the settings of versions for the conditions of a specific application and analyzing the ease of modifying versions of an application software package.

If we are talking about a custom package of application programs, then if the test results are positive, an acceptance certificate for the package of application programs for trial or industrial operation is issued, and the work is considered completed.

Testing of commercial application software packages created on the initiative of developers for a wide range of users in the absence of a specific customer is organized somewhat differently.

For such commercial applications, it is common to conduct testing in two successive stages, called alpha and beta testing.

These tests consist of normal and stress testing by end users of the software product, in accordance with the accompanying documentation, and vary in the number of users involved.

During alpha testing, end users are involved who work for the same company, but were not directly involved in the development of the software package.

For beta testing, voluntary users (potential buyers) are recruited and given a free version of the application software package for trial use. In this case, it is of particular importance to identify competent and friendly users who are able to improve the quality of the tested (for example, five) programs with their recommendations. Their activities are stimulated by the free early receipt and development of a new software product and their own assessment of its quality. These users undertake to inform the developers of information about all identified defects and errors, as well as make changes to the programs and data or replace versions according to the instructions of the developers.

Only after successful use of beta testing by a limited number of users, the project manager or the development company, a decision is made to sell the package of application programs for sale to a wide range of users.

Recently, development companies have begun to post beta versions of programs on their Web sites, allowing everyone to download them, thereby increasing the number of volunteer testers. Feedback is organized either through the conference system on the developer’s website or via email. Debugging the software. Basic Concepts

Only the error that cannot be corrected.

Confucius

Debugging a software tool (SO) is an activity aimed at detecting and correcting errors in a software using the processes of executing its programs. Debugging can be represented as the repeated repetition of three processes: testing, as a result of which the presence of an error in the PS can be determined, searching for the location of the error in the PS documentation programs and editing the documentation programs in order to eliminate the detected error. In other words: Debugging=Testing+Finding errors+Editing.

Sometimes testing and debugging are considered synonymous.

Principles and types of software debugging

The success of software debugging is largely determined by the rational organization of testing. When debugging the PS, errors are found and eliminated, mainly, the presence of which is detected during testing. Testing cannot prove the correctness of the software; at best, it can demonstrate the presence of an error. In other words, it cannot be guaranteed that testing the software with a practically feasible set of tests can determine the presence of every existing error in the software. Therefore, two problems arise. The first task is to prepare such a set of tests and apply it to the PS in order to detect as many errors as possible. However, the longer the testing process (and debugging in general) continues, the greater the cost of the software becomes. The second task: to determine the moment of completion of debugging of the software (or its individual components). A sign of the possibility of completing debugging is the completeness of coverage, passed through the PS, tests of many different situations that arise during the execution of PS programs, and the relatively rare occurrence of errors in the PS at the last stage of the testing process. The latter is determined in accordance with the required degree of reliability of the PS specified in the specification of its quality.

To optimize the test suite, i.e. To prepare a set of tests that would allow, for a given number of them (or for a given time interval allotted for testing), to detect a larger number of errors in the PS, it is necessary, firstly, to plan this set in advance and, secondly, to use a rational strategy planning (designing) tests. Test design can begin immediately after completing the stage of external description of the software. There are different approaches to developing a test design strategy, which can be conditionally graphically placed (Fig. 17) between the following two extreme approaches.

Rice. 17. Approaches to test design

The left extreme approach is that tests are designed only on the basis of studying the software specifications (external description, description of the architecture and module specifications). The structure of the modules is not taken into account in any way, i.e. they are treated as black boxes. In fact, this approach requires a complete search of all sets of input data, since otherwise some sections of the software programs may not work if any test is skipped, which means that the errors contained in them will not appear. However, testing a software system with a full set of input data sets is practically impossible. The right extreme approach is that tests are designed based on studying program texts in order to test all execution paths of each software program. If we take into account the presence of loops with a variable number of repetitions in programs, then there may be many different ways of executing PS programs, so that testing them will be practically impossible.

The optimal test design strategy lies within the range between these extreme approaches, but closer to the cool edge. It involves designing a significant portion of tests according to specifications, but also requires designing some tests according to program texts. In this case, in the first case, the strategy is based on the principles:

for each function or feature used - at least one test;

for each area and for each boundary of change of any input value - at least one test;

for each special (exceptional) situation specified in the specifications - at least one test.

In the second case, the strategy is based on the principle: each command of each PS program must work on at least one test.

The optimal test design strategy can be specified based on the following principle: for each program document (including program texts) included in the software, its own tests should be designed in order to identify errors in it. In any case, this principle must be observed in accordance with the definition of software and the content of the concept of programming technology as a technology for developing reliable software. There are two main types of debugging (including testing): autonomous and complex debugging of the software.

Autonomous debugging of a software system means sequential separate testing of various parts of programs included in the software, searching for and correcting errors recorded during testing. It actually includes debugging each software module and debugging the interface of modules.

Comprehensive debugging means testing the entire software system by finding and correcting errors recorded during testing in all documents (including software program texts) related to the software system as a whole. These documents include the definition of the requirements of the software, the quality specification of the software, the functional specification of the software, a description of the architecture of the software and the texts of software programs.

Commandments for debugging software

General recommendations for organizing software debugging are provided. But first we should note a certain phenomenon that confirms the importance of preventing errors in the previous stages of development: As the number of detected and corrected errors in the PS increases, the relative probability of the existence of undetected errors also increases. This is explained by the fact that with an increase in the number of errors detected by the VPS, our understanding of the total number of errors made by it is clarified, and therefore, to some extent, the number of errors that have not yet been detected.

Commandment 1. Consider testing the key task of software development, entrust it to the most qualified and gifted programmers; It is not advisable to test your own program.

Commandment 2. A good test is one that has a high probability of detecting an error, not one that demonstrates the correct operation of the program.

Commandment 3. Prepare tests for both correct and incorrect data.

Commandment 4: Document missed tests via computer; study the results of each test in detail; Avoid tests that cannot be missed again.

Commandment 5. Connect each module to the program only once; never change a program to make it easier to test.

Commandment 6. Skip again all tests related to checking the operation of any software program or its interaction with other programs if changes have been made to it (for example, as a result of eliminating an error).

Offline software debugging

During offline debugging of a software system, each module is actually tested in some software environment, except for the case when the program being debugged consists of only one module. This environment consists of other modules, some of which are modules of the program being debugged, which have already been debugged, and some are modules that control debugging ( debugging modules). Thus, during offline debugging, some program is always tested ( program under test), built specifically for testing the module being debugged. This program only partially coincides with the program being debugged, except for the case when the last module of the program being debugged is being debugged. In the process of autonomous debugging of the software, the program under test is expanded with debugged modules: when the next module is installed, the last debugged module is added to its software environment. This process of expanding the software environment with debugged modules is called integration programs. The debugging modules that are included around the module being debugged depend on the order in which the program's modules are debugged, which module is being debugged, and possibly which test will be skipped.

During bottom-up testing, this environment will contain only one debugging module (except for the case when the last module of the program being debugged is being debugged), which will be the head of the program being tested. This debugging module is called leading(or driver). The leading debugging module prepares the information environment for testing the module being debugged (i.e., it forms its state required for testing this module, in particular by entering some test data), accesses the module being debugged, and after finishing its work issues the necessary messages. When debugging one module, different leading debugging modules can be compiled for different tests.

During top-down testing, the environment of the module being debugged contains, as debugging modules, debug simulators(stubs) of some not yet debugged modules. These modules include, first of all, all modules that can be accessed by the module being debugged, as well as modules that have not yet been debugged, which can be accessed by already debugged modules (included in this environment). Some of these simulators may change for different tests when debugging one module.

In practice, the module being debugged may contain both types of debug modules around it if a mixed testing strategy is used. This is due to the fact that both bottom-up and bottom-up testing have their advantages and disadvantages.

TO the benefits of bottom-up testing include:

 ease of preparation of tests;

the ability to fully implement the module test plan.

This is due to the fact that the test state of the information environment is prepared immediately before accessing the module under development (the leading debugging module).

The disadvantages of bottom-up testing are its following features:

test data is prepared, as a rule, not in the form intended for the user (except for the case when the last, main, module of the program being debugged is being debugged);

a large amount of debugging programming (when debugging one module, you have to create many leading debug modules that form a suitable state of the information environment for different tests);

need for special testing of module pairing.

TO the benefits of top-down testing its features include the following:

most tests are prepared in a form designed for the user;

in many cases, a relatively small amount of debugging programming (module simulators are, as a rule, very simple and each is suitable for a large number, often for all, tests);

 there is no need to test module pairing.

The disadvantage of top-down testing is that the test state of the information environment is prepared indirectly before contacting the module being tested; it is the result of applying already debugged modules to test data or data produced by simulators. This, firstly, makes it difficult to prepare tests and requires highly qualified testers (test developers), and secondly, it makes it difficult or even impossible to implement a complete test plan for the module being debugged. This disadvantage sometimes forces developers to use bottom-up testing even in the case of top-down development. However, it is more common to use some modification of top-down testing, or some combination of top-down and bottom-up testing. Based on the fact that top-down testing, in principle, is preferable, we will dwell on techniques that allow us to overcome these difficulties to some extent.

First of all, it is necessary to organize program debugging in such a way that the modules that enter data are debugged as early as possible, then test data can be prepared in a user-friendly form, which will significantly simplify the preparation of subsequent tests. This input is not always carried out in the head module, so you have to first debug the chains of modules leading to the modules that perform the specified input. While the modules that perform data input are not debugged, test data is supplied by some simulators: it is either included by the simulator as part of it, or is entered by the simulator.

During top-down testing, some states of the information environment in which it is necessary to test the module being debugged may not occur during execution of the program being debugged under any input data. In these cases, it would be possible not to test the debugged module at all, since the errors detected in this case will not appear during the execution of the debugged program under any input data. However, this is not recommended, since when the program being debugged changes (for example, when maintaining a software system), information environment states that were not used for testing the module being debugged may already arise, which requires additional testing of this module (and this could not be done with a rational organization of debugging , if this module itself has not changed). To test the module being debugged in the specified situations, suitable simulators are sometimes used to create the required state of the information environment. More often, they use a modified version of top-down testing, in which the debugged modules are pre-tested separately before their integration (in this case, a leading debugging module appears around the debugged module, along with module simulators that the debugged module can access). However, another modification of top-down testing seems more appropriate: after completion of top-down testing of the debugged module for achievable test states of the information environment, it should be tested separately for the remaining required states of the information environment.

A combination of bottom-up and top-down testing is also often used, which is called the sandwich. The essence of this method is to simultaneously carry out both bottom-up and bottom-up testing until these two testing processes meet on a module somewhere in the middle of the structure of the program being debugged. This method, with a reasonable testing order, allows you to take advantage of the advantages of both bottom-up and top-down testing, and also significantly neutralize their disadvantages.

Testing the module pairing is very important during offline debugging. The fact is that the specification of each program module, except the main one, is used in this program in two situations: firstly, when developing the text of this module and, secondly, when writing an appeal to this module in other program modules. In either case, as a result of an error, the required compliance with the given module specification may be violated. Such errors need to be detected and corrected. This is why module pairing testing is intended. With top-down testing, pairing testing is carried out along the way with each skipped test, which is considered an advantage of top-down testing. During bottom-up testing, the module being debugged is accessed not from the modules of the program being debugged, but from the leading debugging module. In this regard, there is a danger that the last module may adapt to some “misconceptions” of the module being debugged. Therefore, when starting (during the process of program integration) the development of a new module, you have to test each call to a previously debugged module in order to detect the inconsistency of this call with the corresponding module (and it is possible that the previously debugged module is to blame for this). Thus, it is necessary to partially repeat the testing of a previously debugged module under new conditions, and the same difficulties arise as with top-down testing.

It is advisable to carry out autonomous module testing in four sequential steps.

1. Based on the specification of the module being debugged, prepare tests for each possibility and each situation, for each boundary of the areas of acceptable values ​​​​of all input data, for each area of ​​​​data change, for each area of ​​​​invalid values ​​​​of all input data, and each invalid condition.

2.Check the module text to ensure that each direction of any branching will pass on at least one test. Add missing tests.

3.Check the module text to make sure that for each loop there are tests that ensure at least the following three situations: the body of the loop is not executed even once, the body of the loop is executed once, the body of the loop is executed the maximum number of times. Add missing tests.

4.Check the module text to ensure that there are tests that test the sensitivity to particular specific input values. Add missing tests.

Comprehensive debugging of software

During complex debugging, the software as a whole is tested, and tests are prepared for each of the software documents. Testing of these documents is carried out, as a rule, in the reverse order of their development. The only exception is testing of application documentation, which is developed according to the external description in parallel with the development of program texts - this testing is best done after testing the external description is completed. Testing during complex debugging is the application of software to specific data that the user may encounter (in particular, all tests are prepared in a form designed for the user), but possibly in a simulated (rather than real) environment. For example, some input and output devices that are inaccessible during complex debugging can be replaced by their software simulators.

Testing of PS architecture. The purpose of testing is to find inconsistencies between the description of the architecture and the set of software programs. By the time testing of the PS architecture begins, autonomous debugging of each subsystem must be completed. Errors in the implementation of architecture can be associated, first of all, with the interaction of these subsystems, in particular, with the implementation of architectural functions (if any). Therefore, I would like to check all the ways of interaction between the subsystems of the PS. In this case, it is advisable to at least test all subsystem execution chains without re-entering the latter. If a given architecture represents the PS as a small system of dedicated subsystems, then the number of such chains will be quite foreseeable.

Testing external functions. The purpose of testing is to find discrepancies between the functional specification and the set of software programs. Despite the fact that all these programs have already been independently debugged, these discrepancies may be, for example, due to a discrepancy between the internal specifications of the programs and their modules (on the basis of which the autonomous testing was carried out) with the functional specification of the software. As a rule, testing external functions is done in the same way as testing modules at the first step, i.e. like a black box.

PS quality testing. The purpose of testing is to search for violations of the quality requirements formulated in the software quality specification. This is the most difficult and least studied type of testing. Not every PS quality primitive can be tested. The completeness of the software is checked already when testing external functions. At this stage, testing of this quality primitive can be continued if it is necessary to obtain any probabilistic assessment of the degree of reliability of the software. However, the methodology for such testing still requires development. Quality primitives such as accuracy, stability, security, time efficiency, to some extent memory efficiency, device efficiency, extensibility and, partially, device independence can be tested. Each of these types of testing has its own specifics and deserves separate consideration. The ease of application of the software (a quality criterion that includes several quality primitives) is assessed when testing the documentation for the application of the software.

Testing of documentation on the use of software. The purpose of testing is to find inconsistencies in the documentation for the use of the entire set of software programs, as well as to identify inconveniences that arise when using the software. This stage immediately precedes the connection of the user to the completion of the development of the software (testing to determine the requirements of the software and certification of the software), so it is important for developers to first use the software themselves the way the user will do it. All tests at this stage are prepared based only on documentation on the use of the software. First of all, the capabilities of the software must be tested, as was done when testing external functions, but only on the basis of application documentation. All unclear points in the documentation should be tested, as well as all examples used in the documentation. Next, the most difficult cases of using the software are tested in order to detect a violation of the requirements for the relative ease of application of the software.

Testing the definition of KPS requirements. The purpose of testing is to find out to what extent the software does not meet the stated definition of requirements for it. The peculiarity of this type of testing is that it is carried out by the purchasing organization or the user organization of the software as one of the ways to overcome the barrier between the developer and the user. Typically, this testing is carried out using control (standard) problems for which the result of the solution is known. In those cases when the software being developed must replace another version of the software that solves at least part of the problems of the software being developed, testing is carried out by solving common problems using both the old and the new software (with subsequent comparison of the results obtained). Sometimes a form of such testing is used experienced operation of the PS - limited application of the new PS based on the analysis of the use of the results in practical activities. Essentially, this type of testing has many similarities with testing a PS during its certification, but is performed before certification, and sometimes instead of certification.

Testing of software products Under testing of software products should be understood as the experimental determination of quantitative and/or qualitative characteristics of the properties of a product when it operates in a real environment and/or when simulating the operating environment.

The purpose of the test is to experimentally determine the actual characteristics of the properties of the software product (SP) being tested. These characteristics can be both quantitative and qualitative. It is important that on their basis it can be concluded that the PI is suitable for use for its intended purpose. If the conclusion is negative, then the PI sample is returned for revision.

Testing is the final stage of development. It is preceded by the stage of static and dynamic debugging of programs. The main method of dynamic debugging is testing. In a narrow sense, the purpose of testing is to detect errors, while the purpose of debugging is not only to detect, but also to eliminate errors. However, you cannot limit yourself to just debugging the program if you are sure that all errors have been eliminated. The purposes of debugging and testing are different. A fully debugged program may not have certain consumer properties and thus be unsuitable for use. Verifying the functionality of a program using a test example cannot serve as an alternative to testing, since a program that is workable under the conditions of a test example may turn out to be inoperable under other conditions of use.

In accordance with GOST 19.004-80 under program testing understand the establishment of program compliance with specified requirements and program documents. This definition is based on the assumption that the technical specifications for the development of the program define all the requirements (characteristics), the provision of which guarantees the suitability of the program for its intended use.

In the absence of a technical specification (TOR) for the development of a software tool (SS) or a complete and justified list of requirements for the characteristics of the software being developed, the task of testing the software becomes uncertain and non-constructive.

The duration of the test depends on the type, configuration (complexity) of the software, as well as on the goals and degree of automation of the technological process in question (for example, when testing operating systems from one to six months). Complex software systems may take the longest time to test after integration.

The main types of testing of software products (SP) are:

preliminary;

acceptance;

 operational tests, including trial operation.

Depending on the location, bench and field tests are distinguished.

Under test bench understand a set of technical devices and mathematical models that provide automatic simulation of the operating environment; receipt of input data, distorting influences; registration of information on the operation of the substation, as well as management of the testing process and test object.

If bench tests are based on the principle of modeling, then the corresponding test benches are called modeling.

A testing ground is a place intended for testing under conditions close to operating conditions, and provided with the necessary testing equipment.

Systems operating in real time are subjected to field testing. In testing conditions, field tests are usually combined with the use of real objects of automated systems and modeling of some objects and processes of their functioning.

Based on the degree of dependence of testers on developers, dependent and independent tests are distinguished.

In dependent tests, the main operations of the tested software (preparation for work, preparation and input of initial data, registration and analysis of results) are performed by program developers.

The test results are assessed by a commission with the active participation of developers.

Independent tests are carried out by special units that are not responsible for program development and are not directly subordinate to development managers.

Test flow diagram

The goal of increasing the efficiency of testing, speeding it up and making it cheaper can be achieved by developing a test flow chart that provides for:

 knowledge of the purpose of the tested software, the conditions of its functioning and the requirements for it from users;

automation of the most labor-intensive processes and, above all, modeling of the operating environment, including distorting influences;

 clear presentation of the purpose and sequence of the test;

 purposefulness and non-redundancy of the test, excluding or minimizing the repetition of similar procedures under the same operating conditions of the tested PS;

 systematic monitoring, regular maintenance of test protocols and journals;

consistent definition and implementation of the test plan;

 comparison of available resources with the expected volume of testing;

the ability to provide an objective quantitative assessment of the completeness and reliability of test results at all stages.

Any type of test must be preceded by careful preparation.

The preparation of PS tests includes the following activities:

drawing up and agreeing on a test schedule;

development, acquisition, testing and certification of software and hardware used in testing;

analysis of the suitability of the testing equipment used during preliminary tests for acceptance testing;

 analysis of the suitability of accumulated data on the quality of the PS for use in the final determination of the values ​​of quality indicators of the tested PS;

checking and agreement with the customer’s representative of the design documentation for the substation, presented during testing;

development, coordination and approval of programs and test methods;

certification of specialists for admission to testing;

acceptance of the tested prototype of the PS on a data carrier and documentation;

carrying out activities aimed at ensuring the reliability of tests.

It should be emphasized the need for advance development and testing of software and hardware that will be used during testing.

It should be borne in mind that the level of accuracy and reliability of the measuring equipment must be significantly higher than the corresponding indicators of the tested object.

Based on the above, the following five stages of testing can be determined:

1. Inspection of the designed substation, analysis of design documentation.

2. Determination of the most important subsystems and functions of the designed substation to be tested.

3.Analysis of PS quality indicators and methods for determining their values. Development of programs and testing methods.

4. Development (mastery) of testing software and hardware, test libraries and databases (if they are required).

5. Direct testing, analysis of results, decision making.

Depending on the specifics, conditions of use, and quality requirements for the tested substations, tests can be carried out either by testing, or by statistical modeling of the operating environment, or on the basis of full-scale and mixed experiments. Using all of these methods is often useful. The values ​​of some quality indicators can be obtained expertly.

In Fig.  18 shows a technological diagram in the form of stages of preparation and testing and their connection with the stages of PS development.

Test planning and evaluation

The test plan should be focused on ensuring a comprehensive verification of the software and the specified reliability of the results obtained while using the limited resources allocated for testing. The following approaches are possible to solve this problem:

1) analyze the range of input data. Based on the analysis, many combinations of test data sets are prepared, covering the most characteristic subsets of the input data. The program is treated as a black box. Testing comes down to the sequential input of test data sets and analysis of the results obtained;

Rice. 18. Technological scheme for testing PS

2) analyze many situations that may arise during the operation of the PS. Choose the most characteristic ones. Each of them is expressed through a test set of input data. Next, the process of testing and analyzing the results is reduced to step 1;

3) using a graph diagram, the microstructure of the PS is analyzed. A set of paths is selected that completely covers the graph diagram, and a sequence of test sets of source data, the execution of which will take place along the selected paths. The organization of tests is similar to points 1 and 2;

4)PS are tested in a real operating environment;

5) The PS is tested in a statistically simulated operating environment that is adequate to the real environment.

None of these approaches are universal. Each has its own advantages and disadvantages, depending on the specifics of the PS being tested. For example, approach 1 may be preferable if the range of input data is observable, relatively easy to analyze and systematize, and unacceptable otherwise. The most reliable results are obtained when tested in a real operating environment. But such tests are rarely possible. Therefore, in practice, combinations of all types are used. For example, a mixed method, when the operating environment of the PS is modeled, the reliability of the results is checked by comparison with the results obtained when the PS operates in a real environment.

The method for solving the test planning problem includes the following steps:

finding all ways of implementation;

 selection of a minimum subset of paths that ensure testing of all sections of the program;

development of tests to check selected paths.

Error detection intensity criterion. If we assume that during one experiment no more than one error is detected and each error is eliminated before the start of the next experiment, then we can assume that with a favorable progress in debugging and testing, the value of the error detection intensity criterion N can be calculated using the formula

N=1 - n/TO,

With an increase in the number of experiments, the criterion for the intensity of error detection will asymptotically tend to kedinize.

Then, for example, the following condition can be accepted as a test termination criterion: N  >0.95 when no more than three insignificant errors were detected in the last two hundred experiments. The idea of ​​choosing such a criterion is based on the fact that the frequency of error detection, expressed by the ratio n/TO, as the number of experiments increases, it should decrease and, by the time the tests are completed, take a value close to zero. It should be borne in mind that the assessment of the level of test completion will be reliable only if each experiment is carried out under new conditions and the testers strive to detect errors, rather than prove their absence.

Criterion for a given value of mean time between failures (J.D. Moose criterion). Let us assume that the total number of defects detected and eliminated in the program n(a defect is understood as any reason for dissatisfaction with the properties of the program) is described by an exponential function of the operating time t

Coefficient WITH1 when the absolute reactivity of the program when running tests or statistical tests differs from the absolute reactivity when the program runs under real conditions.

If, for example, in one hour of testing a controlled process is simulated, occurring in real conditions for ten hours, then the compression ratio WITH  is assumed to be equal to 10. The speed of detection and elimination of defects, measured relative to the operating time of the program, is proportional to the failure rate. The number of registered failures m depends on the total operating time of the program as follows:

Proportionality factor IN = n/m called the defect reduction factor.

If errors detected during testing are eliminated, then the current value of the mean time between failures will increase.

Achievement of the required (specified) mean time between failures can be taken as a test completion criterion T 0 .

When planning software debugging and testing, the influence of the following factors should be taken into account:

speed of detection of defects;

 speed of defect elimination;

satisfaction with machine time.

The first factor depends on the staffing and qualifications of the testers, the second - on the staffing and qualifications of the group of debugging programmers, the third - on the technical equipment of the developing (testing) organization.

We bring to your attention magazines published by the publishing house "Academy of Natural Sciences"

The beauty of being a programmer has a lot to do with debugging. Why do programmers violate the requirements known to them? They don’t ask comments, don’t describe in detail the essence of the problem being solved, and don’t follow other useful advice. Most often, the reason is impatience; they want to quickly see how the program works, to see the results of its work. Debugging is a bit of a detective process. We suspect that the newly created program is not working correctly. The presumption of innocence does not work here. If we can present a test on which the program gives an incorrect result, then our suspicions are proven to be correct. We secretly always hope that the program will work correctly the first time. But the purpose of testing is different - to try to refute this assumption. And only then, having corrected all the identified errors, get a correctly working program. Unfortunately, debugging cannot guarantee that the program is correct, even if all tests pass. Debugging can prove that a program is incorrect, but it cannot prove that it is correct.

The art of the tester is to create as complete a test system as possible, checking all possible branches of calculations. Let's explain this with a very simple example. Let the program find the sum of the first N elements of an array X containing M elements. In addition to the "normal" test, which checks the situation in which 1 M. But this is a simple case, and cycles are usually nested, and within them cases are analyzed, within which there are their own cycles.

Earlier we mentioned the “chechako” law - a beginner can hang any system. There is an explanation for this; out of ignorance, he will specify one of the unlikely combinations of input data (working in a visual environment, he will press the most inappropriate button for a given situation). Therefore, the tester conducting debugging must be able to take the position of a beginner; the test system must ensure that the program works correctly not only in “normal situations”, but also has “fool protection” and will not lead to looping or stopping in extreme situations. probable situations.

The difficulty of debugging lies in the fact that, having discovered and corrected an error, you get a new program for which the debugging process must be started again, skipping all the tests again. It is known that in programs there are enchanted places - the correction of one error leads to the appearance of a new one. In such cases, the best way out is to look for another, fundamentally different solution to the problem.

Debugging Tools

Some program errors are caught automatically at the compilation stage. This includes all syntax errors, type mismatch errors, and a few others. However syntactically correct the program needs to be debugged because, although the calculation results have been obtained, they do not meet the required specifications. More often than not, a program that has not yet been debugged works correctly on some input data, but gives an erroneous result on others. The art of debugging is to discover all the situations in which the program's operation leads to erroneous calculations. VBA has very sophisticated tools designed for debugging programs, i.e. to detect errors in programs (testing) and correct them. There are two groups of VBA tools that help the programmer identify and correct errors:

1. The first group allows you to control the progress of the computational process, i.e. the order of operators in procedures, the order in which the procedures themselves are called. If necessary, during the debugging process, you can change this order; you can, for example, skip the execution of some operators, or return to their execution again
2. The second group of tools allows you to control changes in the state of the computational process (values ​​of variables and properties of objects) during execution. And here you can intervene and change the state, setting new values ​​for certain variables along the way.

Before we begin a detailed examination of these tools, recall that during debugging a program can be in one of three states: design, calculation, and interrupt. Having completed the design, you can run the program for execution. By interrupting program execution at a given point and going into the interrupt state, you can monitor the values ​​of variables and properties of objects at a given point and, if necessary, change these values ​​“manually”. In this case, you can change the order of executed statements, specify the next executable statement, and you can edit the program text before continuing the calculation. The transition from the evaluation state to the interrupt state can occur for a variety of reasons, for example, upon reaching a breakpoint, when one of the many interrupt conditions is met, due to step-by-step execution of the program. We will discuss all these possibilities later, but now we will consider one special case. Sometimes a program gets stuck in a loop and needs to be forced into an interrupt state. How to stop a running program? Just press the Ctrl+Break key pair, familiar from working in DOS. The following dialog box will appear on the screen with a stop message.

Debugging(debug, debugging) - a stage of computer program development at which errors are detected, localized and eliminated. To understand where the error occurred, you have to: find out the current values ​​of the variables; find out which way the program was executed.

Debugging process starts with trying to reproduce the problem, which can be difficult when programming parallel processes or with some unusual errors known as Heisenbugs.

Debugging technologies.

1) Use debuggers- programs that include a user interface for step-by-step execution of the program: statement by statement, function by function, stopping at certain lines of source code or when a certain condition is reached.

2) Output the current state of the program using located in critical points programs output statements- to the screen, printer, speaker or file. Outputting debugging information to a file is called logging.

Debugging tools.

1. Debugger- a software tool that allows the programmer to observe the execution of the program under study, stop and restart it, run it in slow motion, change values ​​​​in memory and even, in some cases, go back in time.

2. Profilers– allow you to determine how long a particular section of code takes to execute, and coverage analysis will allow you to identify non-executable sections of code.

3. API loggers– allow the programmer to track the interaction of the program and the Windows API by recording Windows messages in the log.

4. Disassemblers will allow the programmer to view the assembly code of the executable file

5. Sniphers will help the programmer to trace the network traffic generated by the program

6. Hardware interface sniffers will allow you to see the data exchanged between the system and the device.

7. System logs.

Usage high level programming languages, such as Java, usually makes debugging easier because they contain tools such as exception handling, making it much easier to find the source of the problem. In some low level languages ​​such as Assembler, errors can lead to subtle problems - such as memory corruption or memory leaks - and it can be quite difficult to determine what caused the error in the first place. In these cases, sophisticated debugging techniques and tools may be required.

Debugging = Testing + Finding errors + Editing

Types of Debugging Software, including testing (in our country).

1.1. Autonomous debugging Sequential separate testing of various parts of programs included in the software, with search and correction of errors recorded in them during testing. It actually includes debugging each software module and debugging the interface of modules.

1.2. Comprehensive debugging . Testing the software as a whole with the search and correction of errors recorded during testing in all documents (including software program texts) related to the software as a whole. Such documents include definition of software requirements, software quality specification, functional software specification, software architecture description. and software program texts.

2.1. Syntactic debugging Syntax errors are detected by the compiler, so correcting them is quite easy.

2.2. Semantic (semantic) debugging. Its time comes when there are no syntax errors left, but the program produces incorrect results. Here the compiler itself will not be able to detect anything, although in the programming environment there are usually debugging aids, which we will talk about later.

Relationship between testing and debugging processes through the debugging algorithm.

After the working code is written, test runs of the program are carried out on various sets of test data.

In this case, the tester or programmer must receive a control result in advance with which the work of the code being tested will be verified.

If discrepancies are detected between the control and actual results, the search begins for the problematic section of the code and identifies errors using the above methods.

Tools for automatic testing of program source code.

The main technique here is to create source text tests that will be applied to the code section under test, and the testing system will report their results.

Examples of such systems include: the built-in doctest module in Python and the xUnit multilingual testing library, distributed under the terms of GNU/GPL and LGPL. The basis for using all these tools and techniques is to break down one large task into a number of clear and smaller tasks.

23. Basic principles of object-oriented programming: encapsulation, inheritance, polymorphism. The difference between an object-oriented approach and a modular approach when developing programs

Object-oriented programming(OOP) - a programming paradigm in which the main concepts are the concepts objects And classes(or, in a less well-known version of languages ​​with prototyping, prototypes).

Prototype- this is a sample object, in the image and likeness of which other objects are created.

Class

Fields, described in the class, are used to store the components of the object’s state, i.e. fields define the state of objects.

Methods, described in a class, define the behavior of objects. Each method determines the reaction of an object to some external or internal message.

Object- type variable Class

Principles of object-oriented programming.

1. Abstraction. Abstraction is a way to highlight a set of significant characteristics of an object, excluding insignificant ones from consideration. Accordingly, abstraction is a set of all such characteristics.

2. Encapsulation.

Encapsulation is an OOP principle according to which fields and methods are combined in a class.

Encapsulation allows you to differentiate developer access to various fields and properties of a class (approximately the same way as is done in modules Delphi, when only the interface part is visible from other modules). Similarly, within classes, some fields and methods can be made freely available for use (visible) anywhere in the program, while other fields and methods can be made available only within the current module and the class’s own methods. This allows the various characteristics and capabilities of a class to be hidden within the description, so that developers reusing the class can focus on its most important properties.

3. Inheritance. Inheritance is a system property that allows you to describe a new class based on an existing one with partially or completely borrowed functionality.

Inheritance is the ability to construct new, more complex classes from existing ones by adding fields and defining new methods ( principle of hierarchy).

The class from which inheritance is made is called basic, parent or superclass. New class – descendant, inheritor or derived class. The inheritance mechanism provides a descendant class with the ability to use the fields and methods of parent classes. The chains of inheritance can be unlimited length. In this case, different methods for each of the heirs are allowed redefine.

4. Polymorphism. Polymorphism is the property of a system to use objects with the same interface without information about the type and internal structure of the object.

Polymorphism (“variety”) in programming is the ability to change the program code in accordance with the value of some parameters.

4.1. Pure polymorphism- the possibility of different interpretation of the function code depending on the type of arguments.

4.2. Overloading (polymorphic names) of functions- the ability to define several functions with one name; the choice of the desired function can be determined by the types of arguments, scope (within a module, file, class, etc.); if the choice is determined by the type of the arguments, then the overload is called parametric.

4.3. Method Overriding- in OOP - the possibility of different definitions of methods in the descendant class and the parent class; a particular method is determined by the class of the object on which it is called. When overriding methods, a distinction is made between simple and complex polymorphism.

4.3.1. Simple polymorphism are used if, when calling an overridden method, the type of object for which this method is called is precisely known, and, therefore, it is known exactly which method should be connected: the parent method or the child method. In this case, the required method is determined at the stage compilation programs.

4.3.2. Complex polymorphism are used if, when calling an overridden method, it is necessary to clarify which method should be connected: the parent method or the child method, since the object for which the overridden method is called can be either an object of the parent class or an object of the child class. In this case, the required method is determined at the stage execution programs when the type of the object is known exactly.

4.4. Generic functions(templates) - the ability to describe parameterized classes and function templates; the parameters of such descriptions are the types of arguments of methods or functions.

The difference between an object-oriented approach and a modular one.

1) Object-oriented approach to software design products is based on:

− identifying classes of objects;

− establishing the characteristic properties of objects and methods of their processing;

− creating a class hierarchy, inheriting the properties of objects and methods for processing them.

2) Modular programming- this is the organization of a program as a collection of small independent blocks, modules, the structure and behavior of which is subject to certain rules.

It should be noted that the concept of “module” does not coincide in this case with the concept of “module” (in the sense of “library”) of the language Pascal. It should be a simple, closed (independent) program unit (procedure or function), visible, implementing only one function. To write one module, minimal knowledge about the text of others, both calling and called, should be sufficient.

When modular programming of software products, the composition and subordination of functions are first determined, and then a set of software modules that implement these functions is determined.

24. Classes and objects: their definition, the relationship between them. The role of class components - fields, properties, methods. Access specifiers published, public, private, protected. Constructors and destructors, their role. Events and their use in program management

Class is a structural data type that includes a description of data fields, as well as procedures and functions that work with these data fields (methods).

Object- type variable Class- an entity in the address space of a computing system that appears when an instance of a class is created (for example, after launching the results of compilation (and linking) of the source code for execution).

Class-object relationship:

The concept of a class is more general than the concept of an object. An object is an instance of a class. A class can be thought of as a collection of objects (just as a set is a collection of elements). A class can be elementary or subdivided into subclasses (just as a set is subdivided into subsets). For example, the class PERSON contains a subclass STUDENT, which in turn contains the object John_Smith.

Classes have(for example in Delphi):

− Field class ( attribute) in OOP, a variable associated with a class or object. Fields, described in the class, are used to store the components of the object’s state, i.e. fields define the state of objects. Fields are accessed by their name.

− Methods, described in the class (subroutines that process the fields and properties of the class), determine the behavior of objects. Each method determines the reaction of an object to some external or internal message.

− Property- a way to access the internal state of an object, simulating a variable of some type. Access to an object property is implemented through a function call. When you try to set the value of a given property, one method is called, and when you try to get the value of this property, another method is called.

The fields, properties and methods of a class are called class members.

A variable declared as a class is actually pointer per class instance. In an object-oriented program using classes, each object is an "instance" of some particular class, and there are no other objects.

Access specifiers published, public, private, protected.

Encapsulation is a system property that allows you to combine data and methods that work with them in a class and hide implementation details from the user.

By default (in Delphi), the visibility of parent class members is inherited exactly, but is allowed raise visibility - making fields, properties and methods more accessible. Reduced visibility is not allowed.

Encapsulation levels(availability of any class members):

1) Public. Class members located in this section are accessible from anywhere in the program.

2) Private. Members of a class are available only in the module where the class is described. By default, all class fields are considered to be located in the section private.

3) Protected. Members of a class are available in the module where this class is described, as well as inside methods of classes that are descendants of this class and described in other modules.

4) Published. Class members located in this section are accessible from anywhere in the program. This section contains class properties: fields available for editing and changing values ​​during design and from Object inspectors.

5) Automated. Class members located in this section are accessible from anywhere in the program. Descriptions are allowed to be placed in this section only if the class is a descendant of the standard class TAutoObject(in Delphi), designed to create so-called automation servers using technology COM(Competent Object Model).

Constructors and destructors.

When object is created, a special method called designer. It performs various actions to initially initialize the fields of an object.

When the object is destroyed(for example, it was declared inside the procedure as a local variable and is removed from memory when it terminates), another method is called - destructor, which performs various additional actions to free up memory if necessary.

Events and their use in program management.

An event in OOP is a message that occurs at various points in the executable code when certain conditions are met.

Events are designed to be able to predict how software will react.

To solve the problem, we create event handlers: Once the program reaches a given state, an event occurs, a message is sent, and a handler intercepts the message.

25. The main differences between the Object Pascal (Delphi) language and Turbo Pascal. Dynamic arrays in Delphi: description, features, application

Turbo Pascal- an integrated development environment for the Pascal programming language and a programming language in this environment, a dialect of the Pascal language from Borland.

Delphi- a programming environment that uses the Object Pascal programming language.

Object Pascal is the result of language development Turbo Pascal, which in turn developed from the language Pascal. Pascal was a completely procedural language.

Turbo Pascal, starting from version 5.5, added to Pascal object oriented properties, and in Object Pascal - dynamic identification of the data type with the ability to access class metadata (that is, the description of classes and their members) in compiled code, also called introspection - this technology is designated RTTI.

Since all classes inherit the functions of the base class TObject, any pointer to an object can be converted to it, and then use the ClassType method and the TypeInfo function, which will provide introspection.

Object Pascal ( Delphi) is the result of a functional extension of Turbo Pascal.

The Delphi Pascal object model, compared to the model used by Borland Pascal 7.0, is more complete:

− restricting access to fields and methods by defining its own interface for each class field (five types of sections when declaring a class, using properties);

− more developed mechanisms for implementing polymorphic methods (abstract, dynamic methods)",

− tools for working with metaclasses (metaclass variables, class methods, RTTI mechanism).

Dynamic arrays in Delphi.

Array is an ordered set of data. Dynamic is an array whose size can change during program execution.

Announcement array: var My_Array: array of BaseType;

With such a declaration, no memory will be allocated, and the initial size of the array is null.

Setting the size array: SetLength(My_Array,100);

Getting the number of elements array: n:=Length(My_Array);

Accessing the first element array: My_Array:=10; x:=My_Array;

Declaration of two-dimensional array: var A: array of array of BaseType;

If you assign a dynamic array to a variable, its contents are not copied; only a pointer to the array is assigned. But if you apply it to a new array SetLength, then copying will occur.

26. Structure of modules in Delphi. Interface, executable parts, initiating and finishing parts. Procedures and functions: features in Delphi

A project in Delphi is a set of software units - modules.

Delphi allows you to put your functions and procedures in a separate module (Unit), and then use the procedures and functions of the module in your programs, indicating the name of the module in the list of modules required by the program (instruction uses). A module is a file with the *.pas extension.

The module begins with a header - an instruction unit, which specifies the name of the module.

A module consists of a sequence of sections. Each section begins with a keyword and continues until the beginning of the next section. To section implementation(implementation) you need to place the procedures and functions declared in the section interface.

Module structure in Delphi.

Unit<имя>;

Interface <интерфейсная часть>

Implementation <исполняемая часть>

initialization <инициирующая часть>

finalization <завершающая часть>

The initiating and ending parts are most often absent. IN initiating parts contain statements that are executed before control is transferred to the main program and are usually used to prepare its work. IN final parts indicate statements that are executed after the main program has completed its work (they release resources allocated to the program, close files, etc.).

unit ModuleName;

interface// interface section

(Descriptions of module procedures and functions that can be used by other modules.)

const// constant declaration section

(Declarations of module global constants that can be used by module procedures and functions.)

type// distributed type declarations

(Declarations of global module types that can be used by module procedures and functions)

var// variable declaration section

(Declarations of module global variables that can be used by module procedures and functions)

implementation// implementation section

(Descriptions (text) of module procedures and functions!}

Using the module. In order for a program to use the functions and procedures of a module, it is necessary to add this module to the project and specify the module name in the list of modules used. After adding the module name to the list of modules used by the application, the module itself must be added to the project.

Procedures and functions in Delphi.

Procedure– a subroutine that performs some action and which can be called from another place in the program. After the procedure is executed, program execution continues from where it was called.

A procedure is a type of subroutine. Typically, a subroutine is implemented as a procedure in two cases:

− when the subroutine does not return any data to the main program;

− when a subroutine returns more than one value to the program that called it.

Options is the input data.

Procedure declaration

procedure ProcedureName (var parameter1: type1; ... var parameter K: type K);

A typical development cycle, repeated many times over the life of a program, looks something like this:

1Programming - introducing new functionality into the program, correcting errors in existing ones.

2Testing (manual or automated; by a programmer, tester or user; “smoke”, in black box mode or modular...) - detection of an error.

3Reproducing the error - finding out the conditions under which the error occurs. This can be challenging when programming parallel processes and with some unusual errors known as Heisenbugs.

4Debugging - finding the cause of the error.

Debugging the program. Errors may occur when building complex programs. They are usually divided into 3 groups:

syntactic;

runtime errors;

algorithmic.

The simplest of them syntactic– typing errors – they are corrected during program compilation. Runtime errors– errors that occur when the types of entered parameters do not correspond to the types of variables used, or calculation errors (for example, division by 0) – are also easily tracked even at the first launch of the program.

After such an error occurs, you must click “Ok” in the error message window, and then exit the program - menu item « Run/ Programreset» or press the key combination<Ctrl><F2 >. When such an error occurs, the cursor in the program will point to the line that caused the error. Most difficult to detect algorithmic errors. The program compiles without errors, does not give errors during trial runs, but when analyzing the results it turns out that the result is incorrect. It is necessary to manually “scroll” the algorithm - required program debugging.

For traces program (checking the “logic of the algorithm”), i.e., executing the program step by step, you must perform the following steps. In the menu item select " Run/ Stepover" or " Run/ Traceinto"(keys<F8 > and<F7 > respectively), the command " Run/ Traceinto» differ in more detailed tracing. To stop tracing and continue executing the program in automatic mode, select " Run/ Run» or press<F9 >. You can stop the program using the command " Run/ Programreset» or press the key combination<Ctrl><F2 >. Sometimes you need to trace from a specific line of a program. To do this, move the cursor to the line of interest and execute the command “ Run/ Runtocursor» or press<F4 >. Often the possible location of an error in the algorithm is known - then they use breakpoint. The programmer places the cursor on the desired line and places a breakpoint there using the command " Run/ AddBreakpoint» or by clicking<F5 >. The selected line will be marked. To remove a breakpoint, click on this line again.<F5 >. When executed, the program will hit a breakpoint, then the programmer can trace the program using<F7 > and<F8 >. If necessary, you can specify condition stopping at a breakpoint (this setting is made in the " Breakpoints", menu item " Run/ Addbreakpoints»).

When executing a program step by step, it is often necessary not only to check the correctness of the “logic of the algorithm”, but Andknow the values ​​of some variables. To do this, run the command “ View/ Watch/ Addwatch» and enter the name of the variable of interest or move the cursor to the variable whose value you want to view and press<Ctrl><F5 >. When tracing the program in this case in the window " Watchlist» you can observe changes in the values ​​of the variables of interest.

10. Creation and description of new data types.

New data types. When you need to declare your data type, you should place its description in the type description section. In general, the type description looks like this:

Name = Type Description;

Name - new type name;

Type description – description of possible values ​​of variables of the created type.

Comment. When declaring a new type, an equal sign is placed after the type name, followed by a description of the type.

Examples

DayOfWeek= (Monday, Wednesday, Friday);

Day=1..31;

A type of this type is called enumerated; variables of this type can only take enumerated values. In the example, this is one of the names of the day of the week (type DayOfWeek) or one of the numbers from 1 to 31 (type Day). You can use functions with enumerated variables Pred(variable) and Succ(variable) returning previous ( Pred) and the following ( Succ) from the allowed values.

Examples

Let the variables W: DayOfWeek and D: Day be declared. Then:

Succ(W); (The operator will return the value 'Monday')

Pred(D); (The operator will return the value '4')

Notes:

Values ​​of an enumerated type cannot contain Russian letters.

Operator handlingSucc orPred to the last one (for the operatorSucc ) or the first (for the operatorPred ) element will result in an error.