Q: Why are there so many software bugs?
A: Generally speaking, there are bugs in software because of unclear requirements, software complexity, programming errors, changes in requirements, errors made in bug tracking, time pressure, poorly documented code and/or bugs in tools used in software development.
- There are unclear software requirements because there is miscommunication as to what the software should or shouldn’t do.
- Software complexity. All of the followings contribute to the exponential growth in software and system complexity: Windows interfaces, client-server and distributed applications, data communications, enormous relational databases and the sheer size of applications.
- Programming errors occur because programmers and software engineers, like everyone else, can make mistakes.
- As to changing requirements, in some fast-changing business environments, continuously modified requirements are a fact of life. Sometimes customers do not understand the effects of changes, or understand them but request them anyway. And the changes require redesign of the software, rescheduling of resources and some of the work already completed have to be redone or discarded and hardware requirements can be effected, too.
Q: Do automated testing tools make testing easier?
A: Yes and no.
For larger projects, or ongoing long-term projects, they can be valuable. But for small projects, the time needed to learn and implement them is usually not worthwhile.
A common type of automated tool is the record/playback type. For example, a test engineer clicks through all combinations of menu choices, dialog box choices, buttons, etc. in a GUI and has an automated testing tool record and log the results. The recording is typically in the form of text, based on a scripting language that the testing tool can interpret.
If a change is made (e.g. new buttons are added, or some underlying code in the application is changed), the application is then re-tested by just playing back the recorded actions and compared to the logged results in order to check effects of the change.
One problem with such tools is that if there are continual changes to the product being tested, the recordings have to be changed so often that it becomes a very time-consuming task to continuously update the scripts.
Another problem with such tools is the interpretation of the results (screens, data, logs, etc.) that can be a time-consuming task.
Q: What makes a good test engineer?
A: Good test engineers have a “test to break” attitude. We, good test engineers, take the point of view of the customer, have a strong desire for quality and an attention to detail. Tact and diplomacy are useful in maintaining a cooperative relationship with developers and an ability to communicate with both technical and non-technical people. Previous software development experience is also helpful as it provides a deeper understanding of the software development process, gives the test engineer an appreciation for the developers’ point of view and reduces the learning curve in automated test tool programming.
Q: What is a test plan?
A: A software project test plan is a document that describes the objectives, scope, approach and focus of a software testing effort. The process of preparing a test plan is a useful way to think through the efforts needed to validate the acceptability of a software product. The completed document will help people outside the test group understand the why and how of product validation. It should be thorough enough to be useful, but not so thorough that none outside the test group will be able to read it.
Q: What is a test case?
A: A test case is a document that describes an input, action, or event and its expected result, in order to determine if a feature of an application is working correctly. A test case should contain particulars such as a…
· Test case identifier;
· Test case name;
· Test conditions/setup;
· Input data requirements/steps, and
· Expected results.
Q: How do you create a test plan/design?
A: Test scenarios and/or cases are prepared by reviewing functional requirements of the release and preparing logical groups of functions that can be further broken into test procedures. Test procedures define test conditions, data to be used for testing and expected results, including database updates, file outputs, report results. Generally speaking…
· Test cases and scenarios are designed to represent both typical and unusual situations that may occur in the application.
· Test engineers define unit test requirements and unit test cases. Test engineers also execute unit test cases.
· It is the test team that, with assistance of developers and clients, develops test cases and scenarios for integration and system testing.
· Test scenarios are executed through the use of test procedures or scripts.
· Test procedures or scripts define a series of steps necessary to perform one or more test scenarios.
· Test procedures or scripts include the specific data that will be used for testing the process or transaction.
· Test procedures or scripts may cover multiple test scenarios.
· Test scripts are mapped back to the requirements and traceability matrices are used to ensure each test is within scope.
Q: How do you create a test plan/design? (Cont’d…)
- Test data is captured and base lined, prior to testing. This data serves as the foundation for unit and system testing and used to exercise system functionality in a controlled environment.
- Some output data is also base-lined for future comparison. Base-lined data is used to support future application maintenance via regression testing.
- A pretest meeting is held to assess the readiness of the application and the environment and data to be tested. A test readiness document is created to indicate the status of the entrance criteria of the release.
Inputs for this process:
- Approved Test Strategy Document.
- Test tools, or automated test tools, if applicable.
- Previously developed scripts, if applicable.
- Test documentation problems uncovered as a result of testing.
- A good understanding of software complexity and module path coverage, derived from general and detailed design documents, e.g. software design document, source code, and software complexity data.