Information Technology (IT) has become the in-thing in almost all sectors of our economy; today individuals and organizations are investing huge sums of money on the latest IT developments with the main aim of maximizing their work performance rates through induction precision, reliability, consistency, and efficiency attributes that comes with new IT developments. Various innovations have successfully been made in the IT industry and therefore potential users have got a wide variety to choose from depending on their specific needs. However, the choice may sometimes not be possible unless a complete appraisal is carried out to determine the effectiveness/suitability of a new innovation. The appraisal instrument (test) can take the format of a comparison whereby the results got will be compared against an established criterion or a certain performance target that an individual or an organization may have established. Further, the test may be formal, i.e. inclusive of written documents such as tests, quiz, or paper; the test still can be informal meaning that it is done in casual methods such as observations, evaluations and so on. Again, the test can be set and administered internally or externally by contracted agents. No matter what format of test is chosen, the guiding principle is that the test should be reliable and valid. [Yu, Chong Ho, (2005)]
In regards to the brief information in the introduction part, this paper will comprise of a comprehensive test plan targeting to examine the efficiency of new features/programs. The comprehensive test will measure both the qualitative and quantitative aspects of the new features/programs; this will be achieved through the subjection of the results on an established criterion that will be made up of the major attributes of an efficient IT component/program. These attributes includes; speed, precision, efficiency, cost, complexity, reliability, validity, lifespan, and vulnerability. Both formative and summative tests of all the attributes of the new features/ programs will be carried out independently in order to come up with substantive test results which can be used to make a comprehensive conclusion on the suitability of the examined features/programs.
Objectives of the Test
The key objective of the test will be to test the efficiency of new features/ programs that were introduced by eassywiters.net IT department. The test results will also be used as a source of reference especially when doing some improvements to the system.
Attributes to be tested
Precision: the suitability of any IT feature/component lies in its ability to perform its core functions and give out results which are accurate, thus it is wise to conclude that the more accurate a program is the more efficient it is and vice versa. This test will examine the ability of the new features/ programs to process data and give out results that are free from faults.
Speed: Any programs efficiency is highly dependent on how quick it reacts to users commands, some new programs may slow down a system or even make the system faster. The new features/ programs will be tested on their speed relative to the work load that they can accommodate or work on at any given time. The determined speed will then be compared to the set target or standard of measure or even compared with the speed of the programs that were already being used initially.
Efficiency: an efficiency test will also be carried out; however, this will rely on the combination of results got from all other tested attributes of a particular program/feature. The efficiency test will consist of other attributes tests discussed here; a combination of these tests will determine the overall efficiency level of a new program. The sum of all other attributes then will be compared with the set overall standard measure to determine the efficiency of the feature in question.
Cost: the sum of expenses that has been incurred in the purchase, installation, training personnel, future costs for maintenance, and any other costs that may have been incurred in the process of introducing a certain program will be calculated, other anticipated costs which are bound to be incurred in the future will also be considered. This will be compared with the expected benefits that the program may bring to the IT department and the company as a whole.
Reliability; the level of reliability level of a program rests in its ability to perform its core functions precisely and in a consistent manner, as part of the test experiments a reliability test will be carried out for the new features/ programs. The tests will involve the repeating of an experiment several times in order to determine the degree of variance and therefore the authenticity level of results. A reliable program will always give out similar or almost similar results when performing under similar conditions. If for example a new feature is introduced in the software program for computing the cost of orders relative to the orders complexity and duration, it is expected that it should give similar prices for orders with similar characteristics.
Validity: if a program fully performs the task that it was meant to perform without any problems then it is said to be a valid one, for example one of the new programs may have been installed to enable an easy communication between the clients and the writers, therefore a validity test will be carried out to test whether the intended task is being done to the required standards. The new programs will be subjected to a validity test to determine whether their performance is worthy the costs incurred in purchasing, installing, and maintaining them.
Complexity: the interaction of users and a program determines its efficiency, a program that requires a lot of technical training may be not a reliable one it will end up costing a lot of money and time. Complexity may even take the form of the time taken by different audiences to log in into the system or when navigating their ways through the several destinations in the system. The test will seek to evaluate how the system users are interacting with the new features/ programs that were introduced programs.
Lifespan/Vulnerability: whereas some programs may stay for a very long time without the need to upgrade or change them others tend to have short duration. Other programs may also be good in regards to some of the attributes discussed above but they may be too susceptible to risks such as virus, power outages etc., this susceptibility determines a great deal the lifespan of the program. A lifespan test will be carried out to determine whether the introduced features have got a longer lifespan or they are bound to crash within a short time.
NB: the essence of testing all these attributes independently is to exhaustively examine the efficiency of the new programs. The test would not be fully effective if all the attributes were tested at the same time as cases of inhibition would crop up.
Questions the Test will seek to Answer
The test will seek to answer the following four main questions (1) what impact will the new features have on the performance of the core company functions? To answer this question econometric method of determining cost benefits will be used alongside with qualitative methods that will lead to the full appraisal of both expected and unexpected impacts. (2) Will the new programs help to make the interaction with the customers and writers easy? To answer this question observational method will be used to determine whether the desired effects will really took place, experimental designs will also be used to examine whether the effects that are observed are as a result of the implementation of the new programs or the existing ones. (3) Are the new features likely to help cut unnecessary costs which were incurred initially due to factors such as slowness of the system? The answers to this question will be derived through the use of econometric method of determining cost benefits alongside with qualitative methods that will lead to the full appraisal of both expected and unexpected impacts. (4) Will the new features help to create a new image of the company relative to customers, writers, and audiences perceptions? To answer this question a need assessment will carried out that will include analysis of existing data, using of online surveys to get feedbacks from the target audiences, and other methods of extracting tangible information from the target audiences.
As indicated above the new features/ programs will be comprehensively tested in order to come up with reliable findings that can be relied on in making of conclusions about their worth. Therefore best methods will be used to test the features/ programs, conspicuously standing in the test will be formative and summative elements, the features/ programs will be subjected to formative and summative tests so that all the new programs attributes are fully tested. The main reason for this rigorous testing is simply to enable the realization of final results which can be termed as “useful” [Scriven, M. (1991)] The formative part of the test will seek to strengthen the programs/ features being tested by examining their short term delivery, the quality of its implementation, the organization of the personnel, the inputs, and procedures involved in the implementation of the program. On a formative point of view the needs of the program will be evaluated to determine the program target group, the magnitude of the needs, and how best the needs can be satisfied. The new features consistency will also be monitored to determine how well it is meeting the needs of various audiences of the program such as clients, administrators, staff, writers, and others alike. Should the test method being applied be noted to develop a snag, the formative stage allows for an alternative method to be adopted or some modifications done. As the testing process proceeds as per the set steps, (NB: each attribute of the programs will be tested independently) remarks and observations will be made as they will be used in the summative part of the test to make a grand conclusion on the new features/programs efficiency. [Scriven, M. (1991)]
On the other hand, the summative part of the test will seek to examine the overall outcomes of the programs tested after the set time frame; a summary of what was observed in the formative testing will be made. It is in this part of the test plan whereby the overall impact of the new features/programs will be determined and an estimation of the comparative costs incurred in the whole process of implementing the features/programs carried out. The questions as to whether the programs fully met the intended or unintended needs of the target audiences will be answered in this part of the test. To arrive at this, a combination of the formative and summative tests outcomes and those of other related studies will be used. [Scriven, M. (1991)]
In order to test all the above mentioned attributes of the new features/programs effectively and come up with precise results and comprehensive conclusions, the test time frame needs to be reasonable so that all the experimental designs are adequately carried out; it is hereby proposed that the test be carried out for a period of three months. The test will assume the Sir Ronald A. Fisher’s experimental design whereby control experiments will be set to test each of the above mentioned aspects independently, the testing of the attributes independently is necessary as it will reduce potential chances of interference caused by either retroactive or proactive inhibition. According to Sir Ronald A. Fisher the experiments will comprise of the following methods; [Bisgaard, S., 2008]
Comparison; using a set target, findings of other tests alike, or even prior performance of similar features/ programs, the performance of each new feature will be measured and relative to the targets. This is the main experimental design that will be used through out the test, the formative and summative test results that were recorded for each attribute of a particular feature/ program will be compared against the set standard measure. This will be necessary as the features to be tested are new and therefore very little information about their performance is known, save for the manufactures descriptions. All the above explained attributes of each feature will be tested and the results compared against the set standard baseline (intended and unintended expectations). [Bisgaard, S., 2008]
Replication; it is understood that test/experiment measurements may sometimes have great variations, even after repeating an experimental process several times the variations may still continue to come out, therefore there is the need to conduct numerous measurements of replicated entries in order to determine and estimate the variation. This will help to eliminate doubts and therefore increase the tests validity and reliability. The tests experiments designs will be repeated several times with similar variables and the varying results recorded each time, then an average will be calculated, the resultant figure will be deemed to be reasonably inclusive of the inevitable variance and other errors. [Bisgaard, S., 2008]
Randomization; some attributes will be naturally hard to examine because they may require more time to study, it is therefore advised that a random treatment mechanism be used to allocate a performance merit to the programs. Tables of random numbers are recommended as the best instrument for randomization for this case, however great care should be exercised when allocating performance merit to the attributes to avoid the perceived notion that randomization is done messily. For, instance, attributes such as, vulnerability and durability may be hard to examine within a short duration and therefore randomization will be a very appropriate method to measure them. [Bisgaard, S., 2008]
Blocking; this will involve the arrangement of the experimental units into blocks that share some similarity, e.g. when testing for a certain attribute such as speed, a “speed test experiment unit/block” will be created for all the features/ programs being examined. This will help to eliminate known but inconsequential sources of variation that may crop up between the set units, as a result great precision will be achieved especially when it comes to estimation of the source of variation under study. Again, this will also save time which would have been otherwise wasted on repeating the testing procedure for a particular attribute every now and then with the several features/programs in question. [Bisgaard, S., 2008]
Factorial experiments: after all the attributes for each program has been independently tested, they will be further subjected to a joint factorial experiment testing in order to evaluate their joint interactive impacts. The results of the independent experiments carried out earlier will now be compared to those of the factorial experiment and the results used to give out a clear picture of the grand impacts of the new features/ programs, awaiting further analysis. Factorial experiment will help to test collectively the impacts of each feature/ program and therefore any aspect of the features which may have escaped testing during the independent experiments will still get a chance to be tested and therefore this test plan can be said to be a comprehensive one. [Bisgaard, S., 2008]
After the results of the tests have been recorded they will be subjected to a proper analysis procedure, an analysis design that best accommodates all the chunks of results of both the independent and the factorial experiments will be used. The test results for independent attributes for each feature/ program will be analyzed independently using the analysis of variance (ANOVA) method. This will go down well with Fisher’s model of design experiments, he (Fisher) argued that the best analysis method that best suits this kind of experiments is the ANOVA method. This method will duly accommodate all the chunks of results and the variance that was noted so that they are partitioned into components for easy interpretation and estimation. The resultant summative observations will then be used to make conclusions and where possible recommendations will be drawn from them. [Bisgaard, S., 2008]
Bisgaard, S (2008): “Must a Process be in Statistical Control before Conducting Designed Experiments?” Quality Engineering”, ASQ, pp. 143 – 176, 2008, Vol. 20, Nr2, accessed on February 13, 2009
Scriven, M. (1991). Evaluation thesaurus. 4th ed. Newbury Park, CA: Sage Publications. ISBN 0803943644, accessed on February 13, 2009
Yu, Chong Ho (2005). “Reliability and Validity.” Educational Assessment, available at; http://www.creative-wisdom.com/teaching/assessment/reliability.html, accessed on February 13, 2009