Written by Dr. Hannes Nel

Introduction

It is said that Albert Einstein wrote in a letter to his daughter in 1938 that there is an extremely powerful force that governs all the universe, a variable that scientists often forget.

He confessed that he omitted the variable when he developed the relativity theory.

That variable is love.

I don’t know if the story is true.

And if it is true, then Einstein probably wrote a letter to his daughter in which he declared his love for her in a most romantic and creative manner.

Even so, it made me think. When scientists start their findings by writing: “all other factors being constant,” they are admitting that their findings are wrong.

Because other factors are never constant.

I guess I am just being difficult because it is sometimes necessary to investigate the influence of single factors on an event, behaviour or phenomenon.

Even so, the need for considering the interrelationship between different factors are also important.

I will discuss the following issues related to statistical research methods in this article:

1. Determining validity from statistical data.
2. Calculating statistical significance.
3. Analyzing statistics.
4. Coming to conclusions from statistical data.

Determining validity from statistical data

Statistical validity refers to whether conclusions drawn from a statistical study agree with statistical and scientific laws.

There are different kinds of statistical validities that are relevant to research.

The following are examples of such statistical validities.

1. Construct validity. Construct validity ensures that the results of the data that you collected conform to the theory of your research.

For example, a questionnaire on the quality of learning provided by universities and completed by employers must provide a true picture of the value that university studies have for the workplace.

• Content validity. Content validity ensures that the test or questionnaire that you prepared covers all aspects of the variable that is being studied.

For example, if you were to do research on the exam papers that students studying towards a degree in accounting must write, then the exam papers must test all the exit level outcomes of the subject to have content validity.

• Face validity. Face validity is related to content validity and is a quick initial estimate to check if the test that you will conduct is in line with the hypothesis that you are investigating. It is more subjective than content validity.

For example, if you were to do research on the exam papers that students studying towards a degree in accounting must write, and it looks like a good exam paper that meets the requirements for assessment, then on appearance the exam papers can have face validity.

• Conclusion validity. Conclusion validity is achieved when the conclusions that you reach from the data that you collected are accurate and justified.

This will be the case if the sample or samples that you used are large enough, randomly chosen and taken from the population being investigated.

• Criterion validity. Criterion validity measures how closely the results that you obtain with a data collection instrument matches that of a different instrument.

For example, if you use a questionnaire to measure the extent to which university studies add value to the workplace, and you measure the same research question by making use of a different, proven questionnaire that was used for the same purpose previously, then your questionnaire will have criterion validity if it delivers the same results, or at least nearly the same results, as the proven questionnaire.

• Internal validity. Internal validity is achieved if you can claim that  the results that you achieved with your research can be attributed to the factors that you considered and not to other factors which you did not consider.

It is a measure of the inherent cause and effect relationship between the factors that you considered in your research.

For example, if you can prove that a certain symptom is an indication of only one specific illness, then your finding will have internal validity.

• External validity. External validity relates to how you apply the results of your investigation to the wider population.

It tells you if your findings apply generally or only to the target group of your research.

For example, if you can prove that your findings, based on a sample taken from a certain population also apply to any other group from any other population, then your research findings will have external validity.

Calculating statistical significance

Statistical significance means that you are sure that the statistics that you generated are reliable.

It does not necessarily mean that your findings are important.

Statistical significance can, for example, be skewed by the size of the sample that you investigate.

The larger the sample, the more significant will small differences between two variables appear to be.

Analyzing statistics

There are many ways in which statistics can be analyzed.

Numbers as such seldom provide a clear picture of any value for coming to conclusions.

Dedicated computer software will mostly provide you with such numbers.

Ultimately, however, it is up to you to interpret the numbers and to make sense of them.

For this purpose, you might need to summarize the data or rearrange it in tabular or graphic format.

Some computer programmes might do this for you. They might even interpret the data to an extent, but they will not come to conclusions or findings.

You might, for example, need to group statistics in different age groups, gender, nominal ranges, etc. to see the bigger picture from which to come to conclusions.

This can be made visual in the forms of tables, line graphs, bar charts, histograms, etc.

The computer software might do some handy calculations for you, for example calculating averages, also called the mean; medians (the midpoint of the data); mode (most common value in a set of data); range (the difference between the smallest and the largest value); standard deviation (the average spread around the mean); variance (the square of the standard deviation), etc.

Coming to conclusions from statistical data

Statistics are often used as a basis for coming to conclusions about presumed effects and relationships.

There are several principles of statistics that, if violated, can affect the inferences made from results as well as subsequent conclusions of the research.

Sophisticated statistics do not guarantee valid conclusions.

You will, therefore, need to obtain the assistance of an expert statistician to help you interpret statistical data if you are not one.

However, coming to conclusions and developing findings from them are still your responsibility.

Summary

The internal validity of conclusions is tested against statistical and scientific laws.

There are different kinds of validities, depending on how and against what your conclusions are tested.

Kinds of validity include:

1. Construct validity.
2. Content validity.
3. Face validity.
4. Conclusion validity.
5. Criterion validity.
6. Internal validity.
7. External validity.

Statistical significance is achieved if your statistics are reliable.

Reliability is often damaged when the size or composition of your sample is wrong.

Statistics can be analyzed by consolidating them in a table or graphs.

Some analysis and calculations are often done by computer.

You will need to come to your own conclusions based on your interpretation of the data.

Finally, you will need to derive findings from your conclusions.