Worksheet 7 - Gathering and Processing Data

Question 1
You are carrying out research into savings institutions in St Vincent & the Grenadines.
Identify which of the following are primary and which are secondary sources of information. Explain why for each source.

(a) www.firstcaribbean.com

(b) Your own interview with the manager of GECU

(c) A report by the Eastern Caribbean Central Bank on Savings in St Vincent from their website without a date.

(d) Statistics from the Government’s Annual Financial Statement on the Economy on Credit Union business for 2004.

(d) The results of a survey using questionnaires carried out by a PhD student for his thesis on “The growth of credit unions in St Vincent” submitted in 2006

(d) A text book entitled “Caribbean Economics” published in 1997.

(e) An advertisement for Millennium Offshore Bank’s Supergrowth Bond which compares its rate of interest to other savings institutions in St Vincent in last week’s local newspaper


Question 2

Evaluate 3 of the references above in Question 2 for reliability and validity. Make sure you evaluate both source of the data and the data itself.

Question 3

Evaluate the credibility ONE of the following websites.


http://www.guyanacaribbeanpolitics.com/

http://anthurium.miami.edu/home.htm

http://www.geocities.com/CollegePark/Classroom/9912/kamguidepg.html

Question 4

The Principal wishes to research whether changing to an Associate Degree from the current A level / CAPE programme would be successful.

(a) What 3 different methods of research could he use to gather data on which to base his decision? State the method in detail and the source from which the data would be gathered for each method.

(b) What factors might affect reliability and validity of one of the methods you chose. What could be done to minimise the effects of the factors you have identified.

(c) The Community College has 830 students of which 700 are female and 130 are male. Discuss the effect of sample size and demographic representation (include all the factors) if the Principal wishes to ensure a reliable and valid sample of the students.

(d) The Principals is in favour of changing to an Associate Degree. How might this cause bias in interpretation of the data?

Question 5

The following questions are from a survey into herbal medicine. Comment on the effectiveness of each question in generating reliable and valid data

1. Do you use alternative medicine?

2. Is herbal medicine a good or a bad thing?

3. Which of the following ways have you obtained herbal medicine / have you used
herbal remedies
(a) prescribed by a health professional e.g. registered nurse or doctor
(b) herbal remedies suggested by other health practitioners e.g. bush doctor, herbalist
(c) over the counter herbal remedies purchased at a pharmacy
(d) herbal remedies from your own garden/collected by you
(e) any other method of obtaining herbal remedies (Please give details)

4 (a) Have you used herbal medicine Yes / No / Not sure
(b) What illnesses have you taken herbal medicine to relieve?


Question 5

You are carrying out research into attitudes towards recreational use of cannabis.

Other than questions to elicit demographic information suggest 4 questions two open and two closed in the sequence they would appear in the questionnaire to generate data.

Gathering data in research - an evaluation of reliability and validity

Primary and secondary data sources

A research instrument can gather data from either a primary or secondary source

Primary sources
Primary sources are original materials on which other research is based. They are not interpreted or evaluated. Examples of primary sources are data gathered by a questionnaire, statistics of population from the Government census or a first report of a research experiment and its finding

Secondary sources
Secondary sources are less easily defined than primary sources. Secondary sources interpret and analyse primary sources examples might be text books, or journals which review others work. Sometimes is it always easy to distinguish primary from secondary sources. A newspaper article is a primary source if it reports events, but a secondary source if it analyses and comments on those events.

Evaluating Reliability and Validity

In evaluating sources there are two elements reliability and validity. For a data source to be accurate and credible high levels of reliability and validity is the aim. Both elements are equal in importance in judging the accuracy and credibility of a source.


Reliability – Can the source provide the data?

For a source to be reliable we must evaluate the ability of the source to provide the information. We are looking at the question “Is it likely that this source can provide this data?”

The issue is therefore authority. To evaluate authority we can look at several aspects of the data source.

Author - Is the author an expert in the field? What qualifications do they have? For example an article on a website about HIV+ written by medical doctor might have more authority than one written by some one without qualification.

Professional standards. – Does the author have certain professional standards? The example of a doctor immediately comes to mind. Similarly academic writers who are published in academic journals or books have to conform to standards and have their work checked by other academics. Journalists mostly operate within a professional approach especially large international newspapers such as the New York Times or the Guardian (UK)Authority can mean expertise.

Publisher – Is the publisher reputable? Academic publishers need to maintain their reputation for accurate factual information so they also have editors to ensure a high standard. Other publishers such as newspapers, magazines etc need to avoid legal action for libel (telling lies about someone) so also should be careful to print the truth.

Organisation or Institution – If the data is from an organisation, for example the United Nations, we need to evaluate their reputation and their role or responsibilities. For example statistics on the economy from the East Caribbean Central Bank would come from a highly reliable source as the bank use the statistics to conduct the very important business of issuing bank notes and controlling the money supply in the region.

Research method – Could the research method chosen generate the data necessary? For example in researching teen pregnancy would carrying out an interview of an expert generate the data needed or would questionnaires of teens be a better choice.



Validity – Is the data true?

A source could have high levels of reliability. For example, academic research published in an academic journal by the leading expert in the field however the data may have a low level of validity in that it might be very out of date. Equally it may be possible that a source might not be considered highly reliable for example an intenet site which does not have the name of the author, organisation who maintains the site etc however the data is still true or valid.

In evaluating validity we need to look at accuracy and bias.

To evaluate accuracy we can look at several aspects of the data

Currency – When was the data published or gathered? Could the information be out of date? For example statistics on rates of HIV+ infection will need to be up to date to be accurate.

Relevance – Does the information relate to the circumstances you are applying it to? For example, will research carried out in the United States apply to the Caribbean?

Data collection – Was the data collected by reliable methods? Was it accurately recorded?

Sample size – Was the sample size large enough for generalisation to be accurate? For example if a newspaper article has only interviewed one person in a large crowd can we assume that all the points of view are represented? Similarly with social research the sample size is vital to judging whether the data is representative of the population as a whole.

Replicable – Do other sources have similar information? Would another similar piece of research have the same result? This is particularly relevant to sources such as the internet which lack references.

To evaluate bias we can look at:

Representation – Does the sample include all the variables within the population such as age, gender, social class, religion, education level which might affect response? Even with a large sample if the sample is not representative then bias in the data will occur.

Cultural bias – Has the data been collected by someone of the same or a different culture. For example, an Western researcher may misinterpret a non-Western culture and be biased due to racism or other factors. Similarly when researching within ones own culture, being subject to the same values and beliefs as the subjects may cause one not to question certain responses. For example when evaluating religious or other beliefs.

Political bias – Is the data being presented from either a right wing or a left wing perspective. The conservative agenda (e.g. free market economics, personal liberty above all other rights and fundamental religious views) will differ from the liberal agenda (e.g. some control of the market for social gain, social control for the good of society, religious tolerance for different views).

Social bias – Aspects such as gender, race, age and social class may affect the presentation of data. For example a women’s perspective on sexual equality may differ from a man’s views.

Faulty research methods – Even the best academic researchers can make mistakes and inexperienced researchers such as a student may have issues with poorly designed and executed questionnaires and interviews. Mistakes within the research method inadvertently cause bias. This is why academic research is reviewed by several other academics to evaluate the methodology and avoid bias in the conclusions or faulty conclusions.

Aim of the source in presenting the data – The reason for the data being presented will have an effect on bias. For example a Government might present certain statistics on economic performance if they are favourable and might avoid others. Whilst the data is valid, there might still be bias in that other relevant information is not present. If the source’s aim is persuasive again there may be bias. For example commercial sites wishing to sell products.

The Research Process

The research process is systematic and has several stages

(a) Selection of topic
The general area for research will be determined by either academic consideration (e.g. to further knowledge on one particular area) or by a specific need for information (e.g. to make a decision on services for example whether to implement an associate degree)

(b) Research question ­
The research question is the narrowing of focus from a topic. For example the topic Associate Degrees at Community College may have a research question “The impact of implementation Associate Degrees at Community College”.

(c) Thesis
It may be at this stage a thesis will be proposed or it may be that some preliminary data gathering will take place before the thesis. For our example our thesis which we are seeking to either prove or disprove will be “Fewer students will take an Associate Degree compared with the current student numbers taking the A level programme”.

(d) Designing the research instrument
A research instrument is the way in which data (the facts) is gathered. It may be that research will be carried out solely through reviewing others research (library / book review) or combined with data gathering through research instruments such as surveys, experiments or observations. When designing the research instrument, the focus will be the goal of the research question and the thesis.

(e) Gathering data
The research will use the various instruments to collect information about the topic.

(f) Recording data
Accurate recording of data is central to research both primary and secondary research can be biased by incorrect recording.

(g) Analysing and evaluation of the data
The data will be assessed against the thesis as well as other conclusions being sought for the research question. It may be that the research might show fewer students would take the A level programme however other data gathered might show that more students who previously would not have taken the A level programme would take the Associate degree

(h) Presentation of results
The research question is used to provide a framework for presenting the findings.

(i) Review of results
In many instances the research process and results are reviewed by either the researchers peers or in an academic evaluation process.