Entry #10: Validity and Reliability

Bulls-eye!!! 

A good research provides reliable and valid results. Don't you agree?

So why is it very important for a research to have reliability and validity? Before that, we need to know what is validity and reliability.

Validity is the meaningfulness or usefulness of the specific inferences researches make based on the data they collect.

What kinds of evidence might a researcher collect? Essentially, there are three main types.

• Content-related evidence of validity refers to the content and format of the instrument.
• Criterion-related evidence of validity refers to the relationship between scores obtained using the instrument and scores obtained using one or more other instrument or measures.
• Construct-related evidence of validity refers to the nature of psychological construct or characteristic being measured by the instrument.

Reliability refers to consistency of the scores obtained.

Reliability and validity always depend on the context in which an instrument is used.

Research Proposal is in Progress...

65%

Entry #9: Variables

Well, that was exactly that came up into my mind when people started talking about variables. I was so confused as I often mixed up the definitions of dependent and independent variables. Then suddenly they came up with extraneous variables, manipulate variables, mediator variables, criterion variables and the list goes on and on. Pheww..

I realised that I really need to open the research methodology bible aka the 'handbook' written by Fraenkel, Wallen and Hyun (2012) and it is very much of help in understanding the elements in educational research.

Through my reading and lecture notes, I found that variable is actually a set of variation and it could be categorical or quantitative.

So what about the other variables mentioned earlier?
Well, to make it clearer to digest, based on my reading, I have concluded that:

independent variable = experimental variable = manipulated variable = treatment variable
dependent variable = criterion variable = outcome variable


What are the moderator, mediator and extraneous variables?

Extraneous variables
In previous entry, I have mentioned about the extraneous variables. The extraneous variables could influence the effect on the dependent variables or other variables that exist. Examples of extraneous variables are genders, the personality of teachers involved, the experience level of the students which could affect learning in the study. One of the way to avoid or control these variables is by holding them constant.

Moderator variables
It is a special type of independent variable. It is a secondary independent variable that has been selected for study to determine whether it affects or modifies the basic relationship between the primary independent variable and the dependent variable. The key word here is modify.

Mediator variables
It is one that attempts to explain the relationship between the two other variables. Mediator variables provide explanation to the relationship between the independent and dependent variables.

Below is the graphic organizer to explain the relationship between the variables.

The other factor could also be the extraneous, mediator or moderator variables.

Below are the examples for independent and dependent variables.

So I guess I won't get confuse anymore and whenever people ask me about variables, I hope I could explain them pretty well and same goes to answering the test :)

Entry #8: Steps in literature review

According to Fraenkel, Wallen and Hyun (2012), there are six steps in literature review

The first step to literature review is to define the problem as precisely as possible. This is because if the problem statement is too broad, it would become too fuzzy in finding the suitable and appropriate general reference. Therefore, the questions of interest should be narrowed down to specific area of concern.

Secondary sources: secondhand information, such as description of historical events by someone not present when the event occurred.

1. encyclopedia of educational research
2. handbook of research on teaching
3. national society for the study of education
4. review of educational research
5. review of research in education
6. subject guide to books in print (current edition)
7. Malaysian Education Blueprint (this could also be considered as the secondary source)

Primary sources: firsthand information, such as testimony of an eyewitness, an original document, a relic, or a description of a study written by the person who conducted it.

1. journals
2. dissertations or thesis

It is very important to put citation in literature review and to paraphrase the sentences in order to avoid plagiarism. Normally, the references and citation in literature review follows the APA style of referencing.

Entry #7: Type of Research

There are various types of research methodologies. However, I'm going to focus on experimental research since it is related to my research proposal.


Below is the conceptual framework or the research design for experimental research.

This design is the randomized pretest-posttest control design.

Two groups of subjects are used, with both groups being measured or observed twice. the first measurement serves as pretest, the second as the posttest. For my research purpose, I will randomly select 60 students from the average proficiency level and these students then will be divided into two groups: 1)experimental and 2) control group.

Test-retest method is used in order to secure the reliability of the findings.

In a research, there are possibilities that the dependent variables or the results would be affected by other variables. The variables which may influence the dependent variables are called the extraneous variables. It is very important to eliminate or minimize these extraneous variables in order to ensure the validity and reliability of a study.
Variables, validity and reliability will be further discussed in other entries.
:)

Entry #6: Constructing Research Proposal

Construction In Progress!!

Well, the due date for my research proposal submission is getting closer...yet my research is still under construction. I've discussed with my lecturer, Dr. Teoh regarding my research topic. I thought I've came up with good questions, but after the discussion, there are few things that need to be tweak or alter. 

Again, as I've mentioned earlier, you can see that there are parts that I need to reconstruct. Well, I think for my research proposal, I need to list out the keywords which I'm going to focus in my research so that people who read it could clearly understood the direction and the focus of my research. 

Keywords: Reader Response Approach, promoting speaking skills

I guess people are way ahead of me in doing their research proposal. I think I'm going to stop now. Will continue the post later~ :)

Perhaps, in next entry I will write about:

1. type of research
2. definitions (keywords)
3. reference (for my literature review)
4. variables and hypothesis 
5. sample of subject
6. instrumentation
7. descriptions of procedures to be followed (if possible)

The discussion will be based on this coursebook :)

Entry #5: Refining the RQ

Hi there....
 I think there's a need for refining my research question. I asked my friends to read my blog, as they also furthering their master right now and semesters ahead of me, so I could consider them as seniors for this matter. One of my friends suggested to me to refine the question as for Q2 and Q3 could be combined into one.
When I look at the question again, I do agree with her.
Well, before I go further with my RQ, lets look at the criteria of good RQ.

Fraenkel, J. R., Wallen, N. E. & Hyun, H. H. (2011). 

How to design and evaluate research in education. 8th ed. 

McGraw-Hill.

So, there are four criteria for a good research question. 

The four are: 1) feasible, 2) clear, 3)significant and 4) ethical.

Feasible:

The question is feasible meaning that it could be conducted or investigated within the allocated time, cost and energy efficient. In other words, it is easy to conduct. I think, we could determine whether the question is feasible or not if we could identify the mean of collecting the data i.e through surveys or interviews etc.

Clear:

Clear means, the moment people read your research question(s), they are able to comprehend what you are trying to achieve, provided with suitable and appropriate key words.

Significant:

This part is very crucial especially in quantitative research. The questions should display or provide how significant the research is in contributing to the society. 

Ethical:

The questions must not touch on sensitive issues. This involves with ethics and moral values.

So. lets look at my research questions and how could I refine them to make sure these four elements are there.

Question #1

What are the causes of the students' anxiety in participating in oral activity in ESL classroom?

Question #2
What are the students' anxiety level before using RRA?

Question #3
What are the students' anxiety level after using RRA?

I think my Q1 is feasible as I'm trying to find out the causes or factors of students' anxiety especially in ESL classroom during oral activity. I could conduct surveys or questionnaires to find the factors as well as do some interviews with the students and teachers. The question is clear as I've specifically mention students' anxiety in oral participation. I guess the question is not too broad and people would understand it. Is it significant? For me, of course, as most teachers who are teaching English as Second Language (ESL) or even as Foreign Language (EFL) would face the same difficulty especially when it comes to students' participation. Well, mostly those students who are lack in proficiency. I believe that this research would help the teachers, particularly, in understanding the reasons why the reticent students behaving like that. The question is definitely ethical.

As I've mentioned earlier, I think I need to reconsider my Q2 and Q3 and restructure the questions to make it become a good question. As I'm trying to use the RRA approach in my class, in enhancing the students' participation in oral activity and I want to see the result, how this approach could help my students in overcoming their anxiety, I guess one key word that suits this purpose is the significant difference before and after using the approach.

Question #2 (Edited)
Is there any significant difference in students' anxiety level before and after the introduction of RRA in oral activity in ESL?

So is the new edited question deserves to be considered as a good question?

• Is it feasible?
• I think it is feasible as it could be investigated without expanding money or energy. I think I would be able to gather sufficient results within allocated or given time frame. I could carry out pre-test and post-test to measure students' anxiety level. My hypothesis would be that it is going to be a significant difference in students' anxiety level which indirectly could be seen through their participation in oral activity in ESL classroom. I could also conduct interview with the students, on how they feel before and after the teacher uses the approach and I could also interview the teacher whether is there any changes in the students' participation.
• Is it clear?
• I think the question is clear as it helps me in deciding the means of data collection and it helps me to get a very clear direction of my research.
• Is it significant?
• Yes, as this question will lead to the results of using RRA in reducing students' anxiety level, thus enhancing students' participation level in oral activity. If it shows positive results, this approach could be used in helping teachers who are facing the same situation.
• Is it ethical?
• With no doubt, it is ethical. :)