Entry #10: Validity and Reliability

Bulls-eye!!! 

A good research provides reliable and valid results. Don't you agree?

So why is it very important for a research to have reliability and validity? Before that, we need to know what is validity and reliability.

Validity is the meaningfulness or usefulness of the specific inferences researches make based on the data they collect.

What kinds of evidence might a researcher collect? Essentially, there are three main types.

• Content-related evidence of validity refers to the content and format of the instrument.
• Criterion-related evidence of validity refers to the relationship between scores obtained using the instrument and scores obtained using one or more other instrument or measures.
• Construct-related evidence of validity refers to the nature of psychological construct or characteristic being measured by the instrument.

Reliability refers to consistency of the scores obtained.

Reliability and validity always depend on the context in which an instrument is used.

Research Proposal is in Progress...

65%

Entry #9: Variables

Well, that was exactly that came up into my mind when people started talking about variables. I was so confused as I often mixed up the definitions of dependent and independent variables. Then suddenly they came up with extraneous variables, manipulate variables, mediator variables, criterion variables and the list goes on and on. Pheww..

I realised that I really need to open the research methodology bible aka the 'handbook' written by Fraenkel, Wallen and Hyun (2012) and it is very much of help in understanding the elements in educational research.

Through my reading and lecture notes, I found that variable is actually a set of variation and it could be categorical or quantitative.

So what about the other variables mentioned earlier?
Well, to make it clearer to digest, based on my reading, I have concluded that:

independent variable = experimental variable = manipulated variable = treatment variable
dependent variable = criterion variable = outcome variable


What are the moderator, mediator and extraneous variables?

Extraneous variables
In previous entry, I have mentioned about the extraneous variables. The extraneous variables could influence the effect on the dependent variables or other variables that exist. Examples of extraneous variables are genders, the personality of teachers involved, the experience level of the students which could affect learning in the study. One of the way to avoid or control these variables is by holding them constant.

Moderator variables
It is a special type of independent variable. It is a secondary independent variable that has been selected for study to determine whether it affects or modifies the basic relationship between the primary independent variable and the dependent variable. The key word here is modify.

Mediator variables
It is one that attempts to explain the relationship between the two other variables. Mediator variables provide explanation to the relationship between the independent and dependent variables.

Below is the graphic organizer to explain the relationship between the variables.

The other factor could also be the extraneous, mediator or moderator variables.

Below are the examples for independent and dependent variables.

So I guess I won't get confuse anymore and whenever people ask me about variables, I hope I could explain them pretty well and same goes to answering the test :)

Entry #8: Steps in literature review

According to Fraenkel, Wallen and Hyun (2012), there are six steps in literature review

The first step to literature review is to define the problem as precisely as possible. This is because if the problem statement is too broad, it would become too fuzzy in finding the suitable and appropriate general reference. Therefore, the questions of interest should be narrowed down to specific area of concern.

Secondary sources: secondhand information, such as description of historical events by someone not present when the event occurred.

1. encyclopedia of educational research
2. handbook of research on teaching
3. national society for the study of education
4. review of educational research
5. review of research in education
6. subject guide to books in print (current edition)
7. Malaysian Education Blueprint (this could also be considered as the secondary source)

Primary sources: firsthand information, such as testimony of an eyewitness, an original document, a relic, or a description of a study written by the person who conducted it.

1. journals
2. dissertations or thesis

It is very important to put citation in literature review and to paraphrase the sentences in order to avoid plagiarism. Normally, the references and citation in literature review follows the APA style of referencing.

Entry #7: Type of Research

There are various types of research methodologies. However, I'm going to focus on experimental research since it is related to my research proposal.


Below is the conceptual framework or the research design for experimental research.

This design is the randomized pretest-posttest control design.

Two groups of subjects are used, with both groups being measured or observed twice. the first measurement serves as pretest, the second as the posttest. For my research purpose, I will randomly select 60 students from the average proficiency level and these students then will be divided into two groups: 1)experimental and 2) control group.

Test-retest method is used in order to secure the reliability of the findings.

In a research, there are possibilities that the dependent variables or the results would be affected by other variables. The variables which may influence the dependent variables are called the extraneous variables. It is very important to eliminate or minimize these extraneous variables in order to ensure the validity and reliability of a study.
Variables, validity and reliability will be further discussed in other entries.
:)

Entry #6: Constructing Research Proposal

Construction In Progress!!

Well, the due date for my research proposal submission is getting closer...yet my research is still under construction. I've discussed with my lecturer, Dr. Teoh regarding my research topic. I thought I've came up with good questions, but after the discussion, there are few things that need to be tweak or alter. 

Again, as I've mentioned earlier, you can see that there are parts that I need to reconstruct. Well, I think for my research proposal, I need to list out the keywords which I'm going to focus in my research so that people who read it could clearly understood the direction and the focus of my research. 

Keywords: Reader Response Approach, promoting speaking skills

I guess people are way ahead of me in doing their research proposal. I think I'm going to stop now. Will continue the post later~ :)

Perhaps, in next entry I will write about:

1. type of research
2. definitions (keywords)
3. reference (for my literature review)
4. variables and hypothesis 
5. sample of subject
6. instrumentation
7. descriptions of procedures to be followed (if possible)

The discussion will be based on this coursebook :)

Entry #5: Refining the RQ

Hi there....
 I think there's a need for refining my research question. I asked my friends to read my blog, as they also furthering their master right now and semesters ahead of me, so I could consider them as seniors for this matter. One of my friends suggested to me to refine the question as for Q2 and Q3 could be combined into one.
When I look at the question again, I do agree with her.
Well, before I go further with my RQ, lets look at the criteria of good RQ.

Fraenkel, J. R., Wallen, N. E. & Hyun, H. H. (2011). 

How to design and evaluate research in education. 8th ed. 

McGraw-Hill.

So, there are four criteria for a good research question. 

The four are: 1) feasible, 2) clear, 3)significant and 4) ethical.

Feasible:

The question is feasible meaning that it could be conducted or investigated within the allocated time, cost and energy efficient. In other words, it is easy to conduct. I think, we could determine whether the question is feasible or not if we could identify the mean of collecting the data i.e through surveys or interviews etc.

Clear:

Clear means, the moment people read your research question(s), they are able to comprehend what you are trying to achieve, provided with suitable and appropriate key words.

Significant:

This part is very crucial especially in quantitative research. The questions should display or provide how significant the research is in contributing to the society. 

Ethical:

The questions must not touch on sensitive issues. This involves with ethics and moral values.

So. lets look at my research questions and how could I refine them to make sure these four elements are there.

Question #1

What are the causes of the students' anxiety in participating in oral activity in ESL classroom?

Question #2
What are the students' anxiety level before using RRA?

Question #3
What are the students' anxiety level after using RRA?

I think my Q1 is feasible as I'm trying to find out the causes or factors of students' anxiety especially in ESL classroom during oral activity. I could conduct surveys or questionnaires to find the factors as well as do some interviews with the students and teachers. The question is clear as I've specifically mention students' anxiety in oral participation. I guess the question is not too broad and people would understand it. Is it significant? For me, of course, as most teachers who are teaching English as Second Language (ESL) or even as Foreign Language (EFL) would face the same difficulty especially when it comes to students' participation. Well, mostly those students who are lack in proficiency. I believe that this research would help the teachers, particularly, in understanding the reasons why the reticent students behaving like that. The question is definitely ethical.

As I've mentioned earlier, I think I need to reconsider my Q2 and Q3 and restructure the questions to make it become a good question. As I'm trying to use the RRA approach in my class, in enhancing the students' participation in oral activity and I want to see the result, how this approach could help my students in overcoming their anxiety, I guess one key word that suits this purpose is the significant difference before and after using the approach.

Question #2 (Edited)
Is there any significant difference in students' anxiety level before and after the introduction of RRA in oral activity in ESL?

So is the new edited question deserves to be considered as a good question?

• Is it feasible?
• I think it is feasible as it could be investigated without expanding money or energy. I think I would be able to gather sufficient results within allocated or given time frame. I could carry out pre-test and post-test to measure students' anxiety level. My hypothesis would be that it is going to be a significant difference in students' anxiety level which indirectly could be seen through their participation in oral activity in ESL classroom. I could also conduct interview with the students, on how they feel before and after the teacher uses the approach and I could also interview the teacher whether is there any changes in the students' participation.
• Is it clear?
• I think the question is clear as it helps me in deciding the means of data collection and it helps me to get a very clear direction of my research.
• Is it significant?
• Yes, as this question will lead to the results of using RRA in reducing students' anxiety level, thus enhancing students' participation level in oral activity. If it shows positive results, this approach could be used in helping teachers who are facing the same situation.
• Is it ethical?
• With no doubt, it is ethical. :)

Entry #4: Re-defining my research

Hello there,

So I guess since I'm going to change my research topic, I need to redefine the area, objective, statement and the question(s).

As I've mentioned earlier, I realised that the most important thing in conducting or deciding on the research topic is that it must be measurable. Measurable in the sense that how am I going to collect my data, what are the items that I'm going to use for my survey questions for example, that could help me in measuring the targeted response.

From that, I guess I would be able to decide which type of research my research would follow, whether it is going to be qualitative or quantitative, etc.

So, is it Quali or Quanti?

Quantitative research is an inquiry into an identified problem, based on testing a theory, measured with numbers, and analyzed using statistical techniques. The goal of quantitative methods is to determine whether the predictive generalizations of a theory hold true.

By contrast, a study based upon a qualitative process of inquiry has the goal of understanding a social or human problem from multiple perspectives. Qualitative research is conducted in a natural setting and involves a process of building a complex and holistic picture of the phenomenon of interest.

Lecture note for Chapter 4

So, I guess, based on my research proposal, my research would fall under quantitative research since my research is designed to determine the effectiveness of using Reader Response Approach (RRA) in reducing the students' anxiety level. The research would be measured in numbers as I would measure the students' anxiety level pre and post introduction of RRA in oral activities in ESL classroom.

I hope this one would present a clearer view and direction of my research.

Entry #3: Redesigning the research proposal

Well....
when I look back at my research proposal, I think the idea is too idealistic.
I realize that no matter what topic or problem that we want to discuss, the most important thing is it must be measurable.
I also find out that as I try to look for the reading materials, I somehow diverted my ideas and I'm starting to think for another topic.
Instead of focusing on motivation solely,  I think I might be focusing on the relationship between students' anxiety level and their participation in speaking activity.

I think that one would be clearer compared to the previous one.

I think, one of the reason that students' are so reluctant to speak or to get involve in any speaking activity is because of their anxiety level. I do believe that anxiety is one of the factors which demotivating students' in using English as L2 orally. So I guess it is not too far astray from my previous idea yet this one is more focused. 

In general, what causes on to be anxious could be their level of stress, worry, nervousness, over or less self-confidence and perhaps their bad memories with particular matter.

Based on my observations and understanding, students become anxious when speaking other language, which is not their mother tongue as they are afraid that others would judge them. They are afraid of making mistakes as they think their friends would make fun out of it and perhaps the teacher would scold them. I need to know whether these causes of anxiety are true or not. 

Entry #2: Defining area, problem, purpose and questions

I guess my research would be focusing on the relation between motivation and English as Second Language (ESL). Therefore, the area of my research would be ESL.

I always believe that there are multiple ways in doing things, still you can achieve the same objective. Some might think of the problem first and then will proceed to objective and finally come out with research questions. But some might work the other way round. It's like deductive and inductive ways of doing it

In my case, I think of the questions first. What are the things that I want to know or find out. Like I've mentioned in my previous post, I want to know what are the factors that motivate the students in using English as L2 especially as spoken language. But, before finding the factors, I think I also need to know their attitudes and perceptions towards the language, so that I could relate them with the contributing factors of their motivation. Therefore, my research problem and objective are derived from these two questions.

Below is the overview of my research aka Task 1 given by Dr Teoh.

Entry #1: Research Topic

Hi!

I've been blogging for 8 years yet it's more on my personal thoughts and views. That is why it takes me days to get started, writing this entry.

Better late than never, right?

So, before I decide on my research topic, I need to think on area that I would love to touch on.

Motivation. Yes. Even during my interview, when the interviewer asked me what I wanted to do for my research proposal, I did mention about motivation.

I always believe that motivation plays an important and crucial role in determining someone's success. 

It's like behind a successful man, there is a woman; thus, behind a great man lies a great motivation.

I've been teaching for two years now and I notice that my students are very reluctant to speak in English. The most crucial moment was when I started teaching, in 2012. The students even asked me to speak in Malay while teaching them English. I tried my best to get them to talk in English, regardless of their grammar mistakes. I also came up with my own program, which is English Time Zone, where the students have to speak in English during recess time, no matter where they are. I realize that it starts to show some changes and improvement in their willingness to at least try to speak using English.  But of course, there are still other students who I find reluctant to speak. Even when I asked them or spoke to them. So, I wonder what are the things that affect their motivation in using English as L2, orally.

What is motivation?

There are two types of motivation which are intrinsic and extrinsic motivation.


Extrinsic motivation means someone is motivated by outside factors like for the sake of getting rewards or praises from others.
Meanwhile, intrinsic motivation means what motivates a person comes form within the person himself or herself.

There are two theories that could help me to understand more on motivation concept.

Well, I am familiar with the Maslow's theory before but I have never came across Herzberg's theory. I guess I'll have to read more on the theory.

I guess I'll stop now.

Before that, I would to end my entry with this quote.