Computer Science 494: Research Methods

Study Guide

Unit 1: What Is Research? Research Problems, Hypotheses and Literature Reviews

This unit contains the following sections:

• 1.1 Preface
• 1.2 Learning Outcomes
• 1.3 Selecting a Problem
• 1.4 Forming a Research Topic or Question and Conducting a Literature Review
• 1.5 References

1.1 Preface

Research is a unique activity. It exacts testable insights from researchers and uses rigorous methods to reach validated proofs for those insights. At the same time, it is flexible about the selection of methods, the interpretation of results, and the type and scope of problems considered. “Every significant piece of research not only contributes to the evolution of the human knowledge frontier, the “what,” but also enriches the process mechanisms underlying research, the “how” (Kumar, 2011, p. 287).

According to Wikipedia, research is used to “establish or confirm facts, reaffirm the results of previous work, solve new or existing problems, support theorems, or develop new theories” (“Research,” 2014).

The article goes on to state that “[t]he primary purposes of basic research (as opposed to applied research) are documentation, discovery, interpretation, or the research and development (R&D) of methods and systems for the advancement of human knowledge.” Conversely, applied research is designed to develop practical applications of the fundamental scientific principles revealed by basic research.

Thus applied versus basic is one crucial distinction in research. Another is empirical versus rational:

Empiricism is a label for a way of thinking and working with data. It indicates an attitude complex characterized by an utmost faith in the senses, a firm belief in the power of observation, a willingness to be ruled by observable evidence, a belief that scientific conclusions should never get beyond the realm of extrapolation, and a feeling that the rational universe of science is nothing more that the habitual association of certain ideas of a perceiver.

Rationalism represents the opposite way of thinking and working with facts. In rationalistic method the criterion of truth is not sensory but intellectual and deductive. It assumes the universality of natural laws, and hence its appeal to sense perception is in pursuit of particulars. There is a direct preoccupation with conceptual schemes, constructions, and logical manipulations. The supreme example of rationalism is in pure mathematics. (McKinney, 1966, p. 72)

There is a dynamic interplay between the rationalistic and empirical perspectives. Many theories and hypotheses develop deductively; however, those theories and hypotheses must be validated by real-world observations, that is, empirically. According to W. Lawrence Neuman, this combination defines the positivist approach to research:

Positivist social science is an organized method for combining deductive logic with precise empirical observations of individual behavior in order to discover and confirm a set of probabilistic causal laws that can be used to predict general patterns of human behavior. (2006, p. 82)

Note that, while Neuman restricts his consideration to social science research, which is most likely to be the type of research students undertake in Computing and Information Systems, the same general point can be made about scientific research in any area.

Ranjit Kumar (2005) outlines a useful way to distinguish among types of research:

If you examine a research study from the perspective of its objectives, broadly a research endeavour can be classified as descriptive, correlational, explanatory or exploratory. . . .

A study classified as descriptive research attempts to describe systematically a situation, problem, phenomenon, service or program, or provides information about, say, the living conditions of a community, or describes attitudes towards an issue. . . .

The main emphasis of a correlational research study is to discover or establish the existence of a relationship/association/interdependence between two or more aspects of a situation. . . .

Explanatory research attempts to clarify why and how there is a relationship between two aspects of a situation or phenomenon. . . .

[Exploratory research . . . is undertaken with the objective either to explore an area where little is known or to investigate the possibilities of undertaking a particular research study. (p. 10, emphasis in the original)

Kumar (2005) also explains that research can be distinguished according to the “inquiry mode employed” (p. 8).

The structured approach to inquiry is usually classified as quantitative research and unstructured as qualitative research. In the structured approach everything that forms the research process—objectives, design, sample, and the questions that you plan to ask of respondents—is predetermined. The unstructured approach, by contrast, allows flexibility in all these aspects of the process. The structured approach is more appropriate to determine the extent of a problem, issue or phenomenon; the unstructured to explore its nature. (p. 12, emphasis in the original)

In general, according to Colin Robson, social science research should be “empirically based, systematic, sceptical and ethical [and] in some sense influential or effective if it is to be worthwhile” (2002, p. 19).  The requirement for ethical conduct of research is particularly important with research that involves people. (Research ethics forms the topic of Unit 5 of this course.) The question of influence and effectiveness is related to ethics. It is based on the concept that frivolous research is a waste of resources and of the time of the people (interviewees, respondents) who participate.

This introduction has drawn attention to many of the distinctions that can characterize scientific research, but it is important to remember that irrespective of the underlying philosophy, scientific research is about expanding the knowledge frontier.

Unit 1 requires that you engage in reading activities, video viewing activities, and some assessment activities, including contributing to a discussion forum. We anticipate that you will need about 9 hours (540 minutes) to complete this unit.

1.2 Learning Objectives

After completing Unit 1, you should be able to

• define the term research and describe what research involves.
• discuss the importance of research.
• identify different types of research.
• describe the processes by which research is conducted.
• identify methods that can be used to select a problem.
• evaluate the quality of a research problem.
• identify methods that can be used to collect, review, and infer findings from the literature.

1.3 Selecting a Problem

We estimate that this segment of Unit 1 should take about 4 hours (240 minutes) to complete.

Reading Assignment 1.3.R1

This sample research article identifies many of the key elements of research. It serves as an overview of what is involved in research and what is expected as an outcome of research.

Note the sections included in this article: abstract, introduction (this section is untitled in this article, but it does include a literature review), method (including sample, measures, design, and procedures), results, conclusions and references.

Reading Assignment 1.3.R2

This reading introduces the language of research and presents a brief discussion of key aspects of research. Make certain that you follow each of the links, and be prepared to return to them as you work through the course.

Viewing Assignment 1.3.V1

Watch the short video:

Reading Assignment 1.3.R3

In 2009, the European Commission published two reports. In the first report (see below), read the section titled “The Asian Century.” In the second report (see below), read the sections titled “The Role of Socio-economic Sciences and Humanities” and “Conditions for Transition.”

Reading Assignment 1.3.R4

The PowerPoint available on the link below considers the importance of cross-disciplinary research:

Study Questions

Indisputably, the purpose of research is to create new knowledge, but what is done with that knowledge? Respond to the questions below in the context of the articles you have read and the short video you have watched.

1. Researchers, in general, share their work only with colleagues and funders, and the research outcomes are typically not aimed at the public. Why does this kind of exclusivity occur, and what problems result from it?
2. Extending current research requires a thorough understanding of past research in the area as well as contemporary research efforts. Why is it important to put research in context, and what strategies can researchers use to do so? Are there any risks in the emphasis on context?
3. Research means narrow focus, not broader application. Is this statement true? What are the implications, positive and negative, of a narrow focus?
4. It is impossible to apply outcomes of a single research study on a society-wide scale. Is this the case? Why? Is it a good or a bad thing?
5. Research can be interdisciplinary. What are the strengths of cross-disciplinary research, and how can it proceed?
6. Techniques exist to record and communicate research outcomes to a public audience. What are these techniques, and how effective are they?

1.4 Forming a Research Topic or Question and Conducting a Literature Review

We estimate that this segment of Unit 1 should take about 5 hours (300 minutes) to complete.

Dr. W. P. Brinkman offers a course on Empirical Research Methods at the Delft University of Technology (see Brinkman, 2011). He distinguishes three different types of research questions.

Frequency (descriptive) questions

• How often do programmers take a break?

• What are the reasons for programmers to take a break?

  [Further examples]

• How usable do users find software application X?
• How sociable are Facebook users?
• How often do network servers crash?
• What are users’ views on 3D TV?

Difference questions

• Compared to computer artists, do programmers use computers for longer continuous intervals?

  [Further examples]

• Is there difference between male and female users in time spend online?
• Has culture an effect on a person’s preference of web page design?
• Has effectiveness of recognising emotion using computer face recognition technique influence by type of emotion expressed in the face?

Relationship questions

• Is there a relationship between the image quality of the focus of attention and the perceived overall quality of image?

  [Further examples]

• Is there a relationship between perceived usability of system X and computer skills of the user?
• Does an increase in perceived image quality coincide with an increase in compression time?
• Has the degree of presence a positive effect on effectiveness of VR treatment? (Brinkman, 2010a).

In essence, the research question should identify a property (e.g., programmers’ attitudes towards mood-support software) of the element, a property that can vary between elements of the population (e.g., the population of professional programmers) in a given situation or environment (e.g., when debugging or when working in groups).

As another example, a research question could investigate the causal relationship between the antecedent (e.g., attitude towards and social norms governing the design of a mobile phone) and the consequent (e.g., intention to select a mobile phone with a particular design).

Variables associated with the antecedent are called independent variables, and variables associated with the consequent are called dependent variables. A dependent variable changes as the independent variable changes.

An intervening variable “links the independent and dependent variables. In certain situations the relationship between an independent variable and a dependent variable cannot be established without the intervention of another variable” (Kumar, 2005, p. 61). For example, the research question “do high-definition monitors improve the effectiveness of virtual reality treatments,” could be influenced by an intervening variable we will call “virtual presence.” Thus, “high-definition monitor” causally relates to “virtual presence” and in turn “virtual presence” causally relates to “effectiveness of virtual reality treatments.”

Extraneous variables influence the strength of the relationship between the independent and the dependent variables. For instance, the extraneous variable, “gaming experience,” might influence the causal relation between “high-definition monitor” and the “effectiveness of virtual reality treatments.”

Note that, according to William Brinkman (2010b),

Causality requires at least three conditions:

Reading Assignment 1.4.R1

Read about dependent and independent variables in a source of your choice.

One possible source is

Read about statistical significance in a source of your choice.

One possible choice is

Read about research questions in the sources identified below:

Read about literature reviews in the sources identified below.

In particular, read about meta-analysis:

Note: If you are planning to write an essay rather than completing a thesis or a project, you may also wish to watch the video clip identified below:

Universities, societies, and professional associations have extensive libraries, including books, magazines and journals. IT professionals have long used these resources to further their research and development activities. More recently, many of these libraries have moved online. Initially, only the catalogues were available, but now more and more resources have been converted to digital format for online access.

In the beginning, special terminals or local wired connections were required to access the digital information, but now online access to most such databases is available through the Internet, making the information available to anyone in the world with an Internet connection.

Athabasca University has an extensive online library, as befits a distance learning institution. The list of journals in all areas of the arts and sciences is comprehensive. For the CS graduate student, the number of IT journals and magazines is also extensive. Most of these journals and magazines are available online.

Two of the more important societies for IT professionals are the Institute of Electrical and Electronics Engineers (IEEE) and the Association for Computing Machinery (ACM). These two organizations represent much of the “state of the art"” in computing science today, and the many journals published by these organizations are essential tools in the IT professional’s toolkit.

Practice Assignment 1.4.P1

In the mid-nineties, Lycos was one of the preferred search engines. Today there are numerous search engines, of varying effectiveness, but the most popular search engine currently in use is Google™.

In addition to the dedicated search engines, thousands of web pages now offer “portals” to the Internet, featuring search engines that will examine either the local site or the entire Internet. These portal engines are often “powered by Google” or by some other mainstream search engine. For example, the Java site features a search engine that will search the entire Java site.

Most people choose a search engine and then stick with it as long as it continues to deliver valuable links to queries without returning too much “junk.” The biggest problem with web searches is that they tend to return a lot of unrelated pages, as well as pages that no longer exist. This problem is a result both of the mechanism used to gather sites, and of the mechanism used to store and index those sites.

Pick a phrase that relates to your research. Try to choose your phrase so that is not too generic (e.g., “computer programming” or “object oriented” will return millions of hits). Choose any two web search engines (i.e., Google and one other). Enter your phrase and compare the “top 10” pages returned by each engine. Try putting double quotes around the phrase and compare the results to the first search. Explore the “advanced search” features of the two search engines.

Assignment 1

Review Assignment 1 on the course home page, and do whatever you can at this point.

1.5 References