Reflections on Data Collection with an Eye Toward the Future

by Jim Buschman

From comments made at the Council on International Educational Exchange (CIEE) Conference in Chicago, IL, Nov. 11, 1999

The activities accomplished to date in study abroad-related data collection, while extremely important to us all, represent just a small portion of the possibilities for collecting data and answering key questions about our field. The following observations are intended to help us think a bit more widely on the nature of data and how collecting it can ultimately assist us in our professional work.

1. Data collection may be seen as a series of questions and answers. Data do not just jump out and cry, "Analyze me!" First we must decide what we want to know. Then we frame the questions in such a way as to shed light on what we want to know. In order for IIE to collect the study abroad data reported in Open Doors, someone had first to decide, from the universe of all that could be asked, exactly what questions would be asked.
   
   
2. The answers we get depend on the questions we ask. This seems obvious, but the point is that just because we can ask a question and get a fairly reliable answer, this does not mean that the question was good in the first place. "How many students buy Eurailpasses?" is a question that can, through several means, be reliably answered. It may even be a good question, but only if it can be useful to some segment of our enterprise.
   
   
3. Data can be what we collect as a part of our regular jobs, or what we collect because we want to learn something. We are not limited to that data that come in during the process of our preparing students, sending them, teaching them, and bringing them back. The many forms students must fill out for us provide opportunity to ask additional questions simply because the answers to those questions can help us. Or we can collect data in a manner other than the filling out of a form. Few of us have "research" as part of our job descriptions, and this is a limitation on the time we can spend in data collection. But we can be smarter about setting up structures that will bring in the data we need.
   
   
4. Data can be quantitative or qualitative. Practically all of what we have collected for Open Doors and the first IIE e-mail survey is quantitative. The answer is a number. The advantages of numerical data are, among others, that we can cast the net widely and collect results from schools across the country; that we can perform tests of significance and other statistical marvels on them; and that results from one year can be easily compared to results from other years. But there are many valid and important questions that cannot easily be answered quantitatively. We know anecdotally, for example, that our students in places like France and Italy encounter societal practices regarding alcohol use that are different from our own norms. What happens when this occurs? A questionnaire might tell us something, but so might a participant observer who gets to know a group of students over time and documents their experiences. This is fieldwork of the sort done by Margaret Mead, and it has been a part of educational research for at least the last 20 years. It lacks the advantages I just mentioned for quantitative research, but it is wonderful for documenting what happens in a particular context, how it happens, and what participants really believe about what happens.
   
   
5. Sometimes we do not need answers as much as we need better questions. This is, in part, a plea for more fieldwork research in study abroad, since such research can often yield better questions that can in turn drive the collection of more valid quantitative data. A program site would be a good environment for a doctoral dissertation or an anthropology class assignment. Such work might lead us to restructure the ways we collect data on students, with a consequent improvement in what we learn about our students.
   
   
6. The word "data" is a plural noun. "Data are," not "data is."