Image
banner_trends

Download the July 2021 Report

By Donald L. Rubin, PhD, and Leah Mason, EdD

In this edition of Trends & Insights, we explore how international educators can best obtain the information necessary to bolster the future of U.S. education abroad programs following this moment of historic regrouping and revisioning. Our answer is straightforward: Compile and use institutional data already residing on campus to tell compelling stories about the impact of education abroad. This can be accomplished by building capacity and collaborating with data-handling units on campus.

Robust Data Collection in a Time of Change

For many stakeholders at most U.S. campuses, the case for education abroad primarily relies on personal testimonials, anecdotes, and case studies that speak to the benefits of studying abroad. Such informal evidence is easily grasped and can be inspiring for students. But is it sufficient for promoting education abroad in an increasingly challenging landscape for higher education in general and for international education in particular? More convincing are the sorts of larger scale findings recently reported by the Institute of International Education (IIE; Farrugia and Sanger 2017) and NAFSA and Emsi (2020) that offer evidence of how skills developed during education abroad contribute to postgraduate employability.

Our experience implementing two large, federally funded projects about education abroad analyzed hard data among stakeholders—campus executive leadership, faculty curriculum committees, academic advisers, employers, and parents, as well as students—and revealed a strong appetite for rigorously collected data. For example, senior campus leadership wants to know the track record of education abroad in promoting campuswide priorities such as equity of participation and learning outcomes for diverse subgroups. Families want assurance that when they invest in their child’s education abroad, those funds are well spent with respect to timely college completion.

Large-Scale Data Collection: CASSIE and GLO

The two programs on which we base our recommendations were funded from 2017 to 2020 by the U.S. Department of Education’s International and Foreign Language Education Office.

The Consortium for Analysis of Student Success Through International Education (CASSIE) is housed at the University System of Georgia. CASSIE collected detailed student-level data from more than 220,000 undergraduates from 41 U.S. institutions. More than 30,000 of those students had studied abroad. Considering a broad slate of student background variables—such as high school grade point average, financial aid status, and gender—CASSIE found a 6.2 percentage point advantage in four-year degree completion for baccalaureate students who have studied abroad compared with those who have not. For underrepresented minority students, the advantage was nearly double that: 11.6 percentage points.

The Graduate Learning Overseas (GLO) project was conducted by IIE. It measured the scale and scope of U.S. graduate students’ educational activities abroad and sought to understand institutional data collection practices, motivations, and challenges. From aggregate institution-level data on approximately 800,000 enrolled graduate students from 205 U.S. institutions, IIE found that 3.4 percent of all students enrolled in U.S. graduate degree programs participated in an overseas learning activity in the 2016–17 academic year. More than half of all overseas learning was through experiential activities that may or may not have been credit-bearing, such as research and fieldwork, travel seminars and study tours, work, co-op or internships, volunteering, and service learning. While these data show there is room to expand graduate learning opportunities overseas, they also point to a need for improved data collection around the overseas activities that graduate students are engaged in but are not monitored by a department or institution.

Data Collection for CASSIE and GLO

Data collection for CASSIE and GLO was a significant endeavor for many institutions. Though international education champions at more than 100 U.S. institutions expressed initial interest to participate in CASSIE, in the end, only 18 institutions in addition to the public colleges in Georgia (which were automatically enrolled in CASSIE if they had education abroad programs) were able to complete the data compilation.

The GLO survey documented similar challenges from institutions regarding their ability to report data on graduate learning overseas and their confidence in the reported data accuracy. The most significant challenges in collecting GLO data were institutions relying on students to self-report information and institutional offices not maintaining records on student activities that do not count for academic credit. Although more than half of all graduate students participated in an experiential learning activity overseas, less than half of responding institutions felt they had most or all of the data on students engaged in these activity types. As a result, institutions may be missing a significant amount of data on noncredit learning activities because there is no recordkeeping in place for such information.

Barriers and Facilitators to Large-Scale Data Collection on Campuses

During the course of these two projects, we observed common barriers to collecting education abroad data so that they could be integrated with other student information. We also discovered promising practices for facilitating data collection on campuses.

Data Management

Barrier: Decentralized and unconnected data sources. Merging student-level data from multiple sources is a heavy lift. Important data come not only from education abroad but also admissions, financial aid, and the registrar. Often crucial information was housed on separate databases that could not easily “talk” to each other. On a substantial number of campuses, even education abroad information was scattered among different offices.

Facilitator: Existing data warehouse. Some campuses maintain sophisticated data warehouses, making it relatively easy to query comprehensive databases and thus compile complex student-level or aggregate data sets. Evolving a campus data warehouse is a matter of IT policy and resource allocation, so it is important to sit down with a data scientist, IT personnel, or other knowledgeable sources to identify data points and variables to collect, where they are in the system, and how to create the desired reports. If a campus currently lacks such a resource, the education abroad office should make sure that the campus’s chief information officer knows it is a stakeholder that needs a seat on any planning committee.

Intracampus Collaboration

Barrier: Under-resourced institutional research operations. Although institutional research is a vital part of planning and assessment at most institutions, support for this work is not universal within higher education and varies by institution size and type. Institutional research functions are often limited to the minimum reporting requirements or operating over capacity. Additional bandwidth may not be available for special projects like assessing education abroad outcomes.

Facilitator: Existing research or operational collaborations between institutional research and international education offices. Building a data collection team across campus units has the potential to positively affect an institution’s ability to collect comprehensive education abroad data (see the “GLO Toolkit” for details). Collaborations could range from superficial and sporadic to ongoing and innovative. For example, institutions reported regular collaboration on accreditation reporting every 7-10 years, but less common collaboration on strategic initiatives such as student recruitment, inclusion, student success, and financial need. Other collaborations supported efforts such as an education abroad database, data dashboard, and class annotations in student records.

Education Abroad Office Operational Capacity

Barrier: Aspirational education abroad programs. Some institutions are in the early stages of building and professionalizing education abroad on their campuses. As CASSIE was recruiting institutions, quite a few campuses that initially expressed interest dropped out because they simply lacked a history of strong education abroad enrollment, and therefore sample sizes were too small to sustain statistical analysis. Colleges and universities with historically small education abroad enrollments, however, are the very institutions that need to be thinking prospectively about data management.

Facilitator: Education abroad faculty and staff who champion data-intensive activities. Dedicated education abroad professionals recognize the value of collecting and reporting on education abroad data to benefit the broader international education field. Providing education abroad professionals with research training and support—alongside hiring individuals with strong backgrounds in data collection and management—will build capacity for international education offices to use data to tell compelling stories about impact. Among GLO survey respondents, two-thirds of the education abroad offices engaged with other offices at their institution—borrowing data management capacity among those collaborators—to complete the survey.

Institutional Priorities

Barrier: Low priority of outcomes selected for investigation. On some campuses, graduation rates are so high that college completion is not a major research priority. Therefore a project, like CASSIE, that focuses on timely degree completion was of little interest to stakeholders outside international education at those sites. Yet CASSIE’s model of collecting and analyzing rich data to approximate explanatory power remains useful across all sorts of outcomes that may have more currency on campus. For example, one CASSIE institution extended its focus to examine the impact of education abroad on postgraduation employment. By appending data from the college’s career services database to the existing CASSIE data set, administrators can parse out the contribution of studying abroad to employment rates and starting salaries.

Facilitator: Alignment with leading campus priorities. Collecting data for CASSIE and GLO often needed approval at executive levels. Campus administrators interested in participating in these projects obtained the most enthusiastic endorsement when connected to campuswide strategic initiatives. Institutions that explicitly supported global learning goals were inclined to support these data collection efforts, but when research teams linked projects to other equally high-priority strategic initiatives at an institution, such as enhancing learning opportunities for diverse students or linking to the college completion agenda, campus collaboration was more likely to fall into place.

Quality Assurance

Barrier: Uneven data quality and sensitive data. All large-scale data research efforts must contend with missing or miscoded data. When there is a sufficient quantity of useful data, some degree of missing data is manageable. However, some institutions discovered large swaths of missing or implausible values for key variables. For the GLO survey, this was notable around noncredit-bearing learning activities overseas for which institutions did not maintain records or relied on student self-reporting. Furthermore, when institutions reported not having a centralized office or platform for collecting data, they also did not have confidence in their data’s completeness.

Institutions were also concerned about the sensitive nature of data in terms of confidentiality and institutional comparisons. Among the pieces of information requested by CASSIE were financial aid data. Because the data request did not violate federal regulations, most institutions were willing to provide at least some financial aid information. However, data safety assurances about protecting individual students’ anonymity and masking the identity of institutions did not alleviate institutions’ concerns around the risk of potentially unfavorable comparisons with other institutions. In any inter-institutional enterprise, participating colleges will be sensitive about what external benchmarks will be applied to interpret campus-specific results.

Facilitator: Research or IT professionals embedded in international education. International education operations require routine IT services such as website design and management, and often these services comprise a full-time position. A handful of campuses also allocate resources for additional data support to create complex reports and assist in special research projects. Such individuals might, for example, interface with institutional research offices to establish a two-way flow of information between student information systems and international education databases.

Conclusion

As the field of education abroad continues to navigate an era of accountability with a fresh lens applied to risk management, it is more important to establish, refine, and expand data collection and assessment practices. We emphasize the need to build capacity and collaboration among the data-handling units on college and university campuses. In this way, institutions can show the impact of education abroad.

Institutional support for education abroad professionals is key to ensuring they are equipped with the knowledge, skills, and resources necessary to collect and report on education abroad data as well as complementary data collected at their institutions. To set the next generation of education abroad professionals up for success, institutions must consider ways to leverage existing technology, infrastructure, and systems at their institution.

No matter an institution’s starting point, using the existing data to start telling a story is critical for building buy-in and support from stakeholders including students, families, alumni, and campus curriculum and fiscal decision-makers. Aligning data analysis projects with campus priorities, building capacity for data management and analysis within education abroad offices, and especially building partnerships with campus data gatekeepers are all crucial ingredients for promoting education abroad in an evidence-based fashion. Using such evidence to tell the story of education abroad’s impact will help ensure a robust recovery and resurgence following the recent interlude in higher education.

 

References

Emsi and NAFSA. 2020. Developing a Globally Competitive Workforce Through Study Abroad. Washington, DC: NAFSA: Association of International Educators. https://www.nafsa.org/policy-and-advocacy/policy-resources/developing-globally-competitive-workforce-through-study-abroad.

Farrugia, Christine, and Jodi Sanger. 2017. Gaining an Employment Edge: The Impact of Study Abroad on 21st Century Skills & Career Prospects in the United States. New York, NY: Institute of International Education. https://www.iie.org/Research-and-Insights/Publications/Gaining-an-Employment-Edge---The-Impact-of-Study-Abroad.

Institute of International Education. 2021. “Graduate Learning Overseas (GLO).” https://www.iie.org/glo.

Mason, Leah, and Jodi Sanger. 2020. “GLO Toolkit.” Institute of International Education. https://www.iie.org/Research-and-Insights/Graduate-Learning-Overseas/GLO-Toolkit.

University System of Georgia. 2021. “The Consortium for Analysis of Student Success Through International Education (CASSIE).” https://www.usg.edu/cassie.


Don Rubin, PhD, is professor emeritus of communication studies, language and literacy education, and linguistics at the University of Georgia. He teaches intercultural communication, especially in health contexts, and much of his research examines “reverse linguistic stereotyping” of nonmainstream speakers of English. Rubin served as coprimary investigator of CASSIE and as research director for the Georgia Learning of Students Studying Abroad Initiative (GLOSSARI).

Leah Mason, EdD, is research lead at IIE and adjunct professor at Georgetown University. She oversees various multiyear research initiatives, leading IIE’s Project Atlas, the GLO project, and the American Institute for Foreign Study Foundation and IIE’s Global Education Research Reports series and supporting the Open Doors report. Mason’s research focuses on student mobility and its interaction with career readiness.


Authors' note: Partial support for this article was provided by International Research Study grants from the U.S. Department of Education, Office of Postsecondary Education, International and Foreign Language Education division (#P017A170020 to the University System of Georgia, Angela Bell, PI; and #P017A170039 to the Institute of International Education, Leah Mason, PI). The views expressed here are solely those of the authors. The authors acknowledge key contributions of team members Angela Bell, PhD; Rachana Bhatt, PhD; Coryn Shiflet, MA; and Leslie Hodges, PhD (CASSIE); as well as Jodi Sanger, MA (GLO).