Download PDF: Integrating Research and Education

Editors Note: Dave Mogk provides and extensive reflection on his experienes with NSDL. We have excerpted his essay to include the major lessons learned. Please read the PDF for the full richness of Dave’s story.

Introduction

The National Science Digital Library (NSDL) has the potential to be the premier agent of dissemination for instructional purposes the exciting research results that are supported by the disciplinary directorates of the National Science Foundation (NSF). Integrating research and education has long been an important priority of NSF’s mission to support “People, Tools, and Ideas”, and translation of scientific results into instructional practice is increasingly used as evidence of NSF’s “Broader Impacts” review criterion. Digital libraries provide an ideal environment to support the processes of discovery and inquiry that can make Science come alive for learners at all levels and in formal (K-16) and informal (for the interested and inquiring public) instructional settings. The NSDL can play an essential role in NSF’s mission by providing collections and services that directly link scientific results, data and data products, background information on scientific principles and methods, pedagogic strategies, instructional materials, teaching tips, assessments, and human resource development opportunities for students and instructors. Through contributing projects to the NSDL, the DLESE Community Services (DCS) and Microbial Life Educational Resources (MLER) projects, we have experimented with numerous formats to demonstrate ways in which integrating research and education can be achieved in a digital library environment.

Overarching Lessons Learned

The following observations are made from the perspective of one of the co-authors of Pathways to Progress (Manduca, McMartin and Mogk, 2001), co-chair of the initial NSDL Community, Education and Pedagogy committee, co-PI of the original Core Integration Services grant, and co-PI of the MLER collections project.

1) A centralized metadata repository is not as effective for use by self-defined communities of users as specialized portals designed and built to address targeted community expectations, needs, and standards. Current user patterns do not indicate that users go to a central portal and then auger in through many layers and linkages to find what they need. Almost all of our users come into our webpages via searches via the open WWW, and they find our sites based on the metadata about each site that we expose to facilitate search and discovery. We receive less than 1% of the referrals to our site from the NSDL portal. Nor has the development of the “Pathways” projects in the past couple of years improved this situation. We developed the MLER project prior to the advent of the Pathways, but to date have not had these resources harvested by the Biological Sciences Pathway. The issue revolves around similar, but not identical, metadata structures, and the cross-walks needed to access the MLER resources have not been developed. Similarly, the MLER and DCS projects contain content that could be of interest to other Pathway projects (e.g. Materials Science) or other digital library projects (e.g. ReciprocalNet, Analytical Sciences). We have built direct links from our site to these other projects, but through the NSDL infrastructure there is no centralized function that helps direct users in one domain to find collections of related resources in other domains. So, in a “Google” world, one has to ask “What is the value of a centralized metadata repository”? And, the original vision of “The NSDL mak(ing) a substantive contribution towards bridging current disciplinary boundaries by effectively integrating concepts, knowledge and methods across the SMET disciplines” (Pathways to Progress, 2001) is far from realized. Integration and repurposing of related resources from the numerous contributing projects is still a long way off. There are a lot of great digital library projects that have been developed; it’s not at all clear that these projects have come together synergistically into a unified NSDL.

2) Learning resources must be placed in the full context of scientific principles and results, pedagogic practice, teaching activities, assessments, and opportunities for personal development. A huge benefit of digital library technology is found in the ability to allow users to search and browse according to their own interests and abilities. The ability to navigate “vertically” to discover richer material within a topic, and “horizontally” between related topics (either from “sister” disciplines, or linking Science with pedagogy) adds value to the resources themselves and to the experience of the users.

3) Building thematic collections of resources is useful to a point, but instructors are most interested in having access to instructional resources that they can readily download to adopt or adapt for immediate classroom use. Instructional digital libraries can provide a tremendous service by linking scientific principles, methods and content knowledge with recommended pedagogic strategies, teaching activities, and assessments. Reviewing services are also highly valued by educators. Consequently, instructional digital libraries should be developed in close association with curricular development projects (e.g. NSF-CCLI). One area where the NSDL could be more proactive is in support of curriculum development projects by providing technical guidelines and assistance to help make the products of these projects “digital library-ready”, i.e. including the full complement of information needed to rapidly (and automatically?) create required metadata records.

4) Digital libraries can be used to help build and nurture communities of learning, and can also effectively tap these communities for contributions and reviewing of these resources. We have successfully experimented with community-building of digital resource collections (i.e. the ‘barn-raising’ model) in a number of topical areas. This model can be expanded by the NSDL in general to proactively seek out groups who have yet to participate in the NSDL, or who are underrepresented in the current collections, to help them build thematic collections to meet their immediate interests. These efforts can gain value by drawing from, and contributing to, existing collections from related disciplines. This is in accord with the early vision of the NSDL as a “federated” network, with the common goal of supporting excellence in STEM education and adhering to a set of technical standards for interoperability, but with enough autonomy to enable each community to grow its own collections and services as dictated by community standards, practice and expectations.

5) Digital library technology can optimize the use of resources by diverse audiences by facilitating repurposing and enabling bi-lateral links between closely related collections of resources. Once a user enters one of our digital collections, we attempt to keep their interest by either a) embedding direct links to related information, or by b) providing explicit “Related Links” to direct users to other sites or modules where information that will likely be of interest (e.g., Fox et al., 2005). These hard-wired related links have paid great dividends to us in terms of developing a loyal user base, and also for the users who are exposed to new information that should or could be of interest. This is another area where the NSDL could be more effective: helping PIs establish these bi-lateral links among the many NSDL projects. The NSDL Annual Meeting has been partially effective in providing a forum to introduce PIs and their projects to each other via poster sessions, discussion sessions, and oral presentations. The central portal is less effective in this regard. As an example, to learn something about thermodynamics it is not very effective to start at the main portal, and then explore resources in chemistry, back out and dive in again in physics, and then engineering, and then Earth science….whereas treatment of thermodynamic properties could easily be explored side by side with examples from each of these disciplines using bilateral links.

6) Digital libraries can also be used to effectively create collections of non-digital resources (e.g. links to instrumentation and facilities), to make available the “gray literature” of a field, and to capture the experience and advice of a community that is not readily available through other media. The concept of digital resources can be expanded to include instruments, literature, people, and ideas (not formally represented in print or web media) by creating appropriate metadata that makes these “learning objects” discoverable through a digital library environment.

Summary

The NSDL is well-positioned to facilitate integration of research and education in support of NSF’s overall mission. The unique navigational capabilities provided by the digital library environment allows connections to be made directly between scientific content and pedagogic practice, placing research outcomes and instructional materials in close proximity for use by instructors and learners. Digital libraries provide a great environment to promote learning through inquiry and discovery, and can readily transcend disciplinary boundaries by facilitating repurposing and establishment of bi-lateral links between related bodies of knowledge. The most important aspect of digital library collections and services is to place learning resources in contexts that are useful to targeted communities of users. In this regard, a variety of approaches to creation of thematic collections can be used to address the interests, needs and expectations of different learning communities. The concept of a collection in a digital library should be expanded to include instructional activities, web-based resources, data, tools, data products, research methods, non-digital learning resources, community experience (hard-to-find print materials and otherwise undocumented “common wisdom”), and information about people, places and things. The NSDL can play an essential role in support of NSF’s mission by providing dissemination services to help researchers funded by the disciplinary directorates to address the broader impacts of their work, and at the same time, infuse exciting new scientific methods and results into STEM education at all levels and for all audiences.

Download PDF: An NSDL Retrospective: The Case of the Instructional Architect

Introduction

This retrospective essay covers the period from 2001-2008, during which the research group at Utah State University (USU) focused on designing, developing, and evaluating a National Science Digital Library (NSDL.org) web-based service, called the Instructional Architect (IA.usu.edu). Later in this period, the focus was on disseminating the IA service in school contexts by developing and implementing formal and informal teacher professional development opportunities. These efforts have been funded by a series of National Science Foundations grants.

This essay is presented as three sections. In the first section, we describe our efforts to build a simple software system, the Instructional Architect, deploy it with users, and integrate it with the NSDL core technical infrastructure. In the second section, we describe our efforts to better understand the target context of educators, and to develop sustainable and scalable teacher professional development models. The final section reflects on how the IA fit within the NSDL program. Each section also includes a subsection describing evaluation strategies.

This essay also reflects shifts in our thinking over this period. Early efforts reflected a kind of technological determinism (i.e., ‘if we build it, they will come’). This eventually shifted to a more socio-technical approach. An unspoken assumption of early work was that teachers and their students would access and use such technologies in unproblematic and seamless ways. Unfortunately, the history of educational technology suggests that this is seldom the case (Cuban, 2001). Instead, after spending time with ‘real’ people (teachers and their students) in ‘real’ contexts (classrooms), it became clear that we needed to better understand the complex ways in which systems cross institutional boundaries (Agre, 2003).

The Instructional Architect

The Instructional Architect (IA) is an end-user authoring service designed to support the instructional use of online resources in the National Science Digital Library and on the Web. The IA enables users (particularly teachers) to discover, select, sequence, annotate, and reuse online learning resources stored in digital libraries to create instruction (e.g., lesson plans, study aids, homework – collectively called IA projects). In this way, the IA is intended to increase the utility of online learning resources for classroom educators (Recker, 2006).

Two Examples

We begin the description of the Instructional Architect with two examples created by teachers using our tool (see Figures 1 and 2). The foreground of each figure shows one of the teacher’s selected online resources. The background shows the output of using IA: a web page containing the content created by the teacher, consisting of activities and annotations for online resources (referred to by links). Note how the level of detail in the projects varies; the project in Figure 1, intended for middle-school students, provides detailed activities for the students, whereas the project in Figure 2 (intended for kindergarten students) seems to be more of a lesson plan sketch.

Figure 1: Screenshot of an IA project page aimed for middle-school students

Figure 2: Screenshot of an IA project page aimed for kindergarten students

As is apparent from the figures above, teacher-created projects are fairly simple. Teachers are not web designers, nor should we expect them to be. Instead, they are professionals attempting to efficiently and effectively address classroom and learning issues.

Indeed, much of the functionality of IA could be recreated with blog software coupled with a social bookmarking system. However, as previously noted, by following a user-centered design process, we believe the system better meets the basic requirements of teachers who wish to use digital library technology to quickly and easily meet classroom demands.

System Description

From the home page of the Instructional Architect, users can 1) browse projects, 2) register as a new user, or 3) login as a registered user or guest (with reduced functionality).

1. Browse. Users can access IA projects by performing keyword searches or by browsing these IA projects by subject area, grade level, author’s last name, or title (see Figure 3).
   

   Figure 3: Browse IA Projects

2. Register. Users can create a free account, which provides them secure access to their saved resources and IA projects.
3. Login. After the user logs in, the IA offers three major usage modes. First, with the ‘My Resources’ tool, users can search for resources in the NSDL Data Repository. Queries are sent to the NSDL REST-based search interface (Lagoze et al., 2006). Metadata records for matching resources are displayed to users in an abbreviated form (including title, author, brand, description, and date). After browsing these results and viewing resources, users can select desired resources for further use. Users can also select any Web resource including interactive and Web 2.0 content (such as RSS feeds and podcasts), and add it to their list of saved resources. Users can also organize their selected resources into folders (see Figure 4).
   

   Figure 4: My Resources

Second, with the ‘My Projects’ tool, users can create web pages in which they they select a look and feel for their project, input selected online resources, and provide accompanying text in order to create learning activities (called ‘IA projects’).

Finally, users can share their IA projects by ‘Publishing’ them and setting permissions on them, such as a) user-only view, b) users and their students (student view), or c) public view (anyone browsing the IA site). Users can also add basic metadata about their IA projects, including subject area, grade level, and core curriculum standard. These are then used to support browse and search of existing IA projects, as described above.

Evaluation Strategies

Early design and evaluation efforts (2001-2002) focused on measuring usability and utility, referred to as ‘developmental evaluation.’ This included a needs assessment and interface design and development of the IA. Each design cycle was followed by an evaluation that helped inform the design of the subsequent phases. Participants included graduate students as early testers, pre-service teachers, and expert teachers.

Methods included literature reviews, focus group interviews, and expert review of prototype interfaces, early testing by members of the target audience, and analysis of code changes by constituents. Early recommendations included a search tool, and combination tool, and a reflection tool. In addition a more in-depth case study approach was conducted with 8 in-service teachers in Utah. They provided input on how Internet resources were currently used in their teaching practice, and contributed to the needs assessment.

At that time, the design, development, and evaluation of our project were hampered by the fact that the NSDL was co-evolving with our project. This meant that technical standards were in flux, resulting in system instability. In addition, the library collections were simultaneously being seeded and grown, resulting in uneven and sometimes sparse holdings. The latter caused no small amount of frustration among our classroom teachers as they attempted to search for interesting and relevant learning resources. To address this problem, we worked with other educational digital libraries, including SMETE.org, DLESE.org, and the National Library of Virtual Manipulatives (nlvm.usu.edu) to devise means to query their metadata. We were able to greatly benefit from the maturity of these projects.

In 2005-2006, as the NSDL gained maturity, our project worked on tighter technical integration with the NSDL Core Technical Integration. This included queries of the NSDL search service, pilot implementation of community sign-on (CSO) via Shibboleth, and co-branding. At the same, however, the NSDL as a whole, seems to suffer a bit from ‘wheel reinvention.’ For example, many projects are developing tools with similar functionality to the IA. Partly due to the ‘not invented here’ syndrome, projects wanted functionality that differs slightly from what the IA provides. Hence, they found it easier to simple built their own. In general, the NSDL as a whole needs to consider strategies that avoid ‘tool silos’.