Recently, an article titled NSDL Rethinks It’s Digital Library was published in Science that discussed the state of the NSDL and described some of the ‘rethinking’ that has been going on regarding it. While this article has certainly opened up the discussion, it is limited.

As part of the NSDL Reflections project we are inviting you to help us extend and expand that discussion.

We invite you to respond to the question:

How do you see the history of the NSDL and its future differently from that described in the article?

There are a multitude of sub-questions branching out from the main one, and we invite you to examine any and all that you consider important.

Download PDF: Reflections on the NSDL

Introduction

This essay is a reflection on my involvement in the NSDL, which I was lucky enough to be involved with from the beginning. My interests are in using online resources to improve chemical education at both the college and high school level, and this work has led to our current NSDL project, the ChemCollective (www.chemcollective.org).

The NSDL has provided an inspiring home for me as a developer. The structure, including especially in-person meetings and workshops, has created a community of like-minded individuals who have educated me and helped guide my work for the past eight years. In this sense, I believe the NSDL project is an unusually successful NSF research program. Working together on a grand challenge, that of creating a national library, provides a structure that encourages engagement among the participants that is far more substantive than the interactions arising in programs built only around a competitive funding model. These interactions have strongly benefited me as a developer. But this is not the only way to perceive the NSDL. At the NSDL kickoff meeting, an attendee who had spent time in the software industry commented “this is an Internet startup company without a CEO or CIO”. This comment highlights the nature of the NSDL as a coalition of projects. The benefits of the NSDL arise primarily from the value added to those projects, and for me, the added benefit has stemmed primarily from interactions with the NSDL community.

Transformation of Learning and Speed of Change

I firmly believe that the Internet will radically transform the way people learn. I also firmly believe that I do not know how this transformation will occur or what education will look like in ten or twenty years. It is especially difficult to predict which technologies and constructs will have the most lasting impact, beyond saying that the current choices will at most be forerunners to what eventually succeeds. The technological artifacts created by the NSDL may indeed end up being important forerunners. However, the most lasting impact will likely be through the people it inspired. Years from now, when I download the best-selling book “How the Internet transformed education” onto my digital book reader, my involvement in the NSDL will mean that I know some of the main characters in that history book. For me, meeting these dedicated and talented people is by far the largest impact that involvement in the NSDL has had on me and my work. The cross-disciplinary of the project was key to this impact. I have been involved in many cross-disciplinary projects, but the NSDL is unique in bringing me into substantive dialogue with a diverse group of people interested specifically in using network technology to advance education.

A supportive developer community is especially helpful given the challenges facing the NSDL. Many of these challenges arise from juxtaposing the formal education system, which is shockingly resistant to change, with the Internet, which is shockingly able to undergo radical transformations on a moments notice. Life at the interface of these differently-paced worlds can instill a professional version of manic depression. Ideas intended to radically improve education most often end up having incremental impacts. Our hopes to change the world hit against stark realities and end up pushing the boundaries by only a fraction of our initial hopes. An idealistic community, such as the NSDL, is what is needed to inspire individual projects to continue to throw themselves against the wall in hopes of pushing it back further and further. Such an incremental pace is to be expected in research. When it comes to curing cancer or developing nanotechnology, the research enterprise is structured around the anticipation that ultimate success will mostly come as the culmination of small advances. But the rapid transformations we sometimes see on the Internet, with sites growing from initial concept to household words in a few years, builds up hope for similarly rapid advances in education.

An equally challenging aspect of life at the interface of education and technology is the speed at which the Internet transforms. The shared experience provided by involvement in the NSDL has helped me learn how to focus my own efforts on projects that are likely to have impact when they are completed in two or three years, and the Internet landscape has once again been redrawn. In particular, some user needs appear to be education specific on the surface, but turn out to be specific instances of broader issues such as the need for improved communication, search, or rapid software application development. Attempting to meet such broad needs will likely lead to a race with much larger entities, such as the for-profit sector. To avoiding such counterproductive races, I try to put all my ideas through a litmus test for educational specificity. This has shifted my development efforts to projects with a strong focus on the chemical education community.

Download PDF: An NSDL Retrospective: The Case of the Instructional Architect

Introduction

This retrospective essay covers the period from 2001-2008, during which the research group at Utah State University (USU) focused on designing, developing, and evaluating a National Science Digital Library (NSDL.org) web-based service, called the Instructional Architect (IA.usu.edu). Later in this period, the focus was on disseminating the IA service in school contexts by developing and implementing formal and informal teacher professional development opportunities. These efforts have been funded by a series of National Science Foundations grants.

This essay is presented as three sections. In the first section, we describe our efforts to build a simple software system, the Instructional Architect, deploy it with users, and integrate it with the NSDL core technical infrastructure. In the second section, we describe our efforts to better understand the target context of educators, and to develop sustainable and scalable teacher professional development models. The final section reflects on how the IA fit within the NSDL program. Each section also includes a subsection describing evaluation strategies.

This essay also reflects shifts in our thinking over this period. Early efforts reflected a kind of technological determinism (i.e., ‘if we build it, they will come’). This eventually shifted to a more socio-technical approach. An unspoken assumption of early work was that teachers and their students would access and use such technologies in unproblematic and seamless ways. Unfortunately, the history of educational technology suggests that this is seldom the case (Cuban, 2001). Instead, after spending time with ‘real’ people (teachers and their students) in ‘real’ contexts (classrooms), it became clear that we needed to better understand the complex ways in which systems cross institutional boundaries (Agre, 2003).

The Instructional Architect

The Instructional Architect (IA) is an end-user authoring service designed to support the instructional use of online resources in the National Science Digital Library and on the Web. The IA enables users (particularly teachers) to discover, select, sequence, annotate, and reuse online learning resources stored in digital libraries to create instruction (e.g., lesson plans, study aids, homework – collectively called IA projects). In this way, the IA is intended to increase the utility of online learning resources for classroom educators (Recker, 2006).

Two Examples

We begin the description of the Instructional Architect with two examples created by teachers using our tool (see Figures 1 and 2). The foreground of each figure shows one of the teacher’s selected online resources. The background shows the output of using IA: a web page containing the content created by the teacher, consisting of activities and annotations for online resources (referred to by links). Note how the level of detail in the projects varies; the project in Figure 1, intended for middle-school students, provides detailed activities for the students, whereas the project in Figure 2 (intended for kindergarten students) seems to be more of a lesson plan sketch.

Figure 1: Screenshot of an IA project page aimed for middle-school students

Figure 2: Screenshot of an IA project page aimed for kindergarten students

As is apparent from the figures above, teacher-created projects are fairly simple. Teachers are not web designers, nor should we expect them to be. Instead, they are professionals attempting to efficiently and effectively address classroom and learning issues.

Indeed, much of the functionality of IA could be recreated with blog software coupled with a social bookmarking system. However, as previously noted, by following a user-centered design process, we believe the system better meets the basic requirements of teachers who wish to use digital library technology to quickly and easily meet classroom demands.

System Description

From the home page of the Instructional Architect, users can 1) browse projects, 2) register as a new user, or 3) login as a registered user or guest (with reduced functionality).

1. Browse. Users can access IA projects by performing keyword searches or by browsing these IA projects by subject area, grade level, author’s last name, or title (see Figure 3).
   

   Figure 3: Browse IA Projects

2. Register. Users can create a free account, which provides them secure access to their saved resources and IA projects.
3. Login. After the user logs in, the IA offers three major usage modes. First, with the ‘My Resources’ tool, users can search for resources in the NSDL Data Repository. Queries are sent to the NSDL REST-based search interface (Lagoze et al., 2006). Metadata records for matching resources are displayed to users in an abbreviated form (including title, author, brand, description, and date). After browsing these results and viewing resources, users can select desired resources for further use. Users can also select any Web resource including interactive and Web 2.0 content (such as RSS feeds and podcasts), and add it to their list of saved resources. Users can also organize their selected resources into folders (see Figure 4).
   

   Figure 4: My Resources

Second, with the ‘My Projects’ tool, users can create web pages in which they they select a look and feel for their project, input selected online resources, and provide accompanying text in order to create learning activities (called ‘IA projects’).

Finally, users can share their IA projects by ‘Publishing’ them and setting permissions on them, such as a) user-only view, b) users and their students (student view), or c) public view (anyone browsing the IA site). Users can also add basic metadata about their IA projects, including subject area, grade level, and core curriculum standard. These are then used to support browse and search of existing IA projects, as described above.

Evaluation Strategies

Early design and evaluation efforts (2001-2002) focused on measuring usability and utility, referred to as ‘developmental evaluation.’ This included a needs assessment and interface design and development of the IA. Each design cycle was followed by an evaluation that helped inform the design of the subsequent phases. Participants included graduate students as early testers, pre-service teachers, and expert teachers.

Methods included literature reviews, focus group interviews, and expert review of prototype interfaces, early testing by members of the target audience, and analysis of code changes by constituents. Early recommendations included a search tool, and combination tool, and a reflection tool. In addition a more in-depth case study approach was conducted with 8 in-service teachers in Utah. They provided input on how Internet resources were currently used in their teaching practice, and contributed to the needs assessment.

At that time, the design, development, and evaluation of our project were hampered by the fact that the NSDL was co-evolving with our project. This meant that technical standards were in flux, resulting in system instability. In addition, the library collections were simultaneously being seeded and grown, resulting in uneven and sometimes sparse holdings. The latter caused no small amount of frustration among our classroom teachers as they attempted to search for interesting and relevant learning resources. To address this problem, we worked with other educational digital libraries, including SMETE.org, DLESE.org, and the National Library of Virtual Manipulatives (nlvm.usu.edu) to devise means to query their metadata. We were able to greatly benefit from the maturity of these projects.

In 2005-2006, as the NSDL gained maturity, our project worked on tighter technical integration with the NSDL Core Technical Integration. This included queries of the NSDL search service, pilot implementation of community sign-on (CSO) via Shibboleth, and co-branding. At the same, however, the NSDL as a whole, seems to suffer a bit from ‘wheel reinvention.’ For example, many projects are developing tools with similar functionality to the IA. Partly due to the ‘not invented here’ syndrome, projects wanted functionality that differs slightly from what the IA provides. Hence, they found it easier to simple built their own. In general, the NSDL as a whole needs to consider strategies that avoid ‘tool silos’.

Download PDF: “NSDL-Style” Networks: Connecting Across Audiences & Disciplines

Introduction

I begin my personal reflection about NSDL with an analogy that I don’t mean to strain too much but it seems to me that Chicago-style pizza and “NSDL-style” networks share some key characteristics. Both are faithful to core elements of two staples which play a part of modern life in many areas of the world. However, like Chicago’s adaptation of pizza, NSDL-Style networking gains recognition in its own right because of its new and unique contributions to a standard fare in 21st century society (i.e., bringing together different “ingredients” to assemble a new style of networks). While multi-institutional, multidisciplinary networks are recognized as necessary components as we move into cyber-enabled STEM research and learning, NSDL introduced me to a new “flavor” (ok, I promise — I’ll cease…).

My views about “NSDL-style networks” are based upon my experiences and observations from participation in the Evaluation Committee and as a Pathway. In both arenas I’ve had the opportunity to meet and work with people from a very wide range of expertise that I likely wouldn’t have met, had it not been for my involvement in the NSDL.

Different Backgrounds and Different Approaches

It’s not unusual these days to be part of organizations where people from many different backgrounds come together. What seems unique to me is that NSDL embraces its heterogeneity as well as actively strives to ensure that both individual NSDL projects and the overall program benefit from it. In support of EHR’s mission of excellence, the NSDL seeks to bring together all areas, all levels, and all settings in STEM. As projects are welcomed into the NSDL, people with different expertise (e.g., computer science, domain sciences, education, informal learning, information sciences, learning sciences, outreach, publishing), funded in different tracks (e.g., Pathways, Integrative Services, Research, and Tools) are supportively encouraged to learn from each other and work together for greater impact. NSDL leadership has initiated many approaches (e.g., regular and special topic conference calls, “birds of a feather” gathering, mentoring) to identify and foster collaboration among the many distributed projects. NSDL’s common social, governance, and technical framework, brings together its projects representing complementary parts of the STEM community. Through the NSDL framework, these diverse participants cooperate, exchange successful practices, and, where similar concerns exist, together forge fresh ideas and joint efforts for benefits to the individual projects’ community, the STEM community, and the NSDL program overall.

I think people and organizations try new approaches because of need, opportunity, and leadership and so, NSF, the NSDL Program Director, and the Core Integration team (CI) have provided support and guidance to tackle needs and maximize opportunities as the NSDL program and its projects have evolved. In its role as a recognized global leader, NSF consistently sponsors and jumpstarts innovative and promising approaches to advance STEM research & education. NSF decision makers continually seek out efforts to bring research and education together in meaningful ways. Not only does such an approach help to prepare the next generation of scientists but also, it seems to me, to cultivate that synergy for potentially profound impacts on accelerating the development of new knowledge. The NSDL Program Director purposefully molded the program to include projects associated with the broad spectrum of the STEM community to work individually as well as collectively in order to serve STEM and the public good. As NSDL launched, the Core Integration team of UCAR, Cornell, and Columbia shaped their individual areas of effort (outreach, technical, and collection development) to construct an overarching framework to which individual projects could join, contribute, and gain.

To my mind, some end results gained from “NSDL-style networks” are that one develops a greater understanding of the complex, heterogeneous networks needed to conduct STEM research and learning in a cyber-global society, as well as a keen sense that no one group sees the whole picture. When people with different expertise work together, we can start to bring together our complementary strengths and contribute more effectively to this collective undertaking for new, validated forms of STEM research and learning.

My strongest impression of NSDL style networks comes through at its Annual Meetings. Separate from the location and the number of attendees, the meetings themselves convey enthusiasm, energy, and lots of discussions about efforts. Going from session to session, you’re sure to see new as well as old faces and projects – faces and projects that you wouldn’t normally encounter when you attend your “professional” national conference. You can count on running into this mix of people and projects from all parts of the NSDL because the session themes generally and intentionally cut across audiences and disciplines. It takes quite a while to understand the different vocabularies, approaches, and issues reflected by the diverse participants in the discussions. You likely can’t fully appreciate the concerns represented by others outside your audience and discipline, but hearing their perspectives helps you appreciate the complexities facing STEM communities. While it is plain that the questions are plentiful and growing, you take heart in the diversity and commitment of those pursuing some answers.

Download PDF: Collaboration, Alignment and Leadership

Introduction

This essay compares and contrasts—from my personal perspective—four projects funded by the National Science Foundation (NSF) that have depended critically upon medium- to large-scale collaboration. This characteristic distinguishes them from most NSF-funded projects, as typified by the Program Officer for one of the four: “The NSDL program is an unusual program for NSF in that its projects are engaged in building an enterprise much larger than the object of any one grant. Indeed, the success of the program rests squarely on the extent to which the many projects can embrace this collective sense of identity and mission.” [Zia 2001] Though similarly reliant on collaboration, the four projects had significantly different outcomes, and the purpose of comparison is to consider why, with particular emphasis on matters of leadership.

Four Highly Collaborative Projects

The NSF’s research and education portfolio includes many goals and objectives that can be achieved only through collaboration. This requirement can arise from joining multiple disciplines or from matters of scope and scale, as shown by four examples spanning nearly three decades (with dates approximately as indicated, ignoring their incubation phases).

• NSFNET (1984-1995) – NSF’s goal of a single internet, built on lessons learned with ARPANET, BITNET, CSNET and others, but yielding interoperable connectivity across all of academia (and, eventually, the entire non-academic world) [Leiner et al., 2000], required an extraordinary level of collaboration across:
  • Geographic regions as well as institutional and political boundaries.
  • Engineering and research, to create reliable infrastructure from innovation.
  • Established networks on which many constituents already were relying.
  • Public and private institutions, building commerce upon research results.
• Unidata (1983-Present) – In funding Unidata, the goal of the Atmospheric Sciences Division (ATM) was to help universities employ computing and communication technologies to access, display and analyze scientific data [Sherretz & Fulker 1988]. Envisaged as a merging of systems from several universities, this entailed modest collaboration from the outset, but Unidata evolved to be so participatory that it eventually was described as a “collaboratory” [Fulker et al., 1997] and cited as a “virtual community” [Rodrigues et al 2004]. This required collaboration among or across:
  • Institutional boundaries and large distances.
  • Engineering and research, to create reliable systems from university prototypes.
  • Builders of (incompatible) systems, on which constituents already were relying.
  • Researchers and educators, whose data interests are similar but distinct.
• DLESE (1999-2006) – The Geosciences Directorate (NSF/GEO), with assistance from NASA, embraced an ambitious agenda when it established the Digital Library for Earth System Education (DLESE): “… a program plan and structure for the establishment of a national digital library that promises to have a profound impact on geoscience education at all levels. … It will also provide an important vehicle for facilitating the implementation of the Earth and Space Science component of the National Science Education Standards.” [Mayhew 1999] This required collaboration among or across:
  • Institutional boundaries and large distances.
  • Engineering and research, to build reliable systems from prior research work.
  • Geoscience disciplines that, in some cases, had minimal common history.
  • Geoscience researchers, educational researchers and educators.
• NSDL (2000-Present) – The goal of the Education and Human Resources Directorate (NSF/EHR) in forming and funding NSDL was similar to that for DLESE [1] except for its larger scope: “The resulting virtual institution is expected to catalyze and support continual improvements in the quality of science, mathematics, engineering, and technology (SMET) education in both formal and informal settings.” [Zia 2001] Also similar to DLESE, but more explicitly, expectations for NSDL were informed by digital-library research initiatives that began in the early 1990s (under the Computer and Information Science and Engineering Directorate of NSF) [NRC 1998]. The NSDL initiative required collaboration across:
  • Institutional boundaries and large distances.
  • Engineering and research, to build reliable systems from prior research work.
  • Established libraries, digital and physical, on which constituents were relying.
  • SMET disciplines that, in some cases, had minimal common history.
  • Digital-library researchers, educational researchers and educators.
  • Public and private institutions, to evolve publishing practices.

A cursory placement of these four projects in James Austin’s engagement continuum (Figure 1.) indicates that all four rank high, i.e., the collaborators are deeply involved.

Figure 1: Austin Engagement Continuum

Figure 1: Reproduced [Austin 1999] with permission [BEING SOUGHT]. Each of the four projects analyzed here ranks high on several of the listed criteria. In general, project collaborators have integrative relationships, play roles of central importance to the mission, cope with relatively high managerial complexity, and aim for magnified social value.

[1] Though DLESE became one of the libraries that incorporated itself into NSDL, and received NSDL funds, this author sees the two libraries as distinct. Indeed, DLESE was functioning before most of NSDL and relied upon significant independent funding (primarily from the Geosciences Directorate at NSF).

Download PDF: Reflections on NSDL

The picture below is one that I’ve used in virtually every essay or presentation I’ve given about NSDL. As you can see, this essay is no exception.

Twin Jet Nebula

This picture captures three things about our initial vision of NSDL – our aspirations, truly astronomical in scope, and our two goals – making a substantive improvement in the quality of learning in the STEM disciplines and a substantive difference in the extent to which the very best learning reaches all students. The two goals together are what give the vision its character – there are many pockets of very high quality STEM learning and the reach of STEM learning is huge but sadly the reach of the very best STEM learning has been too limited.

“Facility” and “Library”

It is no secret that many of us are disappointed in the results of a project that at its beginning was so promising and had such huge potential. The shortcomings of NSDL can, for the most part, be traced to two related, foreseeable and, indeed, foreseen problems.

• When I was at NSF we had to avoid the use of the word “facility” and we had to work within NSF’s usual grant solicitation mode. Anything else would have lead in the best case to delays of ten or more years. But, NSDL, is very much a facility for research and education and building it through the usual grant solicitation mode without acknowledging that reality is like building a house through many individual grants – you might wind up with 5 beautiful kitchens but no bathrooms. And, that is exactly what happened. Moreover, grantsmanship is essentially competitive and one result was friction among some of NSDLs most enthusiastic and creative supporters.
• NSDL has never had a good sustainability plan. This stems, in part, from our use of the word “library.” The word has many wonderful connotations. Think of how many biographies begin with an ode to the lion statues on the steps of the New York City Public Library; think of the role libraries and librarians play in defending our rights to write and read the most controversial views on the most controversial topics; and think of the way that card catalogs and librarians have helped us find and choose among the millions of available volumes. Ultimately, however, the word “library” is misleading in two very important ways.
  • Many people believe that the word “library” is somehow related to the word “free” but libraries are expensive. When NSDL was first getting started I remember doing a quick calculation, dividing the budget of the Arlington Virginia Public Library by the population of Arlington. I don’t remember the result but through the magic of Google and the Internet I just found the FY 2007 budget of the city-funded Arlington, Texas public library system ($6,703,585) and the 2006 population of Arlington (364,300). A quick division yields $18.40. Post-secondary enrollment in the United States is roughly 17,500,000, so a comparable budget might be on the order of $300,000,000. More significantly, the production of most of the books found in libraries is supported by bookstore sales. This is significant for two reasons – first, the best content in NSDL is likely to be purely digital and not supported by a separate market – and, second, content in NSDL could replace some textbooks. In short, NSDL merits very substantial funding, well beyond current levels.
  • The care and feeding of the contents of digital libraries are closer to the care and feeding of the animals found in zoos than to that of the books found in traditional libraries. This is especially true for the highly interactive contents that are most likely to have a real impact on the twin goals of quality and reach – for example, few of us are able to read floppy disks and many of us spend inordinate amounts of time rewriting old programs in new languages. For this reason, the name National Digital Science Zoo would have been better than National Digital Science Library.
Posted in Friends of NSDL ]]> http://nsdlreflections.wordpress.com/2008/09/18/reflections-on-nsdl-by-frank-wattenberg/feed/ 0 bmuramatsu Frank Wattenberg - Twin Jet Nebula http://nsdlreflections.wordpress.com/2008/09/18/implementation-and-innovation-in-the-nsdl-by-william-arms/ http://nsdlreflections.wordpress.com/2008/09/18/implementation-and-innovation-in-the-nsdl-by-william-arms/#comments Thu, 18 Sep 2008 21:07:04 +0000 Brandon Muramatsu http://nsdlreflections.wordpress.com/?p=11 ]]>


Download PDF: Implementation and Innovation in the NSDL by William Arms

Alternative views of the NSDL

This essay is a personal reflection on how early decisions shaped the National Science Digital Library (NSDL) and how the program has evolved over the past decade. It draws heavily on observations from the planning studies in 1997 and 1998, and my experience as principal investigator of the Cornell University’s part of the Core Integration team until 2005. The thoughts expressed here are purely my own.

The underlying theme of this essay is that the NSDL program has two missions: implementation and innovation. Confusion between these two missions goes back to the beginning of the program. The original concept was to implement a digital library for science education. But the NSF’s principal goal is to support research and the NSDL program also gives grants for innovation in digital libraries and science education.


The first public discussion of a library for science, mathematics, engineering, and technology education (SMETE, as it was originally called) was a National Research Council workshop in August 1997 [1]. This workshop was only partially successful for a reason that has challenged the NSDL throughout its life. Some members of the workshop were genuinely wanted to build a library for scientific education, but the majority were researchers. They welcomed the NSF’s interest in this area because it might provide funding for their personal research. The participants gave insufficient attention to the tough implementation questions. Would a digital library really improve the quality of scientific education? Is this a good way to spend the taxpayers’ money?

The NSF tried again with a workshop in July 1998, which I chaired [2]. The report of the workshop concentrated on the narrow objective of how to build a digital library that would have an impact on science education. While urging that there should be an associated research program, the report emphasized that, “The SMETE Library provides a service; it is not a research project.” It envisioned a central organization that would coordinate a federation of major partners. The importance of these partners was expressed in a sentence that proved prophetic, “The site to which the NSF lends its name and towards which it directs its primary marketing will be considered the central site, but it is unlikely to be the biggest or most heavily used.”

Posted in NSDL Core Integration ]]> http://nsdlreflections.wordpress.com/2008/09/18/implementation-and-innovation-in-the-nsdl-by-william-arms/feed/ 2 bmuramatsu http://nsdlreflections.wordpress.com/2008/09/10/welcome/ http://nsdlreflections.wordpress.com/2008/09/10/welcome/#comments Wed, 10 Sep 2008 16:50:02 +0000 Brandon Muramatsu http://nsdlreflections.wordpress.com/?p=62 ]]>


The collaborative process of building the National Science Digital Library (NSDL) has provided many experiences for members of the NSDL community. The goal of this project (see About) is to capture “reflections” on these experiences by encouraging authors to “tell the story” of how they think NSDL was formed, grew and is continuing to grow.

Please read the essays by NSDL participants, comment on them and participate!

— Susan, Dave, Flora and Brandon

]]> http://nsdlreflections.wordpress.com/2008/09/10/welcome/feed/ 0 bmuramatsu