thing 4: Assessment Overview

Reading

Oakleaf, M. (2014). A roadmap for assessing student learning using the new framework for information literacy for higher education. The Journal of Academic Librarianship, 5(40), 510-514.  http://meganoakleaf.info/framework.pdf [pre-print]

Activity

Please answer the following questions below in a comment:

  • Which of the steps on Oakleaf’s “Roadmap” has your institution completed? Describe a project or partnership that has helped you complete this step.
  • Which steps from Oakleaf’s “Roadmap” will take more work to complete? Identify one next step that would help you make progress toward incorporating the Framework into your assessment work.
Advertisements

18 comments

  1. Our library has instituted pre and post tests for information literacy instruction as an assessment tool. Participation by librarians is voluntary and is not tied to the Framework at this time. We have however, come up with a Library Assessment document that looks at all services, including information literacy outcomes.

    Because our library has yet to take on the Frames, the first step is to identify a Frame on which to focus. The English department is our biggest customer therefore, I would start our disciplinary faculty partnership with them, as we have in the past with the development of the information literacy outcomes rubric that is currently being used the department..

    I think step #9 “Roll it Up & Report It Out” is the most difficult step because it is the end result of having gathered data from a variety of sources.

    Step #2 “Bite the Bullet” by identifying learning outcomes and choosing a formula to articulate said outcomes, would be the next step after choosing a Frame as a foundation for all the work that follows.

    Like

  2. I think we’ve done pretty well with steps 1-3. Most of my teaching isn’t based specifically on the framework but I have incorporated the framework into expressed learning outcomes of the professional programs. The one area that isn’t as specific is with the general undergrad degree. I see potential to have more autonomy in developing those learning outcomes if I can get more time with them.

    I think “Draw the map” will be one of the most difficult areas to address. Right now, we are planning a curriculum mapping that I hope will open doors for library services and instruction but I don’t envision that it will be specifically library related. The focus is on clinical patient care and many clinicians don’t see how the library knowledge can help in patient care. My goal is to be as present as I can be while the mapping is done. I’m hoping instructors will see gaps in their curriculum regarding evidence informed practice and I can help them develop activities to keep those skills alive.

    Like

  3. I think I can safely say we have completed two of these steps, however working in a smaller community college with only two librarians delivering instruction we haven’t really had to struggle with getting on the same page. We do have to comply with a state system set in place where we have to meet certain learning outcomes commiserate with 16 other community college libraries’ instructional outcomes. Those are agreed on at a higher level and represent the minimum outcomes of the instruction we deliver. Those outcomes are Standards based and cannot be easily changed to reflect the Framework until agreed on by certain administrative powers and a consensus among the library directors.

    Using the required system wide learning outcomes for library instruction I have created a rubric which links those to certain frames and adds learning outcomes I would like to implement at a local level. As the article mentioned, this was not a difficult feat, I have a pretty clear vision of what the information literate student should look like upon completing the highest credentials offered by our college, as that would ideally provide them with the greatest exposure to library services and instruction opportunities. My director has been very supportive not only in mapping our system mandated learning outcomes to the Framework but also in experimenting with our instruction and assessment styles to determine best practice for instating this mapping. Support from the director, and a certain level of freedom to experiment, seems imperative to successfully implementing any part of the Framework or any change to the instructional paradigm of an institution.

    Being a new employee, I began working here at the start of the year, I have had to depend on her existing relationships with faculty to get some professors on board with a change in how their usual library instruction is delivered (like using a flipped classroom and active learning activity instead of relying solely on a PowerPoint delivered lecture). Encouraging faculty to participate in implementing the Framework based instructional activities and learning outcomes will most likely be the most time consuming step on the Roadmap (step 3) just because faculty do not regularly take advantage of library instruction. One instructional session a year is not enough to really connect with these instructors to map the framework to the learning outcomes they would prefer and getting feedback on our general information literacy rubric has been nearly impossible. They seem to defer to me as an instructor though I have no hand in designing their curriculum and do not see their students enough to develop any kind of relationship or a full understanding of where they are literacy-wise. Building relationships with faculty, and then earning their trust to really collaborate and streamline instruction to meet their specific students’ and curriculum’s needs will be the biggest but most rewarding step towards embracing the Framework.

    Like

  4. Which of the steps on Oakleaf’s “Roadmap” has your institution completed? Describe a project or partnership that has helped you complete this step.
    I feel that my library has gotten in the zone of acceptance(but has also gotten a tad scared with 1 librarian) about the six frames. The previous standards were nice little boxes that we could check yes or no to, but these frameworks require more institutional thought and self-examination. It helped when a coworker attended an ALA sponsored series of workshops on assessment based on what we hoped would be the start of a new way of teaching. While it did not work as we hoped, the result was the librarian who attended came back with a new sense of purpose that the instruction is critical; it’s not an accessory but it is the coat we wear. Step 2- we have begun to list for our own benefit what the objectives we intend to cover. I strongly agree we knew although we don’t always use the ‘right’ words what we want students to know or be able to do from instruction. Step 3 is one we have not really broached. We intellectually know that the faculty, and their accreditation standards must view these frameworks and their professional standards as important otherwise we are truly ONLY talking to ourselves. Part of me thinks we sometimes still are scared that faculty will completely shut us out if we begin or act like our part is important enough to make some noise. Finally, we have some strong ideas of how to have activities that suggest the outcome is being achieved but again that will require we make some noise, and perhaps suggest that the every library concept imaginable 1 shot model will not be taught. We frequently are not sure what we will do if faculty does not ‘like’ that so we continue with what in retrospect might be teeny weeny steps.
    Which steps from Oakleaf’s “Roadmap” will take more work to complete? Identify one next step that would help you make progress toward incorporating the Framework into your assessment work.
    Steps 5 – draw the map; 6 – get real; 7- hunt and gather; 8- know it when you see it and steps 9 and 10. If I were to push something forward it would be the draw the map one. Maybe we look at the top 3 disciplines that have graduates and find out the threshold concepts for 1 are,etc… then there might be a real sustained basis of instruction and connection to what these students are moving towards.

    Like

  5. I think we’re on Step 2 at my institution, and I have a feeling that steps 3 and 5 are going to be most difficult. We do have as one of our library-wide goals for the upcoming fiscal year to create an assessment plan for the library. It’s one of the parts of our strategic plan that we’re trying to complete this year: “Develop an Assessment Plan to ensure that evaluation of library resources and services is intentional and programmatic.” I think the stepwise approach of this article will be very helpful in that, as we are in the process of getting our information literacy outcomes (closely tied to the framework) adopted by the college and we use those outcomes when planning all library instruction.

    Like

  6. The best way to describe what is occurring at my institution is a simultaneous baby steps into Step 1 and dipping our toes into Step 2. As a library staff we are very interested in the Framework, but as I have mentioned in prior posts the curriculum here leaves very little wiggle room for either the students or the professors. We have friends in the English Writing program with whom we need to cultivate our already established relationships, and are working on getting our foot in the door with both our equivalent of FYE and School of Continuing Education. In our individual instruction sessions we attempt to do Step 4 and as a department we are exploring how to best evaluate the results.

    I am hoping that we can take different steps in not necessarily the exact order as Oakleaf laid out in her article. Much of what we can do is to seize opportunities when they arise. What I need to do is prepare to be ready for any of those moments, as they can occur anywhere on campus at anytime.

    As Cristy Moran posted earlier, just getting on this road is part of the journey. I like how she describes that getting a campus all onto “one page” can take longer than expected. Especially with all the different disciplines and departments working in different ways.

    Like

  7. It is difficult to say, institution wide, what step we are on. In some places, we are still in step 1. But there are areas where we are further along. Our first year experience seminar now has a library unit that was constructed by a group of librarians. The objectives and the assessment pieces are built in. It was a collaborative effort with FYE instructors. They even have options for deploying the instruction based on the needs of the class. So, in that area, it looks like we are in step 6. I think the biggest challenge is that most of us are still doing one-shot instruction. We will have to be very creative in finding ways to collaborate with instructors to make library instruction more integrated, and in finding ways to follow through with authentic assessment pieces, even in a one-shot session.

    Like

  8. I believe we are at Step 1 and also pre-Step 1. One colleague, who is also on this forum, is most likely at Step 2 or beyond at this point, as she is working to adapt our information literacy sessions to the Framework. However, I believe others have not yet addressed the change from Standards to Framework. I am relatively new to my institution and am not sure if the step that will take the most effort to complete will be getting everyone to Step 1 or if it will be Step 3 Agree or Agree to Disagree. For me, developing learning outcomes based on thresholds for my next information literacy session is the next step in incorporating the Framework in my assessment work.

    Like

  9. I feel it is important to incorporate and educate faculty on the Framework. So far, we haven’t really even broached this topic of the Framework or assessing it in our library or how to use it. We are free to use/adopt as needed, but as adjuncts, we aren’t free to work with departments as a whole on how to implement this in a meaningful way for the different disciplines. That said, I do have a goal to start working on Step 2 when I create my learning objectives, or at least working to tie them to the Framework going forward. I would then like to think about how to assess in the right way, although I am not sure that one person assessing is as valuable as having a whole team agree on some basic pre- or post- assessments. As we may be getting a new Assoc. Dean at some point, that may be a place to approach these ideas.

    Like

  10. We are on Step one with a few of our freshman English Comp classes. We adapted a couple of the Frameworks to a 60 minute class although we have no plans to try to incorporate all the framework at the freshman level. With our upper level classes and graduate classes a couple of the other Frameworks are being discussed. Most of the other steps are waiting to see how we survive this first one. Feedback from our administration though wants some sort of quantifiable assessment so I guess that will be our next step.

    Like

  11. I wouldn’t necessarily say that the institution has completed any of the steps on Oakleaf’s Roadmap, since information literacy currently belongs to the English department. However, since library faculty are working on a document to assess the Library, and Information Literacy instruction, we appear to be on Step 2 of the Roadmap.

    The step which will take more work to complete is Step 3 – Agree or Agree to DIsagree. Since academic freedom is constantly mentioned, many want to continue teaching as they have been, and change is sometimes met with resistance. As Rachel mentioned in her post, “it takes a long time to make a change.”

    Like

  12. I really like Oakleaf’s roadmap and I think individual instruction librarians can use this for their own departmental teaching goals. I’m very new at my library, but I think we’re probably between 1 and 2 when it comes to our Freshmen English courses (the only instruction program multiple librarians work with).

    For me, I believe Step 4 will take the most time since each course you teach is different. I could see coming up with a variety of activities related to the learning outcomes for my departments so I have a nice variety to choose from to make my semesters easier.

    Like

  13. I think the hardest part for me will be writing my leaning objectives, though I see the necessity of them with the Frames. We’ve sort of been on automatic pilot with the one-shot, assignment-driven library sessions. Once we better know what we’re looking for in student performance, we can then better know how to access it.

    Like

    • Great point, Andrea. Writing learning outcomes is legitimately challenging! Thing 11 (https://23frameworkthings.wordpress.com/2017/03/21/thing-11/) provides some resources to help with the process of writing them. Personally, I find it really helpful to think of our learning outcomes as a living document that we can always update and change. That way, we don’t get as hung up on getting every word just right, or endlessly editing them.

      I think having learning outcomes that address the broader scope of information literacy (and the big concepts represented by the frames) can help us move beyond those one-off, assignment-focused sessions you mentioned. In many cases, faculty members just aren’t aware of what’s possible! Sharing learning outcomes that reflect the Framework might help you make them aware that there’s more to info lit than learning the databases most useful for a specific project.

      Like

  14. As an institution, I think we’re working toward a cohesive statement about IL that will take the Framework into account but, as yet, we’re at Oakleaf’s first step. I think different librarians are at different steps in their own practices. Many of us have discussed the move from the Standards and the opportunity the Framework presents us to “redefine” IL as an institution; but we’re working on lots of elements (including assessment, a mission statement, an organizational model change, etc.) that I see being informed by the introduction of the Framework.

    I think *getting* on Oakleaf’s road is probably the most difficult, most time-consuming aspect of the whole thing – rather than any isolated step. It is difficult as a large, multi-campus institution to get on any one page, especially when so many other activities and initiatives take priority. But I do think that setting up a committee or taskforce for IL (which we’ve done) is the first step to reevaluating the IL program, thus “getting on Oakleaf’s road.”

    Still…Framework or not, assessment and the collection of learning artifacts from which we can assess in an instructive, ethical, and significant manner will always be difficult – so Step 7 “Hunt and Gather” – is likely to continue to be a challenge. I look forward to continuing to work with that IL committee I mentioned to come up with ways to plan for those roadblocks so that we don’t end up just stalling out. I think working together and brainstorming the best methods to collect, the best contacts in departments/ faculty that can urge other faculty to participate, etc. will be integral in a successful Framework-informed IL program.

    Like

    • Cristy, I totally agree that in many cases, just getting on the road is the most challenging part! Your multi-institution situation sounds challenging and complex. It’s great that you’ve created an IL committee to tackle these roadblocks together. Doing this work right takes time, and thoughtful consideration of your institutional context and the challenges you’re likely to encounter in advance should help the whole process go more smoothly!

      Like

  15. I think we are still at Oakleaf’s Step 1 at our university library. We have done some work with SCONUL to map out our own library’s credit bearing courses and standards for library instruction and outcomes, but I think this is as far as we have gotten. It takes a long time to make change.

    Like

Comments

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s