Faculty member engages students around white board graph

Assessment Brief #84 - Assessing Student Technological Self-Efficacy

Assessment Logo: assessment-revision-outcomes

May 2014

The University Libraries provide services at two facilities with high-end software and technological equipment: The Center for Information Management (CIM) in King Library, and the Digital Den in the Business, Engineering, Science, and Technology (BEST) Library. Both of these facilities also feature service points nearby with staff who can provide additional assistance with technological questions.

While it is known how many people use the facilities during any given time, not much data has been collected regarding the nature of the use.

To address this, users of CIM and the Digital Den were presented with an optional two-question survey when they logged into a machine: "What do you plan to work on today?" and "If this is for a class, which one?" The survey was given during a two-week period in late February and early March. Each user was presented with these questions only once, no matter how many times they used the facilities during that time. The initial survey was also given to users of facilities without the same level of technology and service: Kamm Instruction Room in King Library and other computers on the first floor of BEST Library.

Users who completed the initial survey were asked to complete a longer questionnaire on their technological self-efficacy (the ability to use technology to complete a task). Some questions students were asked include:

  • Using the software enables me to accomplish the task more quickly.
  • I could not have completed the task without this software.
  • Using this room helped me to better complete this task.

757 people completed the initial survey during the testing period. Since initial surveys were optional, they are not indicative of total room use. 311 students were asked to complete the follow-up questionnaire. A total of 144 students completed the follow-up, equating to a 46.3% response rate. Most students who completed the survey indicated that they were completing homework, or an assignment tied to a specific class:

  • Activity
    • Homework - 42%
    • A Paper or Project Involving Research - 34%
    • A Project Involving Elements of Design (website, publication) - 12%
    • A Project involving Geospatial Data - 7%
    • Other - 5%

To compete the tasks at hand, most students indicated that they were using a Microsoft Office productivity tool (42%) or a basic web browser (25%). Not all students denoted a specific piece of software. 

For further analysis, the software was divided into two categories: basic (including browser and Microsoft Office), and advanced software (including design, statistical, geospatial, and other software packages). Overall, 67% of students who participated were using basic software during the study period. 

There was statistically significant difference between rooms: students who used CIM or the Digital Den had the same comfort level with the software as those in control rooms, probably because they used similar kinds of software in all locations. Some interesting trends for software type did occur, as listed below. Means for students using basic vs. advanced software are below (a score of 1 indicates strong disagreement; while 5 indicates strong agreement).  Statistically significant differences are asterisked. 

Prompt:

  • I am familiar with the software.*
    • Basic - 4.8, Advanced - 3.6
  • I have used this software in the past.*
    • Basic - 4.9, Advanced - 4.1
  • I believe I am good at using the software.*
    • Basic - 4.5, Advanced - 3.6
  • The software was difficult to use.*
    • Basic - 1.5, Advanced - 2.4
  • The software is not available elsewhere on campus.*
    • Basic - 1.6, Advanced - 3.0
  • I asked library personnel for help in using the software more effectively.
    • Basic - 1.6, Advanced - 2.3
  • I asked peers for help in using the software.
    • Basic - 1.8, Advanced - 2.6

Conclusions and Implications for Teaching:

Although students who used advanced software reported familiarity with it, they also reported struggling and asking for help. Please consider making them aware of resources that can assist them in successful use of the software, even if learning the software is not part of the assignment.

Some resources are available from library databases:
Atomic Learning Video Tutorials
Safari eBook Collection

Students may also visit the Center for Information Management (CIM), 112 KIng Library. For more information about this study, contact Eric Resnis, University Libraries Instruction Coordinator, eric.resnis@miamioh.edu.