Interactive Design Methods

Symplicity is a SaaS-based student employment solution, which is branded for IUPUI student use as JagJobs.org. The job postings there are specifically targeted at IUPUI students. Many of the jobs listed are for on-campus jobs, but there are also listings for internships and permanent professional positions.

OVERVIEW

About the project

The purpose of this study is to evaluate the usability of Symplicity, a SaaS-based student employment solution, which is branded for IUPUI student use as JagJobs.org.

We conducted onsite usability tests using Windows and Macintosh laptops. We captured the participants’ interactions with the system as on-screen video, and we recorded the audio of the test sessions on our mobile phones. The test administrator and an observer were present in the testing area. The sessions captured each participant’s navigational choices, time on tasks, comments, satisfaction ratings, and interview responses.

In the study, we referred to the current Jagjobs.org implementation of Symplicity as “the IUPUI student job board”. We did this because part of our evaluation was to determine student awareness of JagJobs.org, and we wanted to avoid tipping off potential test participants to the existence of that site.

I took charge of interviewing and it is my first time to communicate with clients here. I learned a lot about how to ask questions and gather information during the interview.

  • Role: 
    User Experience Designer

  • Tool: 
    Quantitative and Qualitative Research

  • Client: 
    JagJobs

  • Timeline: 
    November 2018-December 2018

INTRODUCTION

We conducted a usability study of Sympicity on behalf of the following co-clients:

Zeb Wood, co-director, Media Arts & Sciences Undergraduate Program, IUPUI
David Pierce, director, School of Physical Education & Tourism Management, IUPUI

Our clients expressed a number of concerns about the IUPUI implementation of Symplicity.

1. The primary concern is that the IUPUI students think of JagJobs.org as a resource for finding on-campus jobs exclusively. Students don’t seem to be aware of internship and permanent professional positions on JagJobs.org.
2. Students who have used JagJobs.org have complained about poor usability.
3. Job postings are not being kept up to date.

Symplicity is an expensive service for the university to provide. Our clients feel that the above concerns keep the university from realizing value from Symplicity appropriate to its cost.

Because of these concerns, our clients assembled a team to assess IUPUI’s use of Symplicity and consider a course of action. This team was led by Cicely Noto, a Media Arts and Sciences production manager. We assisted Cicely’s team by conducting this usability study of JagJobs.org in an attempt to answer the following questions:

1. How do IUPUI students find out about off-campus and professional jobs that are being posted for them to fill?
2. How usable is the sign up process for the IUPUI student job board?
3. How usable is the process of finding a job on the IUPUI student job board?

EXECUTIVE SUMMARY

This document describes a usability study conducted on the IUPUI student job board during the fall semester of 2018. Eight participants who had not previously used the site spent approximately 30 minutes each completing test tasks, filling out questionnaires, and responding to interview questions.

We found the following:

1. If students are aware of JagJobs.org, they believe it is only for part-time student jobs.
2. Access to JagJobs.org is forced through an Office of Student Employment web page that has poor usability.
3. JagJobs attempts to force new users to fill out a time-consuming profile before allowing them to use its search feature.
4. JagJobs’ basic search features lack sophistication and could do much to be more useful to the user.
5. JagJobs prominently provides an “Extended Job Search” that is actually a link to a completely different site.
6. We believe our findings support our clients’ concern that IUPUI is not currently getting good value for its investment in Symplicity. We recommend some actions that IUPUI can take to make the site better known, easier to access, and more useful. However, we also recommend that Symplicity be compared with other similar services to see if they provide better value.

I cooperate with my group members. We discussed together and figured out our test plan through the process. It is the time I have to develop my idea and try to be articulate for having consensus.

METHODOLOGY

Test Sessions

We prepared a recruitment script to screen potential participants. Some candidates were rejected because they had previous experience with JagJobs.org. Test sessions for the eight selected participants were scheduled in advance. We gave each of them a $5 Starbucks gift card in gratitude for their assistance.

Test sessions 1-4 were conducted in the campus center lobby, in seating areas without nearby bystanders. Test sessions 5-8 were conducted in a meeting room in the ICTC building. Each test session lasted approximately 30 minutes, and followed this format:

1. The test administrator introduced the study.
2. The participant was provided with a Windows or Macintosh laptop to perform the test, according to their preference.
3. The participant performed three timed test tasks (see below).
4. The test administrator requested a post-task retrospective from the participant after each task.
5. The participant filled out a Systems Usability Scale (see below) to evaluate JagJobs.org.
6. The participant filled out a background and demographic questionnaire.
7. The test administrator conducted a semi-structured interview about the participant’s job search experiences.
8. The test administrator answered the participant’s questions and gave them the gift card.

The test team members took turns acting as test administrator and observer. An audio recording was made of each session, and the laptops recorded the user’s interactions with the web browsers as movie files. We used QuickTime Player for recording Mac OS users and Xbox recorder for Windows users.

I have to search the material we have learned before and decide which usability testing or data analysis is appropriate for our evaluation. I clarified the knowledge I learned during this process.

Test Participants

All of the recruited participants are current IUPUI students, as required for access to JagJobs.org. None of them had signed in to JagJobs.org before.

All of the participants were international students known to members of our team, because they were convenient for recruiting purposes.

FINDINGS

Based on our test results, we have the following answers to our study questions.

1. How do IUPUI students find out about off-campus and professional jobs that are being posted for them to fill?

Students primarily use LinkedIn to find jobs. If they look for an IUPUI-provided source for jobs via one.iu.edu, there is a good chance they will find a resource for the wrong IU campus. If they know they can use JagJobs.org to find jobs for them, it directs them to different web page that has poor usability, and then they must find a buried link that signs them in to a third site with an unfamiliar symplicity.com URL.

2. How usable is the sign up process for the IUPUI student job board?

Due to issues with the design and execution of this test task, we did not find a good answer to this question. However, the site tries to force the user to fill out a complete profile--a time consuming process--before letting them use the its search features.

3. How usable is the process of finding a job on the IUPUI student job board?

JagJob’s basic search capabilities do not allow the user to search by anything but keywords and location: they must get those results before they can filter them by job type. It doesn’t allow users to re-use saved searches. It does not provide information about the employers who post the jobs. If the user attempts to use “Extended Job Search”, as half of our participants did, they are redirected to an entirely different website that is not specifically for their use.

I help comprehend the semi-structured interview data. Because it is semi-structured so it took a little time for to measure and manage the qualitative data. I also do the analysis for quantitative data for others to find the key insights.

RECOMMENDATIONS

Based on our findings, we have the following recommendations to improve users’ experience with JagJobs.org.

1. Promote the JagJobs.org site to IUPUI students as a source for all kinds of jobs, including internships and professional full-time jobs.
2. Redirect the JagJobs.org URL directly to the JagJobs branded Symplicity website rather than going through the Office of Student Employment website.
3. Encourage but do not force JagJobs users to complete their profile information.
4. Remove the “Extended Job Search” link that takes users to an external job site.
5. Add overview information to employer pages.
6. Provide additional options to the basic job search, so that users can filter jobs by their characteristics (e.g. full/part time, on/off campus, application deadline).
7. Provide workshops to teach students how to use JagJobs.

Because I have analysis data we gathering, I could provide some key points in our recommendations. I also considered our clients’ opinion at this stage because I have talked to them face-to-face before.

CONCLUTION

Reflection & Learning

JagJobs is not set up effectively as a resource for IUPUI students to find anything but part-time jobs. Students seeking internships or full-time permanent jobs are probably better off using LinkedIn.com, as, apparently, they do. IUPUI is certainly not using Symplicity to its best capability, and there are things they could do to make the site better known, easier to access, and more useful. It remains to be seen if Symplicity can be configured to improve its own search features and avoid the internal usability issues we discovered. If not, alternative service providers should be investigated to see if they provide better value.