Interactive Design Methods

We evaluated LinkedIn using usability metrics including interviews and observations, inspection methods, issue-based metrics, and performance & self-Report metrics.

OVERVIEW

About the project

We evaluated LinkedIn, a service website provided at LinkedIn.com. As of September 2018, LinkedIn claims to be "the world’s largest professional network, with more than 562 million users." (LinkedIn, 2018) LinkedIn’s stated vision is "Create economic opportunity for every member of the global workforce." The website allows its members to present a public professional profile and connect with others who have done the same. It provides related services that help members to communicate with each other and to connect with employers. Basic membership is free, but additional services can be purchased.

  • Role: 
    User Experience Designer

  • Tool: 
    Quantitative and Qualitative Research

  • Client: 
    LinkedIn Inc.

  • Timeline: 
    September 2018-October 2018

DATA COLLECTION

Interviews and Observations

Our team chose LinkedIn for the task analysis. We set two scenarios for our users. and the second is: searching and applying jobs on LinkedIn. During our contextual inquiry, we would ask user to update LinkedIn profile. At the same time, we would introduce ourselves, explain project and gather demographics like age, gender, professional/student, job title etc. during the observation. We conducted semi-structured interview and we introduce ourselves, explain project and gather demographics, either. The questions we ask users including why do they use it, when do they use it, where do they use it, what do you like or dislike about it, and what would you change about it. We use pen and paper for recording and we use optional audio backup.

After we creating affinity diagraming, we discussed two representative persona for LinkedIn users.

INSPECTION METHODS

User Characteristics

LinkedIn users are students and professionals, as well as hiring personnel at various corporations and industries. Most users are assumed to be within the typical employment age range (18-67). They are likely to be at the start of their career or at a transition point between careers. Their computer skills can be between beginner and experienced users, but the skills needed to perform tasks within LinkedIn varies depending on the amount of time spent engaged with the site.

Task Analysis

Updating LinkedIn Profile

I. Open web browser.
II. Type into the search field linkedin.com to navigate to the site.
III. Sign in with email and password.
        1. Email and password have been saved on the user’s system and logging into the site was pain free.
        2. Can’t remember the password:
                   a. When user can’t remember their password, they click on the “forgot password” link to reset.
                   b. Email is sent to user to reset password.
        3. Can’t remember the email associated to the account.
        4. Create new account.
V. Update profile:
        1. Looking for something that says “update profile”.
        2. Find “Me” within the top navigation offered “View profile”, clicked on view profile.
        3. Looking for edit or update, click on “pencil graphic”, Edit intro window appears.
        4. Change Industry to “Graphic Design”, click on “Save” window closes, done with that change.

Personas

Sam is a 42 year old Software engineer LinkedIn user. He uses the site every evening after work and loves the interaction he has with other users of the site. Sam was downsized recently, and finds himself looking for new employment. Although Sam uses LinkedIn on a regular basis, he has not updated his profile since he started the account five years ago. While attempting to update his profile and resume information, Sam encountered problems updating his information. Remembering his password was an immediate problem since he used his former employer’s computer to set up his initial account. Luckily he used his personal email account instead of his former work account, and he was able to reset his password. Once this was resolved, locating the appropriate menu items and navigation to change and update his information wasn’t as intuitive of a process as he had hoped. Sam also noticed that some of the features he might want to use would come at a cost. The premium service offers individuals the ability to know which companies and individuals viewed his profile. He felt this could be of help in targeting his resume and cover letters more effectively. But he found the price hard to justify, given the limited features offered.

Sam knows he will have to make these changes, and wishes there was a better way to accomplish the task. Sam is old enough to remember the days of sending out resumes and cover letters, one-by-one. Using LinkedIn would make the process easier to get in front of more possible employers, He just wished doing so was easier to accomplish. LinkedIn claims that “I’m the boss of my account”, but it seems to Sam they choose the path of his interaction, rather than let him be in control of how he interacts with his data and their site.

Recommendations and Feedback

I. Require a certain level of knowledge and experience to apply

II. Does not allow to assess the quality of redesign

Prioritized Issues

I. More flexible editing environment for updating profile

II. Personalize job searching experience

III. Revise inconsistency on profile editing

Suggestions For Task Analysis and Personas

I. Avoid identifying the skilled users as the persona in IT, CS, ME field. Try to identify problems for the unskilled group of people.

II. Select new-beginners as typical users for primary persona could be more valuable.

III. The scenarios within task analysis do consider the difficulties for users.

ISSUE-BASED METRICS

Think-Aloud Protocol

For this part of our study of LinkedIn, we conducted a think-aloud protocol to evaluate usability. Participants were asked to complete the following tasks:

Task #1: Perform the steps necessary to update the photo in their LinkedIn profile, using a provided image. (We asked the participants to stop one step short of updating the image, because we didn’t want the test to result in an actual change to their profile.)

Task #2: Find a job listing for an Administrative Support Specialist at IUPUI, and report the required qualifications for that job.

Our criteria for failing either task were as follows:

I. The user takes more than 5 minutes (300 seconds) to complete the task.

II. The user announces they have completed the task but they have not done so.

III. For this protocol, we recruited the following participants:

Raw test results for each participant were as follows:

Outcome Statistics:

I am conducting the think loud and help my group with the quantitative and qualitative findings. I use confidence interval calculator for a completion rate on outcome statistics.

Next Steps


For future think-aloud testing, we would do the following actions differently:

I. Perform a pilot test of the think-aloud protocol, to ensure the tests with recruited participants went more smoothly.

II. Provide the participants with a printed task sheet so that they could easily see the task description instead of having to remember it or read it from our protocol script. This was especially relevant for finding the “Administrative Support Specialist” job.

III. Instruct the participants to announce that they have completed each task when they felt they had done so, in order to have a specific cue to end the timing of the task.

IV. Specify more clearly the starting state of the system for each task. For example, task #2 should have started from the website’s home page, instead of on the profile editing page at the end of task #1, with the photo change still in progress.

PERFORMANCE & SELF-REPORT METRICS

Participants

For this part of our LinkedIn study, we recruited the following participants:

Performance Metrics

To gather performance metrics, we revised the tasks we previously used for our think-aloud protocol. We each performed the tasks ourselves and averaged the results (using a geometric mean) to generate benchmark values.

With 8 participants we had 7 degrees of freedom, and selecting a 0.05 alpha resulted in a critical value of 2.36. From this information and our test times we calculated the following values.

Self-Reported Metrics

After each participant completed the tasks, we asked them to fill out a System Usability Scale (SUS). From these scores we calculated the following values:

Mean: 71.9

Confidence Interval: 61.5 to 82.4

Using an SUS value of 70.0 as a benchmark for acceptability, we are 60% confident that the population would give LinkedIn an acceptable or better SUS score.

Next Steps

Updates to the site could increase the usability and the confidence interval level given the sample size. We feel this could be done by revising the inconsistency associated with profile editing and better automating and personalizing the job searching experience.