.jpg)
Disclaimer: I have no affiliation with Blackboard Inc. and do not wish to undermine the hard work their team has put into their service. I have no access to business analytics that would further inform my design decisions, so the conclusions I come to are based solely off of information gathered from self-initiated research. The solutions I propose in this case study are not meant to be taken as comprehensive or definitive in any way.
February - April 2021
Solo UX Designer
Figma
Blackboard Learn, created in 1997, is a learning management system (LMS) that can be used to supplement or replace traditional in-person learning. As of Fall 2020, Learn had 944 institutions using its platform and 5,232,086 full-time students enrolled. In 2014, Blackboard announced that it would be introducing Learn SaaS and Learn Ultra. The SaaS deployment would allow schools to enjoy a cloud-hosted version of Blackboard, with benefits such as faster delivery of system maintenance and fixes as well as immediate access to the most up-to-date version of Learn with no downtime. From there, schools can choose to enable the Ultra experience, which provides a more modern look to course view and navigation. Learn SaaS and Ultra were launched and ready for piloting in 2016.
There are both personal and business-related incentives that make this redesign relevant.
As someone attending an institution that uses Blackboard, I've noticed that its confusing and outdated interface is a common complaint amongst students and faculty. Blackboard is an online platform made to facilitate the process of learning and teaching. However, I found that it would oftentimes increase the stress levels for my friends and I. As someone new to UX design, I wanted to push myself to get to the bottom of why this was the case and how I could create a design that fostered an efficient user experience.
Additionally, it wouldn't be far-reaching to assume that a primary business goal for Blackboard is to maintain and gain clients. This means not losing the loyalty of the institutions already using it, as well as offering innovative solutions and features that attract new clients. And although Blackboard launched an Ultra experience that promised a cleaner interface with improved workflows and usability, many of their clients are still switching to other LMSes. In fact, Instructure's Canvas definitively surpassed Blackboard as the leading LMS in 2019.
So why, despite the introduction of a cloud-based server and a modern interface, did Blackboard still fall behind? The answer to this is long, but, for brevity, I'll be skipping over multiple issues my research uncovered in order to focus on insights that had a direct impact on my redesign. Additional issues will be discussed later on.
Since Fall 2016, there has been a noticeable shift in clients' LMS needs, one of which being an "increasing desire for more-pleasing student and instructor user experiences that more-closely match consumer-grade software apps." However, it wasn't enough for Learn to just revise their antiquated interface.
e-Literate is an organization committed to aiding higher education institutions and the education companies that serve them. In their article about Learn Ultra's release, they stated the following:
Based on interviews with clients arranged by Blackboard, and based on our own connections at BbWorld, what we consistently heard during dozens of interviews and from listening to panel discussions was that Learn Ultra Course View makes sense primarily for programs or schools that have not been on Blackboard Learn before.
This is because the redesign to the Ultra experience was too drastic—so divergent from the Original experience in both appearance and functionality that loyal clients felt as though they were using an entirely different product. For a real-life example, Framingham State University is one of the many schools that switched to Canvas after piloting both Canvas and Ultra. Robin Robinson, director of education technology and instructional design at FSU, said, "If we were going to move to Blackboard Ultra, it would be pretty similar to moving to a new LMS."
Because of this, I believe a major part of the solution would have been to redesign the interface with a higher level of fidelity to the Learn Original experience.
Redesign goals:
Personal goals:
Nine students were recruited for this study. (While a study including both students and faculty would be more representative of the Blackboard user base, I unfortunately could not get any instructors or administrators to agree to take part in this study. Thus, the redesign will be from the student perspective only.) In the first round of interviews, the goal was to understand how they interact with Blackboard and how they feel about the features they use the most. Although the Blackboard interface slightly differs in appearance depending on which school you go to, many common pain points were observed:
Students didn't like the inconsistencies between course layouts. In particular, the course menu (the black navbar on the left) differed wildly between courses, always dependent on how the professor chose to set it up.
Grades are inconsistent (sometimes it shows a percent, sometimes it shows a point total) and inaccurate (the percent or point total it shows you usually isn't representative of your actual grade). If students want to know their actual grade, they often have to find out the weighting of assignments, quizzes, and tests via the syllabus and then manually input all of their grades into a grade calculator.
5/9 users didn't know there was a calendar until during the interview—even those that had been using Blackboard for more than five years. In fact, before finding out about the calendar, three of those five users said they wished there was a calendar.
* * *
Lastly, I asked students about the effect of Blackboard on their stress and collected KPIs as a benchmark point when comparing the performance of my redesign to that of the original. The results:
* Stress: 0 students said Blackboard decreased their stress; 2 said Blackboard had no effect on their stress; 3 said Blackboard increased their stress; 4 had mixed opinions (Blackboard may increase their stress or have no effect on their stress depending on the instructor, Blackboard increased their stress when they were new to the system but has no effect now, or their stress increases at the beginning of each semester due to inconsistencies between course layouts).
After another two rounds of interviews and usability testing (first on wireframes, then on high-fidelity prototypes) over the course of weeks, a presentable design was finally ready. Each design decision is backed up with extensive justification that gives a why and how to the evolution of features throughout multiple iterations.
Layout
Calendar
What's Due
Edit course title/color
Edit course list
Edit homepage
This feature wasn't in the high-fidelity prototype that users did usability testing on. However, one student suggested that What's Due be replaced with an announcements section. When I brought it up in subsequent interviews, there was a 50-50 split for those who agreed and disagreed. I thought the best middle ground would be an edit homepage option, where you can move, remove, and add widgets to the homepage, giving a more personalized experience.
Layout
Weighted categories
Submission feedback
In Canvas and Learn Ultra, the navigation bar is on the left. After playing around with a version of my redesign where the navbar is on the left, I decided to keep it on the top for two reasons. First, Learn Original has the navbar on the top, and adhering to this approach is in line with my goal of creating a redesign that retains fidelity to the Original experience. Second, I decided there weren't enough navigation links to justify a left navbar. All navbar links in my redesign—which were very few—could easily be nested under the "Student Name" dropdown.
* * *
Performance measurements were collected again at the end of the final round of usability testing in order to compare the performance of my redesign with that of the original design. The results:
* Stress: All 9 students said that they believe this redesign would decrease their stress—that is, if Blackboard operated with a structure and functionality consistent with what they saw in the redesigned prototype, they imagine their stress would be alleviated. When asked why, users said it was due to the redesign's simplicity and immediate access to the most important, student-centered features (courses, to-do list, grades, calendar).
Every KPI showed an increase in the performance and usability of the redesign. CSAT rose from 11.1% to 88.9%, SUS from 38.1 to 90.6, and stress levels from 0/9 users stating that the current Blackboard decreased their stress to 9/9 stating that they believe the redesign would decrease their stress.
These results are impactful because students have a say in which LMS their school uses. I reviewed 15 articles from institutions that switched from Blackboard to Canvas after running pilots of both Canvas and Learn Ultra. These pilot groups included both students and faculty, each an important voice in the final decision. If Ultra had been designed in such a way that produced similar CSAT, SUS, and stress level improvements for both students and faculty, it's very likely that less schools would have decided to make the switch from Blackboard to Canvas. This is incredibly important because customer retention is just as—and sometimes more—crucial to a business's financial success as customer acquisition.
There's more to user experience than user interface and unsolicited redesigns. Through studying articles from 15 institutions that parted from Blackboard and switched to Canvas (even after piloting Ultra), three major themes became apparent:
Fundamentally screwed code - Major impediments to usability are still present in Ultra despite the introduction of a modern interface. Greg Crouch, clinical professor and associate chair for undergraduate studies at Washington State University, said, "the way Blackboard was fundamentally coded is incompatible with the student information system, so issues connecting would continue to arise." Similarly, Amy Greene, executive director for online learning at Ferris State University, said, "Canvas was created on a new and more modern infrastructure, so it integrates much better with existing Ferris systems."
Customer service - Many schools (Arizona State University, Clemson University, Ferris State University, Temple University, etc.) that have switched to Canvas after years—or even decades—of using Blackboard rave about Canvas's incredible 24/7 customer service, hinting at a lack in quality of these services from Blackboard Inc. Greene stated, "Canvas also provides 24-hour support for faculty, staff, and students. They don't have quotas for calls, have no time-limits for addressing problems or answering questions, and are actually fun to talk to."
Chipping away at client trust - When faced with the decision of whether or not to switch LMSes, why would an institution choose to stay with a company that revamped its interface after years and years of refusing to adapt its system to ever-changing user needs and technology advancements? From Framingham, Robinson said, "We've had Blackboard since 2000, and the basic framework has not changed in all of those years." Palomar Community College stated similar grievances: "Palomar has been using Blackboard since 1998. For 18 years Palomar students and faculty have been using the same interface with minimal changes and updates."
Leaving out instructors left a marked gap in my research. A large part of how Blackboard works for students is dependent on how Blackboard works for teachers—you can't have a learning management system that's easy to use for students unless it's easy to use for instructors. Addressing the instructors' perspective is the only way to mitigate major inconsistencies in the organization of courses and grades.
Additionally, as stated in the disclaimer, I have no access to business analytics and the reasoning behind Blackboard's business and design decisions, nor was I hindered by time or financial constraints. I recognize that the harsh winds of reality in the UX field would have blown me towards vastly different conclusions than those I landed upon while researching in a vacuum.
As I had hoped, embarking on this journey helped me grow as a new designer and appreciate nuances of the process by experiencing them firsthand.
Amongst many, many other things, I learned how to pick KPIs that correspond with the goals of the project; I learned that there's no cookie cutter way to go about the design process or structure a case study; I learned how to prioritize pain points and address them accordingly; I learned how to conduct interviews and ask questions to identify and understand the problem, not to validate what I think the problem is.
Most importantly, I learned that "setbacks" aren't really setbacks—they're just part of the design process. Obtaining new information that invalidates the solutions you've already proposed may take you back to square one, but that doesn't devalue the work you've done so far; on the contrary, novel findings and jettisoned proposals work together to enrich your understanding of user and company needs.
Thank you so much for reading!
