Usability Tests
Heuristic Evaluation
In order for us to achieve a renewed platform, while keeping it simple and usable, we performed an heuristic evaluation on the existing platform. Despite being quite old, we were amazed by its usability.
We followed the Jakob Nielsen’s 10 usability heuristics to evaluate the platform. This evaluation method suggests five evaluators as the "magical number" to perform a good heuristic analysis, so all of our team (five people) performed and evaluation. The results are listed bellow.
On the old platform
Problems
| Problem | Description | Heuristic | Severity |
|---|---|---|---|
| Ambiguity in Application Functionality Display | Even with multiple of the functions of the application being disabled or non-functional, they still appear on the website, leading to confusion about the functionalities available. | Error prevention #5 | 3 |
| Cluttered Design | The design is very cluttered and does not feel easy to use for someone with no prior experience. | Aesthetic and minimalist design #8 | 4 |
| Dissertation Creation Form | The dissertation's creation/edition form does not scale well with the number of options available. There isn't much space between each field, which can be overwhelming. | Consistency and standards #4 | 2 |
| Order of columns in different pages | The columns aren't consistent between the main dissertation list and teacher's dissertation list. Despite that, teacher's dissertation list has a weird order and spacing of columns. | Consistency and standards #4 | 3 |
| Uniformity Across Platform | Most of the pages are focused on one center table that shares its look across all of the platform, leading to many pages looking exactly the same despite having completely different data and usage. | Match between system and the real world #2 | 4 |
| Repeated Taks | The website itself is "clunky" to use and does not provide a very efficient way of automatic certain tasks, especially for the teachers. | Flexibility and efficiency of use #7 | 2 |
| Non-Adaptative layout | The platform doesn't account for smaller screens and its fixed at a 16:9 ration. Smaller screens have content cutted and have to scroll. | Flexibility and efficiency of use #7 | 2 |
| Extra menus and information | Pages contain irrelevant information (like the landing page), or empty pages. The side menu has irrelevant tabs. | Aesthetic and minimalist design #8 | 3 |
| Inconsistent Data Presentation in Dissertation List | The dissertation list table presents data in an unusual order, prioritizing courses over dissertation titles. Additionally, the checkbox indicating whether a dissertation is chosen is small and difficult to notice. | Match between system and the real world #2 | 2 |
| Confusion in Academic Year Navigation | On the teachers' "Dissertações submetidas" page, the title correctly displays the academic year of the dissertations shown. However, above the legend, two different academic years are displayed, potentially confusing users about the data shown. | Flexibility and efficiency of use #7 | 3 |
| Lack of Contextual Awareness in Table Design | Most pages use tables to display data, but the table format remains consistent regardless of the data shown, forcing users to rely on page titles or table content to understand their current location and context. | Error prevention #5 | 2 |
| Confusion in Academic Year Navigation | On the teachers' "Dissertações submetidas" page, the title correctly displays the academic year of the dissertations shown. However, above the legend, two different academic years are displayed, potentially confusing users about the data shown. | Flexibility and efficiency of use #7; Visibility of System Status #1 | 3 |
| Lack of Clarity in Initial Login Error Message | The initial login only indicates a problem with the account without specifying the nature of the issue, potentially confusing users who are unsure if the problem lies with their access or another issue. | Help Users Recognize, Diagnose, and Recover from Errors #9 | 3 |
| Visibility Discrepancy in Dissertation Selection | In the dissertations list, some dissertations appear as not chosen (unchecked tick box), but when users click to view more info ("Ver"), they appear as chosen ("Escolhida:Sim"). | Visibility of System Status #1 | 3 |
| Non-Adaptative layout | The platform doesn't account for smaller screens and its fixed at a 16:9 ration. Smaller screens have content cutted and have to scroll. | Flexibility and efficiency of use #7 | 2 |
| Captions instead of recognizable icons/terms | The platforms displays information as a table, and some elements, like in the "Lista de Interessados" page, columns are represented as non-recognizable icons and require a caption | Recognition Rather than Recall #6 | 3 |
| Sidebar Design | The sidebar design lacks organization and spacing, displaying too much information in a cluttered manner. It should be better organized and spaced, with options that are relevant and properly displayed. | Match between system and the real world #2 | 4 |
| Extra information | Pages contain irrelevant information (like the landing page), or empty pages. | Aesthetic and minimalist design #8 | 3 |
Strengths
| Strength | Description | Heuristic |
|---|---|---|
| New Dissertations | There is a very noticeable visual hint indicating if a dissertation is new, so users can easily distinguish which dissertations have been added since they last visited the platform. | Recognition Rather than Recall #6 |
| Error on bad Login | When we, has students of the 3rd year try to login in the website a good error message appears saying that there is a problem with our account and a way to fix it is to talk to the secretary, this message is good but it could be better saying like you don't have access to the website and explain the reason | Error prevention #5; Help Users Recognize, Diagnose, and Recover from Errors #9 |
| Intuitive buttons | Every action button is very intuitive, so users can easily figure what it does. | Recognition Rather than Recall #6 |
| Descriptive legends | Every icon has an associated legend to help lost users to understand the meaning of the column/ field | Help and Documentation #10 |
| Simple Design | The website keeps a simple design (maybe way too simple).The website's information and options are laid out in a way that makes sense (side menus, main table, sort and pagination above) and displays all the necessary information accordingly (but cluttered). | Aesthetic and minimalist design #8; Recognition Rather than Recall #6 |
| Support contacts | Some pages provide a contact to the support team, which can help the user to solve the problem. Most pages also contain a legend to help the user understand the data presented on the tables. | Help and Documentation #10 |
| Captions on icons | The caption that appears, helps the user to know more of how the website is organize | Help and Documentation #10 |
| Consistent vocalbulary | Every term used in the website seems to be in accord with the standards, and it's consistent throughout the platform | Consistency and standards #4 |
| Easy Navigation | Getting in and out of pages is very simple, as well as navigate to desired pages | User Control and Freedom #3 |
On the prototype
After performing the heuristic evaluation on the old platform, we started working on our prototype. As humans, we aren't impune from making bad decisions, so we performed an heuristic evaluation on the prototype. Despite correcting most of the flaws presented by the old platform, we ourselves did our fair quote of usability errors. The results are listed bellow.
Problems
| Problem | Description | Heuristic | Severity |
|---|---|---|---|
| Power-user only search | he search bar is heavily focused on power-users, mainly when applying filters. Basic users will have much more difficulty in applying filters | Flexibility and efficiency of use #7 | 4 |
| Hidden dissertations when user has many alerts | As the "As minhas Dissertações" tab is under the alerts tabs, when it has a lot of notifications it hides the underling tabs. | Visibility of System Status #1 | 3 |
| Extra effort to make an action | In order to accept a dissertation, the admin has to access his profile and then see the list of dissertations. The same happens if a teacher wants to see the list of interested students, he has to access the dissertations and only then see the interested students. | Flexibility and efficiency of use #7 | 3 |
| Overall Design | The aplication still needs to provide a more consistent layout that is both simpler and less clutterred. | Consistency and standards #4 | 3 |
| Lack of errors | More errors and notifications should be added in order to let the user know any mistakes made from his options or from problems with the application. | Help Users Recognize, Diagnose, and Recover from Errors #9 | 3 |
| Position of important buttons | When its presented to a student an agreement to accept a dissertation the options to accept or deny it are in the end of the page and it should be the main focus meaing that it should appear at the end of the page and in the notification it could appear the options to accept or deny it plus the see more option. | Flexibility and efficiency of use #7 | 3 |
| Inconsistent Labeling | Where it shows As minhas escolhas it shoukd appear a different label, depending on the role, like for student it should appear Dissertações com interesse or if he/she already has one dissertation appear Dissertação Escolhida or Atribuída, for a teacher As minhas dissertações is a correct label and for admin it could appear Dissertações pendentes; | Consistency and standards #4 | 4 |
| Lack of Useful Navigation | In the student's profile page t it's missing a quick access button saying "Ver mais dissertações" for the student to add more interest on dissertations, the same goes for the admin navbar, where it's missing a quick access link to see the dissertations that are waiting for approval; | Flexibility and efficiency of use #7 | 2 |
| Role not Displayed | The website doesn't show the user role on the navigation bar , which can lead to confusion about the user's permissions and what he can do on the platform. | Visibility of System Status #1 | 2 |
| Too Many Colors | The amount of different colors and saturation used in the dissertation list page may lead to a unpleasent experience for the user, as it is difficult to focus on the data presented. Colors with more "pop" should be preserved for more important information, while things such as Courses of a dissertation shouldnt use such a strong color. | Aesthetic and minimalist design #8 | 3 |
| Easily Cluttered Layout | The current layout of "Áreas" and the dissertation course are not scalable and may lead to a cluttered UI if more options are added without adjustments done. Same problem for the dissertation details page. Same problem for profiles pages. | Aesthetic and minimalist design #8 | 4 |
| No Captions | There is no legend to indicate what is the green background on the new dissertations list, which may lead to confusion about the meaning of the color. | Help and Documentation #10 | 2 |
| Extra information and inconsistent layout | In the "Profile Student Page Accepted Acordo", the text is not all justified to start on the same position. Curso(s) is also irrelevant information in this page. | Aesthetic and minimalist design #8 | 2 |
Strengths
| Strength | Description | Heuristic |
|---|---|---|
| Clean Layout | The application provides a clean landing page that clearly states the purpose of the application and next steps to be taken by the user in order to achieve his goals. The contrast of the new dissertations makes them easily spotted. | Match between system and the real world #2 |
| Easy Navigation | The search bar provides a quick way to find wanted dissertations, thats intuitive and doesnt require a lot of clicks from the user. | Flexibility and efficiency of use #7 |
| Consistent Color Scheme | The website has a consistent color scheme throughout the pages, which helps the user to understand the importance of the information presented. | Consistency and standards #4 |
| Descriptive Layout | Users can easily see all the crucial information they need with a glance of an eye. | Aesthetic and minimalist design #8 |
| Alerts on Action | When the user makes an important action an alert is shown. This prevents user from making wrong actions and giving confirmation on successful actions. | User Control and Freedom #3; Visibility of System Status #1 |
After compiling the results, we returned to the design table to correct our usability flaws.
All heuristic evaluation reports can be found in our drive