Pathways

Pathways

Mapping Highest Qualification Level

The origin of our Pathways feature started with us trying to map people’s qualifications to careers. Back in September we ran a series of product experiments that gave us a lot of insights. One key insight was that what people would value the most is to map their qualifications, skills, personality and values to careers of interest. So we thought we’d better get started by mapping what level of qualification is required for each of our careers.

Qualification Data

So the first step here was to understand the full range of qualifications and subjects available to people in the UK. And it turns out there’s a lot!

Structured Pathway Data

We then changed tack and started deeply researching one career at a time. With the goal of identifying exactly what was required to get into each one. I started researching 3D Printing Technician and Emma took on Academic Researcher. We decided to go through our careers alphabetically. Then we’d tackle a good, broad spectrum of careers to ensure the structured data we captured would work generally for everything.

A Much Bigger Feature

As we started pulling together this structured data for our first few careers in mid-November, we started to realise that our dataset would support a much richer feature than simply qualification mapping. In fact, the whole idea of “qualification mapping” didn’t quite make sense. People are usually more interested in knowing what qualifications they need, rather than the small list of careers they can access with their existing qualifications. And we were even going far beyond just qualification requirements. We were gathering data on time, cost and funding. As well as non-qualification criteria.

Completing Pathways for 81 Careers

We were also 10 weeks away from the deadline for our final submission of Nesta’s CareerTech prize, which we are a finalist of. Up to that point I thought we had a pretty good chance of winning. But if we pulled off this new Pathways feature, we’d have an amazing chance! I knew we couldn’t research all 504 of our careers in 10 weeks. But maybe we could complete enough to demo and test the feature.

Designing the Pathways Feature

While Emma cracked on with the research, I pivoted towards designing the feature in mid-November.

Employing Professionals

At this point, Anna (our frontend developer, provided by Forward Partners’ Studio team) was already 6 weeks into rebuilding our site with our new visual design. She was about 2 weeks away from completing that work, so we decided to continue to employ her to build this new feature. Anna’s help would accelerate our efforts and ensure we kept the quality really high. And she also helped us move faster with the Nesta deadline looming!

Wireframes

So I started by creating wireframes for what I thought the new flow could look like, factoring in everything I knew about the features and how they would work. This included mapping out the high-level user flow for the current and new design, as well as wireframing each screen. Here’s the flow I mapped and some of the wireframes…

Professional Designs

Josh then spent a week working on pixel-perfect designs for each screen and thinking through the overall flow. As usual, he used Figma, which is an amazing design tool that lets us easily collaborate in real-time. He shared the progress he made each day so that we had a chance to provide further inputs and refine the designs. Here are some of his designs that map to my wireframes above…

Building the Pathways Feature

And then towards the end of November I started writing the detailed specification for the frontend features (how they display visually), how they interacted with the backend (where the magic happens and the data comes from), and the backend design for the Pathways feature (the data structure and algorithm that powers it).

Frontend

Anna and I then worked furiously until the end of the year to build and ship each of the key features. Anna finished the new visual redesign of the site by the end of November. She then created the new input screens to capture location, situation, education and experience information. And then upgraded the final results screens to show a user’s skills match with their careers of interest, a marketplace section to show third-party services to help a user with their career and then the Pathways feature. Anna only took 3 days to complete the frontend part of the Pathways feature. But she had a huge amount of frontend work to do to support the new visual redesign and input screens that were also needed.

Backend

Most of the Pathways engineering work though was on the backend, which was used to power the feature and send the required data to the frontend. This involved turning the 23-page backend design doc into code. Half of the work was actually ingesting the spreadsheet of structured data Emma was putting together into our database and ensuring it was accurate and error-free. I then had to craft algorithms that computed hyper-personalised pathways for users.

Hyper-Personalised Pathways

There is far too much complexity in the Pathways feature to cover in a single blog post. But it’s worth highlighting some of the coolest features that make it so powerful…

Relevant Links

Many of the task cards in a user’s to-do list have hyper-personalised links to help them take the next step to complete that task. Here are some examples…

  • For vocational courses that are typically available in Further Education (FE) colleges, we link to the user’s local college website so they can find and sign up for that particular course. We determine their local college by taking their postcode, converting it into latitude/longitude coordinates and then using trigonometry to compute the distance between their postcode and all 583 FE colleges in the UK. We then link to the closest college to where they live.
  • For courses run by universities (e.g. degree courses and above), we link to Whatuni, which is a university comparison website. We pass in the qualification type, subject and automatically sort by ranking so that in one click the user can see the best universities in the country for that particular course. They can then use Whatuni’s filtering options to further refine their selection based on their past grades for example (i.e. their UCAS points).
  • For tasks that involve joining professional bodies, we link to the specific websites that enable users to immediately sign up and join these bodies.

Time and Doing Tasks in Parallel

We automatically compute the total time it takes to complete all required tasks for a given pathway (ignoring those tasks the user has already completed). In order to do this, we research and record the exact time it takes to complete each task (whether they are qualifications, time spent in a junior role, work experience or other tasks). Some tasks can vary in time, so we capture this nuance too.

Cost and Funding

We automatically compute the total cost to complete all required tasks. Some tasks are free to complete, and others have costs like vocational courses, degrees and joining professional bodies. Emma deeply researched the average costs for these different tasks.

Summarising Time and Cost per Career

We bubble up the time and cost information all the way up to the user’s final results screen next to each of their careers of interest. This enables a user to immediately see how much time and cost the shortest pathway into each career is for them, given their circumstances and background.

Pre-ticking Completed Qualifications

We capture each user’s highest education level, and extra information that might be relevant to pathways. This can include the subjects they studied and the grades they obtained.

Updating Everything as Tasks are Ticked

Pathways is essentially a to-do list of tasks the user needs to complete to get into a career. As the user completes a task, they can tap on it and it is ticked and marked as completed. Once a task is completed, the time and cost for that pathway is updated and immediately reflected to the user. But as we bubble this information up, the user will also immediately see the time and cost information updated in their list of pathways for a particular career. And even on their final results page with the time and cost to get into each career.

User Testing & Refining

We finished and shipped the first version of the Pathways feature on December 31st. Our goal was to finish by the end of the year, so we hit that goal 🎉

  • Refined the order of careers on the final results screen
  • Added instructional screens before and after the quiz to better guide people through the flow
  • Refined the titles of the pathways
  • Updated the home page to include our top features
  • Wrote up and completed 61 Jira tickets
  • Submitted 93 PRs (Pull Requests)

Makeover

We didn’t just work on Pathways over the past four months. We gave our site a full makeover, massively improving its visual design. This was essentially an implementation of the design work Josh did last August (as detailed at the start of my last blog post).

Skills Matching

We started working on the Pathways feature because one of the top pieces of feedback our users voiced was wanting to map their qualifications and skills to careers. Pathways solves for the qualification mapping part, but not skills mapping. So we wanted to try and tackle this too.

Choosing a Skills Taxonomy

We started by reviewing a few different skills taxonomies including O*NET (freely available US-focused data set), EMSI (paid, proprietary data set) and ESCO (freely available European-focused data set). We settled on ESCO in the end because it was free, most relevant to the UK jobs market and it was the only one with a hierarchical skills taxonomy.

The ESCO Data Set

ESCO allow you to export a set of CSV files (spreadsheet format) that contain a list of:

  • 2,960 occupations
  • 13,485 skills
  • 114,393 occupation-skill mapping pairs (so each occupation has on average 39 skills)

Using ESCO Data

The first step we took was to get all of this data into a single Google spreadsheet, split out across multiple tabs. We then put quite a lot of work into stitching the data sets together into the format that enabled us to interrogate it further.

What Matters To Us

An algorithm is only as good as the data you feed it. So the most important first step for us is to prioritise the dataset and ensure it is well understood. It’s then important to clearly identify what problem you’re trying to solve. In our case, we wanted to:

  1. Capture the skills a user has likely picked up during their career so far requiring less than 60 seconds of time investment from the user.
  2. Map these skills to all of their careers of interest to provide a single skills matching score. We only wanted to give a rough guide as to how well each of their careers of interest are to their skills. So a single score best achieves this rather than providing any more granular detail that may be more difficult for the user to parse and make decisions from.
  1. Get a comprehensive list of occupations a user could do (more comprehensive and detailed than our existing list of 504 careers)
  2. Have a mapping of these occupations to skills
  3. Map our career database to this list of occupations so that we can do the final part of the skills mapping to a user’s careers of interest

Our Algorithm

We then designed an algorithm that takes as an input the user’s past jobs and their careers of interest. And outputs a skills match score for each of their careers of interest. Our algorithm essentially computes the overlapping skills from their past jobs with their careers of interest in order to determine a score.

Designing Skills Matching

We then designed the feature visually. Both the inputs where we capture the user’s career history and where we show the skills match to the user in their final results. Here’s how these screens look:

Updating ESCO’s Data Set

We then had to update ESCO’s data set to fully support our algorithm. For example, we had to manually map our 504 careers from our database to their occupation list. We decided to use their list of 2,960 occupations to power the autocomplete input fields where the user specifies their current and past jobs. But in order to map to a user’s final results, we still had to map from their list to our career list. And some didn’t map perfectly. So we then had to create new occupations, along with a manually curated list of skills (which we pulled together by combining the relevant skills from multiple similar occupations).

User Testing Skills Matching

We tested this algorithm with ourselves and a few friends. I was really happy with the results. It worked much better than I expected and produced skills matching scores that seemed fairly accurate. And it’s fairly sophisticated too. To compute the skills match for the average user with 20 careers of interest and a few past jobs, the algorithm executes about 500,000 computations. But thanks to the speed of computing, this happens instantaneously. So it feels like magic as you can see the skills match score for all your careers of interest about 2 seconds after you enter your career history.

Marketplace

We’re a software company. And software is really well suited to solving some of the problems we’re working on. Like helping someone discover and get into a career that will make them happy. But it can’t solve all the problems a user will have on their career journey. For example, many people require some emotional support, encouragement or quite specialised coaching and mentoring. These problems are often best solved with people rather than computers.

Researching Providers

Emma spent some time researching the best services that solve different parts of the career journey and created a curated list of services, with information about each one.

Building & Testing the Feature

So when we built the feature, we decided to only display the curated services which are relevant to a given user based on their age, education level, employment status, living situation and even the specific career they are exploring. This has resulted in a list of high quality, relevant providers. We then show this list in one of two places. On the final results screen for the more general service providers. And a different list on a career-specific screen for providers that help people get into a certain career.

Nesta’s CareerTech Prize

Nesta is an innovation foundation. They provide grants to innovative companies who are working on socially impactful projects. One way they do this is through monetary prizes. We are a finalist in their CareerTech prize.

Engagement

We worked hard to make our app as engaging and easy-to-use as possible. For example, we streamlined all the user inputs that powered all our personalisation features down to four screens. This captures their location information, employment status, salary, age, education information and career history. And from testing, users are able to complete these screens in 1–2 minutes.

Our Engagement Metrics

And here are our engagement metrics, which are all very encouraging:

  • 83% of people (6,485) who started the quiz finished it. They answered 50 or 100 questions.
  • 82% of people (4,484) who started the input screens (as above) completed all four and got to the final results screen.
  • 57% of people (4,484) who started the quiz got to the final results screen. They went through all of our screens, including quality and career selection. Note that we count 4,133 (53%) as “valid” final results users. This is defined as having taken at least 4 minutes to complete the quiz and choosing at least 10 of each of the career buttons across all the quiz questions.
  • 60% of people (2,467) who reached final results clicked “Explore” on at least one career.
  • 28% of people (1,052) who had at least one career with Pathways enabled went on to view at least one Pathway.
  • Of the people who viewed a Pathway, 30% (319) clicked at least one button to take them to the next step. And 16% (170) ticked at least one Pathway task to mark it as complete.
  • 26% of people (647) who explored a career viewed jobs nearby.
  • 24% of people (977) emailed their results to themselves voluntarily. This is a 3x improvement from our product a year ago, showing the dramatic increase in value.

Our Impact Metrics

To measure impact, all our users completed a short survey before they started our quiz. We then asked users to complete a survey a couple of minutes after they reached their final results. If they ignored this quiz, we’d automatically pop it up again every couple of minutes until they completed it. 431 users who reached the final results screen completed our “after” impact survey (13% of all users).

  • 24% (88 people) who didn’t expect to one day find a job they’ll love now do as a result of using Would You Rather Be
  • 82% (355 people) feel they now have more information about careers, received tailored information or had their career horizons expanded
  • 52% (223 people) feel they better understand pathways into a new career
  • 60% (257 people) found the current and future job demand labels helpful

Written Application & Video

Emma and I then spent a full week writing our 7,000 word final application and recording a 3-minute video summarising our solution.

Our Final Solution

I’ve shown you snippets of different parts of our upgraded app throughout this blog post. And of course, you can try it out for yourself here and discover your top careers along with your own hyper-personalised pathways into them. But I thought I’d also share the full end-to-end flow of our app so you can see what it all looks like put together:

Another Nesta Prize

Nesta launched another prize that was relevant to us in October called the Rapid Recovery Challenge. This was in response to the pandemic and its impact on employment. We applied for this prize, but sadly didn’t make the cut. We were probably a few months too early, as this was back in October before we had even updated the visual design of our site or even conceived of Pathways.

Other Work

So you can see we’ve been pretty busy the past four months building Pathways, Skills Matching, Marketplace and pulling together our final application for Nesta’s CareerTech prize. But we’ve also done a few smaller pieces of work in that time too. Here’s a quick snapshot of those…

3 More Product Experiments

My last blog post in October was all about the 15 product experiments we ran in 6 weeks. We ran another 3 product experiments soon after, bringing us to 18 in total.

Job Demand Labels

As I mentioned earlier, we added labels in our app back in October to say if any given career has lots of jobs today or is negatively impacted by Covid. In January, we added a couple more labels to say if any given career is future proof or not. We researched jobs that are likely to be in high demand in the future or at risk, using a number of sources. For example, we looked at:

Psychometric Research

In November we opportunistically hired someone with a PhD in psychometrics to do a week’s worth of research for us. Her name is Cat and she did an excellent job of compiling a summary of all the main research relevant to us.

No Degree Filtering

A lot of users don’t have degrees. In fact, the target user group for Nesta’s CareerTech prize are specifically people without degrees. But it turns out that 292 of our 504 careers (58%) typically require a degree to get into them. So if you don’t have a degree, you only have access to about 42% of available careers.

Supporting “Back”

We’ve always had some minor feedback that users sometimes want to be able to go back to change some of their answers in the quiz questions. But if they try pressing “back” on the browser, it will actually take them back to where they came from prior to Would You Rather Be, which is usually Facebook. And that’s a pretty terrible user experience.

Tracking

So far we track what users do by recording user attributes directly in our database. I then build custom stats pages to pull out what we’re interested in. Every time I’ve reported metrics in past blog posts, including this one, that’s how we did it. And it’s a little time-consuming coding a new stats page each time.

Market Research

One of the target customers we’ll be exploring in the coming few weeks will be schools. We’ll start by talking to a number of career leaders and head teachers of schools. But as a starting point, we decided to do some competitor research to find out what products schools currently use for career provision.

  • Morrisby, which is probably the market leader. Their main focus is private schools. And they recently acquired a company called Fast Tomato who also have a popular product. Morrisby are particularly well-known for their psychometric testing.
  • Cascaid, who have an older product called Kudos and a newer product called Xello.
  • My Career Options, who are a newer company. They don’t have much traction yet, but seem to be doing interesting things in the area of psychometrics.
  • Unifrog, who isn’t strictly a competitor as they focus more on supporting students into universities. But they have good traction in schools and a great product.

Hiring Employee #2

And the last thing we did was hire our second employee!

  • I founded the company in June 2019 and worked part-time on my own for the first year
  • I’ve been full-time since June 2020
  • Emma joined me full-time in August 2020 and there’s only been two of us full-time since
  • Anna also worked with us full-time on frontend development from October 2020 to February 2021

Hiring Process

Emma led the charge with this hire and we posted a role titled “Research and Data Officer”. We paid for prominent placement on Escape the City (which cost £160). As that site yielded the best candidates when I hired for Employee #1 (and it’s how Emma ended up working here!). We had 120 applicants in total, with Escape the City providing the best quality ones again.

Hiring Reflections

We put a lot of effort into hiring, onboarding and all people-related activities. Our people are the heart of our company and I believe it will be the biggest determinant for our long-term success. Of all the things we do and spend time on, hiring is one of the most important. Not to mention that our mission is career happiness, so it’s pretty important that we hire the right people for what we need and do everything we can to make sure they’re really happy in their roles.

What’s Next

We have one very specific goal for 2021. And that’s to make £100k in revenue. Our focus this year is commercialisation so that we can sustain and scale our impact in the future. But we won’t compromise our social mission along the way.

--

--

Tech for Good Entrepreneur

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store