How to assemble a crack AI team
Applied correctly, AI can revolutionize your business, help you streamline processes, create exciting new products, and fully harness the data at your disposal.
So how do you go about setting up a team capable of turning raw data into new technology? What are the skills and types of people you need and where do you find them? And does your approach differ from software engineering projects?
Last week we spoke to The Register about the skills and experiences you need in your team, and what that will help you achieve. Check out the resulting article for a great introduction to the topic; our full interview answers below give you further details about our approach and results in two of our development offices: Hatfield (UK) and Krakow (Poland).
Roland Plaszowski, software development manager, thinks any AI team should have a diverse skill set
It was almost two years ago that we started building our data science team from scratch in Poland. There were some challenges along the way and it took almost a year to build a full team.
There are different ways to build up ML and AI teams in a company; there is usually no one-size-fits-all solution as it strongly depends on the structure of the whole organisation and the business end-goals. In our case, we knew from the beginning that our AI/ML team had to be self-sufficient. We wanted to be able to deliver end-to-end solutions and support them independently. Our priority was to minimize lead time and deliver value in production environments easier.
To achieve that, we had to create a diverse team. We generally believe it’s important to have diverse teams and interdisciplinary skills in all of our software development teams. We were looking for few different skill sets:
- Software engineers that were interested in data science to help us build the platform. Software developers are very important because they can introduce the team to good software practices, both for the development and the later deployment of the solutions.
- Data engineers that could help the team effectively work with the data. Preparing data sets and feature lists is an important part of every ML project, and having people with such expertise makes it a much more efficient and less frustrating process.
- Statisticians and mathematicians that would help the team understand data and theoretical aspects. This is an important part of being sure that the solution makes sense for the data that we have; it is also a big part of the exploratory analysis. Secondly, mathematicians often have a wide knowledge of various algorithms and approaches to many common problems. They also deeply understand how everything works under the hood.
- Business experts: these are not necessary domain experts, but people that can work with stakeholders to understand their goals, and help them understand the approach. Great communication skills are very important for that.
It’s hard to find people that have all skill sets above, but we tried (and succeeded!) in hiring people with at least two of them. It’s also very important to have people that understand and are driven by bringing value to the business. It can be easy to get stuck in research mode for a long time and forget about the value of your work to the business. You always need to make conscious decisions based on the data but also on cost/value analysis.
In our approach, we tried to identify people with potential that could develop over time. Of course, you can now find a whole market focused on data science, but we’ve also noticed that statistician and data analyst positions are also becoming more and more common. In addition, you often find that people from computer science, maths or physics backgrounds can also be well suited (in some extension) for such roles.
People will also use different skill sets with different intensities depending on the state of the project. However, it’s worth keeping in mind that any project has a lifecycle. It’s worth having software engineering people at the beginning, even if you are doing research or analysis. They might help discard options that are not maintainable. On the other hand, even if you are in the late stage of the project doing development work, a statistician or business experts helps a lot to advance the project and understand the different issues that might arise.
My colleague Marcin Druzkowski wrote a very interesting article about his most recent experience with starting a ML project at Ocado Technology. I definitely agree with his view that ML/AI projects need a significant amount of business domain knowledge; having a deep understanding of the wider business is very important. It is crucial for every software project to closely cooperate with the business stakeholders, but for ML/AI projects this becomes especially important. Without their knowledge, it’s much harder to interpret results and propose solutions. Also, ML/AI projects have slightly different needs (as described above): exploratory analysis; research; and usually more trials, versions and experiments are needed.
But generally speaking, the differences between generic software projects and ML/AI projects are not very noticeable. We work hard to make sure that tests, continuous integration, monitoring, automation and documentation are in the project from the beginning, just like any other software engineering project.
Tim Bickley, data science team leader, highlights the importance of creating business value
In the Ocado Technology data science team, we have always had a mix of backgrounds: mathematicians have been the largest single group, but we’ve also had great people from computer science and the sciences. The key things we look for are relatively strong mathematics, a proven track record of independent research and problem solving, and solid programming skills. We find the team benefits from having some people who are particularly strong in one area or another, but doesn’t work so well if someone is outright weak in one.
As the ML/AI field is evolving so quickly, a proven ability (and desire) to quickly learn new technologies is almost more important than pre-existing experience, but of course that experience is very beneficial if we can get it. PhDs are desirable but neither necessary nor sufficient; we’ve had great people without them and the occasional interviewee with them that made us wonder if they found it in a cereal packet.
Finally, our people must be able to look at the problem in a business context: understand the right metrics to look at; how the system will solve a business problem; and come up with the best solution to that problem – even if that means the solution is linear regression rather than a neural network.
Our projects tend to be small in terms of the number of people involved, with one or two people running it to the end. Depending on the situation, software developers from the relevant area will be involved to a greater or lesser extent to ensure that the system will integrate correctly and that we get the necessary data in place. A really important point that people tend to forget is that without data, there can be no AI.
I’d say the main difference between ML/AI and more generic software engineering projects is that what we do is more towards the research end of the spectrum. We are exploring to see if a good solution exists, not simply implementing a requirement. This means our projects have multiple phases of exploration before we either abandon the idea as not currently viable or develop something good enough to trial in production.
We have done pretty well over the years in getting some great applicants coming through, based on a combination of factors: direct recruitment, general awareness in the community, or relying on social network. As we publicise our work more and more, the Ocado Technology brand is an increasingly powerful draw, and the ability for us to start our interviews with a tour of the Hatfield automated warehouse always leaves candidates suitably impressed. We do a fairly rigorous technical interview that gives us a good view of how the applicant thinks and, so far, we’ve been able to keep the bar high and still get enough people past that filter to keep us staffed.
Alex Voica, Head of Technology Communications