Pilots vs Experiments
Updated: Sep 14, 2020
I have recently been working on some internal proposition development, expertly facilitated by Matt Cushen, and he brought something to the table today which really resonated with me: the key differences between pilots and experiments.
Pilots are typically fully formed ideas which can be expensive, involve a lot of resource, are long term, and where the risk of failure is high.
Experiments can be just part of an idea, frequent, quick to deploy, with a focus on learning, and where the risk of failure is low.
Albeit subconsciously, I’ve certainly observed these differences and the associated difference in behaviours during my career, and today this led me to reflect on a recent piece of work we delivered for a client in a way that I hadn’t previously considered.
Right at the start of lockdown I was approached by one of our clients with a simple question; “could we do deliveries?” Sure, delivering stuff to stores, or managing the logistics of a large brand experience event is a critical part of our day job, but we would in no way consider ourselves to be delivery company!
Here was a client presented with the immediate and unexpected problem of getting products into the hands of their B2B customers, and ultimately to their end consumers, in a way that was instantly no longer possible through their usual channels and business model. Their specific needs meant that this wasn’t just a case of figuring out the logistics of shipping a bunch of product through any of the well-known national delivery networks, this needed a much more premium and personal touch right from the point of order through to the point of delivery, and crucially, a way of taking either cash or card payment at point of delivery. As a trusted partner, they turned to us for help. They had the demand, but they needed someone to take the entire problem of fulfilling that demand off their hands and solve it for them end to end, and quickly.
The following statistic seems unbelievable (to the point where I have sometimes extended it to make it sound more plausible): we went from that initial, speculative question from the client, to making our first deliveries to their customers just 4 days later. In the 3 months of lockdown they have now been able to collect and fulfil thousands of orders that would not have otherwise been possible. Maybe even more importantly, it’s transpired that this method of fulfilling their B2B demand is potentially more effective in terms of demand generation, more operationally efficient, delivering a better consumer experience, and ultimately more commercially effective than the way they had been doing business previously.
My reflection today was around how this might have transpired under “normal” circumstances; if the coronavirus impact had never necessitated the immediate pivot that so many businesses have had to employ to survive.
Firstly, it is almost a certainty that the brief would have never crossed my desk. Why would a client with a commercially successful business model feel the need to flip the status quo and divert so drastically from their current operation? The logic just doesn’t make sense on the face of it.
Secondly, if for whatever reason this had been an idea from the client, I can easily see how that plays out in reality; something we would have all seen at some point in our careers. A lengthy process to scope it out with no fixed timeline or end point. Endless project meetings. A huge group of stakeholders. Iteration after iteration of business case. Every possible scenario meticulously thought through and documented in a process flow. Multiple levels of sign-off, maybe finally agreeing to run a pilot with a subset of customers or in a selected geographical region... and so on and so on.
If after all of that it was decided that there might be a commercial case to answer and the brief finally landed with us, we’d no doubt have a similar process to run through. How will this work? Who needs to be involved? How do we recruit? What systems do we need? How do we price it? When could we start it?
Don’t get me wrong, getting this particular job off the ground was a herculean effort by the iD team, particularly under lockdown conditions, trying to bring this to life in a matter of days via video calls! But it was a small team, very much focused on turning an idea into an operation as quickly as possible, with limited resource and budget. Consequently, we’ve all been able to learn as we go, focusing our efforts only on the elements which require it, and optimising the parts which need optimising. We were constrained in many ways, but equally free from the usual constraints in others, it was invigorating!
When we first launched, we were using various Excel spreadsheets to manage the order and delivery status. We begged and borrowed iPads from other projects which we could turn into mobile EPOS systems straight away. We were briefing our frontline team on the fly and flexing the process daily based on their feedback. None of this was perfect, but with a single-minded focus on getting things up and running immediately, it worked and served its purpose phenomenally well. Three months in and we have since invested in using app-based delivery management tools. We’ve automated our reporting. We have glossy briefing packs for new delivery recruits. We’ve established how and where we can be more efficient, whether through direct cost or time. Crucially, we’re focusing only on the 20% of project that needs optimising, rather than the 100% of the project that we’d still be sat around discussing had it not been for the incredibly time-sensitive nature of the Coronavirus impact.
In one way or another, the last three months has been an exceptionally testing time for all of us, but if there’s one lesson that I’ll take away, it’s that there’s a huge opportunity to demonstrate entrepreneurial behaviours with great benefit when we’re constrained, be it by design or by factors beyond our control. Going forward if I have a glimmer of a good idea, I want to remember this experience and employ the same mindset. How can I test this? How can I test this with limited funds? How can I test this with limited funds and start testing it this week!? Great things can happen from an experiment, particularly when the worst case is that you learn something.
If you’ve got the very first spark of an idea; maybe you’re an online brand considering physical retail, an established retailer looking to try something different, or maybe you just have a problem that you’re not sure how to even start solving, then please don’t hesitate to get in touch. I don’t want to charge you for our time, I don’t want to hold you to a commitment of promised work, but nor do I want our collective teams sat through weeks and weeks of project meetings to conclude that a fully formed project scope just doesn’t stack up on paper!
In my line of work, we’re always looking for long term partnerships, but right now nothing gets my adrenaline pumping like an experiment.
Head Of Retail Operations