Menu
Media file

The difficult middle: turning AI pilots into business impact

November 5, 2026 - 10 min read

At the AI Commerce Summit in Porto, presented by Metyis and Google, perspectives from across commerce, retail, technology, analytics and business transformation converged on a common conclusion: proving AI's potential is no longer the challenge. The real test lies in building the conditions that allow it to deliver measurable impact at scale.

An AI model can perform impressively in a controlled setting, supported by selected data, expert supervision, and a clearly bounded task. Production introduces a different standard, where AI must work with imperfect data, existing technology, regulatory requirements, budget constraints, customer expectations, and employees with different levels of confidence and expertise. In that environment, it must create enough value to justify not only the investment required to build it, but also the ongoing cost and organisational effort required to operate it.

This is the difficult middle: the point at which AI stops being an impressive possibility and starts being accountable for a business outcome.

“Most AIs go to the same place where all good demos and pilots go to die: a beautifully designed slide in a beautifully designed deck that nobody ever opens again. This is rarely a failure of the technology. The model was fine. The results were promising. The only thing that was missing was everything else: the right data to make it work, the right processes to run it, and the courage to let it anywhere near a real customer, let alone scale it across an organisation.”André Covas, Associate Partner Design and Portugal Campus Lead at Metyis

The challenge, then, is not simply finding more applications for AI. It is determining which problems justify applying it, understanding the processes and information those applications depend on, and creating an operating model in which successful experiments can become dependable capabilities.

Throughout the AI Commerce Summit, speakers approached that challenge from different perspectives, but many returned to the same themes: starting with a meaningful business problem, understanding the processes and information that AI depends on, balancing experimentation with governance, and measuring value before scaling investment. The lessons below bring those ideas together into a practical framework for organisations looking to move from experimentation to measurable impact.

Practical lessons for scaling AI


1. Start with the business problem, not the availability of AI

The growing number of potential AI use cases can create the impression that progress begins by choosing a tool. In practice, however, lasting value tends to start with a less fashionable question: what problem is worth solving?

A use case anchored in a real point of friction already has a reason to exist because the business understands who experiences the problem, what the current process costs, and which outcome would constitute an improvement. AI can then be assessed against that outcome. When the sequence is reversed, however, organisations risk investing in sophisticated solutions before establishing whether the underlying question is commercially important in the first place.

This discipline becomes even more important as AI lowers the effort required to build prototypes. If more ideas can be tested, organisations need a stronger mechanism for deciding which deserve to progress, because technical feasibility alone is not enough. A use case must also address a recognised customer or operational problem, fit the economics of the business, and improve an outcome that matters.

As a result, organisations may need to think differently about their AI portfolios, because the most valuable applications will not necessarily be the most sophisticated or visible. An application that improves a repetitive operational task, removes friction from a customer journey, or helps an employee make a better decision can create more meaningful value than a technically impressive use case without a clear business need.

The standard, therefore, should not be how advanced an AI application appears, but how clearly the investment connects a meaningful problem to a measurable outcome.

2. Build the foundations before automating the business

Choosing the right problem is only the beginning. While AI can accelerate a process, speed does not correct a process that the organisation itself does not properly understand and can simply allow the wrong work to happen more efficiently.

Previous waves of digital transformation often transferred inherited workflows into new systems without reconsidering the logic behind them. AI makes the same mistake easier to repeat because a model can produce an apparently coherent workflow from incomplete instructions. When the people specifying the task do not fully understand the process, the model inherits an inadequate version of the organisation's own knowledge.

The purpose of understanding the manual process is not to preserve inefficiency. It is to identify which steps matter, where judgement enters, how exceptions are handled, and what a correct result looks like, so the business can decide what should be removed, redesigned, automated, or retained as a human responsibility.

This becomes particularly important with AI agents, where apparent autonomy can obscure how dependent an agent remains on organisational context.

“It’s really about process knowledge and ownership of the process. If you don’t understand the process you’re trying to improve, you’ll fail. The first step is putting people in charge who truly understand the process. The second is context. At HUGO BOSS, we sometimes call it corporate memory. What’s the context that the agent uses to make decisions? Process knowledge and context ultimately determine the outcome of the agent.”Henrik Milthers, Head of Innovation and Project Management of Global eCommerce at HUGO BOSS

Corporate memory is a useful way to frame the challenge. Policies, historical decisions, product knowledge, customer expectations, and exceptions are often distributed across systems and experienced employees rather than captured in one coherent source of truth, and AI cannot compensate for missing organisational context simply by becoming more capable.

Data belongs to the same foundation, yet businesses can discuss increasingly sophisticated AI applications while the product, customer, or operational information those applications depend upon remains fragmented, incomplete, or unreliable.

As AI takes on more decisions and interactions, the consequences of weak information become greater because a data-quality problem that once affected an isolated touchpoint can potentially influence an AI-generated answer, recommendation, or action.

“Many companies first need to fix their foundations. It’s great to talk about all these new capabilities, but if your systems aren’t working, your data is a mess, your systems don’t talk to each other, and nobody can generate usable insights, then don’t invest in the next shiny thing. It won’t bring you anything.”Stijn Groenink, Senior Partner and Global Lead Digital Commerce Solutions at Metyis

A strong foundation is therefore not simply more data. It is accurate, connected, and usable information organised around the decisions the business needs to make, because data quality is no longer an infrastructure concern hidden beneath the experience. Increasingly, it determines whether that experience can be relevant, accurate, and trustworthy

4. Give experimentation freedom, but production discipline

Strong foundations make AI more dependable, but organisations still need enough freedom to discover where it can create value.

Exploration and operational control are sometimes treated as competing philosophies, with one encouraging employees to experiment locally while the other centralises control over tools, models, data access, risk and expenditure. In practice, scaling AI requires both, but at different points in the journey.

Exploration benefits from proximity to the work because people in product, finance, design, operations or customer service encounter different problems and will use the same technology differently. As a result, their experimentation can surface opportunities that a central AI programme may not anticipate while also helping AI develop as a practical capability rather than remain the responsibility of a specialist team.

Production changes the obligation because once an application uses company information, influences a customer interaction, enters an operational system or informs a decision the business may act upon, experimentation requires more explicit boundaries around data access, approved technologies, security, legal requirements, cost and accountability.

The principle is therefore not governance as an approval layer added at the end. It is governance designed around the different risk profiles of learning and operating.

This distinction also preserves human responsibility because AI capability should not be measured simply by how frequently employees use it; the meaningful test remains whether it improves the problem they solve, the decision they make or the value they create.

Critical judgement becomes more important precisely because AI can produce plausible work at speed. Employees need to understand how to use these tools, but also how to challenge their outputs, recognise when context is missing and know which decisions should not be delegated.

The objective is not maximum AI adoption. It is productive experimentation that has a credible route into accountable operation.

5. Use evidence to decide what deserves to scale

Moving from experimentation to business impact ultimately requires an economic logic. An AI initiative should define the outcome it is expected to improve, the cost of pursuing it, and the evidence required to justify the next investment decision.

That does not mean every experiment must generate an immediate financial return because experimentation creates learning, and some tests should fail. The organisation should, however, understand what it is trying to learn and how that evidence will influence what happens next.

“Don’t copy and paste whatever you read online. Think about it. You can ask AI to help you think about it. And then test. Start small and test. And small doesn’t mean you then wait until next year to review the P&L. You can do it every month. If you can close the books faster, you can iterate every week. Some of the ideas we kill. The ones that generate money generate so much money that they become completely self-funded.”Luis Carvalho, Chief Technology Officer at Farfetch

The same discipline matters when evaluating customer and commercial outcomes because visible activity is not necessarily incremental value. Attribution, Marketing Mix Modelling and incrementality answer different questions about how credit is assigned, how investment should be planned and whether an intervention actually caused additional results, but together they can provide a stronger basis for deciding where investment should move. Evidence also influences adoption inside the organisation.

“The biggest success factor is evidence. If you can clearly demonstrate the value created by AI, a tool, or a new capability, people come on board more easily. That’s also the hardest part: proving the actual impact of the investment. But evidence is what drives adoption and creativity.”Dominik Zellner, Head of Engineering at Mercedes-Benz.io

This creates a reinforcing cycle: evidence supports trust, trust makes adoption easier, adoption provides further learning, and that learning gives the organisation a stronger basis for deciding which applications deserve to progress. Rather than scaling AI because an experiment is technically successful, organisations can scale because they have learned enough to demonstrate that the capability solves a meaningful problem under real business conditions.

Why the difficult middle matters now

The gap between pilots and business impact is rarely explained by one missing technology. Instead, it appears when promising applications encounter the organisation around them: processes that need to be understood, information that needs to be dependable, experimentation that needs a path into production, and investment that needs evidence. As a result, scaling AI is not principally a race to deploy more advanced models, but an operating challenge.

“The companies that are successful are not necessarily the smartest companies in terms of tech. They are the smartest companies in choosing the right tech and then adopting it in a way that actually changes how they run their business. So the how is vital.”Stijn Groenink, Senior Partner and Global Lead Digital Commerce Solutions at Metyis

That “how” is the difficult middle. The pilot proves what AI can do, but the harder test is whether the organisation can connect it to a problem worth solving, create the conditions in which it can operate reliably, and demonstrate enough value to justify scaling it. That is where possibility becomes an operating capability and where AI becomes not merely impressive, but useful, accountable, and commercially relevant.

More impact