The problem we still are not seeing…
There are curious ideas. Some appear out of nowhere and disappear just as quickly as they arrived. Others, however, keep circling around in your head for months, even years, waiting for the right moment to take shape. The idea I want to share in this article clearly belongs to the second group.
For quite some time now, I have felt that something does not quite fit within the current conversation around Artificial Intelligence, Agentic AI and autonomous agents. I see organisations developing copilots, virtual assistants, specialised agents, multi agent ecosystems and entire cognitive automation platforms. I see vendors announcing new capabilities practically every week. I see consultancies presenting adoption frameworks. I see analysts talking about productivity.
But there is one question that keeps coming back to my mind again and again: How are we going to govern all of this when it stops being an experiment and becomes real operational practice?
A few months ago, I began exploring this reflection in greater depth. After reviewing various academic papers related to the quality and management of AI agents, including some very interesting approaches focused on defining service level agreements for intelligent agents, and after attending a particularly inspiring session in London hosted by SysAid, I eventually reached the conclusion that it was worth writing this article and sharing these ideas with all of you.
Partly because I had the feeling that many of the conversations we are having about AI are centred on technology, when perhaps the truly important questions lie somewhere else. Not because I have all the answers. Quite the opposite.
In fact, the more I read, the more conversations I have, and the more projects I see emerging, the more questions arise. And when you reach that point, one of two things usually happens… either you keep quiet to avoid complicating your life, or you write an article to see whether, together, we can find better answers. I have chosen the second option, hehe. Perhaps because I have reached “an age” where “complicating my life” has become part of my personal job description, hahaha.
The funny thing is that this feeling is not new. Those of us who have spent quite a few years working around disciplines such as Corporate Governance, Service Management, Organisational Transformation or Enterprise Service Management have seen similar situations many times. The technology changes, the vendors change, the acronyms change and even the colours of the presentations change, but there are certain patterns that repeat themselves with almost mathematical precision. First, a new technological capability appears. Then come the use cases. Later comes the phase of collective enthusiasm where it seems that all of humanity’s problems will be solved sometime next quarter. And finally, a far less attractive but infinitely more important question arrives: how do we manage all of this?
We saw it with ERP. We saw it with ITSM. We saw it with Cloud. We saw it with digital transformation. And now we are starting to see it with Artificial Intelligence. In fact, I would even dare say that a significant number of organisations are making exactly the same mistake they made in previous eras.
We are spending enormous amounts of time talking about capabilities, but very little time talking about responsibilities. We are talking about what agents can do, but not about what they should do. We talk about autonomy, but not about limits. We talk about productivity, but not about accountability.
We talk about automation, but not about governance… and that should concern us a little bit. Quite a lot, actually. Because when you take a closer look at what is happening right now, you discover something particularly interesting… we are no longer talking about simple tools. Nor are we talking solely about conversational assistants capable of answering questions. What is beginning to arrive in organisations are entities capable of interpreting context, making decisions, executing actions, interacting with other systems, coordinating processes and even collaborating with other agents in order to achieve specific goals.
In other words, we are beginning to incorporate new participants into business operations… and this is where my mind starts making a lot of noise. Because, for decades, we have developed mechanisms to govern people, teams, departments, suppliers, services, processes and technology, it seems somewhat strange to assume that we can introduce an entirely new category of organisational actors without developing a specific model to manage them.
Perhaps the problem lies precisely in the way we are seeing them. We continue to talk about agents as if they were simply software. As if they were just another application in the company’s technology catalogue. As if they were a slightly more sophisticated evolution of the tools we already know. Yet I am becoming increasingly doubtful about that interpretation.
A traditional system does what it was programmed to do. An agent, on the other hand, begins deciding how it is going to do what it was designed to do… and that difference is enormously significant. So significant that I suspect we have still not fully understood all of its implications.
For example, when an organisation introduces a self-service portal, a BPM tool or an ITSM/ESM platform, the expected behaviour is usually perfectly defined, or at least that is always the intention, hehe. Processes, workflows, rules and even exceptions are known. However, with an agent, grey areas begin to emerge.
What happens when it misinterprets a situation? What happens when several valid alternatives are available? What happens when information is incomplete? What happens when it must choose between speed and accuracy? What happens when it coordinates actions with other agents? What happens when a decision it makes has financial, operational or even reputational impact? …and above all: Who is responsible for all that? Because let’s be honest, at some point someone is going to ask that question, aren’t they? Sooner or later, it always happens.
Which leads me to think that perhaps we are having the wrong conversation… or at least an incomplete one. Perhaps the real challenge of the next stage of Artificial Intelligence will not be technological. Perhaps the real challenge will be organisational… or perhaps the challenge is not to build increasingly intelligent agents, but rather to learn how to live alongside them in a sensible, controlled way aligned with business objectives.
…and well, that is precisely where the idea I want to develop throughout this article begins to take shape. An idea that is still evolving, that will undoubtedly improve through debate, and that will probably require many more conversations before it reaches the required level of maturity. But every idea needs a starting point… and mine is fairly simple, at least according to me, hehe.
If we have spent years using Service Level Agreements to govern service quality, perhaps the time has come to ask whether organisations will need something similar to govern their Artificial Intelligence agents. Perhaps the time has come to start talking about AI Agent Level Agreements… or simply ALAs.
I believe that “…if I am right”, then the truly interesting thing will not be understanding what they are. The truly interesting thing will be to understand why traditional SLAs are starting to fall short in the world we are building.
SLAs were designed for services, not for agents…
There is something that I find particularly interesting when I observe how organisations are beginning to incorporate Artificial Intelligence into their operations… and that is that, almost without realising it, we tend to interpret any innovation using mental models we already know. It is completely natural. In fact, we do it constantly… more often than we probably should.
When new technologies appear, we usually try to fit them into familiar categories because that helps us understand them more quickly. The problem is that, every so often, something appears that does not quite fit into any previous category… and when that happens, continuing to use old frameworks of thought often marks the beginning of many mistakes, and you know that perfectly well.
Personally, I have the impression that this is exactly what is happening with Artificial Intelligence agents. Because when we talk about them, most organisations seem to assume that we are simply talking about a new generation of digital services… and I am not entirely convinced that is the case.
In fact, the more deeply I explore this topic, the more convinced I become that we are trying to manage a new reality using conceptual tools designed for a completely different, and older, reality.
Let us think for a moment about SLAs. For decades, they have been a fundamental component of service management… and rightly so. They have allowed us to define expectations, establish commitments, measure quality levels and manage relationships between suppliers, customers and business units. Thanks to them, we have professionalised service delivery and transformed many subjective conversations into objective, data driven discussions.
I do not believe that anyone who has worked seriously in service management could question the importance of a good SLA, although they do exist even though many people have rarely seen one, hahaha… I certainly would not. However, I also believe that we must recognise something important. SLAs were designed to govern services. They were not designed to govern agents… and although the difference might seem small, I suspect it is much bigger than it appears.
When we define a traditional SLA, we usually concern ourselves with fairly specific matters. Service availability. Response times. Resolution times. Capacity. Performance. Continuity. User satisfaction… and so on. They are all perfectly reasonable variables when we are talking about services.
But now let us conduct a little thought experiment. Imagine an AI agent participating in the recruitment process. I am not talking about publishing vacancies or sending emails automatically. I am talking about an agent capable of analysing candidates, cross-reference information, identifying patterns and proposing recommendations to hiring managers… now imagine that this agent recommends rejecting a particular candidate.
Which SLA metric helps me evaluate whether that recommendation was appropriate?
Availability does not. Response time does not. Processing capacity certainly does not… because the problem is no longer related to service delivery. The problem is related to the quality of judgement the agent has applied for during the process… and that changes the entire conversation completely, you are getting the idea now, aren’t you?
The same would apply to a financial agent capable of approving certain expenses, a procurement agent interacting with suppliers, or an agent coordinating activities across multiple business areas. In all these situations, we can continue measuring times, availability and performance… and we probably should keep doing so, I would say, wouldn’t you?
But those metrics are no longer sufficient because the real risk no longer lies solely in whether the agent works. The real risk emerges when the agent makes the wrong decision while functioning perfectly… and I believe that distinction is fundamental. It takes us into situations and circumstances that, until yesterday, were neither our concern nor “our problem”.
For years, we have become accustomed to evaluating systems based on their ability to execute predefined tasks correctly. However, agents are beginning to introduce something entirely new into the organisational equation… they introduce judgement, their own judgement perhaps, but judgement nonetheless… and when judgement appears, questions also emerge that SLAs were never designed to answer.
What level of autonomy is an agent allowed to have? What decisions can it make without human supervision? Which decisions require validation? What risks can it assume? What information can it use? What power does it have to act on other systems? What level of traceability must it maintain? What happens when several valid responses exist and it has to choose one of them? And who answers when that choice turns out to be wrong? …and a thousand other questions.
Many years ago, when we first started talking about Enterprise Service Management, many of us tried to explain that service management was not simply about implementing tools. It was about establishing rules, responsibilities, operating models and governance mechanisms that allowed organisations to function coherently. That same reflection now stands before us once again, but within a completely different context.
Because perhaps the real challenge of Artificial Intelligence is not building increasingly intelligent agents. Perhaps the real challenge is learning how to integrate them into organisational structures that were designed over decades to manage people, services and traditional technologies… and that leads us to a particularly uncomfortable and rather “existential” question: if an agent is no longer simply an application, nor exactly a service… but increasingly resembles an entity capable of acting, deciding and generating impact within the organisation… then, what exactly should we be agreeing, measuring and governing?
Because I suspect the answer no longer fits within a traditional SLA… and precisely for that reason I am beginning to believe we need a new conversation. One that does not focus solely on the quality of the service provided, but also on the quality of the expected behaviour… one that talks not only about availability and performance, but also about autonomy, limits, responsibility, risk and value creation. A conversation that will probably lead us towards something that does not yet formally exist in most organisations, but which I would be willing to bet will arrive sooner than we imagine: AI Agent Level Agreements.
How do you implement an ALA in practice?
Up to this point, we have talked about the concept, but if there is one thing I have learned over years working in organisational transformation, it is that every good idea eventually faces the same question: “Okay JuanMa, it sounds great… but how do I actually implement it?” …and, honestly, I think that is exactly the right question.
Because an ALA should not become yet another corporate document nobody reads, another spreadsheet lost in a folder, or a fifty-page policy that only gets consulted when something goes wrong. If ALAs ever come to exist formally within organisations, they should become living governance tools, integrated into operations and directly linked to the way we manage agents throughout their entire lifecycle.
…and well, I am convinced that this is where an undeniable and especially interesting connection appears with #TheESMGuide and the A.R.T.E. methodology. Why? Well, because implementing ALAs would not really be an Artificial Intelligence project. It would be a governance and management project… and that completely changes the perspective, you know that already, don’t you?
The first thing an organisation should do would not be to define metrics. Nor deploy technology. And certainly not start building agents simply because the market is talking about them. The first thing it should do is answer a very simple question:
Where does it make sense to incorporate agents within our operating model?
…and yes, it sounds like an obvious question, especially for you who already knows everything, hehe, but I suspect that over the next few years we will see many organisations doing exactly the opposite… you will see, they will build agents first and only afterwards go looking for problems to assign to them.
We have watched that film several times before with other technologies… and it rarely ends well, even if you find that difficult to admit. That is why, before talking about ALAs, the first step should be clearly identifying which processes, services or capabilities could genuinely benefit from intelligent agents. Because, in reality, not every process needs AI, not every service need autonomy, and not every problem needs an agent, no matter how excited you get or how innovative you feel suggesting it… and recognising that is also part of a good strategy.
Once the use case has been identified, the next step emerges defining the agent’s organisational role… and this is where I think many organisations will discover something interesting, because an agent should not be viewed as a tool. It should be viewed as an operational actor.
In other words, just as we define roles and responsibilities for people, and occasionally we even manage to do it properly, we should define roles and responsibilities for agents. What is its mission? Which process does it participate in? Who owns it? What authority has it been granted? What risks does it assume? What value is it expected to generate? …notice that we still have not talked about prompts, models, tokens or platforms… we are still talking about governance, and of course, that is no coincidence.
Personally, I imagine an ALA functioning as a sort of governance profile attached to the agent. A living document capable of quickly answering fundamental questions about expected behaviour. Questions such as: What objective does it pursue? What data does it use? What actions can it execute? What decisions can I make? Which decision must it escalate? What metrics will evaluate it? Which risks are considered acceptable? Who is responsible for supervising it? How frequently will it be reviewed? …and, as you can imagine, many more besides.
If you think about it, many of these questions look remarkably similar to the ones we already use when designing services, processes or governance models… and that is precisely why I believe #TheESMGuide provides such a strong foundation for initiatives of this kind. Because an organisation that already has governance, a service catalogue, process management, measurement mechanisms and a clear operating structure starts with a huge advantage over one attempting to introduce agents into the middle of chaos.
In fact, I would even dare to state something that may not be especially popular, but then again, it is difficult to be liked by everyone, hahaha. I believe that before implementing intelligent agents, we should be capable of managing our services properly. I know. It is not a particularly futuristic statement. It does not sell many licences. But I genuinely believe it is true… because we already know that automating disorder still creates disorder. Just faster.
So, once the agent has been defined and the ALA documented, the next major responsibility consists of supervising its behaviour during operation… and this is where the ALA stops being a document and becomes a management instrument.
Organisations should, of course, monitor traditional elements such as availability and performance. But they should also monitor much more interesting aspects: level of autonomy exercised, quality of decisions, volume of escalations, exceptions generated, errors detected, deviations from policy, value generated for the business, risks materialised… and so on, and so on.
Because an agent that technically meets its objectives could still be creating significant organisational problems… and this is precisely where traditional models begin to fall short.
Finally, we come to a stage that, interestingly enough, is often forgotten in many AI initiatives… periodic review… and this is where I believe A.R.T.E. once again has a great deal to contribute. Because if there is one thing the methodology advocates, it is that every transformation should be measured, optimised and continuously evolved.
An ALA should never be static. An agent’s autonomy may increase. Its responsibilities may change. Risks may evolve. Business needs may be modified. Therefore, the ALA should evolve as well. Exactly as services, processes and governance models evolve.
We could summarise this entire idea in a single sentence: An ALA should not be used to control agents. It should be used to ensure that agents evolve in alignment with the organisation’s objectives, risks and capabilities.
Because, at the end of the day, that has always been the true objective of governance. Not control for the sake of control. Rather, creating the conditions necessary for things to evolve safely, sustainably and while delivering value… and I suspect that when agents become a common component within our organisations, that is exactly what we will need to do with them.
Thank you for making it this far. I am sure you will be able to take considerable value from this entire approach, and if you feel like becoming a little “existential” and sitting down to philosophise about it, count me in…
I hope you have a fantastic week! ✌🏻😊
