Innovating responsibly in the AI era

For Avencera, AI brings opportunities to reinvent not only how we work, but also what we can achieve. In recent months, we have been working on a wide variety of innovations both for our clients and within our business.  

Through our client-facing tools in Project Shybird, we’ve developed a system that delivers a fundamental rethinking of the education system – creating a pipeline from live labour market data through to course creation to ensure that learning is dynamic and responsive to the skills that are actually required for employment.

At the same time, we’ve revamped a number of internal Avencera processes to streamline administrative work and information access for our staff, and have developed specialised agents to supercharge our consultants working on specific propositions like bid capture.

However, capturing these benefits while minimising downsides requires careful consideration of AI safety, security, and governance. AI carries a swath of real risks, some of which are inherent to the technology and some of which arise during implementation, adoption or deployment.

We are as committed to mitigating AI’s risks as we are to unlocking opportunities with it and are guided by the following principles across our AI development and adoption work.  

1 Build trust

Innovation means change, change creates uncertainty, and uncertainty creates risk. Therefore, responsible AI innovation must reduce uncertainty to support change. Talking about outcomes is necessary, but building trust must come first.

Developers, staff, and users of our products should be able to easily understand the considerations we take in AI development and adoption, and be confident that we always deploy this framework in our innovation work.

This is particularly important in a space as fast-moving as AI, where we’ve seen an impressively fast series of large changes not only in LLM development, but also in adoption and governance work. In response, our framework is designed to be future-proof – that is, flexible enough to ensure our principles are applicable despite changes in AI capability.

2 Establish oversight

Clear, independent oversight is critical to ensuring AI is developed and implemented with impact in mind. To that end, we have established a robust oversight system for our AI innovation work, consisting of an internal AI Board which sets our AI policy, risk appetite, and escalation routes. They are supported in this work by our Responsible AI Panel, which is composed of evolving sector experts who provide domain-specific advice and challenge to the Board.

All AI development at Avencera passes through this governance process. Importantly, the Board’s remit is to focus exclusively on the impact and risks of AI tools we develop. The board has veto power – any development deemed too risky can be halted, paused, or sent back to developers with recommended changes before re-review.

3 Identify the use case

During ideation and solutioning, we clearly define the problem being solved, the intended users, and the outcomes we are trying to achieve. This helps ensure AI is used where it can deliver genuine value and not add unnecessary risk, as well as provide a clear basis for further risk assessment and success criteria.

4 Assess risks and impacts

One of the most important tenets of responsible AI is an attention to the downstream impacts of the technology, with a view to minimising harm and risk. We consider all stakeholders involved in a system’s lifecycle, including the users of our AI products, the users of outputs generated by our AI products, the wider workforce, developers, the creators of data used to train models, and (in the case of Shybird) the education system as a whole.

We ask questions like ‘what would large-scale deployment of this system mean for stakeholders?’, often directly to stakeholders, and we ask multiple times – at ideation, solutioning, development, testing, and deployment.

In identifying impact, we focus on assessing risk across the following risk categories, which are increasingly becoming Responsible AI standards: safety and security, validity and reliability, accountability and transparency, explainability, and fairness and bias prevention.

5 Protect data and privacy

Importantly, before a single line of code is written or system deployed, we assess the data we expect to provide to an AI system to ensure inputs are appropriate, high-quality, lawfully obtained, and proportionate to the intended purpose. This applies equally to training data, operational inputs, and information generated or processed by other AI systems. In particular, we minimise the use of sensitive data and implement privacy and confidentiality safeguards to ensure data is handled in accordance with legal and compliance requirements.

6 Design safeguards

Once risks and impacts are identified, we design appropriate safeguards into the solution. At a minimum, these always include recommended governance controls, security and data protection measures, human-in-the-loop oversight, transparency mechanisms, monitoring processes, and escalation routes for managing risks proportionately.

7 Test and validate

Before deployment, we rigorously test all AI systems to ensure they are safe, reliable, accurate, and fit for purpose. Validation against intended use cases is important, but beyond this, we test for edge cases and challenge assumptions – in other words, we try to break our own tools – to confirm safeguards operate as expected.

8 Deploy responsibly

AI should be introduced in a controlled and transparent manner, with clear ownership, guidance, and support for users. We ensure all stakeholders have clear and easy access to information on the purpose, capabilities, limitations, and governance arrangements for our AI systems so they can be used responsibly in practice. Additionally, we supervise the deployment process to check that there are no further unintended consequences of the deployment of our systems, with rollback mechanisms in place.

9 Curate organisational culture

Responsible AI innovation depends as much on culture as it does on technology. At Avencera, we encourage curiosity about and experimentation with AI, alongside a healthy dose of risk awareness. By creating an environment where questions are welcomed and assumptions and risks are challenged, we ensure AI is adopted thoughtfully and safely across the organisation, and enable everyone to take on the role of human-in-the-loop. We invest strongly in leadership engagement, workforce capability and change support alongside our technology deployments.

10 Monitor, evaluate, and improve

AI systems and the environments in which they operate are constantly evolving. We continuously monitor the performance of our systems, evaluate their impacts, manage emerging risks, and update governance approaches as technology and best practices develop. We build monitoring and feedback functionality into all our AI systems such that we can identify and quickly correct growing issues.

It’s important to note that we haven’t fully reinvented the wheel on responsible AI. A significant amount of excellent work has been done in this space – in particular, we have drawn on the NIST AI Risk Management Framework, the Model for Responsible Innovation from DSIT’s Responsible Technology Adoption Unit, and specific frameworks for sectors we work in, such as Ofqual’s position on regulating AI in qualifications.  

We continue to stay on top of sector best-practice throughout our innovation work.

If you’re interested in hearing more about how we put this into practice, please get in touch by emailing hello@avencera.ai