Dataset evaluation

Sector

Banking

Problem to solve

A team within the organization wanted to pilot a generative chatbot providing first-level support to staff on HR topics. However, given the organization’s role as a leading body in the banking sector, and since this was its first GenAI pilot, a prior assessment of the functional, social, organizational and ethical impacts was needed, along with structured risk management, before rolling the system out to the entire workforce.

Goal

We conducted Conduct a hybrid Ethical Assessment of the chatbot in order to: (1) identify potential ethical and social impacts; (2) assess risks by combining qualitative and quantitative analysis (carried out by MIRAI) on correctness, fairness, coherence and hazard; (3) provide mitigation strategies for the mapped risks, maintaining a balance between innovation and the protection of human and organizational values.

Why it matters

  • High reputational exposure of the Client
  • First GenAI pilot, to be validated ethically as well before going “Live” to the whole company workforce
  • Alignment with guiding values: transparency, inclusiveness, responsibility, ethics, sustainability
  • Need for compliance with the AI Act

Methodologies and frameworks adopted

  • Hybrid qualitative + quantitative approach 
  • Immanence proprietary framework based on 6 Ethical Dimensions (Governance, Data Quality, Technical Robustness, Fairness, Transparency, Human Rights & Sustainability) 
  • Impact & Risk Assessment (theoretical vs. emerging risk; Risk Coefficient)
  • Semi-structured interviews (9 stakeholders), survey of the 40 testers (14 responses)
  • Regulatory references: AI Act, Charter of Fundamental Rights of the EU 

Takeaways

  • The hybrid approach (technical + qualitative) generates more value: tools grow when guided by human dialogue and awareness;
  • The need to strengthen governance, metrics and monitoring became evident, especially regarding hallucinations;
  • The importance of testing the system on a realistic demographic of the company workforce emerged (the datasets were synthetic and “ideal”);
  • Automation bias must be mitigated through design and communication choices (disclaimers, step-by-step launch).

— RICHIESTA INVIATA ✅ ✉️ —

Grazie per il tuo messaggio.