Measuring trust: Why every AI model needs a FICO score

According to the 2023 State of IT study conducted by Salesforce, a staggering 90% of IT leaders predict that Generative AI will soon become widely popularized in the near future.

According to a study conducted by McKinsey, half of all companies integrated artificial intelligence into their operations in the year 2022. Additionally, IDC predicts a remarkable 26.9% growth in global AI investments for the year 2023. A survey conducted among customer service experts revealed an astonishing 88% increase in the implementation of AI between 2020 and 2022. Furthermore, research indicates that a significant 86% of IT executives anticipate a significant role for generative AI in their organizations in the coming years.

Additionally: A majority of experts believe that artificial intelligence (AI) will enhance the worth of their competencies.

However, despite these figures being worrisome, a majority of IT leaders expressed apprehensions regarding the ethical implications of generative AI, with 64% expressing concerns. In addition, 62% also feared the potential impact of this technology on their professional careers. When surveyed, IT leaders highlighted various issues associated with generative AI, such as the high risk of security breaches (79%), the presence of bias (73%), and the negative effect on the environment (71%). Another study revealed that a significant portion of customers, 23%, lack trust in AI technology, while 56% remained neutral. This deficit in trust can be swayed in either direction depending on how companies utilize and deliver AI-driven services.

According to McKinsey's most recent study, the use of generative artificial intelligence has the potential to contribute roughly $2.6 trillion to $4.4 trillion to the economy each year, based on their examination of 63 different application scenarios. In comparison, the entire gross domestic product of the United Kingdom for 2021 amounted to $3.1 trillion. This would result in a 15% to 40% increase in the overall impact of all forms of artificial intelligence. However, the main obstacles preventing the widespread adoption and growth of AI are trust issues. Specifically, the lack of trust and the skills gap among employees pose significant challenges for businesses looking to embrace generative AI technology.

Moreover: Creating AI that generates and the ultimate reason: Establishing a bond of reliance with your consumer.

In order to get a better understanding of how AI companies can build trust with everyone they work with - their employees, customers, partners, and the communities they serve - I recently had an interesting conversation with Richie Etwaru. Etwaru is an expert in data privacy, advanced analytics, AI, and digital transformation, and he is also the co-founder and chief creative officer of Mobeus. He has a number of patents, books, TEDx talks, and industry-first innovations to his credit.

How can we establish credibility with all fundamental frameworks employed for generative AI and more? Here is what Etwaru suggested.

Arthur C. Clarke's famous quote from his book "Profiles of the Future: An Exploration into What is Possible," published in 1962, has proven its timeless wisdom.

Skipping ahead six decades to November 20, 2022, OpenAI introduced ChatGPT, a remarkable technology that appeared to merge the realms of sorcery and enchantment. This was more than just enchantment; it was disconcerting. The connection between humanity and enchantment revolves around equilibrium. We take pleasure in enchanting experiences that captivate and inspire us, but when they surpass our comprehension, breaking the boundaries of our understanding, they become intimidating. The instant enchantment appears too powerful, it pushes us outside of our comfort zones and into a perplexing realm. The uneasiness arises from encountering phenomena that defy logical or scientific explanation.

Additionally: The top-notch AI conversational agents: ChatGPT and other remarkable substitutes.

In the world of technology, both ChatGPT and similar models designed for interactive purposes, as well as creations like DALL-E capable of converting text into multimedia, bring forth something truly surprising. They bring about progress that even science fiction writer Arthur C. Clarke might not have foreseen, reaching a level of complexity that surpasses mere "sorcery." This evokes a sense of instinctive worry because they introduce ideas and abilities that are difficult for us to fully comprehend.

The human mind is sensitive. When confronted with unexplainable events, our natural reaction, influenced by development, tends to be fear. That's why a small, unknown creature can be scarier than a big, familiar animal. ChatGPT and similar AI systems have surpassed this boundary of extraordinary wonder, and their impressive abilities are certainly causing excitement.

We are not afraid of AI due to its abilities; rather, the limited knowledge we have about its functioning and accomplishments is what unsettles us. Our lack of understanding leads us to envision all the possible things AI might be capable of doing.

Additionally: Enhancing the quality of your ChatGPT prompts for optimal outcomes with generative AI

In the blog post titled "Take a Breath Regarding GPT-4 Already," Rodney Brooks presents the viewpoint that we should refrain from mistakenly equating performance with capability. Brooks elaborates on the idea that just because an AI model can accomplish one task, we should not automatically assume it can handle another task as well, merely based on the fact that humans who can do the former can typically accomplish the latter. Our anxiety arises from the tendency to imagine AI having boundless capabilities, while in reality, we tend to overestimate the overall competence of systems that exhibit impressive effectiveness in specific, limited areas.

Making the inner workings of AI clear and understandable can greatly decrease the apprehension associated with it. By transforming AI from a mysterious "black box" into an open and transparent "glass cube," we can fundamentally change the way we embrace this technology as a human race.

Dr. Michael Wu's presentation titled "Unveiling the Inner Workings of Generative AI beyond ChatGPT" delves into the mechanics of how a basic query such as "What shade encompasses the heavens?" truly operates. Wu adeptly demystifies the functioning of generative models, illustrating that their astonishing outputs are generated solely by employing "mere mathematics" and statistical analysis, rather than any form of conscious intellect. By exposing the mathematical underpinnings governing the responses of AI systems, Wu assures his listeners that these systems lack the cognitive awareness present in human beings.

In addition: I came very close to falling for this deceitful scheme involving artificial intelligence-generated crypto invoices, despite being knowledgeable in security measures.

Even though our knowledge of AI is expanding, there is still a long way to go. In an interview with CNBC, AWS CEO Adam Selipsky compared our progress in AI to running a marathon, where we have only taken three steps out of ten thousand. As AI continues to develop, models will surpass their current capabilities. By strengthening our understanding of data, managing models more effectively, integrating with various ecosystems, improving human skills, and constantly innovating in mathematics and statistics, we have the potential to greatly enhance AI in the future.

As we have successfully managed to regulate our concerns regarding previous advancements such as electricity, flight, automobiles, and the internet, it seems challenging to completely regulate our fears surrounding AI. The reason behind this lies in the fact that AI possesses a compounding exponential nature, unlike anything we have encountered before. Ultimately, our unease stems from how AI could potentially impact the progression of humanity as a whole. In extreme cases, we tend to imagine AI leading to the extinction of humans. However, the outcome is unlikely to be as black and white as complete victory or defeat. Rather than viewing the future as a battle between humans winning or losing their existence, we need to discover ways to coexist harmoniously and sustainably with artificial intelligence.

Additionally: The expert identifies the top five hazards associated with generative AI.

In order to ensure coexistence as our main focus, it is necessary to establish a method to assess the level of compatibility between an AI model and this objective. When we encounter an AI system, we should be able to swiftly determine if it is a "good AI" that promotes harmonious human-AI interaction and fulfills human requirements, or if it disregards coexistence and should not be relied upon. We need a simple scoring system that is comprehensible to all, portraying the trustworthiness and dedication of an AI model towards benefiting humanity.

In the absence of such a system, we might become more doubtful of artificial intelligence in general, leading to a lack of trust towards any company using it. A well-structured system to evaluate how well AI aligns with harmonious coexistence between humans and AI is crucial for gaining public trust and maximizing the benefits of this technology.

The AI Act of the European Union has made some initial progress in implementing an AI scoring system. This system mandates that each AI model must have a CE marking and a unique model number, which can be traced back to conformity assessment data. However, the information provided by this CE marking only reveals the details of how the model was trained and created. It does not indicate whether the model is trustworthy. Even if a model meets all the necessary regulations, it may still struggle to gain the trust of the public. This lack of trust can have a significant impact on how consumers, corporations, and countries perceive and use the model for their products and services. Simply meeting the minimum requirements is not enough to ensure widespread acceptance and usage. Therefore, it is crucial that we develop an AI scoring framework that goes beyond technical metrics to evaluate factors such as the model's potential to benefit humans, its level of transparency, and its ability to coexist peacefully with other systems.

Additionally: Morality of AI: Advantages and perils of synthetic intelligence.

Google and OpenAI have recently adopted the use of "model cards" as a means to compile and showcase pertinent details about their models' design, data, training, performance, and limitations. For instance, Google has created a model card specifically for their MediaPipe BlazeFace AI model, which follows a structure and content guidelines primarily set by Google employees in a scholarly paper. Conversely, OpenAI has developed a "system card" for GPT-4, aligning with the sections, data, and formatting outlined in a paper originating from Stanford University.

Although the model/system cards are a positive move, one of the main difficulties is that they have different structures and arrangements of information. However, the major hurdle lies in the fact that the majority of consumers do not possess the time, patience, or ability to comprehend these cards. Therefore, while they are accessible to the public, they are essentially unhelpful to consumers due to their extensive length and complexity.

Additionally, a recent study conducted by IBM reveals that approximately 40% of employees will be required to acquire new skills within the upcoming three years owing to the impact of AI.

To start off, let's create a straightforward and user-friendly measure that indicates how well an AI model serves human needs and promotes a harmonious relationship between humans and AI. We can call this measure the "Human & AI Coexistence score," or HAICO score for short. Now, let's explore how this score could be determined. What information would we need to gather about each AI model? How frequently would this data need to be collected? And what formula should be used to calculate the HAICO score? Ultimately, we want to create a framework that simplifies complex information into a score that can be easily understood by the general public.

Although it may be challenging, it is achievable to implement a scoring system like this. Consider our hypothetical HAICO score, which consists of 50 characteristics of an AI model categorized into the five levels of Maslow's Hierarchy of Needs (see Figure 1).

Image 1: A summary of our illustrative HAICO rating

All 50 qualities would evaluate aspects concerning the harmonious relationship between humans and AI. The information from these qualities would be gathered from every step of the model creation process, including how transparent the routines embedded in the silicon chips are, the consent and ownership of the data used for training, model design, inference effectiveness, retraining, and redistribution.

Instances of model characteristics could encompass qualities such as the model's resilience, accessibility, fairness, consideration towards human independence, inclination towards consensus, perpetual learning capacity, and ability to enhance human existence. Each characteristic would receive a rating ranging from 0 to 5, and subsequently, a calculation would amalgamate them to determine an overall HAICO score between 0 and 100 for every model (Figure 2).

Additionally: An AI ethicist warns that if we fail to take action promptly, the current AI boom will exacerbate societal issues.

The ultimate HAICO scoring system, which consists of three levels:

Image 2: A visual representation of an AI model showcasing ratings ranging from 0 to 5 for each of the 50 characteristics. These ratings are then added together at each level.

This showcases how complex technical information can be represented in a straightforward three-tier model of coexistence and reliability rating. The HAICO exemplar framework offers a foundation for this. Converting it into a practical framework for public use would necessitate inclusive progress and ongoing improvement. However, it proves that creating a nuanced system for scoring the interaction between humans and AI is possible.

The development of our HAICO scoring mechanism is still ongoing, and there is a lot of work remaining. For instance, the allocation of weights for each layer and the range that defines an AI model as non-coexistent may vary for different groups of people. The formula used to calculate the HAICO score could be different for AI models targeted at PG-13 audiences compared to those targeted at Rated R audiences. This example shows that we can create a scoring system for AI models that provides a simple and dependable method to ascertain whether the models can be relied upon to promote harmony between humans and AI (See Figure 3).

Furthermore: In anticipation of artificial intelligence, another technological wave is rapidly engulfing the scene.

We need to move beyond the ongoing debate about who will come out on top and instead focus on finding a way to coexist. Artificial intelligence (AI) is here to stay, and so are we. The work ahead needs to be approached as a collaborative effort within our community. Failure to do so will prompt doubt in the reliability of AI-driven products and services, and this doubt will extend to the consumers, corporations, and countries that utilize AI models. As a society, we run the risk of increasingly mistrusting AI and those who use it. This could ultimately lead us to a point where we miss out on the opportunity to harness the potential of this technology to improve the human condition.

Image 3: Utilizing a mathematical equation to determine the ultimate HAICO score for an artificial intelligence (AI) model, thereby identifying it as being in a state of COEXISTENCE with a rating of 76.

Here's some positive information: Besides the individuals involved in the growing AI community -- like companies that produce hardware (NVIDIA, Intel, Apple, AMD, SambaNova), cloud computing providers (AWS, Google, Azure, Oracle, Alibaba, Salesforce), artificial intelligence models, markets (Cohere, Hugging Face), applications (OpenAI, Antrophic, Stability.ai), and consulting firms specializing in strategy and services (Deloitte, Accenture, IBM, Cognizant, and others) -- we are also witnessing the development of a group of tools focused on measuring AI models.

One way to gain insights about a dataset is by using TensorFlow Data Validation. This tool helps to grasp the dataset's characteristics, uncover any irregularities, and compare different datasets used for model training. To assess the model's resilience, one can employ either CleverHans or the Adversarial Robustness Toolbox (ART) to simulate adversarial attacks. When it comes to addressing biases in machine learning models, there are several options available such as Google's Fairness Indicators, IBM's AI Fairness 360, or Fairlearn. These tools aid in measuring, visualizing, and mitigating biases. Additionally, tools like Google's TFX, Seldon, or Fiddler can effectively monitor the model's performance over time, sending alerts if there are significant changes or deterioration.

Title: MIT Claims Generative AI Tools are Undermining an Essential Element MIT highlights a crucial aspect being compromised by the widespread utilization of generative AI tools.

The puzzle is starting to fit together. Our ultimate goal is to live side by side with AI. We have reached a point where we can work together to create a trust measure for each AI model, indicating how well it aligns with human-AI coexistence. This measure will be simple to comprehend, much like the FICO score we use to gauge a person's financial reliability. The HAICO score discussed in this article is just a taste of what's to come, intended to initiate discussion. There's no time like the present to tackle this challenge.

This blog post was written by Richie Etwaru, one of the co-founders of Mobeus. Etwaru is a versatile executive, entrepreneur, and influential figure on a global scale. Working closely with top executives and boards, he has successfully orchestrated major changes in the financial services and healthcare sectors. Etwaru is the creator of software-enabled spatial computing and is also known for introducing the concept of the 31st Human Right. He has written three books, given three TED talks, and been a speaker at over 100 conferences.