Turingbots 2024 convictions, related to the use of Generative AI within the companies
As we navigate the exciting world of Generative AI, we must embrace a spirit of humility and continual learning, much like the gold diggers. This learning begins with developing Proof of Concepts (POCs), business choices and technological choices that often need to be revised by the constant and rapid evolution of the tools provided by the major suppliers. Reflecting on the initial criticisms faced by these AI models in 2023 (sometimes with legal problems), particularly regarding their tendency to produce hallucinations, we’ve witnessed significant strides in just one year. For instance, Copilot, despite its imperfections, has been adeptly integrated into some of the most challenging environments, marking a shift towards widespread and seamless adoption by millions of users. While we are mindful that our current insights will evolve, we are eager to share our convictions about the strategies and approaches underpinning the successful development of Generative AI solutions in 2024.
Simple Use Cases: The Key to GenAI Growth
We are gradually integrating these tools into our daily operations, a journey filled with learning and discovery. As organisations navigate this terrain, there’s an apparent inclination towards prioritising primary use cases of generative AI. These foundational applications are selected for their straightforward execution and minimal risk. They typically involve more straightforward data to process, further reducing complexity and costs. A prime example is semantic search, demonstrating AI’s proficiency in efficiently understanding and interpreting natural language. Furthermore, these use cases consciously avoid sensitive data, like customers’ personal information, to mitigate risks. This strategy is not just about minimising data misuse or compromise; it’s about adhering to ethical standards and ensuring the secure use of generative AI. As a result, organisations can explore the capabilities of AI while maintaining a solid commitment to data privacy and security.
This leads us to share our first conviction: AI-augmented processesv are becoming increasingly integrated into organisations in fundamental yet unobtrusive ways.
To be integrated into organisations in basic yet impactful ways. The focus will be on augmenting established internal processes, primarily aiding employees rather than interacting with customers. This approach keeps human employees in charge, reflecting a conservative stance towards risk management. Such integration signifies a shift towards AI-assisted workflows where the technology is a supportive tool rather than a replacement for human labour. The priority remains enhancing efficiency and accuracy in internal operations while maintaining ethical standards and human oversight. This AI development and implementation direction represents a balanced approach, blending technological advancement with cautious, responsible application.
Data Indexation: Crafting LLM-Friendly Information
In 2023, our first works with LLMs underscored a critical need for a tailored information management system, particularly when indexing and structuring domain-specific information that LLMs must ingest. An off-the-shelf, RAG architecture failed us in one of our projects. Our working solution relied on a customised parser and chunkier, a combination of semantic and keyword searches, and a re-ranker. Of course, the keyword extraction step was done with an LLM! Our custom code allowed us to gracefully merge information from heterogeneous sources (web pages, documents, OCR). Custom RAGs are essential to enhance the models’ contextual understanding, thus mitigating hallucination issues and improving generation relevance. Key elements underlying custom RAG include the restructuration of databases and information assets, leveraging metadata, and pre-processing and organising the information, aligning more closely with natural human inquiry patterns.
We predict key innovations will focus on refining data pre-processing and search techniques to facilitate this transformation. Pre-processing will go beyond simple data cleaning; it will involve intelligent information segmentation and better embedding calculations that lay the groundwork for precise semantic search. Metadata, once a by-product of data management, will become central (again). Enhanced metadata will include descriptive information (including access) and context and usage patterns, enabling LLMs to understand not just content but the nuances of its application. Metadata will also be generated by LLMs from the content itself.
Data Governance: Back to the Centre of the Game
In 2023, the pioneer LLMs users realised how the output quality depends on the input quality. This understanding stems from practical experiences, such as recognising that, for instance, if a client’s address is incorrect, the model can only correct it if specifically trained and provided with additional relevant information. Thus, after the experimentation phase, the initial sustainable implementations of these models have brought data management up to date. It’s becoming increasingly more apparent that data governance is critical. The questions of data sensitivity, appropriate usage, access control, ownership for remediation, availability, and standard formats are not just technical considerations but fundamental to the responsible deployment of generative AI.
Moving into 2024, it’s expected that information management will emerge as a critical factor in the successful deployment of generative AI projects. After years of investment in data governance initiatives, which often fell short of expectations in demonstrating their added value, a renaissance in the perception and implementation of these practices is expected. Organisations are likely to place a greater emphasis on establishing robust data governance frameworks. This shift is driven by the recognition that effective data management is not merely a compliance or administrative task but a strategic imperative that directly impacts the efficacy and reliability of generative AI applications. The urgency to answer data usage, privacy, and management questions reflects a broader movement towards more ethical, transparent, and accountable AI systems. As such, data governance is poised to move from a background function to a central role in shaping the future of AI-driven innovation.
Explainability: Building Trust through Transparency
The virtues of indexing and data governance are increasingly recognised as key to bolstering trust. Historically, LLMs have faced scepticism due to concerns around accuracy and reliability. However, the increase in their usage indicates a positive shift in perception. Challenges like sourcing and copyright issues, along with the need for relevance and security, are driving organisations and providers to offer more insights into the intrinsic nature of these models. This trend shows a growing demand for confidence in the output of LLMs, reflecting a broader requirement for transparency and explainability in AI systems.
As we progress through 2024, there is an expectation that public regulations surrounding generative AI will continue to evolve, focusing on the transparency and accountability of these systems. The need for explainability in AI is becoming more than a technical requirement; it is emerging as a societal and regulatory expectation. This need will likely lead to more sophisticated AI models that efficiently process and generate data and provide more precise insights into their decision-making processes. Organisations will increasingly prioritise AI solutions that clarify how data is used, and decisions are made. As these developments unfold, generative AI is expected to become more integrated into daily business operations, emphasising ethical considerations and user trust more strongly.
Interactions: New Paradigms of the User Experience
In 2023, the proliferation of Conversational bots, driven by Generative AI, posed a significant challenge in the digital landscape as customers and users were faced with a scattering of resources and attention. With many independent options, the user experience became fragmented, leading to inefficiencies and dissatisfaction. The abundance of disconnected tools and interfaces created a need for a more harmonised, cohesive, and user-friendly interface.
Looking forward, the technology of Conversational bots is expected to accelerate its adoption by recentering the User Experience (UX) and building seamless access to various services through a unified interface. To support the behavioural changes needed by the primary users -Customer (CX) and Employee (EX)- bots need to improve the level of interactions. This improvement will be achieved by enhancing tailored and empathetic experiences. Drawing a parallel, similar to how Google addressed this issue in the early years of the internet by gathering internet research into a single web browser, major players such as Microsoft, Google, and Meta are expected to integrate the diverse service offerings into their popular communication platforms and productivity suites. In 2024, users will be able to open their usual communication platform to access products and services, as information will be delivered to them instead of requiring active searching.
Green GenAI: The Need to Master Energy Consumption by Design
In 2023, the widespread adoption of Generative AI raised concerns about its environmental impact. Observations and metrics revealed three significant sources of carbon emissions linked to Generative AI. Firstly, the energy-intensive process of training large language models, such as OpenAI’s GPT-4, was estimated to emit approximately 300 tons of CO2, significantly exceeding the average annual carbon footprint of an Australian (around 20 tons). Secondly, the model usage through running inference, constituting 80–90% of neural network energy costs, contributed substantially to environmental degradation. Thirdly, manufacturing computing hardware and operating cloud data centres further increased the carbon footprint. Despite challenges on the horizon, there is an opportunity for positive change. The consistent doubling of overall computing power demand every six months signals a pressing need for proactive measures to mitigate the environmental impact of Generative AI.
Moving into 2024 and beyond, Generative AI must undergo a Green transformation to take more responsibility for reducing its environmental impact. The goal of Green GenAI is to prioritise eco-friendly design and usage optimisation. Noteworthy examples of best practices are techniques like quantisation, which reduces the memory footprint of models for compatibility with smaller systems and demonstrates a solid commitment to energy efficiency. Adopting a Mixture of Experts strategy, incorporating multiple sub-models within large models, enables high accuracy at rapid speed while minimising energy consumption. Additionally, the increasing use of specialised hardware (like Google’s TPU), offering improved performance per watt compared to GPUs, is anticipated. Implementing these measures will enable the development of sustainable practices that can guide the growth of Generative AI, ensuring a harmonious balance between technological progress and ecological responsibility, especially as we move towards integrating Generative AI into mobile devices.
OpenAI: The End of a Hegemony?
Undeniably, Open AI has led and is leading the way. And its main shareholder, Microsoft, is infusing the technology into our daily lives through co-pilot, an AI assistant built into its solutions that we have been using for decades. This partnership has set a high bar in the field, demonstrating generative AI’s profound impact and potential in various applications. However, despite their current dominance, there are signs that the landscape is ripe for change, with other players poised to challenge OpenAI’s hegemony.
In 2024, a shift in the leadership of generative AI innovations could occur, driven by several factors. Sovereignty concerns may lead to the adoption of non-American platforms and solutions. Technological advancements and the quest for improved performance, transparency, and control will likely lead organisations to experiment with alternatives. Economically, the concept of “bot stores,” akin to Apple’s successful App Store model, might evolve, giving way to different economic models in the AI space. OpenAI is now facing competition with a diverse array of prominent players, including Google (with Vertex AI/Gemini), Meta (with Llama), X/Twitter (with Grok), Baidu (ERNIE), Anthropic (with Claude), Cohere (with Command), Salesforce and Databricks. Additionally, open-source models, such as the one from Mistral and Stability AI, facilitate self-deployment. This growing competition and diversity in the generative AI market could lead to ground-breaking solutions that address current concerns about security, relevance, and transparency, potentially reshaping the entire landscape of AI technologies.
Looking back at 2023, generative artificial intelligence has made a sensational entrance into our daily lives, paving the way for a future full of exceptional possibilities. After having, like everyone else, followed the movement and tested these technologies, 2024 is shaping to be exciting, full of uncertainties and promises. Whether we are right or wrong in our predictions, we cannot wait to see what 2024 brings, marking a potential turning point in the evolution of AI and its integration into business processes and users’ daily lives. This period of transformation, rich in innovation and learning, promises to be as stimulating as it is rewarding.
Quentin
Quentin leads the initiatives related to Generative AI for onepoint in Asia-Pacific.