What is Universal Service Data for Conversational AI?

The Universal Service Data sits at the core of Conversational AI solutions to discover, ingest, and parse enterprise data from a plethora of different data sources and fuse them together into a universal data schema-less structure that allows Conversational AI NLP/NLU models to learn from. In doing so, the Universal Service Data is called to address three key tasks to propel the continuous and autonomous learning of Conversational AI solutions: data findability, information extractability, and content dynamicity.

Let’s start first with Data Findability. While on the Web, content is well-linked, organized in HTML pages which are well-structured and easy to extract information from, in Enterprises, 80% of the content is typically unstructured and less than 10% of that content can be found on internal websites. The rest of the content is scattered across a very broad digital landfill of internal relational databases and data warehousing, file servers, content management and ticketing systems, or externally to public sites and communities, and so forth. The Universal Service Data of Conversational AI scans continuously and in real-time the vast landfill of Enterprise repositories, public sites, and technical communities to identify content that can be used to learn from, both in terms of user popular questions/issues and corresponding popular and accepted resolutions to their problems.

Second on the list is Information Extractability. Content is trapped into a large variety of different collections, such as web content, email threads, RDMs entries, presentations, business spreadsheets, etc. Most of these collections are rarely in plain text. Instead, they come in a variety of formats, such as HTML, XML, Adobe PDF, Microsoft Word and PowerPoint, and so on, and those formats have huge differences in their structure. Plain text documents may be trivially indexable and searchable by almost all the search engines available in the market, but to convert the many document formats to a universal data format for Conversational AI to use, it requires a heavy investment in data parsing technologies. The Universal Service Data automates the information extraction process by handling simple to very complex data parsing tasks like for the ones required for unfriendly PDF documents, for which it requires advanced models for layout analysis, segmentation, character and symbol recognition, and structure recognition.

Third is Content Dynamicity. Enterprise and public content are very dynamic in nature. Collections grow by the minute — webpage, troubleshooting playbooks, product manuals, FAQs — server locations change and documents are often forgotten during the migration. Content is regularly refreshed to guarantee employees and customers with access to the latest and most accurate information. The same holds true for content available externally to the enterprise, where the content refresh is even more accelerated and frequent. With potential terabytes of content to track, Conversational AI face the challenge of keeping up to date an accurate view of what content is available and trustworthy, whether any duplication which may cause confusion during the learning process, whether any gap between what users are seeking for, and what available to provide answers to their questions.  The Universal Service Data takes care of this challenge by tracking in real-time the content migration, the de-duplication, and freshness of the content and popularity to ensure the AI Service Desk NL/NLU models are always exposed with the most current and accurate information to better serve users’ requests.

To better understand how critical the role of the Universal Data Layer is and how it works in symbiosis with Conversational AI NLP/NLU models, we must discuss three data sources which are common and very important for Conversational AI: Service Desk Tickets, Knowledge Base articles, and Live Sessions between service agents and users.

Service Desk Tickets are the most invaluable source of information as they precisely capture the most popular issues faced by the Enterprise workplace and the variety of ways employees report those issues to Service Desk systems. We call the former user intents and the latter intent phrases.  The Universal Service Data ingest and process both historical and real-time tickets and extract and organize rich ticket content which is then used by NLP/NLU models. Tickets with a strong semantic affinity of intents are automatically grouped together and intent phrases are used to train Conversational AI systems to correctly classify those intents learning directly from how users present those issues. Furthermore, the Universal Service Data extracts critical information directly from ticket resolution notes which are used by NLP/NLU models to learn the most effective modalities of which service agents successfully closed those tickets. When this process is executed continuously and without any interruption, the Universal Service Data enables the Conversational AI models to learn new information every second, either in the form of new user intent, and/or new intent phrases and/or more effective resolutions to user issues.

Knowledge Base: Knowledge articles, FAQs, Technical Forums & Communities, Public Sites, and more – generically known as knowledge base — represent an incredibly rich domain of information and knowledge which is regularly leveraged by Conversational AI solutions. Knowledge articles are written and regularly refreshed to address the most frequent issues reported by users. Forums and technical communities gravitate around active discussions around common technical problems experienced by the vast majority of users. Public Sites provide rich troubleshooting and resolution procedures for popular applications, services, tools, and are adopted by typical Enterprises. The Universal Service Data taps into the wealth of such information to deeply learn. Of course, the task to connect all the dots together is not easy by any means but here is where Conversational AI models come into play, capable of automatically clustering similar issues and resolutions together and properly weighing the order of which recommendations shall be generated and proposed to end-users.

Live Agents Conversations: What happens when advanced Conversational AI agents fall short and are unable to auto-resolve user issues? In those cases, users are redirected to live service agents who guide users through the process of understanding and resolving their problems. When sessions are handed off to live agents, the Universal Service Data Layer switches to a passive listening mode. By parsing and extracting information from every correspondence between users and live agents, the passive listening mode enables Conversational AI agents to learn. NLP/NLU models can then analyze and learn the user issue as well as the precise and end-to-end troubleshooting process used by live agents and the final resolution. In doing so, the Universal Service Data enable Conversational AI agents to learn how and what service agents ask users and how agents follow up based on users’ answers. Large Semantic Neural Graphs are automatically generated and enriched in real-time to capture all this information. That information is then used by Conversational AI Agents to closely mimic — almost “impersonate”— service agents when leading conversations directly with end users. Book a conversational AI platform demo today!

Aisera offers the most feature-comprehensive and technology-advanced Cognitive AI Automation solution on a Universal Service Data layer, perfectly blending together Cognitive AI technology, Supervised and Unsupervised AI Learning, AI Virtual Assistant technology, and Conversational Automation into one SaaS, a turnkey ready solution to drive ROI at lightning speed. Aisera’s proprietary unsupervised NLP/NLU technology, User Behavioural Intelligence, and Sentiment Analytics are protected by several patent-pending applications.