Success in Document Processing Requires an Orchestration Platform

Imagine you’re running the finance department of a large organization. Your department manages the organization’s global finances with multiple banks, hundreds of bank accounts and vendors. So, you receive financial documents of all kinds on a daily basis such as bank statements, letters of credit (LCs), invoices, and more—lots of documents in different formats from lots of sources.

You’d like to work on a real-time basis with the data from these documents, extracted and entered into your system of records and matched with corresponding business activity—quickly, easily, and with high accuracy—so that your department can respond and process in a timely manner. A highly efficient accounts payable (AP) and receivable (AR) operation can play a key role in an organization’s ability to provide and receive on-time deliveries and reap advantages such as lower procurement prices and loyal vendors and customers.

But achieving efficiency with data processing can be challenging, especially for departments that are still employing manual, legacy systems. That’s where intelligent automation, particularly intelligent document processing (IDP), can help move things along.

IDP can accelerate and streamline processes and responses as well as reduce errors associated with manual work. Even so, “digitizing” data in the real world is never as simple as feeding documents to a black box and receiving clean output from the other side. The world of work has become complex. So, make sure you choose an IDP solution that simplifies that complexity. Let’s dig a bit deeper into today’s world of workflows.

Document intake

Business process workflows are orchestrated from myriad communication technologies, core application systems, and data transfer techniques. They’re governed by the security requirements of one or more organizations involved in a process. Workflows today are also often organized and optimized around a human-first system and may govern how documents are collected and processed in an assembly-line-like operation.

In these workflows, a document that requires digitizing may come bundled as part of an email by itself or as part of a docket of multiple documents. Following are some examples. A document may be received over an email with a cover letter, a packing slip, and the invoice. A revenue cycle management company in healthcare sets up a mailroom for receiving all the “Explanation of Benefit” documents from the payers it serves. A bank sends a consolidated pdf file for all the accounts a customer holds, which is transferred over SFTP to a customer’s secure folder. A passport or visa processing center receives a docket with application forms, including supporting documents such as the applicant’s identity, employment, and education proof.

When a file or a docket is received, the first job is to identify for a particular process which document to process. In the IDP parlance, this is called “classification.” Thanks to the evolution of artificial intelligence (AI)/machine learning (ML) technology, one can easily train software bot models that can classify documents into specialized streams for downstream processing. For documents that are not classified correctly, manual intervention may be required.

Digitizing documents

Once the documents are classified, the real “digitization” step begins. An image of a document when presented to a computer is nothing but an image. Computers apply computer vision algorithms to predict what each image of a letter means to a human being, for example, if “o” is an “o” or is it a “0” (numeric zero).

Add in the variations from fonts, background colors, handwriting, and image resolution, and one can imagine the complexity of accurately predicting a letter, word, or sentence. A typical computer-printed page with font 12 might have 250 words, typed double spaced. Thankfully, optical character recognition (OCR) technologies have evolved over the last 20+ years to get to word-level accuracies as high as 90-99%.

The technologies apply human-like intelligence to continue to get better at making better predictions. Still, the printing and scanning quality has a measurable impact on the accuracy of the predictions in reading. Document quality attributes such as blur, rotation, and skew need to be corrected before passing the document to an OCR engine to get an optimal outcome. This step is called pre-processing and is akin to data quality improvement steps in most AI/ML implementations.

Overcoming the schema challenges

Once a word is read with a high confidence score (as in a set of characters typed or written together are recognized as a word), semantic understanding must be applied to the words by the computer—the schema or the relationship between words (key-value pairs, tables, tables without headers, complex tables). This presents infinite variations and challenges to a computer, even though the structure of the information is intuitive to the human. The job of IDP technology is to translate human intuitive information structures to computer intuitive structures without needing to code for each of these structures.

The IDP industry is, therefore, rapidly evolving to grapple with this challenge and developing ways to deal with complex information structures presented by each business domain, industry practices, and document conditions.

Documents in a supply chain or operations environment might have dirty documents and poor scan quality whereas financial statements might present complex tables running over pages with some unstructured text but nicely organized and clean digital PDFs. Specialization is required to tune the AI/ML models to extract data with the highest accuracy that makes operationalizing the technology commercially feasible.

Today, vendors are training software models for large document digitization opportunities such as invoices, mortgage, bank statements, tax statements, or building technology that allows training specific models for a document type. One size does not fit all, which clearly applies to this problem.

Now, one would imagine that the job is done with high-quality extraction achieved; however, there are still challenges in meeting the business requirements before operationalizing this technology. This set of challenges include:

• Identifying and correcting common error patterns in OCR and extraction layers
• Validating data for errors and, correcting data
• Verification of data such as zip code, address, vendor names against enterprise or public domain data

Meeting the challenges

What you want is a flexible, easy-to-use, and fully orchestrated solution to meet those challenges and others—to simplify the complexity for the user. You want a solution can that provide end-to-end automated processing, including helping to build the last mile capability, so to speak, for a meaningful document processing outcome that will improve the effectiveness of a business user. Manually building and stitching all parts of a document processing is cumbersome and expensive without the aid of an enterprise-grade platform.

Imagine a government tax filing processor quickly reviewing all the data submitted without having to key in that data—with no input errors. Or a mortgage processor going through a pile of loan application documents in an organized manner in a matter of few minutes.

IDP has that potential, which is why as a software category, it has seen fantastic growth in the last five years with vendors attempting to solve different facets of the solution with their unique approach to the challenges and for specific domains.

Solving the challenges, however, to achieve digital transformation objectives for a business requires a platform approach. That approach considers the unique characteristics of the business process workflow, document characteristics, accuracy, and validation requirements. An IDP platform, therefore, must provide the following capabilities to orchestrate for any document workload:

• processing the documents for addressing document quality
• classifying the documents
• extracting documents with domain and document specialized engine
• validating and, continuously improving outcomes

 
Organizing for success

Even the best IDP platform requires a human touch. Utilizing a platform and achieving success requires a “hands-on” approach and taking ownership of the solution. It requires a knowledge of how the pipeline of document understanding is set up for specific use cases and building relevant skills for understanding outcomes and troubleshooting problems.

As an example, on a periodic basis, the nature/mix of the document workload may change and require corrective actions such as tuning the pipeline, addressing document quality challenges, or tuning specific models. Modern IDP platforms are building operational analytics to make these assessments easier.

Analytics will help achieve data-driven operational actions in managing the digitized processes and in evaluating the effectiveness of technology to develop conviction and momentum for doing more across the organization. While every platform is different, the following skills are a must-have to run your document operations successfully with IDP solutions:

• Skills in the platform
• Basic understanding of AI/ML
• Data analytics skills

An operational technology such as document extraction requires a carefully designed solution with an understanding of peak workloads, backup plans to process documents when technology can use a human hand, and disaster recovery. Executives who are planning to use document digitization technology as part of their automation journeys to achieve enterprise-scale transformation goals must keep expectations real to deliver meaningful business outcomes and build trust in the technology.

Lastly, as document digitization makes inroads into an organization, one should rethink business process flows and re-organize for the new technology-empowered paradigm.

See How a Platform Can Benefit Your Processing.

START HERE