Webinar Q&A: The Fundamentals of Intelligent Document Processing

IDP Q&A

We recently held an Ask Me Anything (AMA) expert session around Intelligent Document Processing (IDP), where we answered your questions about getting started with the technology. You can watch the recording of the live webinar here.

In this IDP Q&A blog, we have summarized the questions we covered in the live webinar.

Note also that we organize AMA sessions regularly around different Intelligent Automation topics. If you are interested in learning more, you can follow our future events on our events page.

IDP Q&A

WHAT IS IDP?

Can a system review different types of documents coming into a single email? -> Identify the doctype then process
Yes. There are multiple ways of doing this on a technical level.

What is the difference between IDP and OCR technology?
IDP is a stack of technologies, one of which is OCR. OCR is not intelligent by itself. If RPA is the hands of a robot, then OCR is the eyes and Machine Learning is the brain.

Is IDP basically combining OCR with RPA?
IDP is a stack of technologies, one of which is OCR. OCR is not intelligent by itself. If RPA is the hands of a robot, then OCR is the eyes and Machine Learning is the brain.

What is IDP and would it be useful in a public organization such as a municipality?
IDP is a variation of an intelligent data structuring solution. Municipalities can, for instance, utilize it in recruitment/onboarding when handling job application forms, work contracts, etc. Please check our webinar for a more thorough view of the variety of use cases for IDP.

What are the benefits of IDP?
IDP enables you to automate processes and tasks where RPA bots are unable to handle documents by themselves accurately. This brings great reductions in manual handling times in processes where documents are often still being processed by people, like invoice handling.

How do you see the relation between IDP and RPA?
Robotic Process Automation solutions can only work with structured data. But about 80% of all data handled in organizations is unstructured. Before RPA can be used with unstructured data, the data needs to be structured. Intelligent Document Processing is about structuring extracted text from documents so that it can be processed with RPA.

What is IDP and how does it work?
Robotic Process Automation solutions can only work with structured data. But about 80% of all data handled in organizations is unstructured. Before RPA can be used with unstructured data, the data needs to be structured. Intelligent Document Processing is about structuring extracted text from documents so that it can be processed with RPA.

How can I solve business problems with this tool?
The technology does not solve business problems by itself, but it can enable the automation of many processes that are otherwise outside of the scope of robotic process automation. Please see the webinar for an extensive list of use case examples.

Is IDP an RPA Solution?
It is not. But it is often used with robotic process automation. Robotic Process Automation solutions can only work with structured data. But about 80% of all data handled in organizations is unstructured. Before RPA can be used with unstructured data, the data needs to be structured. Intelligent Document Processing is about structuring extracted text from documents so that it can be processed with RPA.

Why should I use IDP?
Robotic Process Automation solutions can only work with structured data. But about 80% of all data handled in organizations is unstructured. Before RPA can be used with unstructured data, the data needs to be structured. Intelligent Document Processing is about structuring extracted text from documents so that it can be processed with RPA.

WHERE TO USE IDP?

Do you have any practical examples of IDP in banking?
By far the most common use cases out there are invoice processing, purchase order handling, and travel expense handling. Please see the webinar for a much longer list of use cases and areas

What is the best IDP tool?
There are dozens of IDP tools on the market (we feature 30 different ones on the webinar…). Don’t look too much into what the tool itself can do – make sure you are working with a partner/provider that can help you solve your business problem and is transparent about project efforts and costs. The majority of the IDP solutions can on a technical level be used in the majority of the IDP use cases. The devil is in the detail, and unsurprisingly, a compromise between the scope and the costs is often mandatory.

What are typical processes/areas of application where RPA and document processing are combined?
By far the most common use cases out there are invoice processing, purchase order handling, and travel expense handling. Please see the webinar for a much longer list of use cases and areas.

Return on Experience [REX] on IDP project with the TOP 5 tech from the market
There are dozens of IDP tools on the market (we feature 30 different ones on the webinar…). Don’t look too much into what the tool itself can do – make sure you are working with a partner/provider that can help you solve your business problem and is transparent about project efforts and costs. The majority of the IDP solutions can on a technical level be used in the majority of the IDP use cases. The devil is in the detail, and unsurprisingly, a compromise between the scope and the costs is often mandatory.

Can you name some less obvious use cases for IDP, apart from the low-hanging fruits (invoice handling, etc.)?
By far the most common use cases out there are invoice processing, purchase order handling, and travel expense handling. Please see the webinar for a much longer list of use cases and areas.

What are the most useful cases or scenarios to use IDP?
By far the most common use cases out there are invoice processing, purchase order handling, and travel expense handling. Please see the webinar for a much longer list of use cases and areas.

How and where can it be applied?
By far the most common use cases out there are invoice processing, purchase order handling, and travel expense handling. Please see the webinar for a much longer list of use cases and areas.

Is there any good way to approach possible incorrect data in documents? Or is it just pure manual correction work ahead?
IDP is used for extracting and structuring data from documents. Any business rules should be left for RPA solutions to adhere to. The key question is: How would a person check or know that any data is invalid? This same logic should be applied to the RPA bot. Very likely it leads to determining with what should the extracted data be compared.

WHAT ARE THE CHALLENGES?

What are the worst practices with IDP?
Insufficient commitment to participate in the project, e.g. the annotation effort, mismanaging stakeholder expectations…

Which major challenges have you seen when customers implement IDP?
1) Customer expectations: IDP solutions will not handle 100% of the cases. Human input will always be needed.
2) Document quality and labeling: These require effort. If the OCR is unable to extract data accurately, the ML component in the stack won’t be able to improve it much either. Sufficient amounts of training data are needed, always.
3) Volumes: In many cases, the business case does not hold water because the document volumes are too low. As a rule of thumb, you should have at least 20 000 documents per year in your process to consider for full-fledged IDP-fuelled automation. There are tools with which you can tackle lower volumes as well, but the gained benefit might be too low for even the initial implementation costs.
…please see our webinar for a more thorough response.

What are the reasons for slow adoption in the market?
The main reasons are not understanding how broadly IDP can be utilized (does not have to be invoice processing…) or too low volumes for the planned use cases.

What are the limitations of IDP?
IDP is used in processes where RPA bots are unable to extract data accurately from documents (which is almost always the case). To be viable, the IDP solution will need to have dozens to hundreds of documents annotated and it must be trained with sufficient training sets (hundreds to thousands of documents), Quite often the limitation is the too heavy implementation/run cost compared to the benefits solving the business problem would bring. As stated in another response, a rule of thumb is to have at least 20 000 documents per year, to have a starting point for an IDP project.

Which are the most important areas to consider setting yourself up for success?
Make sure you have a water-tight business case and understand what is required of you and the solution provider to reach the outcome you need to solve your business problem. Please see the webinar for a more thorough response on a technical and a business level.