Build a document processing pipeline with Workflows

Create an Invoice Parser

Use the Google Cloud console to create a new Invoice Parser. Refer to the creating and managing processors page for more information.

1. In the Google Cloud console navigation menu, click Document AI and select Processor Gallery.

   Processor Gallery

2. In the Processor Gallery, search for Invoice Parser and select Create.

   

3. Enter a Processor name, such as workflows-invoice-processor.

4. Select the region (location in the Document AI API) to host the processor.

5. Click Create. The Processor Details tab appears.

6. Copy your Processor ID. You must use this in your code later.

   

Configure output bucket in Cloud Storage

When using batch (asynchronous) processing, the input and output files must be stored in Cloud Storage. The following bucket contains sample files to use as input for this guide:

gs://cloud-samples-data/documentai/workflows

Configure a bucket to store the output files in your project.

1. Follow the Create Buckets guide in the Cloud Storage documentation to create a bucket, or use a bucket that already exists.

2. Create a folder in the bucket (for example, docai-workflows-output).

Create workflow

1. Open the Workflows dashboard in the Google Cloud console.

   Go to the Workflows dashboard

2. Click Create to create your first workflow. The Create Workflow page appears.

   

3. Rename the workflow to docai-workflows-quickstart.

4. Add a description, and select a service account if you have one for this project.

   1. If you don't have a service account, select Create New Service Account.
      • Name your service account, and select Create and Continue.
      • Give it the following roles:
      • Document AI API User
      • Workflows Invoker
      • Logs Writer
      • Service Usage Consumer
      • 

5. On the Create Workflow page, click Next.

Define workflow

1. In the Workflows editor, clear the default YAML and paste in the code below:

   main:
     params: [input]
     steps:
       - start:
           call: sys.log
           args:
             text: ${input}
       - vars:
           assign:
             - input_gcs_bucket: ${input.inputBucket}
             - output_gcs_bucket: ${input.outputBucket}
             - processor_id: ${input.processorId}
             - location: ${input.location}
             - project_id: ${sys.get_env("GOOGLE_CLOUD_PROJECT_ID")}
       - batch_process:
           call: googleapis.documentai.v1.projects.locations.processors.batchProcess
           args:
             name: ${"projects/" + project_id + "/locations/" + location + "/processors/" + processor_id}
             location: ${location}
             body:
               inputDocuments:
                 gcsPrefix:
                   gcsUriPrefix: ${input_gcs_bucket}
               documentOutputConfig:
                 gcsOutputConfig:
                   gcsUri: ${output_gcs_bucket}
               skipHumanReview: true
           result: batch_process_resp
       - return:
           return: ${batch_process_resp}
   

1. Select Deploy.

Test workflow

Now that the workflow has been deployed, you should be on the Workflow details page.

1. Click Execute at the top of the page to go to the Execute workflow page. This page allows you to test your workflow in the console.

2. Replace the placeholder variables in the following input JSON:

   • OUTPUT_BUCKET_PATH: The full path to output bucket and folder created earlier.
   • PROCESSOR_ID: The processor ID for the processor created earlier.
   • LOCATION: The region for the processor created earlier.

   {
     "inputBucket": "gs://cloud-samples-data/documentai/workflows",
     "outputBucket": "OUTPUT_BUCKET_PATH",
     "processorId": "PROCESSOR_ID",
     "location": "LOCATION"
   }
   

3. Copy/Paste the JSON into the input textbox.

4. Click Execute at the bottom of the workflow to test your input and wait for a moment for the process to complete.

5. Once the workflow finishes, the output JSON should look something like this.

   {
     "done": true,
     "metadata": {
       "@type": "type.googleapis.com/google.cloud.documentai.v1.BatchProcessMetadata",
       "createTime": "2023-02-24T18:31:18.521764Z",
       "individualProcessStatuses": [
         {
           "humanReviewStatus": {
             "state": "SKIPPED"
           },
           "inputGcsSource": "gs://cloud-samples-data/documentai/workflows/baking_technologies_invoice.pdf",
           "outputGcsDestination": "OUTPUT_BUCKET_PATH/11119195367705871803/0",
           "status": {}
         },
         {
           "humanReviewStatus": {
             "state": "SKIPPED"
           },
           "inputGcsSource": "gs://cloud-samples-data/documentai/workflows/generic_invoice.pdf",
           "outputGcsDestination": "OUTPUT_BUCKET_PATH/11119195367705871803/1",
           "status": {}
         },
         {
           "humanReviewStatus": {
             "state": "SKIPPED"
           },
           "inputGcsSource": "gs://cloud-samples-data/documentai/workflows/google_invoice.pdf",
           "outputGcsDestination": "OUTPUT_BUCKET_PATH/11119195367705871803/2",
           "status": {}
         }
       ],
       "state": "SUCCEEDED",
       "updateTime": "2023-02-24T18:31:52.543163Z"
     },
     "name": "projects/PROJECT_ID/locations/LOCATION/operations/11119195367705871803",
     "response": {
       "@type": "type.googleapis.com/google.cloud.documentai.v1.BatchProcessResponse"
     }
   }
   

6. Open your output storage bucket, you will see the output for each invoice processed in a folder named after the Operation ID.

7. Within this folder there will be subfolders, one for each document processed from the input batch containing Document JSON file(s).

8. You may inspect each JSON file or write code to use the relevant fields for your use case. Refer to the handle the processing response guide for more information.

9. You can now link additional steps in the workflow to further process the output, such as inserting the extracted entities into a database for analytics.

10. Refer to the Workflows Documentation for more information on how to trigger workflows.