Image Exists Update Service

Pete Sral

Summary: paperwork is scanned into an image (PDF) files and NetTrax.Net to view these images (pdf files).  In order to improve performance, due to querying the network to determine if an image exists every time (for each document in a grid that contains 10 – 1000+ entries would cause 10 to 1000+ scans on the network).  Workaround:  a flag “Image Exists” is associated with the document in the database that allows NetTrax.Net to quickly identify that an image exists without scanning the network if the file(s) exist.

Current Process

Shut down scanning and process the scanning output text file using an import utility.  An output (manifest) text file contains all the new images (pdf files) created that the utility reads and updates the image exists flag.  This utility is executed twice a day.

New Process

Using a Windows service to watch folders we can eliminate halting scanning and process the images (pdf files) as they are being placed in specific folders.  Now, files are flagged real time as they are created and scanning can continue without having to halt and manually run a utility twice a day.

Settings

A simple XML file is used to define all the folders to watch along with files to exclude by extension.  Any file that is created in a watched folder is processed on its own thread allowing the next file to be processed without waiting for the previous file’s processing to be completed.

<?xml version="1.0"?>
<ServiceSettingItem xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <LoginUserID>PSral</LoginUserID>
  <WatchPaths>
    <string>D:TempNoAccess</string>
    <string>D:TempWatchImageExists</string>
    <string>\ServerSharedNewFiles</string>
    <string>\ServerInvalidShare</string>
    <string>\ServerSharedInvalidDir</string>
  </WatchPaths>
  <ExludedFileExtensions>
    <string>.db</string>
  </ExludedFileExtensions>
</ServiceSettingItem>

 

Utilizing the Windows Event Log, information about the service can be used to monitor the status of the service.  E-mail alerts can be configured when errors occur:

https://i0.wp.com/nettrax.net/wp-content/uploads/2017/07/image003.png?resize=895%2C410&ssl=1

https://i0.wp.com/nettrax.net/wp-content/uploads/2017/07/image004.png?resize=659%2C491&ssl=1

Here are some entries from the event log.

9

Information        7/8/2017 8:00:18 PM     NetTrax.Net Image Exists Update Service              0              None

Service stopped: 7/8/2017 8:00:18 PM

8

Information        7/8/2017 8:00:17 PM     NetTrax.Net Image Exists Update Service              0              None

During the span of 28m : 15s . 172ms  Total files processed:

• Total from watched folders: 14

7

Information        7/8/2017 7:39:03 PM     NetTrax.Net Image Exists Update Service              0              None

Completed loading application objects for Dept A in 4s . 112ms

6

Information        7/8/2017 7:38:59 PM     NetTrax.Net Image Exists Update Service              0              None

Started loading application objects for Dept A

5

Information        7/8/2017 7:32:48 PM     NetTrax.Net Image Exists Update Service              0              None

Completed loading application objects for Dept B in 3s . 421ms

4

Information        7/8/2017 7:32:45 PM     NetTrax.Net Image Exists Update Service              0              None

Started loading application objects for Dept B

3

Information        7/8/2017 7:32:31 PM     NetTrax.Net Image Exists Update Service              0              None

Path watcher is watching the following paths:

• D:TempNoAccess

• D:TempWatchImageExists

• \ServerSharedNewFiles

Unwatched paths:

• (Does Not Exist) \ServerInvalidShared

• (Does Not Exist) \ServerSharedInvalidDir

2

Error      7/8/2017 7:32:14 PM     NetTrax.Net Image Exists Update Service              0              None

Path watcher failed to start watching path: D:TempNoAccess

Exception: System.IO.FileNotFoundException: Error reading the D:TempNoAccess directory.

1

Information        7/8/2017 7:32:02 PM     NetTrax.Net Image Exists Update Service              0              None

  • Service started: NetTrax.Net Image Exists Update Service v2.5.0.0

  

Database Logging

Along with the event logs stored on the machine that the service runs on, each processed file has an entry in a database table used for reporting.

Log Entry Classifications

Table: Log Types

ID

Description

1

Successful

2

Pending

3

Error

4

Information

5

Aged

The log table stores information about where, what and details of any issues with each processed file.

Sample Log Table

Log Type

Dept.

File Path

File Name

Workstation

Name

Description

Is

Processed

Log Date/Time

Barcode

Internal Tracking ID

Parent Barcode

Document Type

ID

1

A

D:TempWatchImageExists

000000000000001N901LOCAGMT_J4_051517_A_TSTPC123-WA012_44881_1.pdf

EXESERVER

NULL

1

7/8/17 3:55 PM

000000000000001N901LOCAGMT

e99d946fxe264x47972f095c2cd46

00000000000000

LOCAGMT

1

1

A

D:TempWatchImageExists

000000000000001N901LOANAPP_J4_051517_A_TSTPC123-WA012_44881_1.pdf

EXESERVER

NULL

1

7/8/17 3:55 PM

000000000000001N901LOANAPP

2xb94c8a974048f29db7d57163709

00000000000000

LOANAPP

2

1

B

D:TempWatchImageExists

00000043326112IM34B01KETRHOLD_B3_030411_B_EXESERVER_7315_1.pdf

EXESERVER

NULL

1

7/8/17 7:10 PM

00000043326112IM34B01KETRHOLD

0000a48x9c004d7ca1fdaxa965x4d

00000043326112IM

KETRHOLD

3

3

B

D:TempWatchImageExists

00000043326112IM34B02KETRHOLD_B3_030411_B_EXESERVER_7408_1.pdf

EXESERVER

Document Barcode: 00000043326112IM34B02KETRHOLD does not exist.

0

7/8/17 7:11 PM

00000043326112IM34B02KETRHOLD

NULL

00000043326112IM

KETRHOLD

4

3

D

D:TempWatchImageExists

000000433261122011030402IREVW_20110304_030411_D_EXESERVER_25910_1.pdf

EXESERVER

Parent barcode 00000043326112 does not exist.

0

7/8/17 7:12 PM

000000433261122011030402IREVW

NULL

00000043326112

IREVW

5

4

D

D:TempWatchImageExists

000000000000002017020601ASSGN_20170206_020617_D_EXESERVER_21406_1.pdf

EXESERVER

Image status already set to True

0

7/8/17 7:32 PM

000000000000002017020601ASSGN

2b204d08beef4f07bbxbe2e1f4eb5

00000000000000

ASSGN

6

1

D

D:TempWatchImageExists

000004847750782016080401CCAPP_20160804_080416_D_EXESERVER_26159_1.pdf

EXESERVER

NULL

1

7/8/17 7:34 PM

000004847750782016080401CCAPP

02a2x400d5704590a2aab52d11790

00000484775078

CCAPP

7

1

D

\ServerSharedNewFiles

000004847750782016072501ODPROT_20160725_072516_D_EXESERVER_8341_1.pdf

EXESERVER

NULL

1

7/8/17 7:37 PM

000004847750782016072501ODPROT

0xx205aa7x544a2xb549xdbe8ea9x

00000484775078

ODPROT

8

3

B

\ServerSharedNewFiles

00000484775078IM7TH01ODPROT_H7_072516_B_EXESERVER_15448_1.pdf

EXESERVER

Filename does not contain the required elements.

0

7/8/17 7:39 PM

00000484775078IM7TH01ODPROT

NULL

00000484775078IM

NULL

9

4

D

\ServerSharedNewFiles

000000000000002017020601ASSGN_20170206_020617_D_EXESERVER_21406_1.pdf

EXESERVER

Image status already set to True

0

7/8/17 7:39 PM

000000000000002017020601ASSGN

2b204d08beef4f07bbxbe2e1f4eb5

00000000000000

ASSGN

10

4

D

\ServerSharedNewFiles

000004847750782016080401CCAPP_20160804_080416_D_EXESERVER_26159_1.pdf

EXESERVER

Image status already set to True

0

7/8/17 7:39 PM

000004847750782016080401CCAPP

02a2x400d5704590a2aab52d11790

00000484775078

CCAPP

11

4

D

D:TempWatchImageExists

000004847750782016080401CCAPP_20160804_080416_D_EXESERVER_26159_1.pdf

EXESERVER

Image status already set to True

0

7/8/17 7:40 PM

000004847750782016080401CCAPP

02a2x400d5704590a2aab52d11790

00000484775078

CCAPP

12

3

B

D:TempWatchImageExists

00000484775078IM7TH01ODPROT_H7_072516_B_EXESERVER_15448_1.pdf

EXESERVER

Filename does not contain the required elements.

0

7/8/17 7:40 PM

00000484775078IM7TH01ODPROT

NULL

00000484775078IM

NULL

13

4

D

D:TempWatchImageExists

000004847750782016072501ODPROT_20160725_072516_D_EXESERVER_8341_1.pdf

EXESERVER

Image status already set to True

0

7/8/17 7:40 PM

000004847750782016072501ODPROT

0xx205aa7x544a2xb549xdbe8ea9x

00000484775078

ODPROT

14

4

D

D:TempWatchImageExists

000000000000002017020601ASSGN_20170206_020617_D_EXESERVER_21406_1.pdf

EXESERVER

Image status already set to True

0

7/8/17 7:40 PM

000000000000002017020601ASSGN

2b204d08beef4f07bbxbe2e1f4eb5

00000000000000

ASSGN

15

3

B

\ServerSharedNewFiles

00000484775078IM7TH01ODPROT_H7_072516_B_EXESERVER_22227_1.pdf

EXESERVER

Filename does not contain the required elements.

0

7/8/17 7:45 PM

00000484775078IM7TH01ODPROT

NULL

00000484775078IM

NULL

16

3

B

\ServerSharedNewFiles

00000484775078IM7TH01ODPROT_H7_072516_B_EXESERVER_22219_1.pdf

EXESERVER

Filename does not contain the required elements.

0

7/8/17 7:48 PM

00000484775078IM7TH01ODPROT

NULL

00000484775078IM

NULL

17

1

B

\ServerSharedNewFiles

00000004021352IM34B01GARCSNOT_B3_030411_B_EXESERVER_31329_1.pdf

EXESERVER

NULL

1

7/8/17 7:50 PM

00000004021352IM34B01GARCSNOT

000348f11b6540b592b0daabdxx2x

00000004021352IM

GARCSNOT

18

3

B

\ServerSharedNewFiles

00000004021352IM34B02GARCSNOT_B3_030411_B_EXESERVER_28822_1.pdf

EXESERVER

Document Barcode: 00000004021352IM34B02GARCSNOT does not exist.

0

7/8/17 7:52 PM

00000004021352IM34B02GARCSNOT

NULL

00000004021352IM

GARCSNOT

19