Duplicati and Backup of live Pervasive Database Missing Data

Question

We have recently started using Duplicati for backup of some of our data systems. We run an ERP solution that uses Pervasive (v10).

When Duplicati begins its backup process, to the best of my understanding, it's using either the file date, and or the file byte size to determine what to back up.

The problem that I see with that solution is that some of the data is missing from the table. For example, the workorders module we are certain had new rows of data on the server (source machine) that were NOT copied over to the new file.

Last night we backed up our ERP platform then restored to a new location so as to do a compare of what was backed up during the evening against what the source machine had. We noticed that there are rows missing from one table in the restored backup, that are there in the source table.

The backup is being created from the data directory. We are NOT using the integrated backup that came with the ERP suite.

What I personally believe is happening is that the database isn't writing out the data to the table until the last client disconnects from the ERP software. Also, the byte size of the file missing data and the source machine are the same, even though the source file holds more data.

Last week we did the same test that we did last night and I noticed when I closed the ERP suite, the file updates its modified stamp and the new rows are added to the table, but not before the client disconnects.

Can someone shed some light on why this is happening?

Answer 1

Are the data files open according to Pervasive when the backup occurs? If so, you should be using some sort of agent to close the files or put them into Continuous Operation mode or Backup Agent. From the docs:

Continuous Operations provides the ability to backup data files while database applications are running and users are connected.

When Continuous Operation mode is started, a delta file (.^^^) is created and the original data file is 'closed' so backup programs can access the file and back it up.

Backup Agent puts a GUI front end to Continuous Operation mode but is only supported with PSQL v11 and newer.

Answer 2

With Duplicati, you can set --disable-filetime-check=true to ignore the timestamps and sizes, and scan each file for changes.

This option is not active by default, because it may take a long time to fully read the file contents. For normal file operations, the OS should set the timestamp, but some applications, like TrueCrypt, will revert the timestamp.