To backup or not to backup?

UPDATED to HF1 on 2017-06-09 go to downloads

We have all been there and done that:

You will know the value of backup when you don’t have one

4D has given us a powerful backup system with a lot of features users of other systems envy us for.

What man do not realize, is that you can modify all parameters of a backup from you code, making changes on the fly and averting user errors.

Modifying the Backup.xml file allows you to set the schedule, files to be included, how to handle failures, basically everything that you can set in the backup settings panel of the database preferences.

Now I know that working with XML files is a bit tedious and not always very straightforward, also all the possible settings are a drag to keep track of.

Thus I have again come to the rescue and hereby publish my latest component:

AC_BackupTools

Just drop it into your components folder and you are ready to go.

There are quite a few commands, but most of them are just wrappers to single calls of DOM SET XML ELEMENT VALUE.

An example

Here is a quick example for changing the schedule to daily at 2 am and adding the logs folder:
$tx_Backup:=BACKUP_LoadPrefs //load the preferences BACKUP_SetScheduleDay ($tx_Backup;1;?02:00:00?) //set a daily schedule at 2am ARRAY TEXT($at_Files;0) $l_Files:=BACKUP_GetInclude($tx_Backup;->$at_Files) //get the files currently included APPEND TO ARRAY($at_Files;Get 4D folder(Logs folder)) //add log folder APPEND TO ARRAY($at_Files;Get 4D folder(Licenses folder)) //add licenses folder BACKUP_SetInclude ($tx_Backup;->$at_Files) //set the files to be included BACKUP_SavePrefs ($tx_Backup;True) //write all changes back to the file and clear the DOM reference

simple isn’t it?

Although I would expect you to be able to figure out the rest, here are all component methods and their explanation:

Basic methods for accessing backup settings

The following methods are used to read and write the actual backup.xml file.
Please not, that all Methods from this component can only be used on 4D Server, Standalone and Volume Desktop, they will simply not work on 4D Client.

xmlBackupRef:=BACKUP_LoadPrefs

Parameters
$0	Text		XML Reference

As the name suggests, this will load the Backup.xml file and return an XML reference to the corresponding DOM. Use this for all other calls.

BACKUP_SavePrefs ($tx_Backup;True)

Parameters
$1	Text	required	XML Reference
$2	Boolean	optional	True to purge the XML DOM from memory. Defaults to False.

This one saves the changes back to the XML file, optionally it purges the XML DOM reference, freeing up memory.

BACKUP_Release ($tx_Backup)

Parameters
$1	text	required	XML Reference

This it clean up, it purges the XML DOM reference

Information on last backup run

As the backup.xml also contains information on the last backup run, we can obviously also read these. You could write to them but why would you want to do that?

Path:=BACKUP_LastPath(xmlBupReference{;What})

This returns the path of the last backup file (.4BK), optionally it can also return the path to the last journal file backup (.4BL)

Parameters
$1	text	required	XML Reference
$2	text	optional	what to get: - data (default if omitted) - log
$0	text		the path to the desired backup file

LastTime:=BACKUP_LastTime(xmlBupReference)

Returns the time of the last backup

Parameters
$1	text	required	XML Reference
$0	time		the time of the last backup, ?00:00:00? if none

LastDate:=BACKUP_LastDate(xmlBupReference)

You might have guessed: the date of the last backup

Parameters
$1	text	required	XML Reference
$0	date		the date of the last backup, !00.00.00! if none

getting and setting the backup parameters

DestinationPath:=BACKUP_GetDestination(xmlBupReference)

where the backup will be stored

Parameters
$1	text	required	XML Reference
$0	text		the path to the backup destination folder

BACKUP_SetDestination(xmlBupReference;newPath)

tell backup where the backups should be stored

Parameters
$1	text	required	XML Reference
$2	text	required	the new path to the backup destination folder

SegmentSize:=BACKUP_GetSegmentation(xmlBupReference)

the currently set size of backup file segments

Parameters
$1	text	required	XML Reference
$0	longint		the backup segment size in MB 0 if no segmentation specified

BACKUP_SetSegmentation(xmlBupReference;SegmentSize)

set the desired set size of backup file segments

Parameters
$1	text	required	XML Reference
$2	longint	required	the desired backup segment size in MB set to 0 if no segmentation is wanted

SetsToKeep:=BACKUP_GetSets(xmlBupReference)

get the number of backup sets to keep around

Parameters
$1	text	required	XML Reference
$0	longint		the number of segments to keep 0 means none

BACKUP_SetSets(xmlBupReference;SetsToKeep)

tell backup the number of backup sets to keep around

Parameters
$1	text	required	XML Reference
$2	longint	required	the desired number of backup sets to keep set to 0 to disable keeping of old sets

ParamaterValue:=BACKUP_GetParameters(xmlBupReference;ItemCode)

Parameters
$1	text	required	XML Reference
$2	text	required	one of the permitted item codes see explanation below
$0	text		the value for the parameter requested see list below

Parameter codes and values
CompressionRate	None	no compression
	Fast	fast but not so small
	Compact	small but slow
Redundancy	None
	Low
	Medium
	High
Interlacing	None
	Low
	Medium
	High

BACKUP_SetParameters(xmlBupReference;ItemCode;ParamaterValue)

Parameters
$1	text	required	XML Reference
$2	text	required	one of the permitted item codes see explanation above
$3	text	required	the value for the parameter requested see list of possible values above

IsSet:=BACKUP_GetOption(xmlBupReference;ItemCode)

Parameters
$1	text	required	XML Reference
$2	text	required	one of the permitted item codes see explanation below
$0	boolean		the value for the option requested

Option codes
IncludeStructureFile	True/False	include structure files in backup
IncludeDataFile	True/False	include data file and external data in backup
IncludeAltStructFile	True/False	include user structure file
BackupJournalVerboseMode	True/False	more detailed backup journal
CheckArchiveFileDuringBackup	True/False	check the backup file while backing up
EraseOldBackupBefore	True/False	erase surplus old files before backing up
BackupIfDataChange	True/False	backup only if data has changed
AutomaticRestart	True/False	automatically restart backup if failed
AutomaticRestore	True/False	automatically restore from backup after error on opening data file

BACKUP_SetOption(xmlBupReference;ItemCode;IsSet)

Parameters
$1	text	required	XML Reference
$2	text	required	one of the permitted item codes see explanation above
$3	bollean	required	set the option to true or false

including files into the backup

you obviously did know that you can add any kind of file or directory into the backup did you?
Here is how to handle that with the component:

FileCount:=BACKUP_GetInclude(xmlBupReference;TextArrayPointer)

Parameters
$1	text	required	XML Reference
$2	pointer	required	pointer to a text array with the full paths to each file or folder to include in the backup

BACKUP_SetInclude(xmlBupReference;TextArrayPointer)

Parameters
$1	text	required	XML Reference
$2	pointer	required	pointer to text array with the paths of all files to include in the backup

What if something goes wrong?

Sometimes backups do go wrong, and we want to tell 4D what to do in such a case:

OptionValue:=BACKUP_GetFailOptions(xmlBupReference;OptionCodes)

Parameters
$1	text	required	XML Reference
$2	text	required	one of the permitted item codes see explanation below
$0	text		the value for the option requested

Parameter codes and values
TryBackupAtTheNextScheduledDate	True or False	as text
TryToBackupAfter	time string	as text "01:23:45"
AbortIfBackupFail	True or False	as text
RetryCountBeforeAbort	a number	as text

BACKUP_SetFailOptions(xmlBupReference;OptionCodes;OptionValue)

Parameters
$1	text	required	XML Reference
$2	text	required	one of the permitted item codes see explanation above
$3	text	required	the value for the option requested see explanation above

Where do those funny numbers come from MyBackup[0123].4BK?

Backup sets are numbered. You can edit these numbers, but be careful not to shoot yourself in the foot.

CurrentSet:=BACKUP_GetCurrentSet(xmlBupReference)

Parameters
$1	text	required	XML Reference
$0	longint		the number of the current backup set

BACKUP_SetCurrentSet(xmlBupReference;currentSetNo)

Parameters
$1	text	required	XML Reference
$2	longint	required	the new number for the current backup set 0 will restart at 1

When is the backup supposed to run?

Ok setting the schedule is a bit more complicated, so I created wrappers for the different options:

ScheduleObject:=BACKUP_GetSchedule(xmlBupReference)

Parameters
$1	text	required	XML Reference
$0	object		an object with the data for the currently set type of schedule, see examples below

this will return an object with the details of the current scheduling settings, see examples below:

Example for object with no schedule

{ 
    "Frequency": "none"
}

Example for object with daily run

{ 
    "Frequency": "Hourly",
    "Every": "1", 
    "StartingAt": "0000-00-00T00:30:00" 
}

Example for object with daily run

{
   "Frequency": "Daily",
   "Every": "1",
   "StartingAt": "0000-00-00T02:00:00"
}

a weekly run on specific days

{ 
    "Frequency": "Weekly",
    "Every": "1", 
    "Monday": { 
        "Save": true, 
        "Hour": "0000-00-00T00:30:00" }, 
    "Tuesday": { 
        "Save": true,
        "Hour": "0000-00-00T00:30:00" }, 
    "Wednesday": { 
        "Save": false, 
        "Hour": "0000-00-00T00:30:00" }, 
    "Thursday": { 
        "Save": true, 
        "Hour": "0000-00-00T00:30:00" }, 
    "Friday": { 
        "Save": false, 
        "Hour": "0000-00-00T00:30:00" }, 
    "Saturday": { 
        "Save": true, 
        "Hour": "0000-00-00T00:30:00" }, 
    "Sunday": { 
        "Save": false, 
        "Hour": "0000-00-00T00:30:00" }
}

a 3 monthly run on the last day at 4am

{ 
    "Frequency": "Monthly", 
    "Every": "3", 
    "StartingAt": "0000-00-00T04:00:00", 
    "Day": "29" 
}

BACKUP_SetScheduleNone(xmlBupReference)

Parameters
$1	text	required	XML Reference

Remove any schedule, no automatic runs anymore

BACKUP_SetScheduleHour(xmlBupReference;Every;StartAt)

Parameters
$1	text	required	XML Reference
$2	longint	required	number of hours between runs 1-23
$3	time	required	the start time for the first run

set an hourly schedule, tell every which hour to run and the time to start on

BACKUP_SetScheduleDay(xmlBupReference;Every;Hour)

Parameters
$1	text	required	XML Reference
$2	longint	required	every which day to run 1 is daily
$3	hour	required	when to run in the day

you guessed it, a daily schedule, tell every which day to run and the time to start on

BACKUP_SetScheduleWeek(xmlBupReference;Frequency;Mon;Tue;Wed;Thu;Fri;Sat;Sun)

Parameters
$1	text	required	XML Reference
$2	longint	required	every which week to run 1 is weekly
$3	text	start time for Monday	either a time string for the desired start date or empty for no run on that day
$4	text	start time for Tuesday	see above
$5	text	start time for Wednesday	see above
$6	text	start time for Thursday	see above
$7	text	start time for Friday	see above
$8	text	start time for Saturday	see above
$9	text	start time for Sunday	see above

This one is not so elegant, could have done it with an object or the like, but thought this was simpler.
Just pass a time string for each day you want to run on or an empty string if not.

BACKUP_SetScheduleMonth(xmlBupReference;Every;Day;Hour)

Parameters
$1	text	required	XML Reference
$2	longint	required	run every which month 1 is monthly
$3	longint	required	the day number in the month to run on Allowed values are 1 to 29, where 29 is always last day in the month
$4	time	required	the time to run the backup on

Do I have to explain?

Some nifty utilities for your pleasure

BackupSize:=BACKUP_RequiredSize({$b_WithLog})

Parameters
$1	boolean	optional	include log file size in calculation
$0	real		the prospective size of the backup, please see comment ...

This one is a bit weak, as it does not take into account the .ExternalData folder or any files associated with the backup. Maybe I will add that in the future.
Also it cannot possibly predict the effects of either compression, interlacing and or redundancy. So take the result with a grain of salt.

SpaceOnDisk:=BACKUP_GetSpace

Parameters
$1	text	required	the path to the backup destination
$0	real		the free space on the backup volume

As is written on the box

Value:=BACKUP_GetData(xmlBackupRef;DOMPath)

Parameters
$1	text	required	XML Reference
$2	text	required	the path to the requested item
$0	text		the value of the item

This one is a direct line into the backup preferences XML and allows you to set any value directly by passing the path to it and a value as text.
Most of the Get/Set methods above are wrappers to these BACKUP_getData and BACKUP_SetData

BACKUP_SetData(xmlBackupRef;DOMPath;Value)

Parameters
$1	text	required	XML Reference
$2	text	required	the path to the requested item
$3	text	required	the value to set for the item

Downloads

You made it so far, so here is your reward:

Even if you already downloaded it, please do again, as this is HotFix 1 that resolves the Problem of the missing Asterisk after Get 4D Folder that prevented the Component from doing it’s job…

More examples

adding the index file to the backup

$tx_Backup:=BACKUP_LoadPrefs   //load the preferences
ARRAY TEXT($at_Files;0)
$l_Files:=BACKUP_GetInclude ($tx_Backup;->$at_Files)  //get the files currently included
$t_Index:=Substring(Data file;1;Length(Data file)-3)+"4DIndx"  //get index file
If (Test path name($t_Index)=Is a document)
   APPEND TO ARRAY($at_Files;$t_Index)  //add index folder
   BACKUP_SetInclude ($tx_Backup;->$at_Files)  //set the files to be included
   BACKUP_SavePrefs ($tx_Backup;True)  //write all changes back to the file and clear the DOM reference
Else
   ALERT("Indexdatei nicht gefunden")
   BACKUP_Release ($tx_Backup)
End if

Final comments

Although the flow of donations for my last large component was – to use an euphemisms – dismal, I have not yet given up hope in the good of mankind and 4D developers in general. So if you are interested in the source code for this component, drop me a donation.

Anything, send food, wine is accepted, funny gifs of cats, whatever, even I need some appreciation from time to time…

Thanks folks,
That’s it for today

7 thoughts on “To backup or not to backup?”

Stefan Kitching June 1, 2017 at 1:08 pm

Notifications please

Reply ↓
1. admin Post authorJune 1, 2017 at 1:42 pm
  
  Did you receive the Confirm your subscription for 4DTidbits E-Mail?
  You need to confirm the subscription for it to work.
  Thanks
  
  Reply ↓
admin Post authorJune 9, 2017 at 8:27 am

Sorry, folks, as no one has yet noticed, there was a tiny bug in the component preventing it from doing its job when used as a component:
I forgot the asterisk in
Get 4D Folder(Logs folder;*)
Please re-download the component if you intend to use it.

Reply ↓
Gerald Balzer July 4, 2017 at 8:32 am

Hi Alex, I assume that this in not correct:
CompressionRate True/False include structure files in backup

It’s in the options of “BACKUP_GetOption”

Reply ↓
1. admin Post authorJuly 4, 2017 at 8:38 am
  
  Thanks, corrected!
  
  Reply ↓
franz September 10, 2019 at 4:36 pm

Hi Alex 🙂
your component is amazing and it works fine. I have two question
1) apparenty the “BACKUP_SetSets(xmlBupReference;SetsToKeep)” doesn’t seem to work.
2) there’s a way to create a new backup setting without doing it from the 4d settings?
thanks

Reply ↓
1. admin Post authorNovember 20, 2019 at 11:05 am
  
  Sorry for being late here…
  I will check this in the next few days while upgrading all my tools to V17 / V18
  
  Reply ↓

4DTidbits

a sporadic source of barely useful 4D stuff

To backup or not to backup?

UPDATED to HF1 on 2017-06-09 go to downloads

AC_BackupTools

An example

Basic methods for accessing backup settings

Information on last backup run

getting and setting the backup parameters

including files into the backup

What if something goes wrong?

Where do those funny numbers come from MyBackup[0123].4BK?

When is the backup supposed to run?

Some nifty utilities for your pleasure

Downloads

More examples

Final comments

Related

7 thoughts on “To backup or not to backup?”

Leave a Reply Cancel reply

UPDATED to HF1 on 2017-06-09 go to downloads

AC_BackupTools

An example

Basic methods for accessing backup settings

Information on last backup run

getting and setting the backup parameters

including files into the backup

What if something goes wrong?

Where do those funny numbers come from MyBackup[0123].4BK?

When is the backup supposed to run?

Some nifty utilities for your pleasure

Downloads

More examples

Final comments

Share this:

Related

7 thoughts on “To backup or not to backup?”

Leave a Reply Cancel reply