To backup or not to backup?

UPDATED to HF1 on 2017-06-09 go to downloads

We have all been there and done that:

You will know the value of backup when you don’t have one

4D has given us a powerful backup system with a lot of features users of other systems envy us for.

What man do not realize, is that you can modify all parameters of a backup from you code, making changes on the fly and averting user errors.

Modifying the Backup.xml file allows you to set the schedule, files to be included, how to handle failures, basically everything that you can set in the backup settings panel of the database preferences.

Now I know that working with XML files is a bit tedious and not always very straightforward, also all the possible settings are a drag to keep track of.

Thus I have again come to the rescue and hereby publish my latest component:

AC_BackupTools

Just drop it into your components folder and you are ready to go.

There are quite a few commands, but most of them are just wrappers to single calls of DOM SET XML ELEMENT VALUE.

An example

Here is a quick example for changing the schedule to daily at 2 am and adding the logs folder:

$tx_Backup:=BACKUP_LoadPrefs //load the preferences 
BACKUP_SetScheduleDay ($tx_Backup;1;?02:00:00?) //set a daily schedule at 2am
ARRAY TEXT($at_Files;0)
$l_Files:=BACKUP_GetInclude($tx_Backup;->$at_Files) //get the files currently included
APPEND TO ARRAY($at_Files;Get 4D folder(Logs folder)) //add log folder
APPEND TO ARRAY($at_Files;Get 4D folder(Licenses folder)) //add licenses folder
BACKUP_SetInclude ($tx_Backup;->$at_Files) //set the files to be included
BACKUP_SavePrefs ($tx_Backup;True) //write all changes back to the file and clear the DOM reference

simple isn’t it?

Although I would expect you to be able to figure out the rest, here are all component methods and their explanation:

Basic methods for accessing backup settings

The following methods are used to read and write the actual backup.xml file.
Please not, that all Methods from this component can only be used on 4D Server, Standalone and Volume Desktop, they will simply not work on 4D Client.


xmlBackupRef:=BACKUP_LoadPrefs

Parameters
$0TextXML Reference

As the name suggests, this will load the Backup.xml file and return an XML reference to the corresponding DOM. Use this for all other calls.


BACKUP_SavePrefs ($tx_Backup;True)

Parameters
$1TextrequiredXML Reference
$2BooleanoptionalTrue to purge the XML DOM from memory. Defaults to False.

This one saves the changes back to the XML file, optionally it purges the XML DOM reference, freeing up memory.


BACKUP_Release ($tx_Backup)

Parameters
$1textrequiredXML Reference

This it clean up, it purges the XML DOM reference


Information on last backup run

As the backup.xml also contains information on the last backup run, we can obviously also read these. You could write to them but why would you want to do that?


Path:=BACKUP_LastPath(xmlBupReference{;What})

This returns the path of the last backup file (.4BK), optionally it can also return the path to the last journal file backup (.4BL)

Parameters
$1textrequiredXML Reference
$2textoptionalwhat to get:
- data (default if omitted)
- log
$0textthe path to the desired backup file

LastTime:=BACKUP_LastTime(xmlBupReference)

Returns the time of the last backup

Parameters
$1textrequiredXML Reference
$0timethe time of the last backup, ?00:00:00? if none

LastDate:=BACKUP_LastDate(xmlBupReference)

You might have guessed: the date of the last backup

Parameters
$1textrequiredXML Reference
$0datethe date of the last backup, !00.00.00! if none

getting and setting the backup parameters


DestinationPath:=BACKUP_GetDestination(xmlBupReference)

where the backup will be stored

Parameters
$1textrequiredXML Reference
$0textthe path to the backup destination folder

BACKUP_SetDestination(xmlBupReference;newPath)

tell backup where the backups should be stored

Parameters
$1textrequiredXML Reference
$2textrequiredthe new path to the backup destination folder

SegmentSize:=BACKUP_GetSegmentation(xmlBupReference)

the currently set size of backup file segments

Parameters
$1textrequiredXML Reference
$0longintthe backup segment size in MB
0 if no segmentation specified

BACKUP_SetSegmentation(xmlBupReference;SegmentSize)

set the desired set size of backup file segments

Parameters
$1textrequiredXML Reference
$2longintrequiredthe desired backup segment size in MB
set to 0 if no segmentation is wanted

SetsToKeep:=BACKUP_GetSets(xmlBupReference)

get the number of backup sets to keep around

Parameters
$1textrequiredXML Reference
$0longintthe number of segments to keep
0 means none

BACKUP_SetSets(xmlBupReference;SetsToKeep)

tell backup the number of backup sets to keep around

Parameters
$1textrequiredXML Reference
$2longintrequiredthe desired number of backup sets to keep
set to 0 to disable keeping of old sets

ParamaterValue:=BACKUP_GetParameters(xmlBupReference;ItemCode)

Parameters
$1textrequiredXML Reference
$2textrequiredone of the permitted item codes
see explanation below
$0textthe value for the parameter requested
see list below
Parameter codes and values
CompressionRateNoneno compression
Fastfast but not so small
Compactsmall but slow
RedundancyNone
Low
Medium
High
InterlacingNone
Low
Medium
High

BACKUP_SetParameters(xmlBupReference;ItemCode;ParamaterValue)

Parameters
$1textrequiredXML Reference
$2textrequiredone of the permitted item codes
see explanation above
$3textrequiredthe value for the parameter requested
see list of possible values above

IsSet:=BACKUP_GetOption(xmlBupReference;ItemCode)

Parameters
$1textrequiredXML Reference
$2textrequiredone of the permitted item codes
see explanation below
$0booleanthe value for the option requested
Option codes
IncludeStructureFile True/Falseinclude structure files in backup
IncludeDataFileTrue/Falseinclude data file and external data in backup
IncludeAltStructFileTrue/Falseinclude user structure file
BackupJournalVerboseModeTrue/Falsemore detailed backup journal
CheckArchiveFileDuringBackupTrue/Falsecheck the backup file while backing up
EraseOldBackupBeforeTrue/Falseerase surplus old files before backing up
BackupIfDataChangeTrue/Falsebackup only if data has changed
AutomaticRestartTrue/Falseautomatically restart backup if failed
AutomaticRestoreTrue/Falseautomatically restore from backup after error on opening data file

BACKUP_SetOption(xmlBupReference;ItemCode;IsSet)

Parameters
$1textrequiredXML Reference
$2textrequiredone of the permitted item codes
see explanation above
$3bolleanrequiredset the option to true or false

including files into the backup

you obviously did know that you can add any kind of file or directory into the backup did you?
Here is how to handle that with the component:


FileCount:=BACKUP_GetInclude(xmlBupReference;TextArrayPointer)

Parameters
$1textrequiredXML Reference
$2pointerrequiredpointer to a text array with the full paths to each file or folder to include in the backup

BACKUP_SetInclude(xmlBupReference;TextArrayPointer)

Parameters
$1textrequiredXML Reference
$2pointerrequiredpointer to text array with the paths of all files to
include in the backup

What if something goes wrong?

Sometimes backups do go wrong, and we want to tell 4D what to do in such a case:


OptionValue:=BACKUP_GetFailOptions(xmlBupReference;OptionCodes)

Parameters
$1textrequiredXML Reference
$2textrequiredone of the permitted item codes
see explanation below
$0textthe value for the option requested
Parameter codes and values
TryBackupAtTheNextScheduledDateTrue or Falseas text
TryToBackupAftertime stringas text "01:23:45"
AbortIfBackupFailTrue or Falseas text
RetryCountBeforeAborta numberas text

BACKUP_SetFailOptions(xmlBupReference;OptionCodes;OptionValue)

Parameters
$1textrequiredXML Reference
$2textrequiredone of the permitted item codes
see explanation above
$3textrequiredthe value for the option requested
see explanation above

Where do those funny numbers come from MyBackup[0123].4BK?

Backup sets are numbered. You can edit these numbers, but be careful not to shoot yourself in the foot.


CurrentSet:=BACKUP_GetCurrentSet(xmlBupReference)

Parameters
$1textrequiredXML Reference
$0longintthe number of the current backup set

BACKUP_SetCurrentSet(xmlBupReference;currentSetNo)

Parameters
$1textrequiredXML Reference
$2longintrequiredthe new number for the current backup set
0 will restart at 1

When is the backup supposed to run?

Ok setting the schedule is a bit more complicated, so I created wrappers for the different options:


ScheduleObject:=BACKUP_GetSchedule(xmlBupReference)

Parameters
$1textrequiredXML Reference
$0objectan object with the data for the currently set type of schedule,
see examples below

this will return an object with the details of the current scheduling settings, see examples below:

Example for object with no schedule

{ 
    "Frequency": "none"
}

Example for object with daily run

{ 
    "Frequency": "Hourly",
    "Every": "1", 
    "StartingAt": "0000-00-00T00:30:00" 
}

Example for object with daily run

{
   "Frequency": "Daily",
   "Every": "1",
   "StartingAt": "0000-00-00T02:00:00"
}

a weekly run on specific days

{ 
    "Frequency": "Weekly",
    "Every": "1", 
    "Monday": { 
        "Save": true, 
        "Hour": "0000-00-00T00:30:00" }, 
    "Tuesday": { 
        "Save": true,
        "Hour": "0000-00-00T00:30:00" }, 
    "Wednesday": { 
        "Save": false, 
        "Hour": "0000-00-00T00:30:00" }, 
    "Thursday": { 
        "Save": true, 
        "Hour": "0000-00-00T00:30:00" }, 
    "Friday": { 
        "Save": false, 
        "Hour": "0000-00-00T00:30:00" }, 
    "Saturday": { 
        "Save": true, 
        "Hour": "0000-00-00T00:30:00" }, 
    "Sunday": { 
        "Save": false, 
        "Hour": "0000-00-00T00:30:00" }
}

a 3 monthly run on the last day at 4am

{ 
    "Frequency": "Monthly", 
    "Every": "3", 
    "StartingAt": "0000-00-00T04:00:00", 
    "Day": "29" 
}

BACKUP_SetScheduleNone(xmlBupReference)

Parameters
$1textrequiredXML Reference

Remove any schedule, no automatic runs anymore


BACKUP_SetScheduleHour(xmlBupReference;Every;StartAt)

Parameters
$1textrequiredXML Reference
$2longintrequirednumber of hours between runs
1-23
$3timerequiredthe start time for the first run

set an hourly schedule, tell every which hour to run and the time to start on


BACKUP_SetScheduleDay(xmlBupReference;Every;Hour)

Parameters
$1textrequiredXML Reference
$2longintrequiredevery which day to run
1 is daily
$3hourrequiredwhen to run in the day

you guessed it, a daily schedule, tell every which day to run and the time to start on


BACKUP_SetScheduleWeek(xmlBupReference;Frequency;Mon;Tue;Wed;Thu;Fri;Sat;Sun)

Parameters
$1textrequiredXML Reference
$2longintrequiredevery which week to run
1 is weekly
$3textstart time for Mondayeither a time string for the desired start date or empty for no run on that day
$4textstart time for Tuesdaysee above
$5textstart time for Wednesdaysee above
$6textstart time for Thursdaysee above
$7textstart time for Fridaysee above
$8textstart time for Saturdaysee above
$9textstart time for Sundaysee above

This one is not so elegant, could have done it with an object or the like, but thought this was simpler.
Just pass a time string for each day you want to run on or an empty string if not.


BACKUP_SetScheduleMonth(xmlBupReference;Every;Day;Hour)

Parameters
$1textrequiredXML Reference
$2longintrequiredrun every which month
1 is monthly
$3longintrequiredthe day number in the month to run on
Allowed values are 1 to 29, where 29 is always last day in the month
$4timerequiredthe time to run the backup on

Do I have to explain?


Some nifty utilities for your pleasure


BackupSize:=BACKUP_RequiredSize({$b_WithLog})

Parameters
$1booleanoptionalinclude log file size in calculation
$0realthe prospective size of the backup, please see comment ...

This one is a bit weak, as it does not take into account the .ExternalData folder or any files associated with the backup. Maybe I will add that in the future.
Also it cannot possibly predict the effects of either compression, interlacing and or redundancy. So take the result with a grain of salt.


SpaceOnDisk:=BACKUP_GetSpace

Parameters
$1textrequiredthe path to the backup destination
$0realthe free space on the backup volume

As is written on the box


Value:=BACKUP_GetData(xmlBackupRef;DOMPath)

Parameters
$1textrequiredXML Reference
$2textrequiredthe path to the requested item
$0textthe value of the item

This one is a direct line into the backup preferences XML and allows you to set any value directly by passing the path to it and a value as text.
Most of the Get/Set methods above are wrappers to these BACKUP_getData and BACKUP_SetData


BACKUP_SetData(xmlBackupRef;DOMPath;Value)

Parameters
$1textrequiredXML Reference
$2textrequiredthe path to the requested item
$3textrequiredthe value to set for the item

Downloads

You made it so far, so here is your reward:

Even if you already downloaded it, please do again, as this is HotFix 1 that resolves the Problem of the missing Asterisk after Get 4D Folder that prevented the Component from doing it’s job…

More examples

adding the index file to the backup

$tx_Backup:=BACKUP_LoadPrefs   //load the preferences
ARRAY TEXT($at_Files;0)
$l_Files:=BACKUP_GetInclude ($tx_Backup;->$at_Files)  //get the files currently included
$t_Index:=Substring(Data file;1;Length(Data file)-3)+"4DIndx"  //get index file
If (Test path name($t_Index)=Is a document)
   APPEND TO ARRAY($at_Files;$t_Index)  //add index folder
   BACKUP_SetInclude ($tx_Backup;->$at_Files)  //set the files to be included
   BACKUP_SavePrefs ($tx_Backup;True)  //write all changes back to the file and clear the DOM reference
Else
   ALERT("Indexdatei nicht gefunden")
   BACKUP_Release ($tx_Backup)
End if

Final comments

Although the flow of donations for my last large component was – to use an euphemisms – dismal, I have not yet given up hope in the good of mankind and 4D developers in general. So if you are interested in the source code for this component, drop me a donation.

Anything, send food, wine is accepted, funny gifs of cats, whatever, even I need some appreciation from time to time…

Thanks folks,
That’s it for today

7 thoughts on “To backup or not to backup?

    1. admin Post author

      Did you receive the Confirm your subscription for 4DTidbits E-Mail?
      You need to confirm the subscription for it to work.
      Thanks

      Reply
  1. admin Post author

    Sorry, folks, as no one has yet noticed, there was a tiny bug in the component preventing it from doing its job when used as a component:
    I forgot the asterisk in
    Get 4D Folder(Logs folder;*)
    Please re-download the component if you intend to use it.

    Reply
  2. Gerald Balzer

    Hi Alex, I assume that this in not correct:
    CompressionRate True/False include structure files in backup

    It’s in the options of “BACKUP_GetOption”

    Reply
  3. franz

    Hi Alex 🙂
    your component is amazing and it works fine. I have two question
    1) apparenty the “BACKUP_SetSets(xmlBupReference;SetsToKeep)” doesn’t seem to work.
    2) there’s a way to create a new backup setting without doing it from the 4d settings?
    thanks

    Reply

Leave a Reply to admin Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.