SQL Structure Access

Sadly I lost a client last week as they decided it would be more expensive to pay for all the missed updates of 4D than to buy something completely new…
At least I got them to pay for me helping to export that data from 4D to something  the new supplier (who is using MSSQL server) could read…

Well I thought, that’s going to be quick money and fired up SQL EXPORT DATABASE

Boy was I wrong…

As long as you stay within the 4D Universe all may be fine, but no other SQL database engine can read these Export.sql files 4D creates, simply due to the way they write the INSERT INTO line…

INSERT INTO [ATY_TITLES] ( [ATT_ID] , [ATT_ATY_ID] , [ATT_TITLE] , [ATT_HANDLE] , [ATT_WTS_ID] , [ATT_WTP_ID] )

This may make sense for 4D, as there are apparently still some people out there who use spaces in field and table names (allowing spaces, special characters etc. in object names was not one of the better ideas Laurent ever had…) but all other SQL systems will just reject this syntax as being utterly ridiculous..

I also found out, that contrary to other SQL dumps, for example by MySQL, the 4D dump does not contain the CREATE TABLE statements needed to recreate a compatible structure.

A quick search revealed there is no such command in 4D, all these nice structure exports to XML, HTML, etc. are absolutely useless in this context (has anyone noticed that you cannot print the structure anymore?).

So, only solution, roll my own.

Result is this little stand-alone component with only a few commands:

SchemaText:=ACSQLDEF_SchemaDump({TargetSystem{;SortTable{;SortFields}}})

It will fill a text variable with the complete schema definition of the host database, including Indexes, primary keys and the UNIQUE constraints set in the database.

You may pass a few optional parameters:

TargetSystem  : text    : The target database system for the Schema export, as different target systems require different syntax for some fields. Currently supported are MySQL4; MySQL5 (different handling of booleans); MSSQL and the default 4D

SortTable     : boolean : Sort tables alphabetically
SortFields    : boolean : well, take a guess

As for the target systems, there is a json file in the resources folder, look at it and you can add more target systems, if you do, please share them back.

Now for those unusable Export.sql files…

ACSQL_CleanDump

Will take care of these. Just point it to the SQLExport folder created by 4D and it will troll through all files transforming this

INSERT INTO [ATY_TITLES] ( [ATT_ID] , [ATT_ATY_ID] , [ATT_TITLE] , [ATT_HANDLE] , [ATT_WTS_ID] , [ATT_WTP_ID] )

into this

INSERT INTO ATY_TITLES ( ATT_ID, ATT_ATY_ID, ATT_TITLE, ATT_HANDLE, ATT_WTS_ID, ATT_WTP_ID )

With a 32 GB Export made up of roughly 18 900 files it took some 21 minutes to run.

Of course you must make sure that your table and field names are SQL compliant… I could have added a check in here, but where would be the fun in this?

If you start the component in compiled mode or as a compiled application, it will automatically run this command.

There are a few more tools in there:

TableDef:=ACSQLDEF_TableDefinition(TablePointer{;SortFields})

Create the table definition for a specific table

$p_FieldPointer:=ACSQL_PrimaryKeyofTable(TablePointer)

Will return a pointer to the PrimaryKey field of a table, if the PrimaryKey is made up of multiple fields, only the first field is returned.

Indexes:=ACSQLDEF_IndexesForTable(TablePointer)

will return the SQL Index creation statements for the the table passed, composite indexes are handled as well, you might get something like this in return:

CREATE INDEX IDX_11_21 ON ADRESSEN (ADR_IS_MAIN);
CREATE INDEX IDX_11_11 ON ADRESSEN (ADR_FIRMA,ADR_FIRMA2);
CREATE INDEX IDX_11_10 ON ADRESSEN (ADR_SORT);
PrimaryKeys:=ACSQLDEF_PKeyForTable(TablePointer)

will return a text with only the field name(s) of the fields that make up the primary key of the table

ADR_IS_MAIN
or
ADR_FIRMA,ADR_FIRMA2

Ok, that’s all folks, enjoy.

The component is V14 and can be downloaded here:

AC_SQLHelper.4dbase

You need the source? Make me an offer I can’t resist…

I’d love to get feedback on this.

7 thoughts on “SQL Structure Access

  1. Pingback: SQL schema definition export that makes sense | die4Dwerkstatt

  2. Mike Peters

    Where is ACSQL_CleanDump? It doesn’t appear to be in the Component. This will be a life saver as we migrate away from 4D.

    Reply
    1. admin Post author

      Just updated the component, looks like I forgot to set the required method property. Should work now.

      Reply
    1. admin Post author

      Well, as 4D cannot really use these fields in any meaningful way, i forgot about those as I never use them (though i’d like to).
      You can add these yourself in the translation table, just take a peek at the resources folder of the component, there’s a self explaining json file you can edit to your hearts contempt.

      Reply
        1. admin Post author

          I do not see why not?
          The data structures have not really changed since V14. The only thing we do not manage are Object fields, these would need to be setup as Text fields and the content must be serialised using JSON Stringify.
          That can easily be added to the code.

          Reply

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.