DataVeil 4.9 Released

DataVeil version 4.9 has been released and is available for download at https://www.dataveil.com/my-downloads/

NEW: The Pre/Post SQL Mask Option

A new mask option has been implemented that allows optional SQL code to be performed on each value:

  • Immediately prior to being masked, and/or...
  • Immediately after being masked.

The Pre/Post SQL option will appear as a new tab in a mask's configuration dialog panel.

n49_prepostsql_option

This option enables the user to enhance and expand masking possibilities efficiently, such as modifying the default output format of DataVeil masks and even performing data type manipulation such as type conversion or length truncation.

As a simple example to illustrate, suppose that the user would like instances of multiple names in the masked output of a Person Given Name mask to be separated by pipe characters instead of the default spaces.
The Post SQL code could simply be REPLACE(#_value, ' ', '|'), as shown below.

Original masked output:

Masked output using Post-Mask SQL:

n49_masked_pipe
n49_postsql_pipe

Another example could be when using the Random Number mask on string values that contain various formatting characters that cannot be implicitly converted to numbers. The Pre-Mask SQL code could specify conversion from a formatted string to a numeric data type and the Post-Mask SQL code would then convert the numeric masked output back to the required string format.

For more complex requirements, the SQL code could be placed into an SQL User Defined Function (UDF) that can be called from the Pre/Post-Mask SQL code. The UDF can be defined and uploaded by the same DataVeil masking project as a Pre-Masking Script and dropped at the end of the masking run using a Post-Masking Script.

You could also define the Pre/Post SQL customized mask as a Mask Component so that it becomes a standard mask within your environment available to all of your masking projects.

For further details and more detailed examples please refer to the User Guide at Pre/Post SQL Option.

NEW: Writer Threads Setting

Previously, only a single setting in Database Settings determined the maximum number of threads that DataVeil would use during a masking run.

Now there are two settings, one that relates to Mapping threads (masked value generation) and the other for Writer threads (table updates). This provides finer control in environments where, for example, it is desirable to have faster mapping (more Mapping threads) but a slower update rate (fewer Writer threads) due to environment limitations such as if active replication is required.

The defaults for new projects are 8 threads for Mapping and Writers. In existing projects, the Writer threads will default to the same value as the existing thread setting.

n49_writer_threads

Revised Batch Settings for Optimal Performance

The general recommendations for DataVeil batch settings have been recently revised.

The default batch settings for new projects have been lowered and can be seen in the screen capture above. Specifically:

Mapping batch size: 100,000 rows
Table update batch size: 400,000 rows.

Existing projects will be unaffected and retain their existing settings.

Previous defaults were significantly higher. This is because they were the result of older benchmarks performed against older versions of DBMS's.

It should be emphasized that a wide range of performance characteristics have been observed depending on DBMS type, version and characteristics of the actual database being masked and its environment. Therefore, it is entirely possible that an existing masking project with higher settings will perform better than these revised default settings.

These new default settings should be regarded as a good starting point and the best approach would be to compare the performance of a masking project using different settings. For example, take a note of a project's current performance and its current batch settings. Next, adjust the batch settings to new values and compare the resulting performance. The automatically-generated masking reports show the performance (duration) of each masking task and therefore comparing such reports is useful in comparing performance differences.

Other Changes

There are several other minor updates and fixes. For details, please refer to the release notes using the Version History link from the Downloads page.