DataVeil home

Home    Table of Contents

Executing Projects

There are four functions that relate to executing a masking project:

 Compile
 Preview Run
 Run
 Stop

Compile

Compiling a Project is an optional step. DataVeil shall automatically perform a Compile whenever you perform a Run or a Preview Run.

The Compile function is provided so that you can verify whether a masking Project is ready to run without having to attempt an actual Run. For example, the compiler shall warn you if Sensitive fields do not have masks defined and detect other such problems to assist you while you are developing your Project.

Compiling is simply a matter of clicking on the Compile icon  in the main toolbar or the "Compile" button in the Execution view.

If the compilation is successful, DataVeil shall populate the Control tab in the Execution view with a representation of the Run Plan as shown in the example below:

Data masking execution control

 

Preview Run

The Preview Run is used to perform the masking functions defined in the Project but without overwriting the actual original values. After a Preview Run completes, you can view samples of the 'before' and 'after' masked values in the the Data Browser view.

The Preview Run is an entirely optional step. It's purpose is to help you review and verify that the configured masks are generating suitable masked values as expected without actually overwriting your original sensitive data.

Important:

Although the Preview Run shall not overwrite any of your original data, it shall still make temporary changes to your database for the duration of the Preview Run. This includes disabling triggers and constraints, creating and dropping temporary tables (and a temporary schema for SQL Server). Furthermore, the structure of some original tables may be temporarily altered.

Therefore, you should only perform a Preview Run on a database to which you have exclusive access.

You can perform a Preview Run by clicking on the Preview Run icon  in the main toolbar or Data Browser toolbar, or clicking on the "Preview Run..." button in the Execution view.

A confirmation box shall be displayed:

Preview run confirmation

  

Masked Value Generation Scope

Generally speaking, the Quick option is adequate. It executes quickly and displays accurate samples of generated masked values, subject to only a few approximations (described below.)

Quick

Masked values shall be generated for only a small subset of all sensitive values. This means that the Preview Run execution time shall usually be quick - typically a matter of a few minutes or even as little as several seconds (depending on the masks that are used).

Most of the generated masked value samples shall be the actual values that would be generated if an actual masking Run was performed; however, in some cases the values are approximated or unavailable because the actual values cannot be determined when processing only a subset of all the sensitive values.

The approximations apply to:

* Shuffle mask.

Only the first 10,000 rows are shuffled (whereas a Complete Preview Run shuffles the entire range of rows).

* Inherit mask.

These are dependents whose parent values may not have been included the masked subset(s) which means that the masked values may not be available. Therefore, the dependent Column values shall be displayed as "NA (Inherit)" indicating that the inherited values from the parent dependencies are not available in Quick preview. If you wish to see the masked values generated for dependants then use Complete scope in Preview Run.

* Duplicates.

The 'Duplicates' option, if configured for a Column, is ignored when generating a Quick Preview Run sample.

Complete

The entire range of sensitive values shall be processed when generating sample masked values. This means that the Preview Run execution time shall be almost as long as an an actual masking Run.

All sample masked values shall be accurate (i.e. there are no approximations such as those that may be used in a Quick Preview Run).

Please note that the sample masked values displayed shall be the actual masked values if a normal masking Run is subsequently performed but only if Deterministic masking mode (the default) is used. If Deterministic mode is not used then the masked values are random upon each execution. The determinism can be set as a Project-level default under the Settings tab in the main Window, or overridden on a per-mask basis within each mask's configuration.

 

Sample row count per table

This is the maximum number of 'before' and 'after' masked value rows shall be shown for each masked Table in the the Data Browser view.

The valid range is from 1 to 100 rows.

 

Run

You can run a DataVeil masking Project by clicking on the Run icon   in the toolbar or clicking on the "Run..." button in the Execution window.

Masking execution buttons

 

DataVeil shall automatically compile the Project and ask you to confirm that you wish to proceed to mask the database(s) defined in the Project. This confirmation dialog also gives you the option to review the actual 'before' and 'after' masked values in the Data Browser view upon Run completion.

Please read the cautions in the confirmation dialog, they are extremely important! 

 

 Data masking run confirmation

 

Please make sure that the databases that you have defined in your Connection panels are pointing to the databases that you want to overwrite with masked values. There is no "undo masking" available from DataVeil. You should have a backup copy of the database that you will be masking, in case you need to recover the original values - or even if you later discover that you need to modify your DataVeil masking project and re-run it.

Also please make sure that no-one else will be accessing the databases that you will be masking for the duration of the masking run. DataVeil will be making broad changes to these databases that will affect anyone else accessing that database. Other users could even cause the masking project to fail.

Finally, you must ensure that the usernames specified in the database Connection panels have broad administrative privileges. DataVeil will creating and dropping schemas (on some database systems, such as SQL Server), tables, columns and other objects. It will also be disabling and enabling triggers and constraints.

After you have confirmed this warning by pressing 'OK', DataVeil will ensure that a valid Project Key has been entered if the project has been loaded from a file (.dvp) and such a key is required. If a Project Key is required and has not yet been entered then you will be prompted to enter it now.

For more information on Project Keys please refer to the Project Security section.

After user confirmation, DataVeil shall begin execution.

The Status area will be updated in real-time to keep you informed of the Project execution progress. An example is shown below:

Masking run in progress

   

Upon completion, you can review an automatically generated data masking report under the Reports tab. Please refer to the Masking Report section.

 

Considerations for Azure SQL Database

DataVeil uses the UPDATE query to overwrite sensitive values with masked values.

DataVeil cannot automatically disable Azure transaction logging. DataVeil generates writes to the transaction log during actual masking runs and also during preview runs.

Considerations for Oracle

DataVeil uses the UPDATE query to overwrite sensitive values with masked values except for Index Organized Tables (IOT). If a primary key column of an IOT is masked then DataVeil shall use the INSERT query to replace the original table with a new table containing masked values.

Oracle maintains an Undo Tablespace which records changes to the database. During the Undo Retention Period (the minimum amount of time that Oracle shall keep an Undo record before overwriting it) it is possible to recover the original sensitive values that have been masked.

Therefore, if you are making a masked Oracle database available to users who must not have access to the original sensitive data, then either ensure that those users do not have permission to access Oracle data recovery features or purge the Undo records. Please consult your DBA.

Considerations for SQL Server

DataVeil uses the UPDATE query to overwrite sensitive values with masked values.

Prior to overwriting sensitive values with masked values, DataVeil changes the SQL Server Recovery Mode to 'Simple'. This has the effect of automatically truncating the transaction log after DataVeil commits its updates. After the Run has completed DataVeil restores the Recovery Mode to its original value prior to the masking Run.

 

Run from Command Line

You can run a DataVeil masking Project from the Windows operating system command line.

A sample batch file "dataveil_cmd.bat" has been provided in the installation folder "dataveil\batch"

Please refer to this sample batch file. It contains a complete description of all command parameters, a sample command to execute a DataVeil Project and how to verify whether the execution outcome was successful or not.

Important

Do NOT run more than one instance of DataVeil concurrently for the same operating system User. DataVeil relies on cached data in a User directory. This does not allow more than one instance of DataVeil from running concurrently for the same User.

Multiple DataVeil installations for the same User will not overcome this limitation as all of the User's DataVeil installations will attempt to use the same DataVeil user directory.

As a precaution, you will notice that the sample batch file contains the START command with a /WAIT parameter to ensure that the DataVeil instance completes before resuming the batch file execution. i.e. You can can have sequential executions of multiple DataVeil Projects in your batch file (but not concurrent.)

If you attempt to run multiple instances then each DataVeil instance will likely report spurious errors and the following dialog may also be displayed:

Multiple instance warning

 

Stop

If you want to gracefully stop an executing project Run or Preview Run then you can perform the Stop command by clicking on the  icon in the main toolbar.

The Stop command can only stop DataVeil projects that were started from the GUI interface. A Stop command cannot be used to stop a DataVeil project that was run from the command line.

A confirmation dialog similar to that shown below shall be displayed. If you wish to continue with the Stop command then confirm by selecting the checkbox and click "Yes".

Stop data masking run confirmation

Upon confirmation DataVeil shall attempt to stop its masking tasks and recover the original data and constraint settings as they were immediately prior to the Run.

Please note that if a DataVeil Run has already entered the Finalize phase then the Stop command shall be ignored.

 

Home    Table of Contents