DataVeil Native Library
The DataVeil software includes what is referred to within DataVeil as 'native libraries'.
A DataVeil native library is available only for Oracle and SQL Server. It is not available for Azure SQL DB or MySQL.
A DataVeil native library consists of files that are installed on the DBMS where data masking shall be performed.
The installation of a DataVeil native library is optional with the exception that it is required for the use of the JSON mask.
The main purpose of the native library is to provide faster masking performance than that of the default SQL functions that are included within the core DataVeil product. The native library also provides support for the DataVeil JSON mask because it provides more sophisticated JSON processing support than was available within the DBMS' pure SQL environment.
When a DataVeil data masking run starts, DataVeil queries the database to check if a compatible version of the DataVeil native library is found on the DBMS. If found then DataVeil shall use the native library for performing masking functions. If not found then DataVeil shall continue to execute using its built-in SQL functions. These are typically much slower than native libraries.
Therefore, it is highly recommended to install the DataVeil native library on databases where you shall be performing data masking.
Note: You should never run DataVeil on a production system. DataVeil requires exclusive use of the database/schema during a masking run because it shall be performing environmental and other broad changes that would adversely affect other applications and users. For example, database logging is temporarily altered (e.g. SQL Server is changed to Simple), triggers are disabled, indexes and constraints are dropped, tables altered, etc.
*** If you are upgrading, please uninstall your current version first. See 'Oracle Uninstallation' below ***
The installation files are found in the delivered software directory dataveil/native/oracle
The native library for Oracle is a Java JAR file called 'DVNative.jar'.
For broadest compatibility, the native library has been compiled to run on Java 1.6 or later (Java 8 recommended). Some older versions of Oracle may have older versions of Java. To check which version of Java JVM is installed on the Oracle DBMS, use the command:
SELECT dbms_java.get_ojvm_property(PROPSTRING=>'java.version') FROM dual;
Installation is simple and consists of only two steps:
Step 1) Upload the supplied DVNative.jar to Oracle using the Oracle 'loadjava' command.
E.g. PS C:\Users\Test> loadjava -user myusername/mypassword@database -r DVNative.jar
Step 2) Create the Oracle Function definitions.
E.g. SQL> @setup_functions.sql
Please refer to the instructions supplied in the delivered software directory dataveil/native/oracle/README.txt.
You can install the DataVeil native library on a different schema to the one that you shall be masking.
For example, if you are masking a schema called 'TESTD' but your DataVeil native library is installed on schema 'SCH01' then you could configure your 'TESTD' masking project to call the DataVeil native functions on 'SCH01' as shown below.
In this case you may need to grant Oracle access privilege, e.g. GRANT EXECUTE ANY PROCEDURE TO TESTD
If you leave the 'Function call prefix' field blank (the default) then the native library should be on the same schema as defined in the DataVeil Database->Connection.
Please also refer to The Native Functions Tab in the Project Settings section.
The uninstallation file is found in the delivered software directory dataveil/native/oracle
Step 1) Drop the Oracle Function definitions.
E.g. SQL> @drop_functions.sql
Step 2) Drop the DVNative.jar from the Oracle database using the Oracle 'dropjava' command.
*** Important: In the dropjava command you must reference the same DVNative.jar that was used by the original loadjava command to install it, not the new DVNative.jar. Therefore, you can either change the shell current directory to the location of the old DVNative.jar that is currently installed or explicitly qualify the DVNative.jar in the command below. This is because the dropjava command shall remove from Oracle DBMS the named objects in the referenced DVNative.jar which may be different to the named objects in the newer DVNative.jar.
E.g. PS C:\Users\Test> dropjava -user myusername/mypassword@database DVNative.jar
SQL Server Installation
The installation files are found in the delivered software directory dataveil/native/sqlserver
The native library for SQL Server is a CLR Assembly in a file called 'DataVeilNativeCLR.dll'.
This DataVeil CLR Assembly requires CLR version 4 and therefore it is compatible with SQL Server 2012 or later.
The installation commands are contained in two files: 'install_01_assembly.sql' and 'install_02_udf.sql'.
After the assembly is successfully created using the commands in the first file, use the second file to create the function definitions.
Please refer to the instructions supplied in dataveil/native/sqlserver/README.txt that describe how to adjust some variables in the setup files for your local environment.
By default, DataVeil shall expect the native library to be on the same database that is being masked and the 'Function call prefix' field shall show 'dbo'. However, you can install the DataVeil native library on a different database. For example, suppose you are masking database 'kona' but you wish to use the DataVeil native library installed on 'refdb.dbo'. You could then set the 'Function call prefix field' as shown below:
Please also refer to The Native Functions Tab in the Project Settings section.
SQL Server Uninstallation
The uninstallation file is found in the delivered software directory dataveil/native/sqlserver
Simply perform the SQL batch file uninstall.sql
The commands in this file shall first drop all of the function definitions and then drop the CLR assembly.
How To Determine if DataVeil Native Libraries Are Being Used During Masking
Query the DBMS for Version Info
You can query the DBMS for the DataVeilNativeVersion function to return the Native Library version number.
If the function isn't found (doesn't return a version number) then DataVeil won't be able to find the functions either. In this case review your setup.
DataVeil Masking Project Compile
Whenever a DataVeil Compile is performed DataVeil will log a message on whether a Native Library was found or not.
For example, if found:
INFO Found DataVeil native function library version 0.9.3 on database [kona]
Or, if not found:
INFO DataVeil native function library is not available on database [kona]
Data Masking Execution
Whenever a data masking run is performed, an implicit Compile is performed. This means that the log messages described above shall be similarly logged.
There is also an option to log a Warning or Error if a DataVeil Native Library is not found during a masking run. This is configured under the Settings->Database->Native Functions tab as shown below:
Please refer to The Native Functions Tab in the Project Settings section.
GUI Control Panel
Whenever a mask is executed, the GUI Control panel's Code column shall show 'Native' if a native function is being used to perform the mask or 'SQL' if not.
Please note: Not every mask has a native function implementation. This is typically because it is more suitable to be executed using SQL or is still in development. Masks that are not implemented in the Native Library include: Dataset, Number Sequence, Randomize DateTime, Random Number, Shuffle and User SQL Value.
Whenever a mask is executed, the log message for a mask completion shall include the keyword 'native' or 'SQL' following the mask name.
INFO ...completed Randomize native mask for column [kona].[sales].[Customer].[nationalID]; 4 of 290 rows processed in 0 seconds
INFO ...completed Shuffle SQL mask for column [kona].[sales].[Address].[addressID]; 10,000 of 19,614 rows processed in 0 seconds