Running Multiple Instances Concurrently

The default installation of DataVeil allows for only a single instance of DataVeil to be running at any given time in the same operating system user account. This is a combined total regardless of whether DataVeil is launched as a GUI or from the command-line.

The reason behind this is that DataVeil is built on the Netbeans Platform framework that uses a single default user cache directory (“userdir”) for the application. Therefore, whenever another DataVeil instance is started, Netbeans attempts to use the same default userdir but detects a lock from another instance of DataVeil and then refuses to run. This can produce an error message in a popup window and/or logged to the command-line console, as shown below. It can also cause the DataVeil instance that was already running normally to terminate unexpectedly.
 

 

For most users, a single instance limit is adequate and this is all that a single-user license allows.

For users who have a multi-user license and would like to run multiple instances of DataVeil from the same operating system user account then the remainder of this article shall explain how this can be done.

As mentioned, the issue is the Netbeans userdir to which a DataVeil instance requires exclusive access. The location of this userdir can be set at DataVeil startup using the Netbeans parameter --userdir. See https://netbeans.apache.org/wiki/main/wiki/DevFaqUserDir/

You can see the location of the userdir of your current DataVeil installation in the About dialog of the DataVeil GUI.
 

 

Running multiple command-line instances of DataVeil

By default, a command-line instance of DataVeil shall attempt to use the same Netbeans userdir as the DataVeil GUI which means that both cannot be running at the same time.

Let’s change this so that the command-line instance uses its own separate Netbeans userdir.

This is done by including the Netbeans --userdir parameter in the DataVeil START command.

As an example, let’s consider the sample command line files in the DataVeil software /batch directory. In the original Window’s batch file, the start command appears as:

START "MyWindowTitle" /WAIT "..\bin\dataveil64.exe" ^
--nosplash --nogui -J-Dnetbeans.logger.console=true -J-Dorg.level=WARNING -J-Xms256m -J-Xmx768m ^
--project="S:/Demo v1/SqlServer/kona_V4.dvp" ^
--key="123456" ^
--refreshschema=true ^
--compilewarning=continue ^
--createdirs=true ^
--log="mybatchlogs/mylog.txt!" ^
--logxml="mybatchlogs/mylog.xml!" ^
--logioprogress=true

 

Notice that there is no --userdir parameter which means that a default location has been chosen where all instances shall try to use.

Let’s change that by explicitly specifying a --userdir directory to use C:\dataveil\userdirs\user02

On Windows, the full directory path shall automatically be created if it doesn’t already exist. On *nix systems the directory path up to but not including the actual user dir (C:\dataveil\userdirs\) should already exist.

Note: It is best practice to specify a userdir folder that does not already exist or is empty. This is because if a different version of DataVeil had previously used that folder then the cache contents could be inconsistent and lead to unexpected behavior. This is shown in the example below with the RMDIR command.

You may notice in the start command that forward-slashes have been used. Windows recognizes these and it avoids issues where in some cases double back-slashes would be needed if enclosed within quotes, which are needed if a space is embedded in the path. Therefore, it is usually easier to use forward-slashes instead and it also makes it easier if a copy of the command is ported to a non-Windows system such as Linux.

Also notice below that the DataVeil --license parameter has been added. This is because each separate userdir will persist its own set of environment variables and settings such as the license location. The default license location is stored in the default userdir typically after the user runs the GUI for the first time and specifies the license location. Therefore, a new userdir location would have an undefined license location and so it must be specified as a command-line parameter. This is true of any other parameters such as the location of component files, if a masking project references these, using --component_home. The location of run reports can be specified with --reportdir otherwise they will appear in your OS user default directory. Please see DataVeil command line options.
 

RMDIR /s /q "C:/dataveil/userdirs/user02"
 

START "MyWindowTitle" /WAIT "..\bin\dataveil64.exe" ^
--nosplash --nogui -J-Dnetbeans.logger.console=true -J-Dorg.level=WARNING -J-Xms256m -J-Xmx768m ^
--userdir "C:/dataveil/userdirs/user02" ^
--license="S:/dataveil/licenses/license.dvl" ^
--project="S:/Demo v1/SqlServer/kona_V4.dvp" ^
--key="123456" ^
--refreshschema=true ^
--compilewarning=continue ^
--createdirs=true ^
--log="mybatchlogs/mylog.txt!" ^
--logxml="mybatchlogs/mylog.xml!" ^
--logioprogress=true

 

Note there is a space (no ‘=’) after --userdir whereas there is an ‘=’ after --license.

The line continuation character ^ requires at least one space before it and absolutely no characters after it, not even whitespace, otherwise Windows will throw an error, resulting in a cryptic error message.

For example, by placing a single space character after the ^ in ‘--userdir "C:/dataveil/userdirs/user02" ^ ‘ above, the following error message would be produced:

--license' is not recognized as an internal or external command, operable program or batch file. Unexpected exit code = 9009, probably a problem with the command or the environment.

 

For *nix systems, the procedure is the same except that the --userdir parameter is set in the relevant *nix start command. For example:

../bin/dataveil --nosplash --nogui -J-Dnetbeans.logger.console=true -J-Dorg.level=WARNING -J-Xms256m -J-Xmx768m --userdir "/export/home/dataveil/userdirs/user02" --license="/export/home/dataveil/licenses/license.dvl" --project="/export/home/test/Documents/dataveil/Demo v1/Oracle/kona_sol1.dvp" --key="123456" --refreshschema=true --compilewarning=continue --createdirs=true --log="mybatchlogs/mylog.txt!" --logxml="mybatchlogs/mylog.xml!" --logioprogress=true

 

Using static batch files

A simple approach would be to create a specific batch file for every masking job. Each batch file would explicitly reference its own private userdir.

For example, if you wanted to run 3 DataVeil masking projects concurrently from the command line, you could create 3 copies of the batch file where each file would specify a distinct userdir, such as:

start_dataveil_01.bat could use --userdir "C:/dataveil/userdirs/user01
start_dataveil_02.bat could use --userdir "C:/dataveil/userdirs/user02
start_dataveil_03.bat could use --userdir "C:/dataveil/userdirs/user03

For clarity, you could name the batch files and userdirs using the same name as the masking project file or target database being masked, such as:

start_dataveil_sales.bat could use --userdir "C:/dataveil/userdirs/sales
start_dataveil_warehouse.bat could use --userdir "C:/dataveil/userdirs/warehouse
start_dataveil_admin.bat could use --userdir "C:/dataveil/userdirs/admin

 

Using dynamic commands

An alternative to using specific batch files where each runs using a fixed name userdir is to use an environment variable.

Therefore, in Windows, you could create an environment variable called DATAVEIL_USERDIR.

The START command could then be modified to use the environment variable. Immediately prior to running the batch file the environment variable can be set to an unused location, for example:

set DATAVEIL_USERDIR="C:/dataveil/userdirs/userdir0148"

 

Now the --userdir parameter in the RMDIR and START commands would reference the environment variable:

 

RMDIR /s /q %DATAVEIL_USERDIR%
 

START "MyWindowTitle" /WAIT "..\bin\dataveil64.exe" ^
--nosplash --nogui -J-Dnetbeans.logger.console=true -J-Dorg.level=WARNING -J-Xms256m -J-Xmx768m ^
--userdir %DATAVEIL_USERDIR% ^
--license="C:/dataveil/licenses/license.dvl" ^
--project="S:/Demo v1/SqlServer/kona_V4.dvp" ^
--key="123456" ^
--refreshschema=true ^
--compilewarning=continue ^
--createdirs=true ^
--log="mybatchlogs/mylog.txt!" ^
--logxml="mybatchlogs/mylog.xml!" ^
--logioprogress=true

 

Therefore, you could have a system that assigns a new folder name to the DATAVEIL_USERDIR variable for each new DataVeil process, such as simply incrementing the allocated userdir name (‘C:/dataveil/userdirs/userdir0148’, ‘C:/dataveil/userdirs/userdir0149’, ‘C:/dataveil/userdirs/userdir0150’, and so on). Prior to starting the DataVeil process, a delete of the DATAVEIL_USERDIR folder should be performed in order to ensure that it doesn't already exist from a previous DataVeil run.

 

Resources

Memory: Each DataVeil instance shall allocate its own memory, as shown by the ‘-J-Xms256m -J-Xmx768m’ parameters in the START command, so please ensure that your system has adequate memory resources.

Compute: A DataVeil instance consumes relatively little compute resources during masking execution. This is because DataVeil uses the DBMS servers to perform all masking computations whereas a DataVeil process acts as a controller and remains mostly idle.

 

Summary

To run multiple concurrent DataVeil processes in the same operating system user account, you just need to allocate a separate Netbeans userdir for each instance.

You can do this by specifying the --userdir parameter in the command line to start DataVeil with an explicit path or by using an environment variable for more flexibility.

Sufficient memory must be available for all concurrent DataVeil processes. DataVeil moves the processing onto the DBMS during masking and therefore the actual DataVeil processes will consume minimal compute resources.