The JSON Mask

 

The JSON mask is designed to mask property values within a JSON record.

A JSON mask can contain one or more path expressions. All of the path expressions and related masks within a JSON mask shall be applied to each JSON record processed.

If a NULL value is found instead of a JSON record then NULL is preserved.

Note: The JSON mask requires the use of the DataVeil Native Library.
  

 

To add a JSON path and its mask, click on the 'Add...' button. The following dialog shall be displayed:
  

 

Path
    

The Path parameter shall be interpreted as a JSONPath expression for which all matching properties shall be masked using the Mask parameter.

There are many resources online that explain JSONPath and its syntax. A concise reference with examples can be found at https://goessner.net/articles/JsonPath/

If a JSON property key name contains an embedded space character then you must use bracket notation. E.g.  $.person.['given name']  

 

Required minimum number of matching properties in each record
  

This is the minimum number of properties that should match the path expression in every JSON record. If a JSON record does not satisfy this condition then a DataVeil runtime error shall be reported. The default is 1.

If it is acceptable for a property to sometimes be absent from a record then this parameter can be set to 0. If you would like to enforce the presence of a property in some records but not for others then then create separate JSON masks and specify a Where condition so that the JSON mask with 0 Required matches applies only to those records where it is acceptable for the property to be absent.

 

Mask
  

To configure a path mask for the JSON path, click on the 'Create...' button. The following mask choices are presented:
  


  

Select the required mask and click OK.

The selected path mask's configuration dialog shall appear. Please refer to the relevant mask's documentation in this section for details on its parameters.

Note: A path mask may contain a Determinism tab to allow an explicit determinism setting for that mask; however path masks will not contain a Where or Join tab because these do not apply to JSON paths. Instead, the Where and Join tabs appear in the parent JSON mask as they apply to the SQL row-level selection of the entire JSON record.

 

Example:

The mask below shows a JSON mask that shall mask every property value whose key is 'firstname' using a Person Given Name mask, every property value whose key is 'surname' using a Person Family Name mask and every property value whose key is 'phone' using a Redact mask.

Note that the paths in this example use the JSONPath '$..' specification. The double dots means 'match recursively'. i.e. match every occurrence in the record below the root ('$' represents to record root).

 

 

Masking Multiple Occurrences
  

A single JSONPath expression can match multiple properties.

Each matched property value is masked independently of the other matched values returned by the same JSONPath expression. This means that the original format of each masked value is also preserved (for masks that supports format preservation, such as Redact).

For example, using the JSON mask shown above, the original record...

{"firstname":"Jo", "surname":"Smith", "phone":"+61 (3) 876-3421", "agent":{ "firstname":"Will", "surname":"Baker", "phone":"(415) 739 1234"} }

...could be masked to something like...

{"firstname":"Mary", "surname":"Kuhn", "phone":"+61 (3) XXX-XXXX", "agent":{ "firstname":"Tony", "surname":"Dalton", "phone":"(415) XXX XXXX"} }

 

Exception 
 

There are a few circumstances where some specific mask configurations shall return the same masked value within a JSON record for all properties returned by a single path expression. Specifically, these are:
   - National Identifier masks (non-deterministic only)
   - Number Sequence mask
   - Primary Account Number mask (non-deterministic only)

All other masks shall mask multiple property occurrences independently.
   

Unsupported Masks
  

The following masks are not supported as child masks of a JSON mask:

* Bank Account Number (Belgium)
 * DataSet
 * DateTime (use Randomize DateTime instead)
 * National Identifiers: Belgium and UK
 * Shuffle
 * User SQL Value

 

Invalid JSON Records
  

If a NULL value is found instead of a JSON record then NULL is always preserved, regardless of the 'Preserve Nulls' setting in the column's Masking tab.

If a non-NULL invalid JSON record is encountered by a JSON mask then DataVeil shall report a runtime error.

An invalid JSON record is one that does not satisfy the DBMS's definition of a properly formatted JSON record. This can be determined by the SQL Server ISJSON() function or Oracle IS JSON condition.

Therefore, if a column may contain invalid JSON records then you should include in the JSON mask's Where tab a condition to select only rows that contain valid JSON records.

For SQL Server, the JSON mask's Where tab should use the SQL Server ISJSON() > 0 predicate to test the column name. i.e. The example shown below shows that "jrec" is the name of the column being masked (replace with your column name):

 

For Oracle, the equivalent DataVeil Where condition would be the Oracle IS JSON condition:
 
 

 

Finally, you should add another mask after the JSON mask to tell DataVeil how to handle all the rows that contain invalid JSON records (i.e. all the rows that weren't selected by the 'is valid json' Where condition above). If you don't add such a mask then DataVeil will log a warning similar to:
 "WARNING Column DVTEST.JTABLE_1.JREC does not have an unconditional mask (a mask without a Where condition). This means that it may be possible that some rows will not be masked. If this does occur during the masking Run then DataVeil shall notify you with another warning upon Run completion. See Help->Masks Reference->Common->Mask Execution Order for details.".
 Details on this can be found in the Mask Execution Order topic.

Therefore, to avoid an 'unmasked rows' warning, you need to explicitly define what to do with the unmasked invalid JSON records. The example below shows that a Preserve mask was added as the last mask that explicitly defines that all remaining unmasked rows (invalid JSON records) are to be preserved. Of course, you can choose any suitable mask instead (e.g. Redact, or a User SQL Value mask to replace with NULL or some other string, etc).