Employing field Normalization to resolve data quality issues in ServiceNow

The purpose of this blog is to illustrate what is ‘Field Normalization’ and how it plays an important role in altering the field values to avoid duplication. This blog will help to understand how to normalize the field values.

Normalization:

The Record value can come from different sources like discovery, importing records, or manual entry. Each of these resources may hold the same record with different explanatory value.

Normalization always search for similar field values with slight differences between them and converts altogether into single standard value. By this, one can avoid duplicate records and provide better search results. One can normalize all the field types available in ServiceNow however field value with more explanatory values are preferred. For ex: company name/manufacturer.

1. Hewlett-Packard Incorporated, Hewlett-Packard and Hewlett-Packard, Inc.to HP.In the above example, Hewlett Packard is more descriptive, hence, it is normalized to HP which is more preferable and most recognizable value. This can provide better search result.

Plugin:

Activate the Field Normalization (com.snc.field_normalization) Plugin. It’ll activate related plugins also. After activating this plugin, various components like Business Rules, Script Include, Field Normalization roles, Field Normalization UI pages are installed.

There are two normalization records in above screenshot. One is for company name and the other for CPU type. While activating plugin, if load demo data is selected, then these two records with demo data for reference purpose are created. But these two records will be in test mode. This needs to be activated explicitly in order to apply normalization. Throughout Normalization, the system uses data jobs to change the field values. The changes applied by data jobs can be rolled-back. The data jobs may include ‘Pending Value Collection’, ‘Normal Value Change’, ‘Alias Application’, ‘Rule Application’, and ‘Coalesce to Normal’.

Raw Field:

Raw Field is a custom field which can hold the original value of record before normalization. One has to create their own custom field by name ‘Raw Value’. This field is selected as raw field while creating Normalization.

Normalization record:

Navigate to Field Normalization > Configurations > Normalizations>Create new 1.

1. Name:- Name of normalization record
2. Table:- Table of field being normalized
3. Field:- Field to normalize
4. Mode:- Mode of normalization. It can be active, off, test. By default it is set to test. Set it to active mode to apply normalization.
5. Normalize query:- Apply to all queries using this field.
6. Raw field:- Field to hold original value.

Normal Value:

Normal value is a standard value which replaces all the similar field values with this value. This helps to avoid ambiguity and eliminate the record duplication.

Click on the Normal values appearing in the related list of normalization record to create new normal value.

In the screenshot below, the Company names like Hewlett-Packard, Inc., Hewlett-Packard, Hewlett-Packard Incorporated are normalized to HP. Once normal value is created, all records appearing in the alias will be replaced by normal value.

Alias:

Aliases are different names to same record value in the table which will be replaced by normal value. We can define our own aliases. Click on the aliases related link in the normal value,

After testing, all aliases are replaced with normal values. A data job is always produced for alias created for a normal value. Start the data job to apply aliases. Mode of Normalization is set to active to run Data jobs.

Rules:

A filter condition can be applied if there are many records with the similar value.

In the above example, filter is applied on name. It’ll list out all the records with this matching condition and place under aliases.

Normalized Filed:-

Normalization icon  will get displayed on fields with an associated normalization. Users can click on normalization icon to access normalization.

In the above figure, normalization icon beside name field is present and below that raw value field holding the original value prior to normalization is available.