CleanEnter Help: Microsoft Dynamics
CleanCRM is a batch data deduping tool for Microsoft Dynamics.CleanCRM is a batch data de-duping tool embedded as a tab in Microsoft Dynamics. CleanCRM can be launched from your Microsoft Dynamics ActivePrime Admin tab.
To use CleanCRM, first select the type of records you wish to de-dupe (Accounts, Contacts, or Leads) from the menu in the upper left corner. It is best to de-dupe Accounts first, followed by Contacts.
After selecting which type of records to de-dupe, you must choose your Settings. CleanCRM will find duplicates based on these Settings.
To define your settings, click Settings in the upper left corner. In this menu, you can select the:
- The records you wish to de-dupe .
- The merge rule for the master record in a duplicate set.
- The match settings to identify duplicates.
- The fields to display on screen.
- The fields to concatenate.
The menu can be collapsed by clicking Settings again or by clicking Close in the corner of the menu.
Select the Records to Dedupe
Select the records to dedupe. You can choose to dedupe a list of CRM record IDs in an externtal CSV, or load an XLSX of duplicate sets to work with.
To dedupe a list of records by IDs in an external CSV file, select the option Use external CSV list of IDs to dedupe. Click the Browse for CSV... button, select the file containing the record IDs to dedupe against, and click Open to upload the list to CleanCRM. After the file is uploaded, use the dropdown menu to select column containing the record ID values.
To dedupe a report of records in Service Cloud, select the option Load an XLSX of duplicate sets to work with. Click the Browse for XLSX... button, select the file containing the duplicate sets to dedupe against, and click Open to upload the sets to CleanCRM.
For more information, see this product page.
Choose a merge rule for the master record in a duplicate set, which defines how master records in duplicate sets are chosen.
There are five standard merge rules available, plus any custom merge rules your company may have ordered. The standard choices are:
|Most Complete Address:||The master record in a duplicate set will be the record with the most complete address.|
|First Created:||The master record in a duplicate set will be the earliest record created based on the Create Date.|
|Last Created:||The master record in a duplicate set will be the most recent record created based on the Create Date.|
|First Modified:||The master record in a duplicate set will be the earliest record modified based on the Modified Date.|
|Last Modified:||The master record in a duplicate set will be the most recent modified based on the Modified Date.|
To select a merge rule, simply choose it from the dropdown menu.
The Master Record is utilized to determine how the final row of field values will be created. For example, if you select the merge rule Last Modified, the last modified record is designated as the master record. It will be used to populate the final field values; any missing values will be populated by the other records in the duplicate set.
After CleanCRM identifies duplicate sets, you may wish to re-run a different merge rule. To do this, simply choose a different merge rule from the menu and click Run Merge Rule.
Choose match settings to identify duplicates. Each data field can be custom defined to match your records using Exact, Starts With, or Fuzzy matching on any combination of data fields.
First, select which fields to use for matching. You can choose from Basic fields which are standard fields in Service Cloud, or expand the list of All fields for all remaining fields including custom fields.
Each field has a sliding scale for you to select the type of matching that you wish CleanCRM to perform with it. The four matching options are:
|Don't Match||This field will be ignored for matching.|
|Exact Match||Data in these fields will be matched only if values are perfectly identical (case sensitive).|
|Starts With||Data in these fields will be matched if values begin with the same first few characters. You can define the number of characters to match via the Advanced... button.|
|Fuzzy Match||Similar data in these fields will be matched to each other. CleanCRM applies matching algorithms and domain knowledge to identify misspellings, abbreviations, and phonetic matches. This is the most used option.|
More fields selected for matching will result in a narrower search for duplicates. Fewer fields selected will result in a broader search. A few trial runs will quickly reveal the best combination of fields for matching your data. It is usually recommended to use the powerful Fuzzy matching capability that ActivePrime's tools are known for, which identifies several types of similar matches. Exact matching is typically used on fields that are distinct character-wise, such as email, website, or customer ID. Starts With matching may be valuable for specific data de-duping needs.
**Please note that when matching on phone number fields, your results will be most accurate using Exact or Starts With matching. Fuzzy matching on phone number fields may yield inadequate results.
If you are new to CleanCRM, try these basic match settings for Contacts and Leads:
Click Advanced... next to any field to choose settings for blank field matching, Starts With matching, and words to ignore:
- Blank field matching defines how CleanCRM will match empty fields. Blank field matching is set to No by default, meaning that empty fields from two records will not match with each other ("___ Smith" does not match with "___ Smith" with blank field matching for First Name off). You can turn on blank field matching by selecting Yes, so that an empty field will match with another empty field ("___ Smith" will match with "___ Smith" with blank field matching for First Name on). Note that neither option allows you to match an empty field to a non-empty field ("___ Smith" does not match with "John Smith" if you match by First Name). This functionality is important in preventing large duplicate sets resulting from fields with missing data. For example, this happens when matching only on First Name and several records are missing an First Name (in this case, it would be preferable to select No for blank field matching).
- Characters for Starts With matching defines the number of characters that CleanCRM will use to perform Starts With matching. The default value is 5 characters. Change this number by clicking More or Less. The minimum number of characters required is 3. Note that this option only appears when you have selected Starts With matching for the field.
- Words to ignore defines which words CleanCRM will ignore when matching your fields. Enter one word per line. These entries are case-insensitive. Note that these words will not be ignored if you use Exact matching.
After you define your match settings and click Find Duplicates, CleanCRM will introspect your data. If any of your displayed fields are over 90% populated and are not used for matching, CleanCRM may recommend that you match with them. A pop-up window will ask you whether you'd like to keep your current match settings or use CleanCRM's match recommendations.
Choose the fields to display onscreen. You can add, remove, and rearrange any fields that exist in Service Cloud including custom fields. All available Service Cloud fields are listed in the Available fields column. Fields that will be displayed in your CleanCRM results are listed in the Show these fields in this order column.
To change the fields that will be shown, click on a field name and use the Add and Remove buttons. To change the order of displayed fields, select a field from the Show these fields in this order column and use the Move up and Move down buttons. To discard any changes you have made, click Cancel.
After CleanCRM identifies duplicate sets, you may wish to change the fields displayed onscreen. To do this, simply make your changes in the Displayed Fields menu and click Apply.
Remember to display all fields that will help you to review your duplicates. For instance, you may want to include custom fields, record owners, or lead sources. Please note that CleanCRM will not overwrite any field that is not shown on screen.
There may be some fields for which you want to retain the values from each record in your duplicate set, for example, descriptions or sales notes. CleanCRM lets you keep all this data by concatenating the values from each record into the final record field.
To concatenate the data for a field, check off the desired field and click in the dropdown to select the separator of your choice.
After CleanCRM identifies duplicate sets, you may wish to change field concatenation settings. To do this, simply make your changes in the Concatenate Fields menu and click Apply.
Saving Your Settings
When you have finished choosing your settings, save them by clicking the Save As... button at the top of the Settings menu. Enter a name for your customized settings and click Save.
Your saved settings are listed in the Current Settings menu in the upper left side of the CleanCRM window. To load your saved settings at a later time, click on the menu next to Current Settings and select the saved settings.
Customizing Merge Rules
Before you start creating new merge rules, please make sure to add to CleanCRM all the display fields that will be used in Merge Rules. (Setting -> Displayed Fields).
After you have done that you can launch Merge Rules manager
Merge Rules Manager
You can launch Merge Rules Manager from Settings and also from Options:
You can activate or deactivate any Merge Rules. If a rule is not active it will not be visible for CleanCRM users.
Creating and updating Merge Rules
When you create a new rule you will first need to select an action “Select Master Record” - Select Master Record will set the master record based on meeting your selected criteria.
The record that meets all Criteria Conditions will be set as the Master Record.
“Block Duplicate Record” - Block Duplicate Record will block a duplicate record to not be merged, based on meeting your selected criteria. The record that meets all Criteria Conditions will be removed from the Duplicate Set and will not be merged. It is the equivalent of using “Remove” link on the left of the record in the gridview.
“Block Duplicate Set” - Block Duplicate Set will block the entire duplicate set from being processed and the records will not be merged. If any record meets all the Criteria Conditions, the whole duplicate set will be removed and will not be merged. This is the equivalent of using the “Remove” link in the gridview on all records in Duplicate Set (note that you cannot remove Master, so if it’s done manually, you will have 1 record in that Duplicate Set).
“Set Field Value” - Set Field Value will set the desired field value based on meeting your selected criteria. If all the Criteria Conditions are met, the specified value will be set to “Final” Master Record. Note: It will be set to Final field value directly and will not affect other conflicting values checkboxes. Sample: We need to set the value of “Required evaluation” to “Yes” if we don’t have data for “Phone”, “Fax”, and “Website”. The Merge Rule can be set up with “Set Field Value”, Action: Set Value to “Required evaluation” = “Yes”, Conditions: “Phone”, “Fax”, and “Website” = “Column Is Empty”. In this case, CleanCRM will put “Yes” to Final value of field “Required evaluation”, but only if whole columns for all 3 fields “Phone”, “Fax”, “Website” is empty for this Duplicate Set.
“Add the desired Set Value actions:”
Note: this will be applied to each Duplicate Set individually.
“Set Value” - Set a value to a specific field. You have to fill in the value to set. “Set Value To Largest” - If a field value is a number, it will be set to the larges number. If the value is text, the largest value will be defined by ASCII ordering (Z is larger than A). All values that contain a mix of numbers and letter will be treated as text.
“Set Value To Longest” - The longest value will be used. Sample: “555-555-5555” vs. “555-555-5555 ext 555”. “555-555-5555 ext 555” will be used.
“Set Value To Most Frequent” - The most frequent value will be selected.
“Set Value To Shortest” - This is the opposite to “Set Value To Longest”. The shortest value will be used. Sample: “555-555-5555” vs. “555-555-5555 ext 555”, “555-555-5555” will be used.
“Set Value To Smallest” - This is the opposite of “Set Value To Largest”. If a field value is a number, it will be set to the smallest number. If the value is text, the smallest values will be defined by ASCII ordering (A is smaller than Z). All values that contain a mix of numbers and letter will be treated as text.
The last part is selecting the criteria for this rule
First, you need to define if this will be “And” or “Or” logic.
Select “All” if you need “And” logic, select “Any” for “Or” logic.
You can add multiple conditions and Sub-Conditions. For each Sub-Condition you have to define the logic as well (All/Any).
For each Condition, you will need to define the field and the criteria.
The first set of fields (before the break) is a pre-defined set of multiple fields.
“Address” - Any Address that is presented in CleanCRM displayed fields. This field can be set to “Is Most Complete” or “Is Least Complete”
“Billing Address” - Only the Billing Address fields that are presented in CleanCRM displayed fields will be processed. This field can be set to “Is Most Complete” or “Is Least Complete”.
“Shipping Address” - Only the Shipping Address fields that are presented in CleanCRM displayed fields will be processed. This field can be set to “Is Most Complete” or “Is Least Complete”.
“Create Date” - The record create Date. This field can be set to “Is Newest” or “Is Oldest”.
“Duplicate Type” - This will evaluate whole Duplicate Sets (groups of records in CleanCRM with common number, 1.1, 1.2, 1.3, 1.4) and the defined duplicate type. This can be set to “Is Conflicting Duplicate” - duplicates that conflicts each other, marked with red background color in CleanCRM, “Is Exact Duplicate” or “Is Non Duplicate”.
“Multiple Fields” - This allows you to evaluate multiple fields at once. It can be very useful if your Master Record preference is not based on Date or Address. You will select the criteria for the fields, and then select multiple fields in any order (Use Ctrl+Mouse Click to select/deselect multiple items).
“Are All Empty” - For fields with no data.
“All Contain Data” - For fields with data.
“Least Filled In” - For fields with less data.
“Most Filled In” - For fields with the most populated data. Most useful for helping select the Master Record.
“All other fields” - You can select a field and its criteria. Criteria can be based on the Duplicate set (“Column…”, “Largest Number”, ”Smallest Number”), or an individual value (“Value..”).
Once your settings have been chosen, click Find Duplicates. CleanCRM will identify duplicates amongst the records in your report and display the results in the main body of the CleanCRM window.
CleanCRM's progress will be displayed at the top of the screen. Segmenting your data and finding matches may take a few moments, especially with large reports of records.
After CleanCRM has identified the records and begun the matching process, CleanCRM will introspect your data. If any of your displayed fields are over 90% populated and are not used for matching, CleanCRM may recommend that you match with them. A pop-up window will ask you whether you'd like to keep your current match settings or use CleanCRM's match recommendations.
Understanding Your Results
After CleanCRM has found duplicates, the results will be displayed in the main body of the CleanCRM window. Your matching results are organized into three categories:
- Duplicates Requiring Review
- Duplicates Not Requiring Review
- Unique Records
You can manipulate the data to fine-tune the duplicate sets before merging duplicate records.
Your matching results are organized into three categories:
|Duplicates Requiring Review:||Duplicate records with conflicting data in the displayed fields. These records should be reviewed before merging.|
|Duplicates Not Requiring Review:||Duplicate records without conflicting data in the displayed fields. These duplicates are exact matches and require little to no review before merging.|
|Unique Records:||Unmatched or non-duplicate records as defined by your settings.|
Click the arrow to expand the grids and review your duplicate sets.
Reviewing Your Data
Click the arrow to expand the grids and review your duplicate sets. Duplicate sets are grouped and numbered in the grids. The Final record represents the final field level values for the surviving merged record for each set.
You can easily review your duplicate sets before merging:
- Remove a record from a duplicate set by using the Remove link.
- Change the master record of the duplicate set by reselecting the Master check box. The master record is the surviving record that will be retained and updated after merging the duplicate set.
- Choose final record data by selecting check boxes in highlighted fields representing conflicting data in a duplicate set. You can also manually edit the final record by clicking in the final record data fields and typing your changes.
These duplicates grids are densely packed with information. Underlined text values are hyperlinks to the existing record in salesforce.com. Additionally, final record fields that appear in green can be edited by clicking into the field and typing your changes.
Duplicate sets highlighted in yellow indicate a display warning or error. You can view the warning by clicking view warnings in the Master field of the final record, or by clicking View Errors in the category grid title bar.
Your duplicate sets can be sorted by the data in any of your displayed fields. Simply click on the column header to enable ascending or descending sorting. Your duplicate sets will be sorted amongst each other by the final record data in the selected column.
If you are not satisfied with the match results, you can fine-tune your settings and re-run CleanCRM until it finds duplicate just the way you want.
Exporting Your Data
Each category of grids can be exported to a spreadsheet. Exporting may be helpful if you require external review of duplicate sets. Click the Export button in the category grid title bar to choose a file name and type. You may filter by owners if desired. Click Save to generate the file.
You can see the statistics of your current CleanCRM run by clicking Statistics in the upper-left corner of the window. You will be shown the number of total duplicates, duplicate sets by category, and unique records (non-duplicates). These values help you do a "health check" of your data to see how much duplication currently exists in your Service Cloud system.
Merging Your Duplicates
When you are satisfied with duplicate identification and have reviewed your data, you can merge duplicate sets in three ways:
- One-by-one by hovering over the Final record arrow and selecting Merge Set.
- Page-by-page by clicking Merge Current Page in the category grid title bar.
- All Records by clicking Merge All in the category grid title bar.
During the merge process, all associated child entities from the duplicate records in the set will be moved to the surviving master record so that no information is lost. The other duplicate records will then be deleted. The behavior is the same as when two records are manually merged in Microsoft Dynamics.
**It is important to note that each Merge process is final and cannot be undone.
Successfully merged duplicate sets will be highlighted in green . Unsuccessful merges will be highlighted in red with errors. You can view error details by clicking View Errors in the Master field of the final record or in the category grid title bar. If you can correct the cause of error, you can re-attempt to merge the duplicate set.
**Please note that CleanCRM will not overwrite the master record for any field that is not displayed onscreen. Therefore, the master record will retain its original value for any field that is not displayed. You may change the displayed fields while reviewing your data by reselecting your Settings.
When reviewing large numbers of records, it is advised that you merge duplicate sets one-by-one or page-by-page to keep your data manageable.
The Microsoft Dynamics account administrator is responsible for maintaining your CleanEnter account options. Administrators can enable CleanEnter setup, manage users, and configure settings.
Update CleanCRM Account
This page is used to update your Microsoft Dynamics administrator information. This is the account that CleanCRM accesses to perform searches. If the admin username or password for the account changes, please update it here so that CleanCRM can continue accessing your Microsoft Dynamics data.
This page allows you to manage the Microsoft Dynamics users who are able to use CleanCRM. To enable a user, check the AP Enabled box next to that user's name. To disable a user, uncheck this box. You can add additional Admin users by checking the AP Admin box. You can only enable as many CleanCRM users as the number of CleanCRM licenses you have in your subscription. If you need more licenses, click Inquire About Adding More Licenses.
Any new users of your Microsoft Dynamics account will appear on this list and can be enabled at any time, provided that you have enough user licenses.
For long lists of users, you may need to search for users. You can use the column filters to search for users. Enter your filter criteria and click Apply Filter to refine the list of users on display. You can also press Ctrl+F to find users.
- Above CRM Status, you can select for active, inactive, or all users.
- After updating users, you must click Save to save your changes. Clicking Apply Filter to search for users will discard any unsaved changes on screen.
- Click Export CSV to export the CleanCRM user list on screen.
This page allows you to save your CleanCRM settings for future use. Click Save Current Settings to save your custom settings that are currently being used in CleanCRM.
All prior saved settings will also appear on this page. You can load any saved setting by clicking Load. You can change the default setting which CleanCRM will automatically load when it starts. You can also delete and rename settings here.
This page allows you to manage the email addresses that will receive notifications about CleanCRM. Enter one email address per line in the text box, and click Save Emails to save your changes.
Update CRM Fields
Click Update CRM Fields to update the fields and labels from your CRM into CleanCRM. If a user changes any labels, adds new custom labels, or removes labels, clicking this button will refresh their appearance in CleanCRM instantly.
Update CRM Fields
Is there anything new you would like to see in CleanCRM? Anything that ActivePrime could do differently?
Click Feature Suggestion to send a feature suggestion to ActivePrime. This link will lead you to our support portal, where you can submit your suggestion to the ActivePrime team.
Frequently Asked Questions
For further assistance, visit our Support Portal at www.activeprime.com/support/.
Interested in more data quality solutions?
ActivePrime has developed an array of complementary tools including ActivePrime Search, CleanEnter, CleanCRM, CleanCSV, and CleanImport that are embedded in Microsoft Dynamics. ActivePrime Search is a real-time, fuzzy, CRM search solution. CleanEnter is a real-time duplicate detection service. CleanCRM is a batch data de-duping tool. CleanCSV dedupes CSV files of records. CleanImport dedupes CSV files against your CRM records and imports them. To learn more about these products, visit https://activeprime.com/platforms/activeprime-for-microsoft/. Sign up for a free trial through CleanCRM Free Trial.
Last Revision: 23/12/2018