CleanCRM Help: salesforce.com
CleanCRM is a batch data de-duping tool embedded as a tab in salesforce.com. CleanCRM can be launched from your salesforce.com ActivePrime Admin tab.
To use CleanCRM, first select the type of records you wish to de-dupe (Accounts, Contacts, or Leads) from the menu in the upper left corner. It is best to de-dupe Accounts first, followed by Contacts. Leads can be cleansed at any time since they exist independently in salesforce.com.
After selecting which type of records to de-dupe, you must choose your Settings. CleanCRM will find duplicates based on these Settings.
To define your settings, click Settings in the upper left corner. In this menu, you can select the records to de-dupe, the merge rule for the master record, your custom match settings, and the displayed fields. The menu can be collapsed by clicking Settings again or by clicking Close in the corner of the menu. Please watch this overview video on how to select your settings.
Select the Records to Dedupe
Select the records to dedupe. You can choose to dedupe a list of CRM record IDs in an externtal CSV, or compare against a report in the CRM.
To dedupe a list of records by IDs in an external CSV file, select the option Use external CSV list of IDs to dedupe. Click the Browse for CSV... button, select the file containing the record IDs to dedupe against, and click Open to upload the list to CleanCRM. After the file is uploaded, use the dropdown menu to select column containing the record ID values.
To dedupe a report of records in salesforce.com, select the option Compare against the report in my CRM. Click the + button and enter a Report Name and Report Description. Next to Report URL, paste a link to salesforce.com report you have created and saved. Click Save to use this list in CleanCRM.
For more information, see this product page.
Reports that you upload will be saved for future use in CleanCRM. You may edit or delete a report at any time. If you have multiple reports uploaded in CleanCRM, simply select the one you would like to use for the current session by clicking on it.
You can create a report via the salesforce.com Reports tab. The report can include any number of filters to target a desired segment of records. If you make any changes to an existing report in salesforce.com, be sure to save the report again before using it in CleanCRM.
**Important note: You must include the ID field (AccountID, ContactID, or LeadID) in the first column of your report. Once you have added the appropriate ID field and filters, save the report in order to select it in CleanCRM. No other fields are required, but you may add any fields you wish to better visualize your data.
**Important note: CleanCRM reads reports as if logged in to the CleanCRM admin user of salesforce.com. If another user, besides the CleanCRM admin user, creates the report, the Report Options need to be set to Show All Accounts (not just My Accounts) in order for CleanCRM to read the records in the report.
If you are new to CleanCRM, it is best to start with small data sets and work up to larger data sets as you master the de-duping functionalities.
Choose a merge rule for the master record in a duplicate set, which defines how master records in duplicate sets are chosen.
There are five standard merge rules available, plus any custom merge rules your company may have ordered. The standard choices are:
|Most Complete Address:||The master record in a duplicate set will be the record with the most complete address.|
|First Created:||The master record in a duplicate set will be the earliest record created based on the Create Date.|
|Last Created:||The master record in a duplicate set will be the most recent record created based on the Create Date.|
|First Modified:||The master record in a duplicate set will be the earliest record modified based on the Modified Date.|
|Last Modified:||The master record in a duplicate set will be the most recent modified based on the Modified Date.|
To select a merge rule, simply choose it from the dropdown menu.
The Master Record is utilized to determine how the final row of field values will be created. For example, if you select the merge rule Last Modified, the last modified record is designated as the master record. It will be used to populate the final field values; any missing values will be populated by the other records in the duplicate set.
After CleanCRM identifies duplicate sets, you may wish to re-run a different merge rule. To do this, simply choose a different merge rule from the menu and click Run Merge Rule.
Choose match settings to identify duplicates. Each data field can be custom defined to match your records using Exact, Starts With, or Fuzzy matching on any combination of data fields.
First, select which fields to use for matching. You can choose from Basic fields which are standard fields in salesforce.com, or expand the list of All fields for all remaining fields including custom fields.
Each field has a sliding scale for you to select the type of matching that you wish CleanCRM to perform with it. The four matching options are:
|Don't Match||This field will be ignored for matching.|
|Exact Match||Data in these fields will be matched only if values are perfectly identical (case sensitive).|
|Starts With||Data in these fields will be matched if values begin with the same first few characters. You can define the number of characters to match via the Advanced... button.|
|Fuzzy Match||Similar data in these fields will be matched to each other. CleanCRM applies matching algorithms and domain knowledge to identify misspellings, abbreviations, and phonetic matches. This is the most used option.|
More fields selected for matching will result in a narrower search for duplicates. Fewer fields selected will result in a broader search. A few trial runs will quickly reveal the best combination of fields for matching your data. It is usually recommended to use the powerful Fuzzy matching capability that ActivePrime's tools are known for, which identifies several types of similar matches. Exact matching is typically used on fields that are distinct character-wise, such as email, website, or customer ID. Starts With matching may be valuable for specific data de-duping needs.
**Please note that when matching on phone number fields, your results will be most accurate using Exact or Starts With matching. Fuzzy matching on phone number fields may yield inadequate results.
If you are new to CleanCRM, try these basic match settings for Accounts:
|Account Billing City:||Fuzzy|
|Account Billing State:||Fuzzy|
If you are new to CleanCRM, try these basic match settings for Contacts and Leads:
Click Advanced... next to any field to choose settings for blank field matching, Starts With matching, and words to ignore:
- Blank field matching defines how CleanCRM will match empty fields. Blank field matching is set to No by default, meaning that empty fields from two records will not match with each other ("___ Smith" does not match with "___ Smith" with blank field matching for First Name off). You can turn on blank field matching by selecting Yes, so that an empty field will match with another empty field ("___ Smith" will match with "___ Smith" with blank field matching for First Name on). Note that neither option allows you to match an empty field to a non-empty field ("___ Smith" does not match with "John Smith" if you match by First Name). This functionality is important in preventing large duplicate sets resulting from fields with missing data. For example, this happens when matching only on Account Name and several records are missing an Account Name (in this case, it would be preferable to select No for blank field matching).
- Characters for Starts With matching defines the number of characters that CleanCRM will use to perform Starts With matching. The default value is 5 characters. Change this number by clicking More or Less. The minimum number of characters required is 3. Note that this option only appears when you have selected Starts With matching for the field.
- Words to ignore defines which words CleanCRM will ignore when matching your fields. Enter one word per line. These entries are case-insensitive. Note that these words will not be ignored if you use Exact matching.
After you define your match settings and click Find Duplicates, CleanCRM will introspect your data. If any of your displayed fields are over 90% populated and are not used for matching, CleanCRM may recommend that you match with them. A pop-up window will ask you whether you'd like to keep your current match settings or use CleanCRM's match recommendations.
Choose the fields to display onscreen. You can add, remove, and rearrange any fields that exist in salesforce.com including custom fields. All available salesforce.com fields are listed in the Available fields column. Fields that will be displayed in your CleanCRM results are listed in the Show these fields in this order column.
To change the fields that will be shown, click on a field name and use the Add and Remove buttons. To change the order of displayed fields, select a field from the Show these fields in this order column and use the Move up and Move down buttons. To discard any changes you have made, click Cancel.
After CleanCRM identifies duplicate sets, you may wish to change the fields displayed onscreen. To do this, simply make your changes in the Displayed Fields menu and click Apply.
Remember to display all fields that will help you to review your duplicates. For instance, you may want to include custom fields, record owners, or lead sources. Please note that CleanCRM will not overwrite any field that is not shown onscreen.
There may be some fields for which you want to retain the values from each record in your duplicate set, for example, descriptions or sales notes. CleanCRM lets you keep all this data by concatenating the values from each record into the final record field.
To concatenate the data for a field, check off the desired field and click in the dropdown to select the separator of your choice.
After CleanCRM identifies duplicate sets, you may wish to change field concatenation settings. To do this, simply make your changes in the Concatenate Fields menu and click Apply.
Saving Your Settings
When you have finished choosing your settings, save them by clicking the Save As... button at the top of the Settings menu. Enter a name for your customized settings and click Save.
Your saved settings are listed in the Current Settings menu in the upper left side of the CleanCRM window. To load your saved settings at a later time, click on the menu next to Current Settings and select the saved settings.
Customizing Merge Rules
Before you start creating new merge rules, please make sure to add to CleanCRM all the display fields that will be used in Merge Rules. (Setting -> Displayed Fields).
After you have done that you can launch Merge Rules manager
Merge Rules Manager
You can launch Merge Rules Manager from Settings and also from Options:
You can activate or deactivate any Merge Rules. If a rule is not active it will not be visible for CleanCRM users.
Creating and updating Merge Rules
When you create a new rule you will first need to select an action “Select Master Record” - Select Master Record will set the master record based on meeting your selected criteria.
The record that meets all Criteria Conditions will be set as the Master Record.
“Block Duplicate Record” - Block Duplicate Record will block a duplicate record to not be merged, based on meeting your selected criteria. The record that meets all Criteria Conditions will be removed from the Duplicate Set and will not be merged. It is the equivalent of using “Remove” link on the left of the record in the gridview.
“Block Duplicate Set” - Block Duplicate Set will block the entire duplicate set from being processed and the records will not be merged. If any record meets all the Criteria Conditions, the whole duplicate set will be removed and will not be merged. This is the equivalent of using the “Remove” link in the gridview on all records in Duplicate Set (note that you cannot remove Master, so if it’s done manually, you will have 1 record in that Duplicate Set).
“Set Field Value” - Set Field Value will set the desired field value based on meeting your selected criteria. If all the Criteria Conditions are met, the specified value will be set to “Final” Master Record. Note: It will be set to Final field value directly and will not affect other conflicting values checkboxes. Sample: We need to set the value of “Required evaluation” to “Yes” if we don’t have data for “Phone”, “Fax”, and “Website”. The Merge Rule can be set up with “Set Field Value”, Action: Set Value to “Required evaluation” = “Yes”, Conditions: “Phone”, “Fax”, and “Website” = “Column Is Empty”. In this case, CleanCRM will put “Yes” to Final value of field “Required evaluation”, but only if whole columns for all 3 fields “Phone”, “Fax”, “Website” is empty for this Duplicate Set.
“Add the desired Set Value actions:”
Note: this will be applied to each Duplicate Set individually.
“Set Value” - Set a value to a specific field. You have to fill in the value to set. “Set Value To Largest” - If a field value is a number, it will be set to the larges number. If the value is text, the largest value will be defined by ASCII ordering (Z is larger than A). All values that contain a mix of numbers and letter will be treated as text.
“Set Value To Longest” - The longest value will be used. Sample: “555-555-5555” vs. “555-555-5555 ext 555”. “555-555-5555 ext 555” will be used.
“Set Value To Most Frequent” - The most frequent value will be selected.
“Set Value To Shortest” - This is the opposite to “Set Value To Longest”. The shortest value will be used. Sample: “555-555-5555” vs. “555-555-5555 ext 555”, “555-555-5555” will be used.
“Set Value To Smallest” - This is the opposite of “Set Value To Largest”. If a field value is a number, it will be set to the smallest number. If the value is text, the smallest values will be defined by ASCII ordering (A is smaller than Z). All values that contain a mix of numbers and letter will be treated as text.
The last part is selecting the criteria for this rule
First, you need to define if this will be “And” or “Or” logic.
Select “All” if you need “And” logic, select “Any” for “Or” logic.
You can add multiple conditions and Sub-Conditions. For each Sub-Condition you have to define the logic as well (All/Any).
For each Condition, you will need to define the field and the criteria.
The first set of fields (before the break) is a pre-defined set of multiple fields.
“Address” - Any Address that is presented in CleanCRM displayed fields. This field can be set to “Is Most Complete” or “Is Least Complete”
“Billing Address” - Only the Billing Address fields that are presented in CleanCRM displayed fields will be processed. This field can be set to “Is Most Complete” or “Is Least Complete”.
“Shipping Address” - Only the Shipping Address fields that are presented in CleanCRM displayed fields will be processed. This field can be set to “Is Most Complete” or “Is Least Complete”.
“Create Date” - The record create Date. This field can be set to “Is Newest” or “Is Oldest”.
“Duplicate Type” - This will evaluate whole Duplicate Sets (groups of records in CleanCRM with common number, 1.1, 1.2, 1.3, 1.4) and the defined duplicate type. This can be set to “Is Conflicting Duplicate” - duplicates that conflicts each other, marked with red background color in CleanCRM, “Is Exact Duplicate” or “Is Non Duplicate”.
“Multiple Fields” - This allows you to evaluate multiple fields at once. It can be very useful if your Master Record preference is not based on Date or Address. You will select the criteria for the fields, and then select multiple fields in any order (Use Ctrl+Mouse Click to select/deselect multiple items).
“Are All Empty” - For fields with no data.
“All Contain Data” - For fields with data.
“Least Filled In” - For fields with less data.
“Most Filled In” - For fields with the most populated data. Most useful for helping select the Master Record.
“All other fields” - You can select a field and its criteria. Criteria can be based on the Duplicate set (“Column…”, “Largest Number”, ”Smallest Number”), or an individual value (“Value..”).
Once your settings have been chosen, click Find Duplicates. CleanCRM will identify duplicates amongst the records in your report and display the results in the main body of the CleanCRM window.
CleanCRM's progress will be displayed at the top of the screen. Segmenting your data and finding matches may take a few moments, especially with large reports of records.
After CleanCRM has identified the records and begun the matching process, CleanCRM will introspect your data. If any of your displayed fields are over 90% populated and are not used for matching, CleanCRM may recommend that you match with them. A pop-up window will ask you whether you'd like to keep your current match settings or use CleanCRM's match recommendations.
Understanding Your Results
After CleanCRM has found duplicates, the results will be displayed in the main body of the CleanCRM window. Your matching results are organized into three categories:
- Duplicates Requiring Review
- Duplicates Not Requiring Review
- Unique Records
You can manipulate the data to fine-tune the duplicate sets before merging duplicate records.
Your matching results are organized into three categories:
|Duplicates Requiring Review:||Duplicate records with conflicting data in the displayed fields. These records should be reviewed before merging.|
|Duplicates Not Requiring Review:||Duplicate records without conflicting data in the displayed fields. These duplicates are exact matches and require little to no review before merging.|
|Unique Records:||Unmatched or non-duplicate records as defined by your settings.|
Click the arrow to expand the grids and review your duplicate sets.
Reviewing Your Data
Click the arrow to expand the grids and review your duplicate sets. Duplicate sets are grouped and numbered in the grids. The Final record represents the final field level values for the surviving merged record for each set.
You can easily review your duplicate sets before merging:
- Remove a record from a duplicate set by using the Remove link.
- Change the master record of the duplicate set by reselecting the Master check box. The master record is the surviving record that will be retained and updated after merging the duplicate set.
- Choose final record data by selecting check boxes in highlighted fields representing conflicting data in a duplicate set. You can also manually edit the final record by clicking in the final record data fields and typing your changes.
- Create a task for later review by hovering over the Final record arrow and selecting Create Task.
These duplicates grids are densely packed with information. Underlined text values are hyperlinks to the existing record in salesforce.com. Additionally, final record fields that appear in green can be edited by clicking into the field and typing your changes.
Duplicate sets highlighted in yellow indicate a display warning or error. You can view the warning by clicking view warnings in the Master field of the final record, or by clicking View Errors in the category grid title bar.
Your duplicate sets can be sorted by the data in any of your displayed fields. Simply click on the column header to enable ascending or descending sorting. Your duplicate sets will be sorted amongst each other by the final record data in the selected column.
If you are not satisfied with the match results, you can fine-tune your settings and re-run CleanCRM until it finds duplicate just the way you want.
Exporting Your Data
Each category of grids can be exported to a spreadsheet. Exporting may be helpful if you require external review of duplicate sets. Click the Export button in the category grid title bar to choose a file name and type. You may filter by owners if desired. Click Save to generate the file.
You can see the statistics of your current CleanCRM run by clicking Statistics in the upper-left corner of the window. You will be shown the number of total duplicates, duplicate sets by category, and unique records (non-duplicates). These values help you do a "health check" of your data to see how much duplication currently exists in your salesforce.com system.
Merging Your Duplicates
When you are satisfied with duplicate identification and have reviewed your data, you can merge duplicate sets in three ways:
- One-by-one by hovering over the Final record arrow and selecting Merge Set.
- Page-by-page by clicking Merge Current Page in the category grid title bar.
- All Records by clicking Merge All in the category grid title bar.
You can choose to merge your duplicates now, or merge the records later using ActivePrime Scheduler. See the Scheduler section of CleanCRM Help help file for more information.
During the merge process, all associated child entities from the duplicate records in the set will be moved to the surviving master record so that no information is lost. The other duplicate records will then be deleted. The behavior is the same as when two records are manually merged in salesforce.com.
**It is important to note that each Merge process is final and cannot be undone.
Successfully merged duplicate sets will be highlighted in green . Unsuccessful merges will be highlighted in red with errors. You can view error details by clicking View Errors in the Master field of the final record or in the category grid title bar. If you can correct the cause of error, you can re-attempt to merge the duplicate set.
**Please note that CleanCRM will not overwrite the master record for any field that is not displayed onscreen. Therefore, the master record will retain its original value for any field that is not displayed. You may change the displayed fields while reviewing your data by reselecting your Settings.
When reviewing large numbers of records, it is advised that you merge duplicate sets one-by-one or page-by-page to keep your data manageable.
Use ActivePrime Scheduler to run CleanCRM whenever you want, even remotely! Schedule analysis events to routinely scan for problematic data. Use merge events to clean out pesky dupes from your CRM.
Access ActivePrime Scheduler by clicking Schedule CRM Scan from CleanCRM. Here, you can view a timeline of scheduled CleanCRM events, schedule new scan events, and edit or delete scheduled events.
Schedule a CleanCRM Scan
Set up a future CleanCRM scan using ActivePrime Scheduler by clicking Schedule CRM Scan from CleanCRM. On the Day calendar view, add a new event by clicking New Event or by clicking on the timeline.
Choose the settings for your scan event. Select an Entity to scan (Accounts, Contacts, or Leads), the Settings to Run (any saved settings that you have created in CleanCRM), and Recurrence (One time event, Daily, Weekly, or Monthly). You can also add a note about the scan event.
Your new event will be scheduled at the first available two-hour time slot after the present time. If you created a new event by clicking on the timeline, the new event will be scheduled for the time you selected, or the first available two-hour time slot thereafter.
CleanCRM will scan your records for duplicates at the scheduled time.
Schedule a CleanCRM Merge
After running a CleanCRM scan, you can merge your duplicates. Whether you are merging sets one-by-one, page-by-page, and all at once, you have the option to merge the records now, or merge then later using ActivePrime Scheduler. To schedule a future merge, simply select the option to Merge records later to optimize speed, select a time, and click Yes.
CleanCRM will merge your records at the scheduled time.
Viewing and Editing Scheduled Events
Access the calendar of scheduled events by clicking Schedule CRM Scan from CleanCRM. ActivePrime Scheduler will show you a calendar of scheduled Scan, Merge, and Recurring events, as well as any scheduled Maintenance events. You can view the calendar by Day, Week, or Month. Scroll through time using the arrow buttons, or click Today to return to the present day's schedule.
Click on any scheduled event to view details. Edit the event details and be sure to Save any changes. Delete a scheduled event by clicking Delete event. Deleting a recurring event will delete all events in the series.
Your salesforce.com account administrator is responsible for maintaining your CleanCRM account options. Your account options can be accessed by clicking Options in the upper right corner of the CleanCRM window. Here, you can update your CleanCRM account, manage users, manage saved settings, manage email notifications, update CRM fields, and make feature suggestions.
Update CleanCRM Account
This page is used to update your salesforce.com administrator information. This is the account that CleanCRM accesses to perform searches. If the admin username or password for the account changes, please update it here so that CleanCRM can continue accessing your salesforce.com data.
This page allows you to manage the salesforce.com users who are able to use CleanCRM. To enable a user, check the AP Enabled box next to that user's name. To disable a user, uncheck this box. You can add additional Admin users by checking the AP Admin box. You can only enable as many CleanCRM users as the number of CleanCRM licenses you have in your subscription. If you need more licenses, click Inquire About Adding More Licenses.
Any new users of your salesforce.com account will appear on this list and can be enabled at any time, provided that you have enough user licenses.
For long lists of users, you may need to search for users. You can use the column filters to search for users. Enter your filter criteria and click Apply Filter to refine the list of users on display. You can also press Ctrl+F to find users.
- Above CRM Status, you can select for active, inactive, or all users.
- After updating users, you must click Save to save your changes. Clicking Apply Filter to search for users will discard any unsaved changes on screen.
- Click Export CSV to export the CleanCRM user list on screen.
This page allows you to save your CleanCRM settings for future use. Click Save Current Settings to save your custom settings that are currently being used in CleanCRM.
All prior saved settings will also appear on this page. You can load any saved setting by clicking Load. You can change the default setting which CleanCRM will automatically load when it starts. You can also delete and rename settings here.
This page allows you to manage the email addresses that will receive notifications about CleanCRM. Enter one email address per line in the text box, and click Save Emails to save your changes.
Update CRM Fields
Click Update CRM Fields to update the fields and labels from your CRM into CleanCRM. If a user changes any labels, adds new custom labels, or removes labels, clicking this button will refresh their appearance in CleanCRM instantly.
Is there anything new you would like to see in CleanCRM? Anything that ActivePrime could do differently?
Click Feature Suggestion to send a feature suggestion to ActivePrime. This link will lead you to our support portal, where you can submit your suggestion to the ActivePrime team.
Frequently Asked Questions
For further assistance, visit our Support Portal at www.activeprime.com/support/.
Interested in more data quality solutions?
ActivePrime has developed an array of complementary tools including ActivePrime Search, CleanEnter, CleanCRM, CleanCSV, and CleanImport that are embedded in salesforce.com. ActivePrime Search is a real-time, fuzzy, CRM search solution. CleanEnter is a real-time duplicate detection service. CleanCRM is a batch data de-duping tool. CleanCSV dedupes CSV files of records. CleanImport dedupes CSV files against your CRM records and imports them. To learn more about these products, visit www.activeprime.com/platforms/salesforce/. Sign up for a free trial through CleanCRM Free Trial.
Last Revision: 4/11/2014