What is Entity Resolution and How Does It Transform Data Into Value?

Data should yield opportunities — but for many organizations, more data means more problems. From duplicate customer records to never-ending data science projects, there are numerous challenges. Trying to overcome these challenges is difficult: Connecting disparate datasets into a comprehensive single view of data is time-sensitive and labor-intensive, and brings a maze of security issues.

Without a reliable data foundation, it’s impossible to build a 360-degree view of customers, prospects, partners, and organizations. And without this complete view, there’s no way to turn this data into insights.

Why is connecting data so difficult?

Siloed data
Data is trapped in silos across internal and external systems, resulting in teams repeating work and customers receiving a disjointed experience. Siloed data makes it impossible to see a full picture — which leads to inaccurate decision-making.

Varying requirements
Each unique user or use case requires specific data sources and has different requirements about how fuzzy or strict the matching may be. Sensitivity of sources (in financial crime use cases, for instance) complicates these requirements further. Multiple views are necessary for different requirements.

Data quality challenges
Poor data quality results from duplicate entities, missing information, and intentional manipulation by criminals. Traditional record-to-record matching struggles to deal with sparse, inaccurate data and is ill-suited to natural variations in data sources.

Turning volumes of data into commercially-useful, business-oriented, valuable insights is a significant challenge.

But it’s one that Entity Resolution can help resolve. Here’s how.

Turning records into reality

This animation shows three unique customer records in an organization’s database: John Smith, John Citizen, and J. Citizen. Using Entity Resolution, the organization can connect these records to gain a single view of the data, showing that John Citizen and J. Citizen are actually the same person.

Entity Resolution overcomes variations in addresses, phone numbers, and company name records to match them — and then draws on these attributes to merge John Citizen’s and J. Citizen’s customer records into a single entity. The process also reveals a connection between John Citizen and John Smith that wasn’t evident before.

Internal versus external data

Entity Resolution brings together data from a number of different sources — both internal and external.

Importing data on a project-by-project basis isn’t efficient. Instead, use a central integration point to make integrated internal and external data available to all decision-makers.

Current approaches to data matching

With ever-expanding volumes of data, a lack of consistent data quality, and a wide range of use cases, traditional approaches just aren’t good enough.

Traditional data matching

• Compares data records directly using a paired matching approach

• Relies on many attributes matching to produce a match score that’s high enough

• Struggles with sparsely populated records or those where there is variance in the information

Entity Resolution

• Uses an iterative matching approach that continually enriches records with additional data to provide the most accurate view possible

• Makes connections between data even when quality is low or where there has been manipulation

How Entity Resolution differs from traditional matching

Traditional matching

• Compares data records directly using a paired matching approach

• Relies on many attributes matching to produce a match score that’s high enough

• Struggles with sparsely populated records or those where there is variance in the information

Entity Resolution

• Uses an iterative matching approach that continually enriches records with additional data to provide the most accurate view possible

• Makes connections between data even when quality is low 
or where there has been manipulation


There are a few principles to follow when you look for an Entity Resolution tool. Some may be more important for your organization’s needs than others — but having all seven will give you a solid foundation.

The best Entity Resolution tools resolve entities in the same way a human would — but automatically, at massive scale, and with a full understanding of the data. They don't need to be programmed to know how common a name is or how large a business is. Instead, they mine this information from the data itself.

Accuracy

Above all, your Entity Resolution tool must be accurate. Look for independent validation, client testimonials and proven metrics to ensure it meets the highest accuracy standards. Look for a solution that’s been proven in the fraud and financial crime space, as these are built to overcome challenges like intentionally manipulated or poor-quality data

Transparency

With a trusted foundation of data, you can do everything from improving operational agility to automating decision-making. But you need to understand and trust how your system works. Choose an Entity Resolution tool that’s white-box by design, ensuring that the underpinning logic is accessible, transparent, and explainable.

Real-time and batch ingestion

Batch ingestion enables large scale resolution for data science use cases, while real-time ensures you’re always getting the most up-to-date and accurate view possible. To ensure you get the best of both worlds when it comes to data processing, choose a tool that offers both real-time and batch.

Granular security

Security is one of the main reasons why Entity Resolution tools are deployed within specific areas of the business rather than across the entire enterprise. Look for software that supports dynamic processing, which resolves entities based on the data each use case requires and ensures the user has the right to access.

Scalability

Entity Resolution is designed to bring all your data together into a single view, so it's crucial that your software doesn’t hit limits as you bring in more data. Look for a solution that's proven and in production at large Tier 1 organizations. Also, ensure it can scale linearly with hardware via a distributed architecture — otherwise, you'll end up with long batch times, delayed insights, and ugly interim processing workarounds.

Time to value

One of the main challenges with Entity Resolution is the time required to onboard new data. Some solutions require all data to be normalized into a standard schema, which wastes time. An Entity Resolution tool that is able to accept data in almost any format and contains out-of-the-box integrations to a wide range of trusted external data sources will give you a quick, accurate, complete view of every entity.

Use case flexibility

While many organizations use Entity Resolution for one initial use case, it's actually a foundational layer suited to many parts of the business — from faster customer onboarding, to sharper financial crime detection, to single view augmentation for MDM. As each use case has different requirements, it's vital to implement Entity Resolution software that can support different matching and data source requirements.

Real-time Entity Resolution may keep your single entities up to date, but it doesn’t have the flexibility to provide value across multiple use cases. Dynamic Entity Resolution is the next evolution of real-time Entity Resolution.

Rather than keeping an existing single view of an entity up to date, Dynamic Entity Resolution re­-generates the entity in real-time from the underlying raw data.

This lets the software dynamically include or exclude particular data points and allows users to specify the match confidence they require for their specific use case.

This unique capability allows you to deploy Entity Resolution across your entire organization for any use case.

The evolution of data matching

1. Traditional matching
   Focuses on direct matching, which results in poor match quality

2. Batch Entity Resolution
   Offers improved matching, producing a single-entity view across all data

3. Real-time Entity Resolution
   Keeps the single view up to date as new data comes in

4. Dynamic Entity Resolution
   Adds the ability to re-generate entities at the time of request from the underlying data

What makes Dynamic Entity Resolution different?

“One size fits all” is rarely a good thing — especially when it comes to Entity Resolution. Other systems assume that a single view of “John Citizen” can be utilized for every use case within your enterprise.

However, it can't. Here are two reasons why relying on that single view isn't sufficient:

1. Different use cases require different levels of matching and have varying restrictions around different data sources. For example, if you use a highly sensitive source in the Entity Resolution process, the resulting entities will also be classified as highly sensitive-and therefore can't be used elsewhere. The capability to distinguish source sensitivity is critical if you have external data that's only been licensed for use within one part of the business.

2. Multiple use cases lead to multiple instances. Other systems can support different use cases — but only by replicating data. This might seem like a reasonable approach, but as you start using Entity Resolution more broadly across your enterprise, replication quickly becomes unmanageable. Plus, when you add the matrix of data source permissions to the mix, a “multiple instances” approach quickly becomes unwieldy.

A Dynamic Entity Resolution tool builds on demand, at the time of request. 
This means you can:

1. Deploy one instance of the platform to serve all the needs in your organization

2. Specify the level of fuzziness required per use case, at the time of request

3. Control access to data sources depending on what the user or use case can see

Dynamic Entity Resolution is the only way to create an enterprise-wide, trustworthy, resolved data foundation that can support multiple use cases. It helps you resolve a growing number of use cases in a rapid and secure way, harnessing and connecting all the data to which you have access.

The results speak for themselves. This isn't just about creating a single view across your enterprise — it's about improving efficiency, laying the foundations for accurate analytics, and scaling to your needs.

Dynamic Entity Resolution lets you:

• Create a single, complete view of customers, prospects, and organizations across your business.

• De-duplicate records across your business, giving you a single source of truth.

• Improve the quality of your data and automatically fill in missing information.

• Centralize access to both internal and external data throughout your organization.

• Drive up productivity by providing access to accurate, consolidated information.

• Create the foundation for data-driven decisions and automated/augmented decision-making.

Gain a single customer view across your enterprise to drive growth

Here’s a snapshot of what’s possible with a complete view of your customers and prospects:

Entity Resolution lets you connect and resolve tens of billions of internal and external data points in one place. This means your organization gets an enterprise-wide view of people, places, organizations, and events across supply chains, empowering you to make faster, more accurate decisions throughout the customer lifecycle.

With an accurate, trusted single view in place, you can:

• Understand the connections and relationships between entities, and form networks of connected data.

• Use both the entities and networks within analytical models to enable better decision-making across your enterprise.

With Decision Intelligence, your organization can:

• Drive automation and deliver greater business value from enterprise data.

• Make faster and more accurate decisions.

• Spot hidden risks and identify high-value growth opportunities.