Real-Time Entity Resolution At Scale

USCIS’s core mission is to oversee the lawful immigration process for the United States. On average, USCIS adjudicates (decides) 45,000 filings a day. One pain point is how long people must wait for filings to be processed. Currently, the backlog of filings – while form dependent – are too long and USCIS announced the desire to drastically reduce both the existing backlog and case processing times.

One common and critical component is the identity verification and consolidation of information of the filer. Not only does US CIS need to confirm the person’s identity, US CIS also needs to confirm eligibility, all of which requires a complete view of the filer’s history of interactions with USCIS in real-time.

The challenge is that current processes for case processing are distributed amongst many systems, making it difficult to get a complete view. Adjudicators are required to several minutes validating and researching to confirm eligibility. These systems are data silos, and the data is shared using system-to-system APIs or API aggregators.

Confluent has transformed how the 360-degree view of a person is created the systems are able to send filing updates in real-time to a new system – Person-Centric Identity Services – in order to match and aggregate using Confluent Kafka, AI/ML. With Confluent, we are now able to ingest, match and store data in minutes.

Other benefits:
1. Real-time notification of case processing changes
2. Application telemetry data to identify data gaps
3. Data synchronization using Confluent
4. Data quality enforcement
5. Data freshness monitoring
6. Developer quality-of-life – (ksqlDB and Kafka Streams to solve data processing needs
7. Data Security – topic ACLs and RBACs and encryption
8. Tiered Storage give us flexibility in terms of cheap/efficient storage
9. Confluent connectors allow us to also provide the aggregated identity information to the data lake to support enterprise data analysts using Databricks