Thrivent is a client-owned financial services organization that provides a range of financial solutions, including banking, investments, insurance, and generosity programs.
Thrivent Enterprise Integration Datastore (TEID)
Improving partnered with Thrivent to develop the Thrivent Enterprise Integration Datastore (TEID). This project aimed to consolidate and streamline data from various systems of records, enhancing the accessibility and usability of data for front-end systems, thus improving user experience on their website and mobile applications. The scope of the project included data ingestion, organization, and real-time updates, ultimately modernizing Thrivent's data infrastructure and operations.
Thrivent faced significant challenges with their existing data management systems, which were slow, disjointed, and relied heavily on multiple third-party data hosts. The siloed nature of data storage across different systems and departments resulted in inefficiencies and slow data access, impacting their ability to provide a responsive and cohesive user experience on their digital platforms. Additionally, maintaining and integrating legacy systems like SIFT and the mainframe added complexity and cost, hindering their modernization efforts.
Improving's approach involved creating a unified data integration platform using Confluent's Kafka for real-time data streaming and DynamoDB for fast data storage and retrieval. We collaborated with various teams at Thrivent, including those handling front-end systems, APIs, and data records, to design and implement a robust data pipeline. This pipeline aggregated and normalized data from multiple sources, ensuring real-time data synchronization and seamless integration with new systems like Salesforce. We also leveraged GraphQL for efficient data querying, enhancing the usability of the datastore for front-end developers.
Kafka: For real-time data streaming and integration.
DynamoDB: For fast and efficient data storage and retrieval.
GraphQL: For flexible and efficient data querying.
AWS: Used for S3 buckets and other cloud services.
Kubernetes and OpenShift: For container orchestration and management.
Hashicorp Vault: For secure management of secrets and access control.
Terraform: For infrastructure as code and consistent state management.
Improved Data Accessibility: Centralized data store enabled quick and easy access to comprehensive data sets.
Enhanced User Experience: Faster, more responsive web and mobile applications.
Operational Efficiency: Streamlined data processes and reduced redundancy across departments.
Cost Savings: Potential reduction in API usage fees from third-party data providers.
Modernized Infrastructure: Transitioned from legacy systems to modern, scalable solutions.
Increased Trust in Data: Reliable and real-time data updates allowed for more confident business decisions.
The project involved collaboration with several technology partners. Key partnerships included Confluent for Kafka implementation and support, Apollo for GraphQL development, and Datadog and Elastic for observability and logging. These partnerships provided essential tools and expertise that contributed to the successful implementation of the TEID project.
Data Governance: Implementing a robust schema registry was crucial for managing data integrity and consistency.
Real-Time Processing: Real-time data updates significantly improved operational efficiency but required careful handling of batch processes.
System Integration: Bridging communication gaps between teams was essential for seamless integration and project success.
Technology Choices: While DynamoDB provided excellent response times, it posed challenges for data publishing and required additional tooling for observability.
Legacy System Migration: Gradual migration from legacy systems to modern platforms is a complex and ongoing process.
Scalability: Ensuring the solution could handle large data volumes and maintain performance was critical for long-term success.
The Thrivent Enterprise Integration Datastore project exemplifies Improving's capability to deliver innovative, scalable, and efficient data solutions. By leveraging modern technologies and fostering strong partnerships, we helped Thrivent achieve significant improvements in data accessibility, user experience, and operational efficiency. Our collaborative and adaptive approach ensured the successful implementation of a complex data integration project, setting the stage for future advancements and continued modernization efforts.
Improving embedded a full-time AI engineer and structured training inside First United Bank’s SDLC teams, raising AI maturity by ~80%, cutting idea-to-user-story time from 70 hours to 11, and saving an estimated $40,000 on a single migration project.
Improving's 12-week Deep Learning Program helped Topgolf move engineering teams from ad hoc AI experimentation to disciplined, AI-assisted delivery, driving a 15% increase in self-assessed AI adoption and confidence.
Find out how Improving partnered with a leading American energy company to design and deliver a deep learning upskilling program focused on agentic software development tools.
