Message capture platform

Our client is a leading provider of information management solutions. The company assists in collecting, archiving, classifying, and migrating corporate content.

The client’s tool for collecting data from various communication and collaboration channels needed enhancement. A few notable downsides included the absence of integration with popular messaging systems and a limited number of archiving platforms supported by the application. The tool’s downtime and poor performance also negatively impacted the business. Our engineers were chosen to refine the tool (further referred to as data unifier) and integrate new features.

Data unifier is a universal message capture platform for retrieving data from a great variety of content sources, be it social media or enterprise collaboration platforms.

The system consolidates different data types, including chats, IM, and text, collects data and stores it on an archive platform or email server for further discovery.

This solution is designed for corporations operating in highly regulated industries, such as healthcare, financial services or energy, to comply with different regulations like SEC rule 17-a4, CFTC rule 1.31, Dodd-Frank requirements, FINRA and other regulatory agencies’ recommendations.

Data unifier greatly reduces legal risk and helps organizations with internal investigations, lawsuits, and audits.

Data unifier is configured as a Windows Service, so it starts when the machine is switched on without user intervention and runs in the background.

The tool includes 4 major components:

• Connector retrieves data from the content source.
• Filters are used for filtering messages and routing them to a particular target.
• Targets are message archiving solutions where the retrieved data is stored for further review and search.
• Importers configure how the data is loaded from the content source, what filters are applied, and what the target is. Thus, the importer connects all the above-mentioned components into a single system performing the actual data migration.

The system is designed for large data volumes processing and with ease of deployment in mind.

The number of connectors to third-party services can be infinitely extended. New connectors allow customizing the tool for a particular company and accumulating the data stored in the client’s collaboration applications.

Implementation

To implement the customer’s requirements, the engineers at EffectiveSoft thoroughly examined the existing application to improve its performance. Big Data processing optimization was achieved with multithreading integration. We used the stored procedures to split the server side load and utilize database server capacity.

Each connector that retrieves data from a certain service was developed separately by using the corresponding API and embracing the peculiarities of the service.

As an important part of the development process, the QA team did functional, regression, and performance testing.

Windows Installer built with the WiX toolset was added for fast deployment, consistent internal database, and reliable operation.

After the project was successfully completed, the customer contracted EffectiveSoft to provide technical support of the application, including adding new features and bug fixing.