A Low-Cost In-House Tool for Data Integration and Migration
· 4 min read
Data migration and integration are often the hidden cost in an enterprise transformation project. A large share of the implementation effort goes into connecting legacy systems, external platforms and modern applications, especially when heterogeneous data sources are involved.
At Emeis Technologies, we ran into this problem on our own projects, so we built a tool for it.
That tool is B2B, our in-house integration platform. Its job is to make enterprise data migration and integration faster and cheaper.
Why B2B?
A common and painful problem is consolidating data from multiple legacy and third-party systems into a centralized application or cloud platform.
B2B handles this by:
Supporting both homogeneous and heterogeneous data systems
Handling high-volume data reliably
Managing complex schema variations
Addressing security, scalability, and performance bottlenecks
Typical jobs include modernizing a legacy ERP, syncing CRM records and consolidating customer data from several sources. In each case B2B cuts cost, risk and manual effort.
How It Works
B2B works in two modes:
Push Mode (Real-Time)
Synchronous communication with destination endpoints
Delivers immediate feedback from the integrated systems
Ideal for transactional systems that need instant data updates
Also supports distribution to multiple destinations in real time without affecting the performance of the primary application
Pull Mode (Scheduled)
Used for periodic syncs and migrations
Suited to bulk data loads and nightly jobs
Keeps data in sync across systems on a defined schedule
Both modes use the same mapping engine, which defines how source data is transformed into the target format.
Source & Destination Flexibility
B2B supports a wide variety of systems and formats:
Excel/CSV/JSON (via FTP, SFTP, shared folders)
Relational Databases
APIs (REST/GraphQL)
CRM Platforms
Custom Plug-ins for unique business logic
Smart Mapping Engine
You configure mappings in a simple UI. Main features:
Source-to-destination column binding
Data type and length management
Default value handling for nulls
Lookup and relationship mapping
Field-level validation (regex, custom rules)
Plug-and-play custom logic for advanced transformations
Automation, Monitoring, and Reporting
Background Services
Scheduled services run in the background, syncing data hourly, nightly or in real time, as configured.
Notifications
Automated emails are sent for job success, failure, or warnings. Distribution lists are managed at the job level.
Logs & Reconciliation Reports
Track success, warning, and error counts for each sync
Detailed logs available via UI for debugging and monitoring
Reconciliation reports can be sent as email attachments for audit compliance
The Bottom Line: Affordable + Efficient
B2B is built to keep costs low without giving up capability or scale. Our clients, from growing businesses to enterprises with complex data estates, use it to save time, lower integration risk and go live sooner.
Moving data between systems that store it differently, for example from spreadsheets, legacy databases and APIs into a new application, with the data mapped and transformed along the way.
Why use an in-house integration tool instead of an ETL suite?
Enterprise ETL suites are powerful but expensive and slow to set up. A focused in-house tool covers the common source and destination types, with a smart mapping engine, at a fraction of the cost.
Can the same tool keep systems in sync after migration?
Yes. Once mappings are defined, the tool can run on a schedule to integrate data between systems, not just move it once.
Contains information related to marketing campaigns of the user. These are shared with Google AdWords / Google Ads when the Google Ads and Google Analytics accounts are linked together.
90 days
__utma
ID used to identify users and sessions
2 years after last activity
__utmt
Used to monitor number of Google Analytics server requests
10 minutes
__utmb
Used to distinguish new sessions and visits. This cookie is set when the GA.js javascript library is loaded and there is no existing __utmb cookie. The cookie is updated every time data is sent to the Google Analytics server.
30 minutes after last activity
__utmc
Used only with old Urchin versions of Google Analytics and not with GA.js. Was used to distinguish between new sessions and visits at the end of a session.
End of session (browser)
__utmz
Contains information about the traffic source or campaign that directed user to the website. The cookie is set when the GA.js javascript is loaded and updated when data is sent to the Google Anaytics server
6 months after last activity
__utmv
Contains custom information set by the web developer via the _setCustomVar method in Google Analytics. This cookie is updated every time new data is sent to the Google Analytics server.
2 years after last activity
__utmx
Used to determine whether a user is included in an A / B or Multivariate test.
18 months
_ga
ID used to identify users
2 years
_gali
Used by Google Analytics to determine which links on a page are being clicked
30 seconds
_ga_
ID used to identify users
2 years
_gid
ID used to identify users for 24 hours after last activity
24 hours
_gat
Used to monitor number of Google Analytics server requests when using Google Tag Manager
1 minute
Marketing cookies are used to follow visitors to websites. The intention is to show ads that are relevant and engaging to the individual user.
Formilla powers the live chat in our Talk to Experts menu and the related on-site messages.
Name
Description
Duration
formillaEdgeLastDisplay
Stores when a Formilla on-site message was last shown to you.
400 days
formillaEdgeDisplayedList
Records which Formilla on-site messages have already been shown to you, so they are not repeated.
400 days
YouTube hosts the videos embedded on our site. Its cookies are set when you play a video.
Name
Description
Duration
VISITOR_PRIVACY_METADATA
Set by YouTube when you play an embedded video, to store your cookie consent choices for YouTube.
6 months
VISITOR_INFO1_LIVE
Set by YouTube when you play an embedded video, to estimate your connection speed and remember your player preferences.
6 months
YSC
Set by YouTube when you play an embedded video, to keep track of your viewing session.