This blog is being reprinted with permission from OpenText + Liaison Technologies. Article was written by Ehi Binitie.
In the age of big data, most people understand the importance of gathering and using data to make decisions, drive business processes and understand what is happening throughout the entire enterprise. They know that people with very specific skills are essential to effectively capturing, managing and analyzing data for use, but the roles within the data team are not well understood – especially mine.
I am a data engineer. I work mostly behind the scenes to design and build the objects on the integration platform that house the data that must be captured, stored and made accessible to users. The architecture that I develop allows data analysts to access information in specific ways to analyze, create reports and provide insights that are used to drive business decisions. Simply described – data engineers are the designers, builders and managers of the information or “big data” infrastructure.
The role of data engineer has changed over the years. Before cloud computing was widely adopted as a solution for integration, information was housed in data warehouses that were difficult, time-consuming and expensive to expand when the business’ data needs to be changed. That began to change in the late 1990s when the concept of cloud computing expanded.
I joined Liaison, now part of OpenText, 14 years ago and was excited to be part of an innovative approach to data integration and management. By offering a cloud-based integration platform and a team of data experts to build and manage the service, we were offering a scalability and flexibility that met the data management challenges faced by our customers. Our capabilities to provide the same flexibility have expanded as we use new tools to manage the growing number of complex data sources.
One change that I’ve noticed is acceptance of “the cloud.” When Liaison first approached companies, there was a concern about the ease of accessing data once it moved offsite, the security of data and the reliability of the concept. Today, the cloud is part of everyone’s life, so these concerns no longer keep companies from looking at a cloud-based platform for their needs.
Another significant change in the way that data is viewed is the move from data-driven to data-centric. When I build a data-centric architecture, I can assure the business that the data will always be there in one location for multiple, task-specific applications to access. Applications can come and go as needed, but the data is always there. This future-proofs the company’s investment in the platform by providing the opportunity to add new capabilities as needed, without investing in a new way to collect or manage data.
Data engineers build the pipelines that transform and connect data from a myriad of sources – sensors, connected devices, social media and legacy systems – and ensure that the data is available in real- or near real-time to support insights that enable companies to effectively compete in their industry. Of course, we don’t just support the collection of data – we make sure that all users can access it regardless of the department, geographical location or application they are using to perform their analysis.
While all members of the data team are critical (in fact, I often say that data science is a team sport), I am biased toward my role. Data engineers make sure that information is always available in whatever format to any user to help the company be competitive and successful.
Why not contact GraceBlood and find out about our Managed Services approach to “cloud” EDI services, as well as a new EDI visibility tool? We work behind the scenes to allow you to do what you do best to maintain and build your competitive advantage without getting bogged down in the intricacies of day-to-day EDI.
Learn more about the relationship between ERP providers and EDI partnerships in our free whitepaper: EDI Partnerships Deliver Value to ERP Solutions Providers
