Companies working with big data must be strategic about their plan to tackle the mass amount of information at their fingertips. Many job descriptions posted online for “data scientist” include an extensive list of required skills unrealistic for a single candidate. Instead of an endless search for the perfect single candidate, it’s better to split the skills needed to harness big data into two jobs: data scientist and big data engineer.
The data scientist has a background in statistics, data modeling and data visualization. This person can analyze, model and show actionable trends in data.
The big data engineer has a background in software engineering, system administration, scalability, data governance and security. This person can build a secure, stable system for data to be held and analyzed.
These two skillsets work together very well, but are unlikely to be found in a single candidate. While having a larger team is a larger investment, having the right employees work to their strengths will pay off in the long-run.
