Senior Data Engineer, Boston, MACompensation Commensurate with experience, bonus, benefits, EOE, diverse highly educated culture Candidates must be a U.S. citizen or national, refugee, asylum, or lawful permanent resident.

H1b candidates will be consideredOur client is seeking a Senior Data Engineer to be key player in their Product & Technologies group. You will be the primary person building out the data integration framework capability within the data hub architecture. The data hub uses a modern cloud based distributed computing architecture that utilizes modern technologies such as AWS S3, AWS EMR, Apache Spark, Redshift etc.

Responsibilities You will work on the foundational infrastructure components within this architecture to expand the capability and make the system more robust.You will collaborate with senior engineers to design and engineer a robust, scalable integration framework that can transform the variety of incoming data formats from our customers.You will provide post-production monitoring and support for the components you build to ensure the system performs to specification.

Develop a sound understanding of the architecture already in placeDevelop additional data hub infrastructure components to make the architecture more robust and scalableBuild frameworks, tooling, pipelines that address the partner data integration requirementsProvide operational support for the framework/tooling in production, especially in the early nascent stages till the capability maturesUse agile/scrum methodology to ensure sprint commitments are met regularlyPerform PR reviews for other engineersKeep abreast of new services and technologies that can be utilized to evolve the data hub architectureQualifications The ideal candidate will have a passion for the data domain, a strong background in data engineering, having designed and implemented data pipelines that can handle complex large-scale data transformationsBackground in Computer Science or related field with experience doing data engineering work working with newer data lake type architecturesSolid understanding of algorithms and data structuresStrong understanding of relational databasesProficient with ANSI SQL and procedural SQLProficient with at least one of following programming languages - Python, Java, ScalaDemonstrable experience working with Apache Spark for large-scale data processingExperience working with AWS services such as AWS Glue, EMR, S3 is a strong plusExperience with data lake type architecture is desirableGood understanding of MPP architectures and the Hadoop Big data ecosystemExperience working with unstructured data is a plusBlueSkyClarityBlueSkyClarity (a Delaware LLC) is a search firm focused on retaining smart, passionate and talented people within the marketing, creative, analytic, product, sales, software engineering and information technology domain disciplines for web-to-consumer, digital agency, consulting, start-up, or iconic brand clients in all industries.Posted byDom Costagliola, Principal, m 1-617-899-5094http://www.blueskyclarity.com/Type: direct hire.