Contact Us 877.823.3669

Data Lake Architect in Arlington, VA at SNI Technology

Date Posted: 11/7/2019

Job Snapshot

Job Description


Design, implement and maintain a Datalake based on AWS technologies. This includes storing, cleansing, preparing and securing data to be used by various internal customers.

Primary Responsibilities:
• Implement and maintain an "Infrastructure as Code" approach to deploying AWS resources in the Lake. This includes using technologies such as Git, CloudFormation, Terraform, AWSCLI and AWS CDK
• Develop workflows based on AWS DMS (Data Migration Service) that extract data from various sources including Oracle and populate the Lake.
• Develop workflows that transform data from one format to another using Glue and PySpark jobs. Experience with AWS EMR is equivalent.
• Recommend and implement approaches to secure the Datalake in general and implement a role based approach at the user level to allow only authorized users to access sensitive data.
• Develop approaches monitor the Datalake using technologies such as AWS Cloudwatch.
• Develop lambda and fargate programs to perform maintenance tasks on the Lake.
• Develop and maintain an AWS Aurora Postgres environment. This includes setting up Postgres and developing SQL.
• Evaluate the best type on columnar database to implement, this includes AWS Redshift and Snowflake.
• Develop Athena tables both in CSV and Parquet formats.
• Assist internal users with developing queries to drive Tableau and other 3rd party tools to access the Datalake via Athena and Postgres.
• Participates in recruiting, hiring, onboarding and performance management of new team members.
• Participates in special projects and performs other duties as assigned.

Job Requirements:
• Knowledge and hands-on experience with the AWS data services including several of the following technologies: S3, CloudFormation, Terraform, AWSCLI, Redshift, EMR, Glue, Snowflake, Cloudformation, Postgres, Athena, fargate, RDS, AWS Aurora
• Experience working on a large scale data warehouse environment with thorough knowledge of database and data technologies in general.
• Excellent interpersonal skills, including the ability to work effectively with persons on all levels.
• Excellent troubleshooting skills under deadline pressure in a production environment.
• Knowledge of and ability to use database-modeling software such as Erwin.
• Strong record of project execution and completion with experience using Scrum and agile development practices.
• Excellent written, verbal, and interpersonal communication skills including the ability to interact with all levels of employees and customers throughout the organization.

Education and Experience:
• Bachelor's degree in Computer Science or a related discipline, or equivalent experience. Master's degree preferred.
• A minimum of 7+ years' experience as a software engineer or architect for large data projects such as data warehousing.
• Minimum 4+ years' experience developing applications in the AWS environment.
• Experience using a variety of languages and technologies to develop web solutions.
• Experience working in an iterative or agile development environment, preferably Scrum.