Data Engineer Job at Anblicks, Dallas, TX

bVhHVzVpbGJKdm5FZXMwK3pKUmVCb3BsM3c9PQ==
  • Anblicks
  • Dallas, TX

Job Description

Role: Data Engineer

Location : Dallas, Texas

Job Type: Hybrid

The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working directly with product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.

Responsibilities:

Responsible for design and development of big data solutions. Partner with domain experts, product managers, analyst, and data scientists to develop Big Data pipelines in Hadoop

Responsible for moving all legacy workloads to cloud platform

Work with data scientist to build Client pipelines using heterogeneous sources and provide engineering services for data science applications

Ensure automation through CI/CD across platforms both in cloud and on-premises

Define needs around maintainability, testability, performance, security, quality and usability for data platform

Drive implementation, consistent patterns, reusable components, and coding standards for data engineering processes

Convert SAS based pipelines into languages like PySpark, Scala to execute on Hadoop and non-Hadoop ecosystems

Tune Big data applications on Hadoop and non-Hadoop platforms for optimal performance

Evaluate new IT developments and evolving business requirements and recommend appropriate systems alternatives and/or enhancements to current systems by analyzing business processes, systems and industry standards.

Applies in-depth understanding of how data analytics collectively integrate within the sub-function as well as coordinates and contributes to the objectives of the entire function.

Produces detailed analysis of issues where the best course of action is not evident from the information available, but actions must be recommended/taken.

Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.

Qualifications:

8+ years of total IT experience

5+ years of experience with Hadoop (Cloudera)/big data technologies

Advanced knowledge of the Hadoop ecosystem and Big Data technologies Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Hive, Pig, Impala, Spark, Kafka, Kudu, Solr)

Experience on designing and developing Data Pipelines for Data Ingestion or Transformation using Java or Scala or Python.

Experience with Spark programming (pyspark or scala or java)

Expert level building pipelines using Apache Spark Familiarity with core provider services from AWS, Azure or GCP, preferably having supported deployments on one or more of these platforms

Hands-on experience with Python/Pyspark/Scala and basic libraries for machine learning is required;

Exposure to containerization and related technologies (e.g. Docker, Kubernetes)

Exposure to aspects of DevOps (source control, continuous integration, deployments, etc.)

Proficient in programming in Java or Python with prior Apache Beam/Spark experience a plus.

System level understanding - Data structures, algorithms, distributed storage & compute

Can-do attitude on solving complex business problems, good interpersonal and teamwork skills

Possess team management experience and have led a team of data engineers and analysts.

Experience in Snowflake is a plus.

Education:

Bachelor’s degree/University degree or equivalent experience

Job Tags

Similar Jobs

Russell Tobin

Financial Analyst Job at Russell Tobin

 ...Financial/Operations Analyst Entry Level - Contract Opportunity. Recent Grads are more than welcome! Location: Columbus, OH Duration: 6-12 months contract Pay: $19$23/hour Are you a recent graduate in Finance, Accounting, Economics, or... 

STEL Design®

Junior 3D Visual Designer Job at STEL Design®

 ...join our talented team. As a 3D Artist, you will be responsible for creating visually stunning and technically accurate 3D designs, animations, and visualizations that meet the high standards of our clients. Take a look at the requirements below if you think this is... 

Clean Harbors

CDL Vacuum Truck/ Equipment Operator Job at Clean Harbors

 ...HPC Industrial , powered by Clean Harbors, in Meraux, LA i s looking for a Class A/B CDL Operator with Vacuum Truck Experience to join their safety conscious team! The CDL Operator is responsible for the safe and proper execution of HPC Industrial jobs across all... 

Odin Properties

Groundskeeper/Light Maintenance Tech Job at Odin Properties

 ...disability Retirement Plan - 401(k) Plan Paid Time Off 10 paid holidays! $500 Sign On Bonus!! Employee Referral bonuses! Student loan repayment program Employee Assistance Program Employee recognitions and MUCH MORE! PM19 PI470ba3ab3b0e-30492-36867279... 

The LiRo Group

Construction Scheduler, Project Manager Job at The LiRo Group

We have an immediate need for a Construction Scheduler, Project Manager in Buffalo, NY . Come join our team! We are looking to build services and capabilities through the growth of our key asset- our staff. Ranked among the nation's top A/E firms by Engineering...