Data Engineer

Requisition Number: 200927

External Description:

Think Big Analytics Role Description

Data Engineer

Data Engineers define the architecture and design of data ingestion and ETL / ELT processing to meet functional and non-functional requirements and objectives. They are broadly aligned with the Data Acquisition & Integration sub-practice, although some may be aligned instead with the Testing, Tools & Methods sub-practice. Data Engineers will generally be expected to be “T-shaped” professionals who have a broad understanding of the data acquisition and integration space and to be able to weigh the pros and cons of different architectures and approaches.

A Data Engineer works on implementing complex big data projects with a focus on ingesting, parsing, integrating, and managing large sets of structured and unstructured data. Data Engineers embrace the challenge of dealing with petabytes of structured and unstructured data and associated metadata daily. Data Engineers are involved in the design of big data solutions, leveraging the experience they have with Teradata and Hadoop based technologies such as Hive, Cassandra, and MapReduce. Strong communication skills are required to work in a team environment and with business sponsors of programs.

Principal Data Engineers may function as Tech Leads.

 Key Areas of Responsibility

• Clarify the client’s business problem for data analysis.
• Participate in requirements and design workshops with our clients.
• Translate the business requirements into technical requirements.
• Design and develop code, scripts, and data pipelines that leverage structured and unstructured data.
• Oversee the design, evaluation, and selection of major databases and metadata structures.
• Develop project deliverable documentation
• Must be adept at sizing and estimating new projects, as well as supporting new business development
• Help establish thought leadership in the big data space by contributing to white-papers, technical commentary, and representing Teradata at industry conferences
• All consultants are expected to build value in themselves. Teradata’s extensive library of both instructor led and web based training provides ample opportunity for the consultant to build and maintain marketable skills. Time has been allocated specifically for this task and each consultant is expected to have a ‘Learning Plan’. Progress against the learning plan is part of the annual appraisal.
• All consultants are expected to build value in their practice through the contribution and reuse of consulting assets. After each assigned project, a consultant is expected to evaluate the project deliverables and to contribute those items that may be useful to other consultants that may be assigned similar projects. When a new project is assigned, the consultant will search the asset repository for assets that may improve or accelerate the project delivery.

Skills

• Proven expertise in production software development
• Experience with large data sets consisting of structured and unstructured data and associated metadata and distributed computing with Teradata, Hadoop, Hive, HBase, Pig, etc.
• Proficiency in SQL, NoSQL, relational database design and methods
• Linux / Unix experience
• Experience with Avro, Thrift
• JMS: ActiveMQ, RabbitMQ, JBoss, etc.
• Experience with structured and unstructured data, and metadata.
• Work with the appropriate project management methodology (Agile or Waterfall) based upon customer and project requirements
• Experience leading technical teams
• Ability to initiate and drive projects to completion with minimal guidance
• Excellent verbal and written communication skills
• Knowledge of Architecture Principles, Advocated Positions, Design Patterns, and Implementation Alternatives.

CountryEEOText_Description:

City: Shanghai

State: Shanghai Shi

Community / Marketing Title: Data Engineer

Job Category: Consulting

Company Profile:

Our Company

At Teradata, we believe that people thrive when empowered with better information. That’s why we built the most complete cloud analytics and data platform for AI. By delivering harmonized data, trusted AI, and faster innovation, we uplift and empower our customers—and our customers’ customers—to make better, more confident decisions. The world’s top companies across every major industry trust Teradata to improve business performance, enrich customer experiences, and fully integrate data across the enterprise.

LinkedIn Remote:

Location_formattedLocationLong: Shanghai, Shanghai Shi CN