Senior Data Engineer

Requisition Number: 200779

External Description:

Title: Senior Data Engineer I                                                    

Primary Responsibilities:

As a Senior Data Engineer I, you will provide technical leadership to clients in a team that designs and develops path-breaking large scale cluster data processing systems. You will mentor sophisticated organizations on large scale data and analytics and work with client teams to deliver results.

Additionally, as a senior member of our Consulting team, you will help Think Big, A Teradata Company establish thought leadership in the big data space by contributing white papers, technical commentary and representing our company at industry conferences.

Secondary Responsibilities:

Design and develop code, scripts and data pipelines that leverage structured and unstructured data integrated from multiple sources. Software installation and configuration. Participate in and help lead requirements and design workshops with our clients. Develop project deliverable documentation. Lead small teams of developers and coordinate development activities. Mentor junior members of the team in software development best practices. Other duties as assigned.

Job Qualifications:

• Proven expertise in production software development
• 7+ years of experience programming in Java, Python, SQL, or C/C++
• Proficient in SQL, NoSQL, relational database design and methods for efficiently retrieving data
• Strong analytical skills
• Creative problem solver
• Excellent verbal and written communications skills
• Strong team player capable of working in a demanding start-up environment
• Experience building complex and non-interactive systems (batch, distributed, etc.)

Preferred Knowledge, Skills and Abilities:

• Prior consulting experience
• Experience with Hadoop, Hive, Pig, Avro, Thrift, Protobufs and JMS: ActiveMQ, RabbitMQ, JBoss, etc.
• Dynamic and/or functional languages (e.g., Python, Ruby, Scala, Clojure)
• Experience designing and tuning high performance systems
• Prior experience with data warehousing and business intelligence systems
• Professional or academic background that includes mathematics, statistics, machine learning and data mining
• Linux expertise
• Prior work and/or research experience with unstructured data and data modeling
• Familiarity with different development methodologies (e.g., agile, waterfall, XP, scrum, etc.)
• Broader experience with spring ecosystem including spring-batch, spring-mvc, and spring-hadoop
• Standards-based REST implementation
• Configure a Jenkins build, create/update a Jira ticket, enable Automated Tests in gradle/maven build
• Vagrant, Docker
• Familiar with OSI stack and proper use of HTTP verbs etc.
• Hive Tuning, Hive physical design (file formats, compression, partitioning, bucketing), Hive DSS Queries, Hive for Data Science, Hive for running R/Python (Streaming), Hive Transactions, Hive-HBase
• Knows how to tune a job including parameters, more efficient API calls. Understands Spark SWL. Understands Spark Streaming and can transform a DStream.
• Ability to write advanced UDFs, Serdes, input-loaders, log analysis, how the logical operators map to the lower, level physical implementation
• Setup and leverage output from Ganglia, Nagios, Ambari, Cloudera Manager, etc.
• Able to link Kerberos KDC to a backing LDAP or Active Directory authentication provider
• Advanced understanding of access-driven key design, appropriate denomalization, use of co-locating records of differing schemas in a single table etc.
• Understanding of best practices for Hive schemas. Denormalization, partitioning and bucketing, file formats.
• Able to create, write to and read from Kafka topic. Understanding of key partitioning (just how it works), able to maintain an offset in the topic for consistent reading

Job Abilities:

Must be able to sit for long periods of time working on computers. Must be able to travel to client sites at least 50% of the time. Must be able to interact and communicate with the client in meetings. Must be able to write programming code in applicable languages. Must be able to write project documentation in English.

Education:

Bachelor's Degree or foreign equivalent in Computer Science or related technical field followed by six (6) years of progressively responsible professional experience programming in Java, Python or C/C++. Experience with production software development lifecycle. Experience with Linux, SQL, relational database design and methods for efficiently retrieving data. Experience building complex and non-interactive systems (batch, distributed, etc.).                  

OR

Master's Degree or foreign equivalent in Computer Science or related technical field. Four (4) years of experience programming in Java, Python or C/C++. Experience with production software development lifecycle. Experience with Linux, SQL, relational database design and methods for efficiently retrieving 

CountryEEOText_Description: Why We Think You’ll Love Teradata We prioritize a people-first culture because we know our people are at the very heart of our success. We embrace a flexible work model because we trust our people to make decisions about how, when, and where they work. We focus on well-being because we care about our people and their ability to thrive both personally and professionally. We are an anti-racist company because our dedication to Diversity, Equity, and Inclusion is more than a statement. It is a deep commitment to doing the work to foster an equitable environment that celebrates people for all of who they are.

City: Tokyo

State: Tokyo

Community / Marketing Title: Senior Data Engineer

Job Category: Consulting

Company Profile:

Our Company

At Teradata, we believe that people thrive when empowered with better information. That’s why we built the most complete cloud analytics and data platform for AI. By delivering harmonized data, trusted AI, and faster innovation, we uplift and empower our customers—and our customers’ customers—to make better, more confident decisions. The world’s top companies across every major industry trust Teradata to improve business performance, enrich customer experiences, and fully integrate data across the enterprise.

LinkedIn Remote:

Location_formattedLocationLong: Tokyo, Tokyo JP