Bestjobs Philippines

Don't miss any news or updates from BestJobs

Not now Allow

HR Network Inc.

Verified Employer

This seal certifies that the data and activity of this company have been meticulously verified by Bestjobs

56 reviews

Data Engineer

Pasig, National Capital Region ·  Yesterday (updated)

  • Description

  • Minimum Qualifications:

    • Experience in distributed architecture and big data technologies / systems such as but not limited to the following:

    o Informatica or Talend

    o Cloudera

    o Hadoop Files System (HDFS)

    o Processing frameworks MapReduce

    o Spark

    o Resource Allocator/YARN

    • Knowledgeable in designing sustainable data architecture for Big data systems. That is balance partitioning with respect to target block sizes of the HDFS, compaction techniques, bucketing etc.

    • Experience in managing Small Files in HDFS, Storage formats such as columnar format (Parquet and ORC) and Transmit formats (Avro)

    • Knowledgeable on handling semi-structured data formats – JSON, XML, HTML etc.

    • Experienced in using various ETL querying tools for distributed systems such as Hive, Impala and other SQL clients (RDBMS, Phyton, Scala etc,)

    • Ability to handle and understand data transformation requirements to suit business needs.

    • Experience in handling tools for data transformation and/or ETL through tool-based and code- based development – ex. Informatica or Talend for tool based / Spark or MapReduce framework in Python or Scala/Java for code-based development.

    • Knowledgeable in resource allocation and proper resource sizing for distributed jobs. Ex. Spark executor and driver sizes

    • Background on data warehousing, master data management and reporting technologies.

    • Experience in handling streaming technologies ex. Kafka. Understands concepts of Kafka topics, consumer groups and producers and Flume.

    • Required Technical Skills:

    o Experience in any of the following languages Python, Java, Scala

    o Experience in Talend, Informatica, Ab Initio and other ETL GUI based tools

    o Strong Systems Administration and SQL skills

    o Strong Spark API skills including proper resource allocation and optimizations

    o Strong Kafka Producer and Consumer API skills

    o Comfortable in working with Hive partitioning and bucketing

    Desirable Job requirements but not required:

    • Experienced in end-to-end DevOps techniques is a plus. Able to create Jenkins pipelines and create unit test cases for automated testing. Experience in Git version control is a plus.

    • Experience in Docker containers and container technologies is a plus.

    • Experience in Flink Streaming framework.

    • Experience in using Cloud Technologies (AWS RedShift, S3 etc.)

  • Requirements

  • Minimum education level: Bachelor´s Degree
  • Years of experience: 2
  • Language(s): English
  • Availability for travel: Yes
  • Availability for change of residence: Yes
  • People with disabilities: Yes

Similar jobs

Important company in the sector - National Capital Region, Taguig

Casual - Temporary contract - Negotiable -

IT Engineer - BGC

2 days ago

Full Time - Permanent contract - Negotiable -

Geodetic Engineer

4 days ago

Full Time - Other type of contract - Negotiable -

Home Based - Permanent contract - Negotiable -

Get new jobs on Facebook Messenger

Send to Messenger

Job summary

  • Data Engineer

  • Pasig, National Capital Region

  • Company

    HR Network Inc.
  • Type of contract

    Permanent contract

  • Work type

    Full Time

  • Apply