Lead Software Engineer (Big Data) Kuala Lumpur, Malaysia
Lead Software Engineer (Big Data) Description
Job #: 74609
EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.
We are looking for an experienced Lead Software Engineer (Big Data) to be placed in our Office in Bangsar South MY. This As a Lead Engineer will be roll-up the functional and technical aspect of Software Development with a big data technology in Project space to cloud services.
EPAM Systems, Inc. (EPAM) is a global product development, digital platform engineering, and digital and product design agency headquartered in the US. EPAM has been named 5 times in Forbes’ 25 Fastest Growing Public Tech Companies. Kuala Lumpur is the most recent location in which EPAM has acquired a new business. This is your opportunity to join an organization in start-up mode that plans to (at least) double in size every 2-3 years and the team that you are joining will be responsible for that.
DESCRIPTION
We are looking for an experienced Lead Software Engineer (Big Data) to be placed in our Office in Bangsar South MY. This As a Lead Engineer will be roll-up the functional and technical aspect of Software Development with a big data technology in Project space to cloud services.
EPAM Systems, Inc. (EPAM) is a global product development, digital platform engineering, and digital and product design agency headquartered in the US. EPAM has been named 5 times in Forbes’ 25 Fastest Growing Public Tech Companies. Kuala Lumpur is the most recent location in which EPAM has acquired a new business. This is your opportunity to join an organization in start-up mode that plans to (at least) double in size every 2-3 years and the team that you are joining will be responsible for that.
Responsibilities
- Lead, design and implement innovative analytical solution using Hadoop, NoSQL and other Big Data related technologies, evaluating new features and architecture in Cloud/ on premise/ Hybrid solutions
- Work with product and engineering teams to understand requirements, evaluate new features and architecture to help drive decisions
- Build collaborative partnerships with architects and key individuals within other functional groups
- Perform detailed analysis of business problems and technical environments and use this in designing quality technical solution
- Actively participate in code review and test solutions to ensure it meets best practice specifications
- Build and foster a high-performance engineering culture, mentor team members and provide team with the tools and motivation
- Write project documentation
Requirements
- Candidate must possess at least Bachelor’s Degree in preferably Computer Science/ Information Technology or equivalent
- Candidates with a minimum of 6 years solid experience in Big Data technologies and Enterprise Software Development
- Engineering experience and practice in Data Management, Data Storage, Data Visualization, Disaster Recovery, Integration, Operation, Security
- Strong experience building data ingestion pipelines (simulating Extract, Transform, Load workload), Data Warehouse or Database architecture
- Strong experience with data modelling; hands-on development experience with modern Big Data components
- Cloud: experience in designing, automation, provisioning, deploying and administering scalable, available and fault tolerant systems
- Good understanding of CI/CD principles and best practices
- Analytical approach to problem-solving with an ability to work at an abstract level and gain consensus; excellent interpersonal, leadership and communication skills
- Motivated, independent, efficient and able to handle several projects: work under pressure with a solid sense for setting priorities
- Ability to work in a fast-paced (start-up like) agile development environment
- Strong experience in high load and IoT Data Platform architectures and infrastructures
- Vast experience with Containers and Resource Management systems: Docker, Kubernetes, Yarn
- Experience in direct customer communications
- Experience in technology / team leading of data-oriented projects
- Solid skills in infrastructure troubleshooting, support and practical experience in performance tuning and optimization, bottleneck problem analysis
- Experienced in different business domains
- English proficiency
Technologies
- Programming Languages: Java/ Scala; Python; SQL; Bash
- Big Data stack: Hadoop, Yarn, HDFS, MapReduce, Hive, Spark, Kafka, Flume, Sqoop, Zookeeper
- NoSQL: Cassandra/ Hbase; MongoDB
- Queues and Stream processing: Kafka Streams; Spark Streaming; Event Hub; IOT Hub; Storage Queues; Service Bus; Stream Analytics
- Data Visualization: Tableau, QlikView
- ETL & Streaming Pipelines: Pentaho; Talend; Apache Oozie, Airflow, NiFi; Streamsets
- Operation: Cluster operation, Cluster planning
- Search: Solr, Elasticsearch/ELK
- InMemory: Ignite, Redis
- Solid Cloud experience with more than 2 leading Cloud providers (AWS/Azure/GCP): Storage; Compute; Networking; Identity and Security; NoSQL; RDBMS and Cubes; Big Data Processing; Queues and Stream Processing; Serverless; Data Analysis and Visualization; ML as a service (SageMaker; TensorFlow)
- Enterprise Design Patterns (ORM, Inversion of Control etc.)
- Development Methods (TDD, BDD, DDD)
- Version Control Systems (Git, SVN)
- Testing: Component/ Integration Testing, Unit testing (JUnit)
- Deep understanding of SQL queries, joins, stored procedures, relational schemas; SQL optimization
- Experience in various messaging systems, such as Kafka, RabbitMQ
- Rest, Thrift, GRPC, SOAP
- Build Systems: Maven, SBT, Ant, Gradle
- Docker, Kubernetes, Yarn
We offer
- Friendly team and enjoyable working environment
- Work-life balance and flexible schedule
- Online training library, mentoring, career development and potential partial grant of certification
- Unlimited access to LinkedIn learning solutions
- Referral bonuses
- Compensation for sick leave and paid time off
- Opportunities for self-realization
