Data Engineer in India | 40 hours a week
Braintrust
This job is no longer accepting applications
See open jobs at Braintrust.See open jobs similar to "Data Engineer in India | 40 hours a week" SKALE Network.Data Science
India
Posted on Feb 6, 2025
Job Description
Needs to hire 3 Freelancers
Job Title: Data Engineer (Full-Time, Freelance)
Job Type: Full-Time (Freelance, 40 hours per week)
Location: Remote (Primarily IST, with occasional Pacific Time meetings)
Job Summary:
We are seeking a highly skilled Data Engineer to design, implement, and optimize robust data pipelines for structured, semi-structured, and unstructured data. The ideal candidate will have extensive experience in ETL processes, database architecture, distributed data processing, and cloud platforms (AWS, Azure, GCP). This role is a full-time freelance position, requiring 8-hour workdays as part of a Scrum team, with occasional meetings in Pacific Time but most work done in IST.
Key Responsibilities:
Data Pipeline & Architecture:
Design and architect scalable data pipelines for various data types.
Develop, manage, and optimize databases, including RDMS (MySQL, PostgreSQL), NoSQL (MongoDB), and data lakes (S3).
Implement efficient ETL processes using PySpark and Hadoop to transform and prepare data for analytics and AI use cases.
Optimize database performance, including query tuning, indexing, and caching strategies using Azure and GCP caching databases.
Cloud & Deployment:
Leverage AWS services, particularly S3 and other data storage tools.
Develop and maintain CI/CD pipelines, manage YML files, and work with GitHub for version control.
Use Docker for containerized deployment, with hands-on experience executing Docker commands for database and pipeline management.
Best Practices & Collaboration:
Ensure solutions adhere to best practices in system design, security, performance, and scalability.
Monitor, maintain, and troubleshoot database infrastructure to ensure high availability and performance.
Collaborate with engineering teams to design scalable solutions for large-scale data processing.
Stay updated on the latest database technologies and implement best practices for database design and management.
Qualifications:
4+ years of experience in database architecture and optimization.
Expertise in RDMS, NoSQL, and semi-structured databases (MySQL, PostgreSQL, MongoDB).
Proficiency in programming languages for database integration and optimization (Python preferred).
Strong knowledge of distributed data processing tools like PySpark and Hadoop.
Hands-on experience with AWS services for data storage and processing, including S3.
Strong familiarity with Redis for caching and query optimization.
Proven experience with Docker for containerized deployments and writing CI/CD pipelines using YML files.
Work Schedule:
40 hours per week (Full-Time Freelance).
Part of a Scrum team, working 8 hours per day.
Most work can be done in IST, but occasional meetings in Pacific Time may be required.
Needs to hire 3 Freelancers
Job Title: Data Engineer (Full-Time, Freelance)
Job Type: Full-Time (Freelance, 40 hours per week)
Location: Remote (Primarily IST, with occasional Pacific Time meetings)
Job Summary:
We are seeking a highly skilled Data Engineer to design, implement, and optimize robust data pipelines for structured, semi-structured, and unstructured data. The ideal candidate will have extensive experience in ETL processes, database architecture, distributed data processing, and cloud platforms (AWS, Azure, GCP). This role is a full-time freelance position, requiring 8-hour workdays as part of a Scrum team, with occasional meetings in Pacific Time but most work done in IST.
Key Responsibilities:
Data Pipeline & Architecture:
Design and architect scalable data pipelines for various data types.
Develop, manage, and optimize databases, including RDMS (MySQL, PostgreSQL), NoSQL (MongoDB), and data lakes (S3).
Implement efficient ETL processes using PySpark and Hadoop to transform and prepare data for analytics and AI use cases.
Optimize database performance, including query tuning, indexing, and caching strategies using Azure and GCP caching databases.
Cloud & Deployment:
Leverage AWS services, particularly S3 and other data storage tools.
Develop and maintain CI/CD pipelines, manage YML files, and work with GitHub for version control.
Use Docker for containerized deployment, with hands-on experience executing Docker commands for database and pipeline management.
Best Practices & Collaboration:
Ensure solutions adhere to best practices in system design, security, performance, and scalability.
Monitor, maintain, and troubleshoot database infrastructure to ensure high availability and performance.
Collaborate with engineering teams to design scalable solutions for large-scale data processing.
Stay updated on the latest database technologies and implement best practices for database design and management.
Qualifications:
4+ years of experience in database architecture and optimization.
Expertise in RDMS, NoSQL, and semi-structured databases (MySQL, PostgreSQL, MongoDB).
Proficiency in programming languages for database integration and optimization (Python preferred).
Strong knowledge of distributed data processing tools like PySpark and Hadoop.
Hands-on experience with AWS services for data storage and processing, including S3.
Strong familiarity with Redis for caching and query optimization.
Proven experience with Docker for containerized deployments and writing CI/CD pipelines using YML files.
Work Schedule:
40 hours per week (Full-Time Freelance).
Part of a Scrum team, working 8 hours per day.
Most work can be done in IST, but occasional meetings in Pacific Time may be required.
This job is no longer accepting applications
See open jobs at Braintrust.See open jobs similar to "Data Engineer in India | 40 hours a week" SKALE Network.