*Candidates must be local to the Washington D.C. metro area. About Infinitive: Infinitive is a data and AI consultancy that enables its clients to modernize, monetize and operationalize their data to create lasting and substantial value. . We possess deep industry and technology expertise to drive and sustain adoption of new capabilities. We match our people and personalities to our clients' culture while bringing the right mix of talent and skills to enable high return on investment. Infinitive has been named "Best Small Firms to Work For" by Consulting Magazine 6 times most recently in 2023. Infinitive has also been named a Washington Post "Top Workplace", Washington Business Journal "Best Places to Work", and Virginia Business "Best Places to Work." We are seeking a highly skilled and motivated Data Engineer to join our dynamic team. As a Data Engineer, you will play a crucial role in designing, developing, and maintaining our clients data infrastructure. Your expertise in Python, PySpark, ETL processes, CI/CD (Jenkins or GitHub), and experience with both streaming and batch workflows will be essential in ensuring the efficient flow and processing of data to support our clients. Responsibilities: Data Architecture and Design: <ul><li>Collaborate with cross-functional teams to understand data requirements and design robust data architecture solutions.</li><li>Develop data models and schema designs to optimize data storage and retrieval.</li></ul> ETL Development: <ul><li>Implement ETL processes to extract, transform, and load data from various sources.</li><li>Ensure data quality, integrity, and consistency throughout the ETL pipeline.</li></ul> Python and PySpark Development: <ul><li>Utilize your expertise in Python and PySpark to develop efficient data processing and analysis scripts.</li><li>Optimize code for performance and scalability, keeping up-to-date with the latest industry best practices.</li></ul> Data Integration: <ul><li>Integrate data from different systems and sources to provide a unified view for analytical purposes.</li><li>Collaborate with data scientists and analysts to implement solutions that meet their data integration needs.</li></ul> Streaming and Batch Workflows: <ul><li>Design and implement streaming workflows using PySpark Streaming or other relevant technologies.</li><li>Develop batch processing workflows for large-scale data processing and analysis.</li></ul> CI/CD Implementation: <ul><li>Implement and maintain continuous integration and continuous deployment (CI/CD) pipelines using Jenkins or GitHub Actions.</li><li>Automate testing, code deployment, and monitoring processes to ensure the reliability of data pipelines.</li></ul> Qualifications: <ul><li>Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.</li><li>Proven experience as a Data Engineer or similar role.</li><li>Strong programming skills in Python and expertise in PySpark for both batch and streaming data processing.</li><li>Hands-on experience with ETL tools and processes.</li><li>Familiarity with CI/CD tools such as Jenkins or GitHub Actions.</li><li>Solid understanding of data modeling, database design, and data warehousing concepts.</li><li>Excellent problem-solving and analytical skills.</li><li>Strong communication and collaboration skills.</li></ul> Preferred Skills: <ul><li>Knowledge of cloud platforms such as AWS, Azure, or Google Cloud.</li><li>Experience with version control systems (e.g., Git).</li><li>Familiarity with containerization and orchestration tools (e.g., Docker, Kubernetes).</li><li>Understanding of data security and privacy best practices.</li></ul> Applicants for employment in the U.S. must possess work authorization which does not require sponsorship by the employer for a visa. Infinitive is an Equal Opportunity Employer.

*Candidates must be local to the Washington D.C. metro area and possess work authorization which does not require sponsorship by the employer for a visa. About Infinitive: Infinitive is a data and AI consultancy that enables its clients to modernize, monetize and operationalize their data to create lasting and substantial value. . We possess deep industry and technology expertise to drive and sustain adoption of new capabilities. We match our people and personalities to our clients' culture while bringing the right mix of talent and skills to enable high return on investment. Infinitive has been named "Best Small Firms to Work For" by Consulting Magazine 6 times most recently in 2023. Infinitive has also been named a Washington Post "Top Workplace", Washington Business Journal "Best Places to Work", and Virginia Business "Best Places to Work." Job Summary: We are seeking a skilled and experienced Data Engineer to join our team. The ideal candidate will have a strong background in data engineering and be proficient in using Logstash and Databricks for data processing and analytics. This role is critical in helping us manage and optimize our clients data pipelines, ensuring the availability, reliability, and performance of our data infrastructure. Responsibilities: <ul><li>Design, develop, and maintain data pipelines using Logstash and Databricks to ingest, process, and transform data from various sources.</li><li>Collaborate with data scientists and analysts to understand their data requirements and ensure the data engineering infrastructure can support their needs.</li><li>Optimize data pipelines for efficiency, scalability, and performance to meet business objectives.</li><li>Troubleshoot and resolve issues related to data processing and pipeline failures.</li><li>Ensure data security and compliance with relevant data protection regulations.</li><li>Document data engineering processes and procedures for future reference and knowledge sharing.</li><li>Stay up-to-date with the latest trends and best practices in data engineering and cloud-based data processing technologies.</li></ul> Requirements: <ul><li>Bachelor's degree in computer science, information technology, or a related field. A master's degree is a plus.</li><li>Proven experience as a data engineer with a focus on data processing and integration.</li><li>Strong knowledge and hands-on experience with Elasticsearch.</li><li>Proficiency in Databricks for data analytics, ETL, and data engineering.</li><li>Familiarity with big data technologies such as Apache Spark is a plus.</li><li>Solid understanding of data warehousing concepts and relational databases.</li><li>Strong programming skills in languages such as Python, Java, or Scala.</li><li>Knowledge of data modeling and data warehouse design principles.</li><li>Excellent problem-solving and debugging skills.</li><li>Strong teamwork and communication skills to collaborate effectively with cross-functional teams.</li></ul> Preferred Qualifications: <ul><li>AWS Certified Data Analytics - Specialty or relevant certifications.</li><li>Experience working with data streaming technologies like Apache Kafka or Amazon Kinesis.</li><li>Knowledge of DevOps practices for CI/CD pipelines.</li><li>Familiarity with data governance and data quality best practices.</li><li>Experience with version control systems like Git.</li></ul> Infinitive is an Equal Opportunity Employer.

Data Engineer Jobs in Ashburn, VA

Data Engineer Pythonpysparkaws
November 13, 2024

Senior Data Engineer
November 9, 2024

Remote Sme Data Engineer
November 9, 2024

Data Engineer Pythonpysparkaws
November 9, 2024

Data Engineer Elasticlogstash Experience
November 9, 2024

Data Engineer Elasticlogstash Experience
November 5, 2024

Senior Data Engineer Databricks
November 4, 2024

Principal Data Engineer
November 2, 2024

Quick filters

Top companies

Data Engineer Jobs in Ashburn, VA

Data Engineer PythonpysparkawsNovember 13, 2024

Senior Data EngineerNovember 9, 2024

Remote Sme Data EngineerNovember 9, 2024

Data Engineer PythonpysparkawsNovember 9, 2024

Data Engineer Elasticlogstash ExperienceNovember 9, 2024

Data Engineer Elasticlogstash ExperienceNovember 5, 2024

Senior Data Engineer DatabricksNovember 4, 2024

Principal Data EngineerNovember 2, 2024