Position Details:Client: Banking & Finance Role Title: Senior Data Scientist / EngineerLocation: McLean, VA 22102 (Hybrid - Tue to Thu Onsite)Duration: 6 Months Possible Extension & ConversionSchedule: Mon-Fri : Basic business hoursPay Range: $78/hr. - $84/hr. Job Description:Enterprise Business Technology Office is looking for a Data Engineer Senior who enjoys data and building data platforms.Our ideal candidate is passionate about data management and partners closely with various stakeholders to influence decisions with data in making key business decisions in a timely manner.You will be part of a group that supports a variety of internal customers.You will analyze, design, and develop data storage and data analytic solutions using data lake patterns, that help our customers run more effective operations and make better business decisions.Our Impact:We build various data governance solutions which run as cohesive unit within the overall data platform strategy.We bring pioneering technology to bear on business challenges to delight and excite our customers.We collaborate with other IT teams to advance our technical capabilities.We live to automate and optimize processes and technologies.Your Impact:As a Senior Data Engineer, you are expected to work under limited direction and independently determine and develop approaches to solution. Understand requirements and design for the build and deployment.Scrum/agile based development and maintain efficient data pipelines and ETL process for large-scale data integration.Collaborate with data analysts and other engineer to define data needs and build solution that drive business insights.Develop, test, and deploy code for data extraction, transformation and loading processes (ETL/ELT).Develop and maintain automated test scripts using Pytest to ensure high-quality software releases.You will be a mentor and bring up-to-speed junior resources and consultants on technologies/products that the team works on and Represent and Demonstrate your team's work to customers.Will demonstrate your coaching and mentoring skills to help teams continuously grow and improve in delivering high quality software solutions.Support line of business areas with more advanced statistical and quantitative analysis. Ability to run models, look for exceptions and take corrective actions where necessary.Use technology tools to conduct analysis; apply techniques such as SQL querying and macro development to extract data for populating models.Release Support: Work closely with internal/external teams to help with the release process, ensuring smooth and error-free releases to production.Interact directly with customers to gather feedback, troubleshoot issues, and ensure that the product meets their expectations.Qualifications:Bachelor's degree in computer science or related discipline; advanced Studies/ Degree preferred.5-7 years of experience in data engineering with strong proficiency in Python programing language (Nice to have: Scala, Java).Strong proficiency in spark SQL, SQL, gremlin, GraphQL and database management (Snowflake cloud-base warehousing).Strong experience with data processing frameworks, DataFrames, Apache Spark, Graph DBsExperience with cloud platform AWS (EMR, EKS, Lambda).Experience writing statistical and/or optimization programs to develop models and algorithms.Programming languages may include but are not limited to Python, RSolid understanding of SDLC practices including development, testing, and release management.Experience with version control systems such as Git, BitBucketExperience with RESTful API design and development.Familiarity with PySpark for large-scale data processing and analysis.Keys to Success in this Role:Ability to use data to help inform strategy and direction.Ability to translate complex statistical models and methodologies into simple business terms.Help with pytests, testing frameworks, SDLC expertise.Help with data modeling, data engineering efforts, proper python expertiseInteract with customers and stay on top of issues, understand our platform, and debug issues.Help with talend, control-m, APIs.Hands-on help with our team release, customer release etc.Pyspark, spark, sql knowledge.Strong leadership, planning and communication skills.

Role: Data EngineerLocation: Mumbai and Bengaluru(Onsite) You will have the following responsibilities:<ul><li>Design Analyse relevant internally and externally sourced data (raw data) to generate BI and Advanced Analytics datasets based on your stakeholders’ requirements</li><li>Design data pipelines to curate sourced data into the inhouse data warehouse</li><li>Design data marts to facilitate dataset consumption out of the inhouse data warehouse by business and IT internal stakeholders</li><li>Design data model changes that align with the inhouse data warehouse standards</li><li>Define migration execution activities to move data from existing database solutions to the inhouse data warehouse Engineer</li><li>Regular housekeeping of raw data and data stored in the inhouse data warehouse</li><li>Build and maintenance of data pipelines and data platforms</li><li>Build data solution prototypes</li><li>Explore ways to enhance data quality and reliability</li><li>Identify and realize opportunities to acquire better data (raw data) </li><li>Develop analytical tooling to better support BI and Advanced Data Analytics activities</li><li>Execute data migration from existing databases to the inhouse data warehouse</li><li>Promote and champion data engineering standards and best-in-class methodology</li></ul>You will have the following qualifications:<ul><li>Bachelor’s or master’s degree in computer science, Information Technology, Engineering or related quantitative discipline</li><li>AWS Certified Data Engineer – Associate</li><li>Snowflake SnowPro Core Certification</li><li>4+ years of experience in data engineering or relevant working experience in a similar role, preferably in the financial industry</li><li>Strong understanding or practical experience of at least one common Enterprise Agile Framework e.g., Kanban, SAFe, SCRUM, etc.</li><li>Strong understanding of ETL, data warehouse and BI and Advanced Data Analytics concepts</li><li>Deep knowledge of cloud-enabled technologies – AWS RDS and AWS Fargate, etc.</li><li>Experience with databases and data warehouses - Snowflake, PostgreSQL, MS SQL</li><li>Strong programming skills with advanced knowledge of Java and/or Python</li><li>Experience in building data models</li><li>Strong critical-thinking, analytical and problem-solving skills</li><li>Excellent communicator with team-oriented approach</li></ul>

Position Details:Client: Banking &amp; Finance Role Title: Senior Data Scientist / EngineerLocation: McLean, VA 22102 (Hybrid - Tue to Thu Onsite)Duration: 6 Months Possible Extension &amp; ConversionSchedule: Mon-Fri : Basic business hoursPay Range: $78/hr. - $84/hr. Job Description:Enterprise Business Technology Office is looking for a Data Engineer Senior who enjoys data and building data platforms.Our ideal candidate is passionate about data management and partners closely with various stakeholders to influence decisions with data in making key business decisions in a timely manner.You will be part of a group that supports a variety of internal customers.You will analyze, design, and develop data storage and data analytic solutions using data lake patterns, that help our customers run more effective operations and make better business decisions.Our Impact:We build various data governance solutions which run as cohesive unit within the overall data platform strategy.We bring pioneering technology to bear on business challenges to delight and excite our customers.We collaborate with other IT teams to advance our technical capabilities.We live to automate and optimize processes and technologies.Your Impact:As a Senior Data Engineer, you are expected to work under limited direction and independently determine and develop approaches to solution. Understand requirements and design for the build and deployment.Scrum/agile based development and maintain efficient data pipelines and ETL process for large-scale data integration.Collaborate with data analysts and other engineer to define data needs and build solution that drive business insights.Develop, test, and deploy code for data extraction, transformation and loading processes (ETL/ELT).Develop and maintain automated test scripts using Pytest to ensure high-quality software releases.You will be a mentor and bring up-to-speed junior resources and consultants on technologies/products that the team works on and Represent and Demonstrate your team's work to customers.Will demonstrate your coaching and mentoring skills to help teams continuously grow and improve in delivering high quality software solutions.Support line of business areas with more advanced statistical and quantitative analysis. Ability to run models, look for exceptions and take corrective actions where necessary.Use technology tools to conduct analysis; apply techniques such as SQL querying and macro development to extract data for populating models.Release Support: Work closely with internal/external teams to help with the release process, ensuring smooth and error-free releases to production.Interact directly with customers to gather feedback, troubleshoot issues, and ensure that the product meets their expectations.Qualifications:Bachelor's degree in computer science or related discipline; advanced Studies/ Degree preferred.5-7 years of experience in data engineering with strong proficiency in Python programing language (Nice to have: Scala, Java).Strong proficiency in spark SQL, SQL, gremlin, GraphQL and database management (Snowflake cloud-base warehousing).Strong experience with data processing frameworks, DataFrames, Apache Spark, Graph DBsExperience with cloud platform AWS (EMR, EKS, Lambda).Experience writing statistical and/or optimization programs to develop models and algorithms.Programming languages may include but are not limited to Python, RSolid understanding of SDLC practices including development, testing, and release management.Experience with version control systems such as Git, BitBucketExperience with RESTful API design and development.Familiarity with PySpark for large-scale data processing and analysis.Keys to Success in this Role:Ability to use data to help inform strategy and direction.Ability to translate complex statistical models and methodologies into simple business terms.Help with pytests, testing frameworks, SDLC expertise.Help with data modeling, data engineering efforts, proper python expertiseInteract with customers and stay on top of issues, understand our platform, and debug issues.Help with talend, control-m, APIs.Hands-on help with our team release, customer release etc.Pyspark, spark, sql knowledge.Strong leadership, planning and communication skills.

Key Responsibilities<ul><li>Design, develop, and maintain scalable data pipelines and ETL processes to support data integration and analytics.</li><li>Collaborate with cross-functional teams to gather data requirements and deliver effective solutions.</li><li>Implement and manage data warehouse solutions using Snowflake.</li><li>Develop and optimize data models to ensure high performance and scalability.</li><li>Ensure data accuracy, integrity, and security across all data pipelines and storage.</li><li>Monitor and troubleshoot data pipelines to ensure seamless data flow and availability.</li><li>Leverage AWS services to build and deploy cloud-based data solutions.</li><li>Document data processes, architectures, and pipelines.</li></ul> Essential Skills & Certifications<ul><li>AWS Certified Data Engineer – Associate (Mandatory)</li><li>Snowflake SnowPro Core Certification (Mandatory)</li><li>Strong expertise in AWS services, such as:</li><li>AWS Glue</li><li>AWS Lambda</li><li>Amazon Redshift</li><li>Amazon S3</li><li>AWS Data Pipeline</li><li>Proficiency in Snowflake for data warehousing and analytics.</li><li>Hands-on experience with SQL and database optimization.</li><li>Knowledge of ETL/ELT frameworks and data integration tools.</li><li>Experience with data visualization tools (e.g., Tableau, Power BI) is a plus.</li><li>Familiarity with scripting languages like Python or Scala.</li></ul> Qualifications<ul><li>Bachelor’s degree in Computer Science, Data Engineering, or a related field.</li><li>4-7 years of experience in data engineering roles.</li><li>Strong analytical and problem-solving skills with attention to detail.</li><li>Excellent communication skills and the ability to collaborate with technical and non-technical stakeholders.</li></ul> Preferred Skills<ul><li>Experience with data governance and data security best practices.</li><li>Knowledge of DevOps practices and CI/CD pipelines for data solutions.</li><li>Familiarity with big data tools and technologies like Apache Spark, Kafka, or Hadoop.</li></ul>

Role: Data EngineerProffered Client Location: Mumbai and Bengaluru(Onside) You will have the following responsibilities:<ul><li>Design Analyse relevant internally and externally sourced data (raw data) to generate BI and Advanced Analytics datasets based on your stakeholders’ requirements</li><li>Design data pipelines to curate sourced data into the inhouse data warehouse</li><li>Design data marts to facilitate dataset consumption out of the inhouse data warehouse by business and IT internal stakeholders</li><li>Design data model changes that align with the inhouse data warehouse standards</li><li>Define migration execution activities to move data from existing database solutions to the inhouse data warehouse Engineer</li><li>Regular housekeeping of raw data and data stored in the inhouse data warehouse</li><li>Build and maintenance of data pipelines and data platforms</li><li>Build data solution prototypes</li><li>Explore ways to enhance data quality and reliability</li><li>Identify and realize opportunities to acquire better data (raw data)</li><li>Develop analytical tooling to better support BI and Advanced Data Analytics activities</li><li>Execute data migration from existing databases to the inhouse data warehouse</li><li>Promote and champion data engineering standards and best-in-class methodology</li></ul> You will have the following qualifications:<ul><li>Bachelor’s or master’s degree in computer science, Information Technology, Engineering or related quantitative discipline</li><li>AWS Certified Data Engineer – Associate</li><li>Snowflake SnowPro Core Certification</li><li>4+ years of experience in data engineering or relevant working experience in a similar role, preferably in the financial industry</li><li>Strong understanding or practical experience of at least one common Enterprise Agile Framework e.g., Kanban, SAFe, SCRUM, etc.</li><li>Strong understanding of ETL, data warehouse and BI and Advanced Data Analytics concepts</li><li>Deep knowledge of cloud-enabled technologies – AWS RDS and AWS Fargate, etc.</li><li>Experience with databases and data warehouses - Snowflake, PostgreSQL, MS SQL</li><li>Strong programming skills with advanced knowledge of Java and/or Python</li><li>Experience in building data models</li><li>Strong critical-thinking, analytical and problem-solving skills</li><li>Excellent communicator with team-oriented approach</li></ul> About Us:Collabera Digital is a new age digital engineering company delivering unique experiences, actionable insights, and intelligent products & platforms at the intersection of talent & technology. With a global presence spanning 11 countries throughout APAC and Europe, our network of over 25 global offices serves as the epicentre of digital innovation. We proudly cater to a prestigious clientele, counting Fortune 500 companies amongst our 300+ esteemed clients. With Collabera Digital (www.collaberadigital.com), you:<ul><li>Will get to work on numerous challenging and exciting projects on our various offerings including Salesforce, AI/Data Science, Generative AI/ML, Automation, Cloud Enterprise and Product/Platform Engineering. At Collabera Digital you have high chances of project extension or redeployment to other clients. Will have endless opportunities to learn new technologies through our Training arm – Cognixia. Additionally, you can also share CV of anyone you know. We have a referral policy in place.</li></ul>

<ul><li>You will have the following responsibilities:</li></ul>Design <ul><li>Analyse relevant internally and externally sourced data (raw data) to generate BI and Advanced Analytics datasets based on your stakeholders’ requirements</li><li>Design data pipelines to curate sourced data into the inhouse data warehouse</li><li>Design data marts to facilitate dataset consumption out of the inhouse data warehouse by business and IT internal stakeholders</li><li>Design data model changes that align with the inhouse data warehouse standards</li><li>Define migration execution activities to move data from existing database solutions to the inhouse data warehouse</li></ul>Engineer<ul><li>Regular housekeeping of raw data and data stored in the inhouse data warehouse</li><li>Build and maintenance of data pipelines and data platforms</li><li>Build data solution prototypes</li><li>Explore ways to enhance data quality and reliability</li><li>Identify and realize opportunities to acquire better data (raw data) </li><li>Develop analytical tooling to better support BI and Advanced Data Analytics activities</li><li>Execute data migration from existing databases to the inhouse data warehouse</li><li>Promote and champion data engineering standards and best-in-class methodology</li></ul>You will have the following qualifications:<ul><li>AWS Certified Data Engineer – Associate</li><li>Snowflake SnowPro Core Certification</li><li>4+ years of experience in data engineering or relevant working experience in a similar role, preferably in the financial industry</li><li>Strong understanding or practical experience of at least one common Enterprise Agile Framework e.g., Kanban, SAFe, SCRUM, etc.</li><li>Strong understanding of ETL, data warehouse and BI and Advanced Data Analytics concepts</li><li>Deep knowledge of cloud-enabled technologies – AWS RDS and AWS Fargate, etc.</li><li>Experience with databases and data warehouses - Snowflake, PostgreSQL, MS SQL</li><li>Strong programming skills with advanced knowledge of Java and/or Python</li><li>Experience in building data models</li></ul>

Description Home
Search Jobs
Job Description Big Data Engineer Contract: Charlotte, North Carolina, US Salary: $65.00 Per Hour Job Code: 354462 End Date: 2024-12-12
Days Left: 9 days, 2 hours left Apply Experience in MapR, Hadoop, Scala, Pyspark and Java
08- 10+ years of application development and implementation experience
Experience in Hadoop ecosystem tools for real-time batch data ingestion, processing and provisioning.
Experience delivering ETL, data warehouse and data analytics capabilities on big-data architecture such as Hadoop
Experience of Java or Python experience
Agile project methodologies
Job Requirement BIg Data
Hadoop
Mapr
Java
Spark
Reach Out to a Recruiter
Recruiter
Email
Phone
Jatin Deshmukh
jatin.deshmukh@collabera.com
9803814097 Apply Now

Title: Senior Data Engineer IICompany: Confidential until initial phone screeningLocation: Denver, CO | Hybrid - 4 days onsite, Fridays remoteSalary: $160,000 - $190,000 Base + 20% bonus + fully paid benefits for employees and dependents + unlimited PTO + 3% 401K match + product discounts + $30/day Grubhub allowance for in-office lunchFull-Time Direct Hire W2 Our client is a global leader in connected TV (CTV) and cross-screen advertising. They pride themselves on delivering state-of-the-art advertising solutions that integrate seamlessly with today's ever-evolving digital media landscape. The Opportunity:We are seeking a highly skilled and experienced individual with a focus on data engineering to join our client's team. You will play a pivotal role in designing and developing innovative features on their data pipeline end to end, which processes around a few petabytes of data on a daily basis. You will collaborate with cross-functional teams to drive technical innovation and deliver robust solutions that meet business needs. Responsibilities:<ul><li>Designing and developing resilient data pipelines for massive datasets</li><li>Developing extensible data models that promote accuracy, consistency, and quality</li><li>Optimizing code, processes, and infrastructure</li><li>Championing best practices</li><li>Mentoring those around you</li><li>Collaborating with all of your colleagues</li></ul> Mindset:<ul><li>You prefer data that is measured in petabytes</li><li>Some say you have a need for speed</li><li>Anything worth doing is worth doing well</li><li>You leave code better than you found it</li><li>You love coaching and helping the team succeed</li><li>Coding is a highlight of your day</li></ul> Qualifications:<ul><li>Data engineering experience</li><li>Code regularly in at least one of the following: Python, Java, Scala</li><li>Routinely use and optimize Spark / PySpark</li><li>Proficient in using Kafka or similar technology</li><li>Comfortable developing both streaming and batch jobs</li><li>Writing clear technical documentation</li></ul> Plusses:<ul><li>You’ve worked in Ad Tech</li><li>Experience designing systems to be compliant with CCPA, GDPR, or SOC2</li><li>Experience working securely with PII data</li><li>Experience with Databricks</li><li>Experience with Infrastructure as Code (IaC)</li></ul>

Data Engineer Collabera Jobs

Data Engineer
December 19, 2024

Data Engineer
December 18, 2024

Data Engineer
December 16, 2024

Data Engineer
December 12, 2024

Data Engineer
December 7, 2024

Data Engineer
December 6, 2024

Big Data Engineer
December 5, 2024

Senior Data Engineer Ii
November 24, 2024

Quick filters

Top companies

Data Engineer Collabera Jobs

Data EngineerDecember 19, 2024

Data EngineerDecember 18, 2024

Data EngineerDecember 16, 2024

Data EngineerDecember 12, 2024

Data EngineerDecember 7, 2024

Data EngineerDecember 6, 2024

Big Data EngineerDecember 5, 2024

Senior Data Engineer IiNovember 24, 2024