JOB DESCRIPTION
Pay Range: $75-85/hr on W2
Team: Team is Client delivery platform and Cumulus - lots of data exchanges: AWS platform using Redshift as data warehouse - Dynamo of meta-data. They are working on evolving code for digital SAT products and need dynamic Senior Data engineer who will also design, build and write test automation.
Profile: Senior Data Engineer with SME in data analytics end AI work within our tech stack; 5+ years of experience.

Summary Data Engineer must be a skilled engineer with an unwavering passion for excellence in Software Engineering and drive to transform that passion into consistent excellence in product development. The candidate must be customer-obsessed, have strong analytical and communication skills, enjoy working with native AWS services, and thrive on solving challenging business problems. The candidate will partner with Product Owners/Architects, Developers, and other Software Engineers to help design, define, implement functional features, system level validation and verify features of product development. Qualified candidates will show an aptitude learning and implementation of advanced software engineering practices in Data Analytics, AWS, Microservices environment to solve complex problems with a mission to release high quality software that is resilient and optimally performing. Candidates will be engaged in both data and analytics reports development and quality engineering, testing and automation as a data engineer.

Responsibilities
Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions in full-stack development tools and technologies
Builds, tests, deploys and maintains production code for complex transactional applications using event driven microservices, node.js, Python languages. Incorporates Cloud technologies on new application development to include micro-services, SQL and AWS services such as lambda, S3, SQS, SNS, DynamoDB, Quicksight, AWS Redshift, Opensearch.

Perform unit tests and conduct reviews with other team members to make sure your code is rigorously designed, elegantly coded, and effectively tuned for performance
Required Skills
• Expert in writing and optimizing complex SQL queries for data extraction, transformation, and reporting (e.g., SELECT, JOIN, GROUP BY, HAVING, WINDOW functions).
• Expert integrating Redshift Serverless with AWS services such as S3, Glue, Lambda, Athena, and Kinesis BI tools like Tableau, Quicksight, or Looker for real-time analytics and dashboards
• Expert in building serverless ETL pipelines with Lambda to automate data ingestion and transformation into Redshift.
• Experience in integrating Redshift Serverless with
• Experience of NodeJS and Python
• Working knowledge of any AI technologies
• Experience in working with AWS services, deploying applications to AWS
• Exposure to DevOps tools and automation
• Effective communication skills are a must along with a strong customer service orientation, and the ability to clearly discern client needs.

DETAILED LIST OF JOB DUTIES AND RESPONSIBILITIES:    Involve in Design and development of cloud-based data management systems.
Proficiency and practical experience in Design and implementation of data pipelines in both centralized and decentralized Data architectures.
Ensures established data security policies and standards are followed while designing data solution for cloud-based platforms.
Collaborates with cross-functional teams to architect highly scalable, highly available, and highly performant cloud data solutions.
Evaluates and recommends new cloud products and services for efficient data curation, storage and processing.
Monitors the performance of cloud data solutions and strategizes optimization measures.
Ensures compliance with data governance policies and best practices in the cloud.
Implements data management and data governance processes for cloud data.
Conducts thorough analysis and documentation of cloud data engineering strategies and designs.
Establish regular overall system performance assessment processes and action plans. Expertise in Data Integration leveraging Informatica (IICS), Snowflake, SQL and Python code to solve complex use cases
Provide direction for the design and development of the data layers including review of ETL requirements.
Capable of leading team design sessions and facilitating decisions.
Ensure timely delivery of solutions meeting requirements and expectations

QUALIFICATIONS:  Minimum requirements to submit candidates   5+ years of experience in Design and development of Data pipelines, Cloud Data Architecture, Data warehousing, Data modeling, ETL, enterprise and business information system architectures, and system engineering
Thorough knowledge and Implementation experience of data warehouse and Enterprise Data architecture principles, specifically Decentralized data architectures like Data Mesh.
3-5 years of experience developing Cloud-based (AWS) data solutions.
5+ years of Cloud database platform experience such as Snowflake.
Experience in creating and maintaining end-to-end data pipelines.
Strong communication skills comfortable explaining/proposing solutions to technical stakeholders, vendors, and business consumers.
Proficiency with Unix, Linux, and shell scripting
Experience working with a large, multi-terabyte enterprise data platform in Cloud.
Clear understanding on system administration, database administration, storage architectures, backup & recovery, and networking.
3-5 years of experience in Informatica (IICS) ETL, GitHub(CI/CD) and Airflow (Orchestration) experience.
Experience working with highly sensitive health care information and familiar with HIP nd other data privacy controls.

PREFERRED CERTIFICATIONS AND EXPERIENCE:     Amazon Web Services (AWS) Certification or Cloud Data Engineer Certification Informatica (IICS) and Airflow (Orchestration) experience
Snowflake certification
Cognos and/or Tableau reporting experience

DETAILED LIST OF JOB DUTIES AND RESPONSIBILITIES:<ul> <li>Involve in Design and development of cloud-based data management systems. </li> <li>Proficiency and practical experience in Design and implementation of data pipelines in both centralized and decentralized Data architectures. </li> <li>Ensures established data security policies and standards are followed while designing data solution for cloud-based platforms. </li> <li>Collaborates with cross-functional teams to architect highly scalable, highly available, and highly performant cloud data solutions. </li> <li>Evaluates and recommends new cloud products and services for efficient data curation, storage and processing. </li> <li>Monitors the performance of cloud data solutions and strategizes optimization measures. </li> <li>Ensures compliance with data governance policies and best practices in the cloud. </li> <li>Implements data management and data governance processes for cloud data. </li> <li>Conducts thorough analysis and documentation of cloud data engineering strategies and designs. </li> <li>Establish regular overall system performance assessment processes and action plans. Expertise in Data Integration leveraging Informatica (IICS), Snowflake, SQL and Python code to solve complex use cases </li> <li>Provide direction for the design and development of the data layers including review of ETL requirements. </li> <li>Capable of leading team design sessions and facilitating decisions. </li> <li>Ensure timely delivery of solutions meeting requirements and expectations </li></ul> QUALIFICATIONS: Minimum requirements to submit candidates<ul> <li>5+ years of experience in Design and development of Data pipelines, Cloud Data Architecture, Data warehousing, Data modeling, ETL, enterprise and business information system architectures, and system engineering </li> <li>Thorough knowledge and Implementation experience of data warehouse and Enterprise Data architecture principles, specifically Decentralized data architectures like Data Mesh.</li> <li>3-5 years of experience developing Cloud-based (AWS) data solutions.</li> <li>5+ years of Cloud database platform experience such as Snowflake. </li> <li>Experience in creating and maintaining end-to-end data pipelines. </li> <li>Strong communication skills comfortable explaining/proposing solutions to technical stakeholders, vendors, and business consumers. </li> <li>Proficiency with Unix, Linux, and shell scripting </li> <li>Experience working with a large, multi-terabyte enterprise data platform in Cloud. </li> <li>Clear understanding on system administration, database administration, storage architectures, backup & recovery, and networking. </li> <li>3-5 years of experience in Informatica (IICS) ETL, GitHub(CI/CD) and Airflow (Orchestration) experience.</li> <li>Experience working with highly sensitive health care information and familiar with HIP nd other data privacy controls. </li></ul> PREFERRED CERTIFICATIONS AND EXPERIENCE: <ul> <li>Amazon Web Services (AWS) Certification or Cloud Data Engineer Certification Informatica (IICS) and Airflow (Orchestration) experience </li> <li>Snowflake certification </li> <li>Cognos and/or Tableau reporting experience</li></ul>

JOB DESCRIPTION
Pay Range : $65-75/hr on W2

Team: Interflow /DAP Reporting: DAP Reporting is responsible for reports, analytics data on the digital test administration. Interflow works on generating reports for all the digital test administration. These reports are used to understand the health of the administration and report stats for leadership. This team is responsible for providing live reports to check on the status of the exams and provide canned post admin reports to analyze the admins in detail.
Profile: Data Analyst will work closely with Architects and leads in the Interflow team, key business stakeholders including program staff to support the organizations effort in the areas of data Analysis and information management. Role should have strong experience in writing super complex SQL queries to extract data for reports. Candidate should have experience working with large set of data. These reports are very critical for the company, and we are looking for an expert who is capable of handling, consuming and extracting large sets of data for analysis and reports.

We use Tableau and Quicksight for Data Analysis; Apache Spark for Big Data Processing; and they should have used at least one AI tool: Amazon Q; Google Cloud AI; Google Cloud Smart Analytics; Tableau AI
Responsibilities
Expertise to understand complex queries, proof test, and tweak for the intended use. Critical analysis skills to extract data and analyze the data for inconsistencies and issues.
Good hands-on experience of working with python.
Must have led implementation of any market popular AI tools for analyzing the data and creating trend reports.
Good analytical skills and experience in handling large sets of data
Demonstrate the ability to access, trace, analyze, and remediate data issues leveraging SQL and applicable AWS tools such as Athena and Redshift.
Strong SQL skills and ability to work with different data sources.
Automate the data extract pull and able to visualize these on charts/graphs/Tableau
Gather, analyze, and interpret data to solve specific use cases. Develop and participate in the review of requirements, data mappings, database specifications documentation and other artifacts and clearly communicate to both the business and technical team members the interpretation of the data.
Design and manage information systems, conduct analysis, and generate accurate and comprehensive reports on operational data.
Very strong in documentation of data artifacts, design and reports details.
Support Architects, Leads and product owners to ensure that all aspects of the information analysis and requirements gathering process are completed with the highest degree of accuracy and quality, including the development and socialization of key project artifacts.
Support ad hoc data requests as required as part of DevOps team.
Review, extract, and analyze data and information on system processes and procedures.
Experience developing testing strategies and identifying comprehensive test scenarios based on business requirements.
Demonstrated experience with AWS Lambda, NodeJS, SNS/SQS, S3, IAM, CloudWatch, RDS, DynamoDB, Redshift and Athena.
Willingness and ability to perform in multiple roles on a team, including testing and production support in addition to data analysis.
Strong ability to understand and internalize the big picture and broader implications.
Communicate and understand business goals and requirements, and work to create data solutions that add value to the business.
Support team with functional, regression and end to end testing efforts with a focus on flow of data ensuring that features deliver the expected functionality with high quality.
Self-starter with ability to set priorities, work independently and attain goals.
Develop data-driven reporting solutions from concept to deployment.
Focus on designing and developing effective user-friendly reporting assets including interfaces and outputs.
Cultivate and maintain knowledge of *** products, data systems and information assets.
Work with a highly proficient team using Tableau/Tableau Server, SQL, Relational/Columnar Databases, Data Modeling, MS Office
Develop scripts to extract data in Athena, Timestream and Redshift.
Exhibit good communication and practical decision-making skills, a believer in good feedback and documentation.
About You
BA/BS required (major in an analytical field desired)
A minimum of 5 to 10+ years work-related experience.
Experience in hands-on development of reporting applications for the Web in a professional environment
Passion for new technology and adapting to the latest tools in data analysis and reporting.
The Ethos of continuous improvement and interest in learning new things.
More about you
Strong analytical thinking and structured problem-solving ability
Ability to handle multiple projects and assignments simultaneously and effectively in a cross-functional team environment.
Versed on the agile methodology and best practices.
Excellent interpersonal and collaboration skill with the ability to work with a diverse set of colleagues, across functions, from different organizations, disciplines, etc.
Self-starter, ability to set priorities, work independently and attain goals

Role: Senior Data Engineer (Apache Flink Expert)Duration : PermanentLocation: Madrid, Spain (Hybrid) Description: We are seeking a highly skilled Senior Data Engineer with more than 10 years of experience to join our team. The ideal candidate will have extensive experience in Apache Flink and other stream processing technologies, along with a strong background in designing and building large-scale data pipelines within the banking and financial services industry. You will be responsible for driving the architecture, development, and optimization of data processing systems, ensuring high-quality, real-time data solutions to support business intelligence, analytics, and machine learning initiatives. Requirements:<ul><li>Experience:</li><li>10+ years of experience as a Data Engineer or in a similar role, with at least 5+ years of experience working with Apache Flink and related stream processing tools.</li><li>In-depth experience in designing, developing, and optimizing real-time data pipelines for banking, financial services, or similarly regulated industries.</li><li>Language: Fluent Spanish & English are required</li><li>Apache Flink Expertise: Extensive hands-on experience with Apache Flink, including stateful processing, event time handling, windowing, and scaling Flink applications.</li><li>Big Data Technologies: Proficiency with other big data frameworks and tools like Apache Kafka, Hadoop, Spark, Hive, ElasticSearch, and Cassandra.</li><li>Data Integration & ETL: Solid understanding of ETL/ELT processes, data integration, and the ability to work with both real-time and batch data processing.</li><li>SQL & NoSQL: Strong experience with SQL and database design. Familiarity with NoSQL databases like Cassandra, HBase, or MongoDB is a plus.</li><li>Cloud Platforms: Experience with cloud services such as AWS, Azure, or Google Cloud for hosting data pipelines and scalable storage.</li><li>Security & Compliance: Expertise in implementing security controls for data privacy and compliance with financial industry standards (e.g., PCI-DSS, GDPR).</li><li>Programming Skills: Proficiency in Java or Scala for building data processing applications. Knowledge of Python is a plus.</li><li>Problem-Solving: Strong problem-solving and troubleshooting skills, with a proven ability to work in high-pressure, fast-paced environments.</li><li>Collaboration: Excellent communication and teamwork skills, with experience working in cross-functional teams and mentoring junior engineers.</li></ul>Preferred Qualifications:<ul><li>Machine Learning Integration</li><li>Cloud Certifications</li><li>Advanced Data Technologies</li><li>Financial Services Regulations</li></ul>

Data Engineer Lancesoft, Inc. Jobs

Search for Job title, position, company and/or location…

Senior Data Engineer
March 9, 2025

Ehs Cloud Data Platform Senior Data Engineer
March 3, 2025

Ehs Cloud Data Platform Senior Data Engineer
February 25, 2025

Senior Data Engineer
February 25, 2025

Senior Data Engineer
February 15, 2025

Quick filters

Top companies

Data Engineer Lancesoft, Inc. Jobs

Search for Job title, position, company and/or location…

Senior Data EngineerMarch 9, 2025

Ehs Cloud Data Platform Senior Data EngineerMarch 3, 2025

Ehs Cloud Data Platform Senior Data EngineerFebruary 25, 2025

Senior Data EngineerFebruary 25, 2025

Senior Data EngineerFebruary 15, 2025