Key Responsibilities: Design, build, and manage Azure Data Factory pipelines for data ingestion. Handle both batch and near real-time data ingestion using Azure Service Bus. Integrate structured, semi-structured, and unstructured data sources. Utilize formats such as Parquet, Avro, JSON, etc., and manage loading into the Bronze layer. Build and manage data engineering pipelines using Databricks (PySpark, SparkSQL). Implement Spark-based data transformation jobs from Bronze to Silver and Gold layers. Establish and manage Data Quality Framework using Databricks and Data Factory. Set up data security using RBAC and ACLs to manage permissions on sensitive datasets. Implement automation frameworks for pipeline execution using Databricks Workflows. Monitor pipelines using Azure Monitor, set up alerts and ensure SLAs are met. Optimize storage in ADLS Gen2 and use Azure Key Vault to secure credentials. Optimize costs using Azure Cost Management. Technical Skills: Azure Data Factory for ETL/ELT operations. Databricks (PySpark, SparkSQL) for data transformations. Azure Data Lake Storage (ADLS Gen2). SQL Server, PostgreSQL, Cosmos DB, CRM, and ERP systems. Data Governance, RBAC, and ACLs for managing permissions. CI/CD tools: Azure DevOps, GitHub Actions, Terraform. Azure Monitoring, Alerting, and Logging tools. Public Partnerships is an Equal Opportunity Employer dedicated to celebrating diversity and intentionally creating a culture of inclusion. We believe that we work best when our employees feel empowered and accepted, and that starts by honoring each of our unique life experiences. At PPL, all aspects of employment regarding recruitment, hiring, training, promotion, compensation, benefits, transfers, layoffs, return from layoff, company-sponsored training, education, and social and recreational programs are based on merit, business needs, job requirements, and individual qualifications. We do not discriminate on the basis of race, color, religion or belief, national, social, or ethnic origin, sex, gender identity and/or expression, age, physical, mental, or sensory disability, sexual orientation, marital, civil union, or domestic partnership status, past or present military service, citizenship status, family medical history or genetic information, family or parental status, or any other status protected under federal, state, or local law. PPL will not tolerate discrimination or harassment based on any of these characteristics. PPL does not discriminate based on race, color, religion, or belief, national, social, or ethnic origin, sex, gender identity and/or expression, age, physical, mental, or sensory disability, sexual orientation, marital, civil union, or domestic partnership status, protected veteran status, citizenship status, family medical history or genetic information, family or parental status, or any other status protected under federal, state, or local law.

Alternate Locations: Work from Home Work Arrangement: Remote : Work at home employee residing outside of a commutable distance to an office location. Relocation assistance: is not available for this opportunity. Requisition : 73683 The Role at a Glance We are excited to bring on a Sr. Data Engineer/Sr. Analyst, Application Development to our Customer360 Master Data Management and Search Services teams within Lincolns Workplace Solutions organization. This position will provide subject matter expertise and direction on complex projects/initiatives related to the assigned area(s) of application development responsibility. The candidate must have experience leading teams within an operational organization by suggesting new and innovative ways of gaining data processing efficiencies and participating in architectural and modeling discussions. You will be responsible for maintaining and evolving the cloud-based enterprise data platform. To ensure the technology components of the platform are correctly configured and optimized to meet the functional, non-functional, and operational requirements. The ideal candidate will have experience securely moving and processing large volumes of data across on-premises and cloud environments. What you'll be doing Support the development and maintenance of ETL data pipelines in a state-of-the-art AWS environment, encompassing the full data pipeline lifecycle, including acquisition, transformation, management, export, and analytics. Gain expert-level understanding of the entire MDM and Search technology portfolio. Provide first-level support for production issues and complete RCA. Maintain best practices to facilitate optimized software development, continuous improvement/continuous delivery (CI/CD), and Observability. Analyzes, designs, debugs, integrates, and tests complex software application solutions for systems. Serves as a technical expert and provides complex technical guidance to junior team members. Provides direction to development teams in areas of project scheduling, technical direction and application development standards. Provides input on the development of project plans, test plans and implementation plans including determining estimates on time frames, needed resources and budgets. Maintains knowledge on current and emerging developments/trends for assigned area(s) of responsibility, assesses the impact, and collaborates with senior management to incorporate new trends and developments in current and future solutions. Investigates and resolves problems including providing support for production systems in a timely manner. Partners with internal stakeholders to ensure that systems are designed, developed, and implemented from a business point of view. What were looking for Must haves: Technically proficient in the following languages/frameworks, application platforms and developer tools (5-7 years of experience):: Amazon Web Services (EMR, EC2, Elastic Beanstalk, Lambda, Cloudwatch, SNS, SQS, Lambda, CloudWatch, Secret Manager, IAM, S3, RDS) Unix shell scripting Hadoop HDFS Rest API Java Nice-to-haves: Technically proficient in the following languages/frameworks, application platforms and developer tools: Apache Big Data tools (Spark, Hive, Sqoop, Pig) Python Presto SQL Informatica AutoSys DICE Application Deadline Applications for this position will be accepted through October 23, 2024, subject to earlier closure due to applicant volume. Whats it like to work here? At Lincoln Financial Group, we love what we do. We make meaningful contributions each and every day to empower our customers to take charge of their lives. Working alongside dedicated and talented colleagues, we build fulfilling careers and stronger communities through a company that values our unique perspectives, insights and contributions and invests in programs that empower each of us to take charge of our own future. Whats in it for you: Clearly defined career tracks and job levels, along with associated behaviors for each Lincoln leadership Attribute. Leadership development and virtual training opportunities PTO/parental leave Competitive 401K and employee benefits (https://hrdirectdocs.lfg.com/misc/HR/Recruiting/BenefitsResourcesGuide.pdf) Free financial counseling, health coaching and employee assistance program Tuition assistance program A leadership team that prioritizes your health and well-being; offering a remote work environment and flexible work hybrid situations Effective productivity/technology tools and training The pay range for this position is $90,300 - $164,700 with anticipated pay for new hires between the minimum and midpoint of the range and could vary above and below the listed range as permitted by applicable law. Pay is based on non-discriminatory factors including but not limited to work experience, education, location, licensure requirements, proficiency and qualifications required for the role. The base pay is just one component of Lincolns total rewards package for employees. In addition, the role may be eligible for the Annual Incentive Program, which is discretionary and based on the performance of the company, business unit and individual. Other rewards may include long-term incentives, sales incentives and Lincolns standard benefits package. About The Company Lincoln Financial Group helps people to plan, protect and retire with confidence. As of Dec. 31, 2023, approximately 17 million customers trust our guidance and solutions across four core businesses annuities, life insurance, group protection and retirement plan services. As of December 31, 2023, the company had $295 billion in end-of-period account balances, net of reinsurance. Headquartered in Radnor, Pa., Lincoln Financial Group is the marketing name for Lincoln National Corporation (NYSE: LNC) and its affiliates. Learn more at LincolnFinancial.com. Lincoln is committed to creating a diverse and inclusive (https://www.lfg.com/public/aboutus/companyoverview/diversityinclusion?audience_page_id1422918942386) environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. Follow us on Facebook (http://www.facebook.com/lincolnfinancialgroup) , X (https://mobile.twitter.com/lincolnfingroup) , LinkedIn (https://www.linkedin.com/company/4307?trkInfotas%3Alincolnfinancial%2Cidx%3A3-1-3&trktyah) , and Instagram (https://www.instagram.com/lincolnfingroup/) . For the latest company news, visit our newsroom (https://www.lincolnfinancial.com/public/aboutus/newsroom) . Be Aware of Fraudulent Recruiting Activities If you are interested in a career at Lincoln, we encourage you to review our current openings and apply on our website. Lincoln values the privacy and security of every applicant and urges all applicants to diligently protect their sensitive personal information from scams targeting job seekers. These scams can take many forms including fake employment applications, bogus interviews and falsified offer letters. Lincoln will not ask applicants to provide their social security numbers, date of birth, bank account information or other sensitive information in job applications. Additionally, our recruiters do not communicate with applicants through free e-mail accounts (Gmail, Yahoo, Hotmail) or conduct interviews utilizing video chat rooms. We will never ask applicants to provide payment during the hiring process or extend an offer without conducting a phone, live video or in-person interview. Please contact Lincoln's fraud team at fraudhotlinelfg.com if you encounter a recruiter or see a job opportunity that seems suspicious. Additional Information This position may be subject to Lincolns Political Contribution Policy. An offer of employment may be contingent upon disclosing to Lincoln the details of certain political contributions. Lincoln may decline to extend an offer or terminate employment for this role if it determines political contributions made could have an adverse impact on Lincolns current or future business interests, misrepresentations were made, or for failure to fully disclose applicable political contributions and or fundraising activities. Any unsolicited resumes or candidate profiles submitted through our web site or to personal e-mail accounts of employees of Lincoln Financial Group are considered property of Lincoln Financial Group and are not subject to payment of agency fees. Lincoln Financial Group (LFG) is an Equal Opportunity employer and, as such, is committed in policy and practice to recruit, hire, compensate, train and promote, in all job classifications, without regard to race, color, religion, sex (including pregnancy), age, national origin, disability, sexual orientation, gender identity and expression, Veteran status, or genetic information. Applicants are evaluated on the basis of job qualifications. If you are a person with a disability that impedes your ability to express your interest for a position through our online application process, or require TTY/TDD assistance, contact us by calling 260-455-2558. This Employer Participates in E-Verify. See the E-Verify (https://www.e-verify.gov) notices. Este Empleador Participa en E-Verify. Ver el E-Verify (https://www.e-verify.gov/es) avisos. Lincoln Financial Group ("LFG") is an Equal Opportunity employer and, as such, is committed in policy and practice to recruit, hire, compensate, train and promote, in all job classifications, without regard to race, color, religion, sex (including pregnancy), age, national origin, disability, sexual orientation, gender identity and expression, veterans status, or genetic information. Opportunities throughout LFG are available to employees and applicants and are evaluated on the basis of job qualifications. We have a drug free work environment and we perform pre-employment substance abuse testing.

Job ID Number R4040 Employment Type Full time Worksite Flexibility Remote Job Summary We are looking for a motivated Data Engineer ready to take us to the next level If you have 5 or more years of experience with Google Cloud Platform services, Hadoop and Hive, and are looking for your next career move, apply now. Job Description We are looking for a Data Engineer to development, and implement new systems, and major enhancements to existing systems, as well as provide production support for systems with high complexity. This position will be Full-Time and Remote as a 6-month contract with the ability to renew. Candidates must reside in and be legally authorized to work in the U.S. to be eligible for this role. What Youll Do Develop and implement new data systems to support business objectives Design and execute major enhancements to existing systems to improve performance and functionality Provide production support for systems with high complexity, ensuring minimal downtime and quick resolution of issues Collaborate with cross-functional teams to understand data requirements and deliver solutions that meet business needs Optimize and maintain data pipelines to ensure efficient data processing and storage Ensure data quality and integrity through rigorous testing and validation processes Stay up to date with the latest industry trends and technologies to continually improve our data engineering practices What You'll Need Required: Bachelors degree in computer science, Data Analytics or a related field At least five years of experience performing data engineering tasks, containing but not limited to the following experiences Data engineering tools and technologies, such as Apache Spark, Hadoop, and Hive as well as their APIs such as PySpark Cloud computing platforms, such as Amazon Web Services (AWS), Microsoft Azure, or Google Cloud Platform (GCP) Google Cloud Platform services (GCP), such as: Google Cloud Storage, Cloud Functions, Pub/Sub, Cloud Scheduler, Cloud Run, BigQuery, and Cloud SQL Experience with Google Cloud Platform APIs and SDKs Familiarity with cloud computing concepts, such as virtual machines, storage, and networking Familiarity with data modeling and database design Strong problem-solving and analytical skills Excellent communication and teamwork skills Preferred: Experience writing Python scripts for data engineering tasks Familiarity with Python libraries for data manipulation and analysis, such as NumPy, Pandas, and SciPy Experience with Hadoop and Hive Physical Demands Ability to safely and successfully perform the essential job functions consistent with the ADA and other federal, state, and local standards Sedentary work that involves sitting or remaining stationary most of the time with occasional need to move around the office to attend meetings, etc. Ability to conduct repetitive tasks on a computer, utilizing a mouse, keyboard, and monitor Must be able to communicate with customers/team members over the phone and in person Reasonable Accommodation Statement If you require a reasonable accommodation in completing this application, interviewing, completing any pre-employment testing, or otherwise participating in the employment selection process, please direct your inquiries to application.accommodationscai.io or (888) 824 8111. Equal Employment Opportunity Policy Statement It is the policy of CAI not to discriminate against any employee or applicant due to race, color, religion, sex, sexual orientation, gender identity, national origin, age, marital status, genetic information, disability or being a protected veteran. It is also the policy of CAI to take affirmative action to employ and to advance in employment, all persons regardless of race, color, religion, sex, sexual orientation, gender identity, national origin, age, marital status, genetic information, disability or being a protected veteran, and to base all employment decisions only on valid job requirements. This policy shall apply to all employment actions, including but not limited to recruitment, hiring, upgrading, promotion, transfer, demotion, layoff, recall, termination, rates of pay or other forms of compensation and selection for training, including apprenticeship, at all levels of employment. Employees and applicants of CAI will not be subject to harassment due to race, color, religion, sex, sexual orientation, gender identity, national origin, age, marital status, genetic information, disability or being a protected veteran. Additionally, retaliation, including intimidation, threats, or coercion, because an employee or applicant has objected to discrimination, engaged or may engage in filing a complaint, assisted in a review, investigation, or hearing or have otherwise sought to obtain their legal rights under any Federal, State, or local EEO law is prohibited.

<h2>About Lumen</h2>Lumen connects the world. We are igniting business growth by connecting people, data and applications – quickly, securely, and effortlessly. Together, we are building a culture and company from the people up – committed to teamwork, trust and transparency. People power progress.Lumen’s commitment to workplace inclusion and employee support shines bright. We’ve made the Newsweek 2024 Greatest Workplaces for Diversity list and achieved a perfect score of 100 on the Human Rights Campaign Corporate Equality Index (CEI) for the fifth consecutive year. Plus, we’re the top employer in the communications and telecom industry, ranking 12th overall across all industries in The American Opportunity Index.We’re looking for top-tier talent and offer the flexibility you need to thrive and deliver lasting impact. Join us as we digitally connect the world and shape the future.<h2>The Role</h2>If you're a student eager to explore the exciting world of digital innovation, Lumen is the perfect place for you. Lumen offers a fully immersive, 10-week summer internship program. Our interns demonstrate curiosity, innovation, and a passion for advancing technology. We believe that empowering our people and helping them reach their full potential is essential for the long-term success of both Lumen and our customers.Program dates: May 30 – August 8, 2025.<h2>The Main Responsibilities</h2>Lumen’s Mass Markets organization is looking for a Data Engineer intern on our Applications Engineering team. The intern will be responsible for building machine learning models, developing data pipelines and creating innovating AI applications.Other responsibilities of intern will include:<ul><li>Supporting digital channel for mass markets customers </li><li>Research AI solutions to reduce cost and improve customer experience </li><li>Manage data pipelines</li></ul><h2>What We Look For in a Candidate</h2>Program eligibility is contingent on the candidate’s commitment of the entire 10-week program. No exceptions will be made.Required qualifications<ul><li>Enrolled at a 4-year accredited college or university, rising senior or graduate level education status at the start of the internship </li><li>Graduating August 2025 – May 2027 </li><li>Preferred fields include burt not limited to: – Computer Science, Business Analytics, Data Science, Information Systems. </li><li>Intern must be able to work 40 hours per week during the 10-week program (May 30 – August 8, 2025) </li><li>Proficiency and understanding of: </li><li>MS Office Suite – Excel and PowerPoint </li><li>Python expertise </li><li>Machine Learning frameworks ( TensorFlow, PyTorch, scikit-learn) models </li><li>Python libraries ( NumPy, Pandas, Matplotlib, Seaborn etc…) </li><li>Cloud platforms like GCP </li><li>React.Js </li><li>Node.Js </li></ul>Preferred qualifications<ul><li>Analytical Skills: Ability to analyze and interpret data to provide actionable insights </li><li>Experience with statistical analysis and data visualization tools is a plus. </li><li>Attention to detail and a proactive approach to learning and development. </li><li>Ability to translate technical information into non-technical terms </li><li>Strong problem-solving abilities to identify and resolve issues related to data accuracy, report performance, and user requirements </li><li>Eagerness to learn and adapt to new technologies and methodologies. </li><li>Strong organization skills </li><li>Ability to multitask while working in fast paced environment </li></ul><h2>Compensation</h2>Internship compensation ranges depend on each individual’s level of education, geographic location, and experience/qualifications aligned to the role.Hourly Based Pay Range: 
Min: $26.00/hour 
Max: $43.00/hour<h2>What to Expect Next</h2>After completion and submission of your application, you will be asked to participate in a virtual video interview. This on demand interview will be a way for Lumen to learn more about you and your experience as it aligns to the internship position. A separate email invitation will be sent to you (check your spam) within 6 hours of application. For continued consideration in the summer internship program, please ensure you complete the video interview within 5 business days of application.Application & Interview Timeline:<ul><li>October – First Round Interviews with top, qualified candidates</li><li>November - Interview panel with work team</li><li>December - All Summer 2025 offers will be extended by end of month</li></ul>Requisition #: 335207Background ScreeningIf you are selected for a position, there will be a background screen, which may include checks for criminal records and/or motor vehicle reports and/or drug screening, depending on the position requirements. For more information on these checks, please refer to the Post Offer section of our <a href="https://jobs.lumen.com/global/en/faq">FAQ page</a>. Job-related concerns identified during the background screening may disqualify you from the new position or your current role. Background results will be evaluated on a case-by-case basis. 
Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.Equal Employment OpportunitiesWe are committed to providing equal employment opportunities to all persons regardless of race, color, ancestry, citizenship, national origin, religion, veteran status, disability, genetic characteristic or information, age, gender, sexual orientation, gender identity, gender expression, marital status, family status, pregnancy, or other legally protected status (collectively, “protected statuses”). We do not tolerate unlawful discrimination in any employment decisions, including recruiting, hiring, compensation, promotion, benefits, discipline, termination, job assignments or training.DisclaimerThe job responsibilities described above indicate the general nature and level of work performed by employees within this classification. It is not intended to include a comprehensive inventory of all duties and responsibilities for this job. Job duties and responsibilities are subject to change based on evolving business needs and conditions.<h4>Salary Range</h4>Salary Min:26Salary Max:43This information reflects the anticipated base salary range for this position based on current national data. Minimums and maximums may vary based on location. Individual pay is based on skills, experience and other relevant factors.This position is eligible for either short-term incentives or sales compensation. Director and VP positions also are eligible for long-term incentive. To learn more about our bonus structure, you can view additional information <a href="https://jobs.lumen.com/global/en/compensation-information">here.</a> We're able to answer any additional questions you may have as you move through the selection process.As part of our comprehensive benefits package, Lumen offers a broad range of Health, Life, Voluntary Lifestyle and other benefits and perks that enhance your physical, mental, emotional and financial wellbeing. You can learn more by clicking <a href="https://centurylinkbenefits.com">here.</a>Please be advised that Lumen does not require any form of payment from job applicants during the recruitment process. All legitimate job openings will be posted on our official website or communicated through official company email addresses. If you encounter any job offers that request payment in exchange for employment at Lumen, they are not for employment with us, but may relate to another company with a similar name.Note: For union-represented postings, wage rates and ranges are governed by applicable collective bargaining agreement provisions.Application Deadline 
10/31/2024

Our company is seeking a Data Engineer with expertise in Microsoft Power Platform and Power BI to build solutions for internal and external projects, as well as maintaining and enhancing Power BI reports and dashboards. This role also involves developing and documenting processes for Power BI reports and dashboards, and leading special projects to ensure Power BI solutions meet business needs. The ideal candidate should have a bachelor's degree or equivalent experience, 3 years' experience with Microsoft Power BI, and experience with database development and software development life cycle. This is a remote work position with occasional travel required for meetings. Compensation ranges from $76,800 to $85,000, and includes benefits such as medical and dental care, 401k, PTO, parental leave, and bereavement leave. We are an equal opportunity employer committed to creating a culture of inclusion and celebrating diversity.

Data Engineer Jobs in Pierre, SD

Data Engineer
November 6, 2024

Sr Data Engineer Remote
November 5, 2024

Data Engineer
November 5, 2024

Data Engineer
November 1, 2024

Intern Data Engineer Summer 2025
October 30, 2024

Data Engineer Power Platformpower Bi
October 27, 2024

Data Engineer
October 27, 2024

Data Engineer Digital Solutions
October 23, 2024

Quick filters

Top companies

Data Engineer Jobs in Pierre, SD

Data EngineerNovember 6, 2024

Sr Data Engineer RemoteNovember 5, 2024

Data EngineerNovember 5, 2024

Data EngineerNovember 1, 2024

Intern Data Engineer Summer 2025October 30, 2024

Data Engineer Power Platformpower BiOctober 27, 2024

Data EngineerOctober 27, 2024

Data Engineer Digital SolutionsOctober 23, 2024