C2C Employers please DO NOT apply to this job posting. This is a W2 opportunity. Thank you! Role: Sr Data EngineerLocation: Englewood Cliffs, NJ – onsite 3 days/weekVisa: Must be able to work on a W2 without visa sponsorshipDuration: 6-month contract to hireRate: 75-90/hr on W2 Must-haves <ul><li>5-8+ years of experience as a Data Engineer</li><li>Python and PySpark experience at the programming (OOP) level</li><li>Experience with Kafka for batch processing and/or streaming pipelines</li><li>Relative experience with GCP Dataproc, GCP Pub/Sub, RabbitMQ, AWS Redis, or Spark Streaming is okay</li><li>Proficiency with AWS utilizing Lambda, S3, SQS, SNS, and Kinesis</li><li>AWS EMR or Databricks experience</li></ul> Plus<ul><li>Airflow and DAG – AWS Services</li><li>AWS Certification</li><li>Lead experience and ability to mentor junior developers</li></ul> Day-to-Day A Large Media Client is seeking a Data Engineer with strong Python and AWS technical expertise to join their Data Engineering team. The Data Engineer will play a crucial role in designing, implementing, and maintaining scalable data pipelines and solutions on the AWS platform. Your work will involve handling large volumes of diverse data, ensuring its accuracy, and providing real-time streaming pipelines and batch processing. The day-to-day will involve building and optimizing pipelines for processing terabytes of data on a varying increments, including monthly, weekly, daily, hourly. This includes both batch processing (daily schedules, hourly updates) and real-time streaming pipelines. The Data Engineer will work closely with cross-functional teams (Cyber, Cloud Ops, AWS, Databricks, partners, and vendors) to address technical issues and resolutions. The Data Engineer will utilize Terraform to write infrastructure code and AWS services, including but not limited to Lambda, SQS, SNS, S3, and Airflow.

<h3>Job Description</h3><div>Job Description<div>Job Title: Data Engineer - Highly Skilled Location: Englewood Cliffs, NJ (onsite) Pay rate: $ 76 an hour Benefits: Medical, Dental, and Vision coverage Contract-to-hire: 12 months (possibility of conversion to permanent)  Work Hours: 9 AM - 6 PM Qualifications:<ul><li>Embrace a ‘builder’ mentality to push the business forward</li><li>8+ years experience in a software developer role</li><li>4+ years of experience as a data engineer</li><li>Proficient with AWS, Python, and Pyspark</li><li>Experience with EMR, S3, SNS, Kafka, Databricks, Athena, Kinesis, Lambda and Airflow</li><li>Experience with distributed data technologies (e.g. Spark)</li><li>Proficient with Linux environment</li><li>Passion for media and news</li><li>B.S. degree in Computer Science, Information Technology, or equivalent experience</li></ul>Responsibilities:<ul><li>Create new ingest batch/streaming pipelines from the ground up or maintain existing pipelines. Recommend tools/tech stack that will improve the effectiveness of existing pipelines or reduce the overall operating cost of the existing pipelines</li><li>Possess an in-depth understanding of the data structures and governance Deep understanding of modern cloud computing platforms and concepts (AWS preferred)</li><li>Work with modern schema-less big data storage solutions  </li><li>Work closely with machine learning and data science teams to create scale and efficiency</li><li>Work closely with DevOps to foster a culture of collaboration</li><li>Demonstrate critical thinking for potential roadblocks; comprehend a bigger picture of the business and effectively communicate these issues to stakeholders</li><li>Work closely with internal stakeholders to implement solutions that adhere to solution designs and schema</li><li>Work in an agile software development team to create new batch/streaming ingest data pipelines from the ground up, and/or expand upon those created by other team members.</li><li>Act as a lead, identifying and assigning new work for other engineers.</li><li>Test and create automated unit/integration tests for your code, ensuring every function, service, and object is compatible with your team’s work and with the many systems within the Group system portfolio.</li><li>Act as a development leader, help to formulate best practices, and set up toolsets and procedures for the team, leveraging internally available tools and communicating with other internal development & product teams.</li><li>Stay up-to-date on new development and platform technologies, make recommendations for the right tool for the job, and take the lead in setting up those tools and training other developers on them.</li><li>Create documentation for developers as well as business users to help them understand your products.</li><li>Maintain cloud-based platforms and environments of supported applications, troubleshooting and patching functional issues and data issues in lower or production environments when necessary.</li></ul></div>Company Description<div>Overview TechLink Systems is a national provider of staffing services with offices all over the USA as well as development and support centers globally. We are a MBE (NMSDC), WBE (WBENC), and WOSB certified minority business that has been awarded 2017 TAPFIN Premier Partner, 2016 Manpower Excellence Award, 2016 TAPFIN Elite Partner, are a Known Tier Supplier for Agile 1, and have been ranked as a Top Women and Minority Owned Company since 2004 by several businesses including Philadelphia Business Journal, Diversity Business, Inc 5000, Inc 500, San Francisco Business Times, Diversity2000, and Los Angeles Business Journal. We serve the contract staffing, direct hire, staff augmentation, recruitment, HR management, and project outsourcing needs of clients nationwide. Striving to create personal relationships with clients, TechLink Systems becomes a virtual extension of clients’ own in-house capabilities to help achieve new levels of growth, productivity, and competitiveness. We also have national recruiting centers that service our enterprise and MSP/VMS clients. We are a Secret Cleared Facility and our core competencies are Information Technology, Engineering, Customer Service/Call Center, Admin/Clerical, Finance/Accounting, Scientific/Clinical, Sales/Marketing, Legal, Human Resources, Light Industrial (limited), and Professional across a range of industries. The company’s talented Recruiters and HR Specialists have backgrounds in many cutting-edge, high-performance industries making TechLink Systems an ideal partner for recruitment process outsourcing. We look forward to our partnership with you!</div></div><h3>Company Description</h3><div>Overview\r\n\r\nTechLink Systems is a national provider of staffing services with offices all over the USA as well as development and support centers globally. We are a MBE (NMSDC), WBE (WBENC), and WOSB certified minority business that has been awarded 2017 TAPFIN Premier Partner, 2016 Manpower Excellence Award, 2016 TAPFIN Elite Partner, are a Known Tier Supplier for Agile 1, and have been ranked as a Top Women and Minority Owned Company since 2004 by several businesses including Philadelphia Business Journal, Diversity Business, Inc 5000, Inc 500, San Francisco Business Times, Diversity2000, and Los Angeles Business Journal. \r\n\r\nWe serve the contract staffing, direct hire, staff augmentation, recruitment, HR management, and project outsourcing needs of clients nationwide. Striving to create personal relationships with clients, TechLink Systems becomes a virtual extension of clients’ own in-house capabilities to help achieve new levels of growth, productivity, and competitiveness. \r\n\r\nWe also have national recruiting centers that service our enterprise and MSP/VMS clients. We are a Secret Cleared Facility and our core competencies are Information Technology, Engineering, Customer Service/Call Center, Admin/Clerical, Finance/Accounting, Scientific/Clinical, Sales/Marketing, Legal, Human Resources, Light Industrial (limited), and Professional across a range of industries. \r\n\r\nThe company’s talented Recruiters and HR Specialists have backgrounds in many cutting-edge, high-performance industries making TechLink Systems an ideal partner for recruitment process outsourcing. \r\n\r\nWe look forward to our partnership with you!</div>

Locations: VA - McLean, United States of America, McLean, VirginiaSenior Data Engineer (Remote-Eligible)Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you''ll be part of a big group of makers, breakers, doers and disruptors, who solve real problems and meet real customer needs. We are seeking Data Engineers who are passionate about marrying data with emerging technologies. As a Capital One Data Engineer, you''ll have the opportunity to be on the forefront of driving a major transformation within Capital One. What You''ll Do:Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions in full-stack development tools and technologiesWork with a team of developers with deep experience in machine learning, distributed microservices, and full stack systemsUtilize programming languages like Java, Scala, Python and Open Source RDBMS and NoSQL databases and Cloud based data warehousing services such as Redshift and SnowflakeShare your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the engineering communityCollaborate with digital product managers, and deliver robust cloud-based solutions that drive powerful experiences to help millions of Americans achieve financial empowermentPerform unit tests and conduct reviews with other team members to make sure your code is rigorously designed, elegantly coded, and effectively tuned for performanceCapital One is open to hiring a Remote Employee for this opportunity.Basic Qualifications: Bachelor''s Degree At least 4 years of experience in application development (Internship experience does not apply)At least 1 year of experience in big data technologies Preferred Qualifications: 5+ years of experience in application development including Python, SQL, Scala, or Java2+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud)3+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL)2+ year experience working on real-time data and streaming applications 2+ years of experience with NoSQL implementation (Mongo, Cassandra) 2+ years of data warehousing experience (Redshift or Snowflake) 3+ years of experience with UNIX/Linux including basic commands and shell scripting2+ years of experience with Agile engineering practices At this time, Capital One will not sponsor a new applicant for employment authorization for this position.No agencies please. Capital One is an Equal Opportunity Employer committed to diversity and inclusion in the workplace. All qualified applicants will receive consideration for employment without regard to sex, race, color, age, national origin, religion, physical and mental disability, genetic information, marital status, sexual orientation, gender identity/assignment, citizenship, pregnancy or maternity, protected veteran status, or any other status prohibited by applicable national, federal, state or local law. Capital One promotes a drug-free workplace. Capital One will consider for employment qualified applicants with a criminal history in a manner consistent with the requirements of applicable laws regarding criminal background inquiries, including, to the extent applicable, Article 23-A of the New York Correction Law; San Francisco, California Police Code Article 49, Sections 4901-4920; New York City''s Fair Chance Act; Philadelphia''s Fair Criminal Records Screening Act; and other applicable federal, state, and local laws and regulations regarding criminal background inquiries.If you have visited our website in search of information on employment opportunities or to apply for a position, and you require an accommodation, please contact Capital One Recruiting at 1-800-304-9102 or via email at [Click Here to Email Your Resum]. All information you provide will be kept confidential and will be used only to the extent required to provide needed reasonable accommodations.For technical support or questions about Capital One''s recruiting process, please send an email to [Click Here to Email Your Resum]Capital One does not provide, endorse nor guarantee and is not liable for third-party products, services, educational tools or other information available through this site.Capital One Financial is made up of several different entities. Please note that any position posted in Canada is for Capital One Canada, any position posted in the United Kingdom is for Capital One Europe and any position posted in the Philippines is for Capital One Philippines Service Corp. (COPSSC).

We create world-class content, which we distribute across our portfolio of film, television, and streaming, and bring to life through our theme parks and consumer experiences. We own and operate leading entertainment and news brands, including NBC, NBC News, MSNBC, CNBC, NBC Sports, Telemundo, NBC Local Stations, Bravo, USA Network, and Peacock, our premium ad-supported streaming service. We produce and distribute premier filmed entertainment and programming through Universal Filmed Entertainment Group and Universal Studio Group, and have world-renowned theme parks and attractions through Universal Destinations & Experiences. NBCUniversal is a subsidiary of Comcast Corporation.Here you can be your authentic self. As a company uniquely positioned to educate, entertain and empower through our platforms, Comcast NBCUniversal stands for including everyone. Our Diversity, Equity and Inclusion (https://www.nbcuniversal.com/diversity-equity-inclusion) initiatives, coupled with our Corporate Social Responsibility (https://www.nbcuniversal.com/csr) work, is informed by our employees, audiences, park guests and the communities in which we live. We strive to foster a diverse, equitable and inclusive culture where our employees feel supported, embraced and heard. Together, well continue to create and deliver content that reflects the current and ever-changing face of the world.We are seeking a Staff Data Engineer looking to build the next generation of data pipelines and applications across the development of innovative new systems and solutions using a rapidly changing landscape of emerging technologies including generative AI and large language models. Working across the practices, techniques and tools used for the operational management of large language models in production environments the Staff Data Engineer role is proper for you if you're a subject matter expert in designing data integration frameworks and pipelines and still love to jump in and be "hands-on" when needed. This team is focused on proving the value of new tech and bringing it to production quickly.In the Staff Data Engineer role, you'll have the opportunity to partner with internal stakeholders, data engineers, visualization experts, data scientists, and other technologists across the businesses. You've come to the right place if you love to take large, disparate data sets and build them into flexible and scalable analytics applications and warehouses. Here, you can create the extraordinary. Join us. The ideal candidate should be well-versed in designing, building, and supporting APIs, machine learning services and frameworks, LLMs, lang-chain, and foundational data warehousing technologies. In addition, the candidate should be excited about the possibilities that Generative AI can be leveraged to accelerate various parts of the business.Your primary focus will be building reliable, scalable, and efficient pipelines for use in LLMs and crafting our vision for LLM analytics. You will be essential in defining the team's strategy, evaluating, and integrating data patterns and technologies, and building pipelines alongside domain experts and data scientists.Responsibilities:Design, build, and scale data pipelines across a variety of source systems and streams (internal, third-party, and cloud-based), distributed/elastic environments, and downstream applications and self-service solutions.Deep understanding of Machine Learning best practices (e.g., training/serving, feature engineering, feature/model selection, imbalance data, RAG patterns) and algorithms (e.g., deep learnings, optimization)Solid understanding of data modeling, warehousing, and architecture principles.Implement appropriate design patterns while optimizing performance, cost, security, and scale and end-user experience.Collaborate with cross-functional teams to understand data requirements and develop efficient data acquisition and integration strategies.Interface with other technology teams to extract, load, and transform data from a wide variety of data sources using cloud-native data engineering principles.Become a subject matter expert for data engineering-related technologies and designs.Coach and guide others within the organization to build scalable pipelines based on foundational data engineering principles.Participate in development sprints, demos, and retrospectives alongside releases and deployment.Build and manage relationships with supporting engineering teams to deliver work products to production effectively.Have worked well with data scientists, business analysts, and machine learning infrastructure to connect the dots between business and technology partners.Develop automated tests for your code, ensuring every function, service, and object is compatible with your team's work and with the many systems within the NBCUniversal system portfolio and cross-device and browser compatibility.Create documentation for developers and business users to help them understand our products.Work collaboratively with a multidisciplinary team within a matrixed organization, leveraging strong interpersonal skills to navigate system complexities and deploy solutions efficiently.Deploy to cloud-based platforms and troubleshoot application, cloud, and configuration issues when necessary.Utilize tools for code & test generation to dramatically accelerate the delivery of features and components you create.6+ years of experience in a data engineering role, with a strong emphasis on leading data engineering teamsAbility to think critically about problems, decipher user preferences versus challenging requirements, and effectively use online and onsite resources to find appropriate solutions.Proven ability to thrive in an agile development environment, adept at incorporating feedback and adjusting to changing priorities.Understanding REST-based APIs, vectorized embeddings, and other Retrieval Augmented Generation AI workload components.Direct experience with data modeling, ETL/ELT development principles, cloud development, and data warehousing conceptsKnowledge of cloud technologies such as AWS, Azure, GCPKnowledge of data management fundamentals and data storage principlesExperience in building data pipelines using Python/SQL or similar programming languages.General understanding of cloud data engineering design patterns and use casesBachelor's degree in Computer Science, Data Science, Statistics, Informatics, Information Systems or related field.Desired Characteristics:Familiarity with integrating large language models and AI-generated content technologies into applications.Familiarity with the development ecosystem evolving around LLM integration, such as langchain.Proven adaptability in a fast-paced, evolving technology landscape, with a strong problem-solving ability and quick learning curve.Effective communication skills, capable of working collaboratively across diverse teams and navigating a large, matrixed organization efficiently.Ability to translate business needs into clear technical requirements.Analytical You have experience in delivering self-service analytics solutions that promote data discovery.Experience with Snowflake, Amazon Web Services, or related cloud platforms a plusUnderstanding of big data technology stacks (Hive / Spark etc) is a plusExperience moving on prem technologies to the cloud is a plusAction-oriented You're constantly figuring out new problems and are regularly showing results with a positive attitude, always displaying ethical behavior, integrity, and building trustStrong understanding of Agile principles and best practicesYouve dealt with ambiguity and can make quality decisions in a dynamic, fast-paced environmentAdditional Requirements:Fully Remote: This position has been designated as fully remote, meaning that the position is expected to contribute from a non-NBCUniversal worksite, most commonly an employees residence.This position is eligible for company sponsored benefits, including medical, dental and vision insurance, 401(k), paid leave, tuition reimbursement, and a variety of other discounts and perks. Learn more about the benefits offered by NBCUniversal by visiting the Benefits page (https://www.nbcunicareers.com/benefits) of the Careers website. Salary range: $130,000 - $170,000 (bonus eligible)We are accepting applications for this position on an ongoing basis.NBCUniversal's policy is to provide equal employment opportunities to all applicants and employees without regard to race, color, religion, creed, gender, gender identity or expression, age, national origin or ancestry, citizenship, disability, sexual orientation, marital status, pregnancy, veteran status, membership in the uniformed services, genetic information, or any other basis protected by applicable law. NBCUniversal will consider for employment qualified applicants with criminal histories in a manner consistent with relevant legal requirements, including the City of Los Angeles Fair Chance Initiative For Hiring Ordinance, where applicable.If you are a qualified individual with a disability or a disabled veteran, you have the right to request a reasonable accommodation if you are unable or limited in your ability to use or access nbcunicareers.com as a result of your disability. You can request reasonable accommodations by emailing AccessibilitySupport@nbcuni.com.

Data Engineer Jobs in Englewood Cliffs, NJ

Senior Data Engineer
May 31, 2024

Data Engineer
May 25, 2024

Sr Data Engineer
May 23, 2024

Data Engineer Jm2024
May 23, 2024

Staff Data Engineer Emerging Platforms
May 21, 2024

Staff Data Engineer Emerging Platforms
May 21, 2024

Staff Data Engineer Emerging Platforms
May 18, 2024

Senior Data Engineer Remoteeligible
May 17, 2024

Staff Data Engineer Emerging Platforms
May 9, 2024

Quick filters

Top companies

Data Engineer Jobs in Englewood Cliffs, NJ

Senior Data EngineerMay 31, 2024

Data EngineerMay 25, 2024

Sr Data EngineerMay 23, 2024

Data Engineer Jm2024May 23, 2024

Staff Data Engineer Emerging PlatformsMay 21, 2024

Staff Data Engineer Emerging PlatformsMay 21, 2024

Staff Data Engineer Emerging PlatformsMay 18, 2024

Senior Data Engineer RemoteeligibleMay 17, 2024

Staff Data Engineer Emerging PlatformsMay 9, 2024

Quick filters

Top companies

Senior Data Engineer
May 31, 2024

Data Engineer
May 25, 2024

Sr Data Engineer
May 23, 2024

Data Engineer Jm2024
May 23, 2024

Staff Data Engineer Emerging Platforms
May 21, 2024

Staff Data Engineer Emerging Platforms
May 21, 2024

Staff Data Engineer Emerging Platforms
May 18, 2024

Senior Data Engineer Remoteeligible
May 17, 2024

Staff Data Engineer Emerging Platforms
May 9, 2024