Job Title: Active TS/SCI Data Engineer with a Level III IAT certification
Location:  On-site at Fort Eisenhower, Augusta, GA
Clearance Requirement:  Active TS/SCI Clearance Required
Summary:
We are seeking a skilled  Data Engineer  to join a high-performing IT team focused on supporting intelligence applications. This role involves collaborating with other engineers to develop solutions for data ingestion, transformation, and scaling. The ideal candidate will bring expertise in ETL processes, containerization, and Linux administration to enhance and maintain mission-critical systems.
Key Responsibilities:
Data Engineering & ETL:
Develop and implement custom ETL scripts to process and transform raw data into usable formats.
Ensure data quality, consistency, and integrity throughout the pipeline.
Rapidly create and adjust data pipelines to support training exercises and operational needs.
System Administration & Automation:
Perform Linux system administration, database management, and software installations/updates.
Utilize Infrastructure as Code (IaC) tools like Puppet or Ansible for automation and compliance.
Containerization & Orchestration:
Manage containerized environments using tools like Docker or Podman.
Migrate applications to containers and transition legacy systems to microservices.
Custom Development & Troubleshooting:
Work with proprietary data ingestion systems and develop rapid solutions for unique system requirements.
Interface with customers to create ingestion feeds and resolve technical issues with minimal documentation.
Security Compliance & Cybersecurity:
Ensure systems comply with DISA and GISA security regulations.
Collaborate with cybersecurity personnel to address vulnerabilities and document risk mitigations.
Qualifications:
Minimum Requirements:
Active TS/SCI clearance.
Level II IAT certification with the ability to obtain a Level III required; Level III preferred
Extensive experience with ETL processes, data ingestion, and database management.
Proficiency in Linux system administration and scripting (Python, Groovy, or JavaScript).
Experience with Apache NiFi for data flow management.
Knowledge of containerization and orchestration tools (e.g., Docker, Kubernetes).
Preferred Qualifications:
Familiarity with DevSecOps processes and Agile development.
Ability to work with structured and unstructured data (e.g., XML, JSON).
Familiarity with pub-sub messaging systems like Kafka or RabbitMQ.
Knowledge of Oracle databases and mapping tools like Sysicium.
Certifications such as IAT Level III or equivalent, with flexibility for candidates pursuing certification.
Travel:
Travel may include domestic and international locations, averaging 1-3 trips per year for up to 2 weeks per trip.
Work Environment & Team:
This role involves working with a small, fast-paced team of 4 individuals.
Collaboration is key, as the team supports mission-critical operations with international partners, including NATO allies.
Why Join?
This position offers an opportunity to work in a dynamic, high-stakes environment while contributing to national security. If you have the expertise and drive to manage complex data operations in an agile, high-impact setting, this role is for you.
To Apply:
Please submit your resume detailing your experience with ETL processes, NiFi Apache experience, containerization, and Linux administration.

Summary:The Data Engineer is part of a skilled IT team that will work independently and collectively with other engineers to identify and test potential enhancements and resolve technical issues with the rapid scaling of an intelligence application.Essential Functions:<ul><li>Collaboratively develop and implement custom-developed ETL scripts; and support software products as needed.</li><li>Ensure data quality, consistency, and integrity throughout the pipeline.</li><li>Create, monitor, and adjust data ingestion flows.</li><li>Process raw data by transforming it into a usable form (ETL).</li><li>Rapidly create data pipelines to support training exercise with little prior notification to connect fielded systems to cloud-based infrastructure as well as other fielded systems.</li><li>Develop for and work with the ZIngest system, an ingestion system developed by Zapata Technology.</li><li>Perform System Administration functions such as database management, data management, and fixing issues that arise, including the installation of peripheral software and software updates.</li><li>Implement rapid features and API capabilities to meet evolving customer needs for data access and dissemination.</li><li>Interface with the customer(s) to create ingestion feeds for unique systems with ability to implement solutions with few prior known details/documentations about said systems.</li><li>Use source-code control to track and protect changes to the code baseline.</li><li>Manage containerization efforts (Docker, Podman) and orchestration. Upgrade and migrate applications to containers to allow the required scalability and move legacy monolith applications to a microservices type environment. Document Infrastructure administration, tear down, and creation.</li><li>Assist with Automation and Security tasks: Implement Infrastructure as Code (IaC) using Puppet, Ansible or similar technology to meet security mandated change management guidelines. Ensure application and database compliance with DISA and GISA standards and regulations. Work with Cyber-Security personnel to document security vulnerabilities and any mitigations to reduce risks that cannot be removed.</li></ul>Job Qualifications:<ul><li>Knowledge of Agile programming and development concepts including Jira or similar.</li><li>Knowledge of DevSecOps processes including Infrastructure as Code using Puppet, Ansible, etc.</li><li>Knowledge of containerization and container orchestration including networking using Docker, Podman, or Kubernetes.</li><li>Experience with the full DataOps lifecycle including ingestion and ETL processes into databases, data lakes, or data warehouses.</li><li>Detail-oriented and organized; able to understand information systems and ensure accuracy of work.</li><li>Grasp of Software Engineering concepts, especially scripting in Python and Groovy.</li><li>Linux System administration knowledge: know the Linux command line, service management, software configuration concepts in lieu of virtual machine constraints.</li><li>Ability to convert data and fix malformed data. This requires an understanding of XML, JSON, and regular expressions.</li></ul>Preferred Qualifications:<ul><li>An understanding of pub-sub messaging systems including RabbitMQ, Pulsar and Kafka.</li><li>A solid understanding of and experience in NiFi data flow, from configuration to creating new flows and making new processors.</li><li>Experience with troubleshooting the NiFi software.</li><li>Knowledge of NiFi, with ability to keep the cluster running.</li><li>Ability to write scripts in Groovy and Python within NiFi and Linux.</li><li>Ability to write new processors in NiFi using Java.</li></ul>Certifications Required:<ul><li>IAT Level III certification is needed for this role, but having a level II with ability to obtain level III in agreed upon time may be accepted.</li></ul>Education/Experience include:A bachelor’s degree with 9 to 12 years of experience in performing similar or related work is required. Four years of experience may be substituted for a bachelor’s degree.Clearance Type:TS/SCI clearance required.Travel:Travel may be required for this role but estimated at 2 weeks or less per TDY and an average of 1-3 times per year.AAP/EEO Statement:Zapata Technology is an Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, genetic information, creed, religion, sex, pregnancy, sexual orientation, gender identity, national origin, age, protected veteran status, or disability status.AAP/EEO Employer: Minorities/Females/Disabled/Veterans
#J-18808-Ljbffr

Job Title: Active TS/SCI Data Engineer with a Level III IAT certificationLocation: On-site at Fort Eisenhower, Augusta, GAClearance Requirement: Active TS/SCI Clearance Required Summary:We are seeking a skilled Data Engineer to join a high-performing IT team focused on supporting intelligence applications. This role involves collaborating with other engineers to develop solutions for data ingestion, transformation, and scaling. The ideal candidate will bring expertise in ETL processes, containerization, and Linux administration to enhance and maintain mission-critical systems. Key Responsibilities:<ul><li>Data Engineering & ETL:</li><li>Develop and implement custom ETL scripts to process and transform raw data into usable formats.</li><li>Ensure data quality, consistency, and integrity throughout the pipeline.</li><li>Rapidly create and adjust data pipelines to support training exercises and operational needs.</li><li>System Administration & Automation:</li><li>Perform Linux system administration, database management, and software installations/updates.</li><li>Utilize Infrastructure as Code (IaC) tools like Puppet or Ansible for automation and compliance.</li><li>Containerization & Orchestration:</li><li>Manage containerized environments using tools like Docker or Podman.</li><li>Migrate applications to containers and transition legacy systems to microservices.</li><li>Custom Development & Troubleshooting:</li><li>Work with proprietary data ingestion systems and develop rapid solutions for unique system requirements.</li><li>Interface with customers to create ingestion feeds and resolve technical issues with minimal documentation.</li><li>Security Compliance & Cybersecurity:</li><li>Ensure systems comply with DISA and GISA security regulations.</li><li>Collaborate with cybersecurity personnel to address vulnerabilities and document risk mitigations.</li></ul> Qualifications:Minimum Requirements:<ul><li>Active TS/SCI clearance.</li><li>Level II IAT certification with the ability to obtain a Level III required; Level III preferred</li><li>Extensive experience with ETL processes, data ingestion, and database management.</li><li>Proficiency in Linux system administration and scripting (Python, Groovy, or JavaScript).</li><li>Experience with Apache NiFi for data flow management.</li><li>Knowledge of containerization and orchestration tools (e.g., Docker, Kubernetes).</li></ul>Preferred Qualifications:<ul><li>Familiarity with DevSecOps processes and Agile development.</li><li>Ability to work with structured and unstructured data (e.g., XML, JSON).</li><li>Familiarity with pub-sub messaging systems like Kafka or RabbitMQ.</li><li>Knowledge of Oracle databases and mapping tools like Sysicium.</li><li>Certifications such as IAT Level III or equivalent, with flexibility for candidates pursuing certification.</li></ul> Travel:<ul><li>Travel may include domestic and international locations, averaging 1-3 trips per year for up to 2 weeks per trip.</li></ul> Work Environment & Team:<ul><li>This role involves working with a small, fast-paced team of 4 individuals.</li><li>Collaboration is key, as the team supports mission-critical operations with international partners, including NATO allies.</li></ul>Why Join?This position offers an opportunity to work in a dynamic, high-stakes environment while contributing to national security. If you have the expertise and drive to manage complex data operations in an agile, high-impact setting, this role is for you.To Apply:Please submit your resume detailing your experience with ETL processes, NiFi Apache experience, containerization, and Linux administration.

Referral Eligible Summary: Zapata Technology is seeking an experienced Software / Data Engineer for a remote role The Data Engineer will lead the transformation of our data infrastructure, utilizing a custom ASP.NET application, Postgres stored procedures, and migrating existing VBA scripts to modern platforms to build and manage ETL processes. This position is required to meet the customer's current needs but must also be prepared to transition to modern ETL tools such as Apache NiFi and Airflow in the future. The ideal candidate will have extensive experience in data engineering, database management, and designing efficient ETL workflows without relying on traditional ETL tools. Check out the details below and our awesome benefits here. Job Qualifications: Qualifications and skills include: Design and develop modular ETL solutions that ingest and deliver data from various systems to analytics platforms, using custom ASP.NET applications and Postgres stored procedures. Experience in developing and maintaining scalable data storage systems, particularly in Postgres or other relational databases. Proficiency in writing efficient SQL queries and stored procedures to process and transform structured and unstructured data. Experience in designing scalable data pipelines for reporting and analytics, using manual coding techniques rather than third-party ETL applications. Strong ability to collaborate with cross-functional teams to create and implement custom data solutions. Skilled in developing custom scripts and applications to support data transformation and management needs. Experience documenting data processes, managing compliance requirements, and conducting quality assurance tasks. Ability to troubleshoot and resolve data quality issues and ensure system integrity. Familiarity with metadata management, ensuring compliance with data governance frameworks, including HRPP/Privacy/HIPAA/SOP. CompTIA Security certification, demonstrating a commitment to security and compliance standards. Computing Environment Certification. Preferred Qualifications and Skills include: Experience with ASP.NET, C#, and related web application development frameworks. Advanced knowledge of Postgres, particularly in writing complex stored procedures, query optimization, and transitioning data infrastructure to Postgres. Strong familiarity with AWS cloud environments, with experience in migrating data systems to AWS. Experience with low-code/no-code platforms and distributed data processing tools such as Spark, Databricks, or similar. Knowledge of real-time data processing, streaming applications, and data warehousing concepts, especially in cloud-native environments. Proficiency in shell scripting and automation to support cloud migrations and system integrations. Experience working in Agile methodologies, particularly within cloud-based or hybrid cloud environments. Education/Experience includes: 4 years of Software Development Working Conditions: Prolonged periods sitting at a desk and working on a computer Position Type/Expected Hours of Work: Full time position Clearance: This position requires an active Secret clearance with the ability to obtain TS/SCI. AAP/EEO Statement: Equal Opportunity Employer - M/F/Disabled/Veteran Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Zapata Technology, Inc. provides reasonable accommodation to applicants who are veterans or who have disabilities and are unable to fully use our company application system. If you need a reasonable accommodation for any part of the application and hiring process, please notify Christina Hall, EEO Coordinator at ChristinaHallZapataTechnology.com

Summary:

The Data Engineer is part of a skilled IT team that will work independently and collectively with other engineers to identify and test potential enhancements and resolve technical issues with the rapid scaling of an intelligence application.

Essential Functions:
Collaboratively develop and implement custom-developed ETL scripts; and support software products as needed.
Ensure data quality, consistency, and integrity throughout the pipeline.
Create, monitor, and adjust data ingestion flows.
Process raw data by transforming it into a usable form (ETL).
Rapidly create data pipelines to support training exercise with little prior notification to connect fielded systems to cloud-based infrastructure as well as other fielded systems.
Develop for and work with the ZIngest system, an ingestion system developed by Zapata Technology.
Perform System Administration functions such as database management, data management, and fixing issues that arise, including the installation of peripheral software and software updates.
Implement rapid features and API capabilities to meet evolving customer needs for data access and dissemination.
Interface with the customer(s) to create ingestion feeds for unique systems with ability to implement solutions with few prior known details/documentations about said systems.
Use source-code control to track and protect changes to the code baseline.
Manage containerization efforts (Docker, Podman) and orchestration. Upgrade and migrate applications to containers to allow the required scalability and move legacy monolith applications to a microservices type environment. Document Infrastructure administration, tear down, and creation.
Assist with Automation and Security tasks: Implement Infrastructure as Code (IaC) using Puppet, Ansible or similar technology to meet security mandated change management guidelines. Ensure application and database compliance with DISA and GISA standards and regulations. Work with Cyber-Security personnel to document security vulnerabilities and any mitigations to reduce risks that cannot be removed.
Job Qualifications:     Knowledge of JavaScript frameworks and API development.
Knowledge of Agile programming and development concepts including Jira or similar.
Knowledge of DevSecOps processes including Infrastructure as Code using Puppet, Ansible, etc.
Knowledge of containerization and container orchestration including networking using Docker, Podman, or Kubernetes.
Experience with the full DataOps lifecycle including ingestion and ETL processes into databases, data lakes, or data warehouses.
Detail-oriented and organized; able to understand information systems and ensure accuracy of work.
Grasp of Software Engineering concepts, especially scripting in Python and Groovy.
Linux System administration knowledge: know the Linux command line, service management, software configuration concepts in lieu of virtual machine constraints.
Ability to convert data and fix malformed data. This requires an understanding of XML, JSON, and regular expressions.
Preferred Qualifications:      An understanding of pub-sub messaging systems including RabbitMQ, Pulsar and Kafka.
A solid understanding of and experience in NiFi data flow, from configuration to creating new flows and making new processors.
Experience with troubleshooting the NiFi software.
Knowledge of NiFi, with ability to keep the cluster running.
Ability to write scripts in Groovy and Python within NiFi and Linux.
Ability to write new processors in NiFi using Java.
Certifications Required:   IAT Level III certification is needed for this role, but having a level II with ability to obtain level III in agreed upon time may be accepted.

Education/Experience include:

A bachelor's degree with 9 to 12 years of experience in performing similar or related work is required. Four years of experience may be substituted for a bachelor's degree.

Clearance Type:

TS/SCI clearance required.

Travel:

Travel may be required for this role but estimated at 2 weeks or less per TDY and an average of 1-3 times per year.

AAP/EEO Statement:

Zapata Technology is an Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, genetic information, creed, religion, sex, pregnancy, sexual orientation, gender identity, national origin, age, protected veteran status, or disability status.

AAP/EEO Employer: Minorities/Females/Disabled/Veterans

Data Engineer Jobs in Augusta, GA

Data Engineer
December 14, 2024

Data Engineer 1410
December 9, 2024

Data Engineer
December 1, 2024

Software Data Engineer 1400
November 26, 2024

Data Engineer 1410
November 26, 2024

Quick filters

Top companies

Data Engineer Jobs in Augusta, GA

Data EngineerDecember 14, 2024

Data Engineer 1410December 9, 2024

Data EngineerDecember 1, 2024

Software Data Engineer 1400November 26, 2024

Data Engineer 1410November 26, 2024