The Sysdig Data Engineer interview guide, interview questions, salary data, and interview experiences.

Sysdig Data Engineer Interview Guide

Getting ready for an Data Engineer interview at Sysdig? The Sysdig Data Engineer interview span across 10 to 12 different question topics. In preparing for the interview:
<ul>
<li>Know what skills are necessary for Sysdig Data Engineer roles.</li>
<li>Gain insights into the Data Engineer interview process at Sysdig.</li>
<li>Practice real Sysdig Data Engineer interview questions.</li>
</ul>
Interview Query regularly analyzes interview experience data, and we've used that data to produce this guide, with sample interview questions and an overview of the Sysdig Data Engineer interview.


Getting ready for an Data Engineer interview at Sysdig? The Sysdig Data Engineer interview span across 10 to 12 different question topics. In preparing for the interview:

- Know what skills are necessary for Sysdig Data Engineer roles.
- Gain insights into the Data Engineer interview process at Sysdig.
- Practice real Sysdig Data Engineer interview questions.

Interview Query regularly analyzes interview experience data, and we've used that data to produce this guide, with sample interview questions and an overview of the Sysdig Data Engineer interview.
        

Overview

Sysdig Data Engineer Salary

We've gathered this data from parsing thousands of interview experiences sourced from members.

We've gathered this data from parsing thousands of interview experiences sourced from members.

Sysdig Data Engineer Interview Process

You are given a dictionary with a key-value of <code>{string: number}</code> where values in the dictionary could be duplicates. You are required to extract the unique values from the dictionary where the value occurred only once.

Return a list of values where they occur only once.

Note: You can return the values in any order.

Input:

<pre tabindex="0" class="chroma"><code>dictionary = {&#34;key1&#34;: 1, &#34;key2&#34;: 1, &#34;key3&#34;: 7, &#34;key4&#34;: 3, &#34;key5&#34;: 4, &#34;key6&#34;: 7}
</code></pre>

Output:

<pre tabindex="0" class="chroma"><code>find_unique_values(dictionary) -&gt; [3,4]

#Only 3 and 4 occurred once.
</code></pre>

Dictionary Unique Values

You’re tasked with designing a database system for Swipe Inc., a company that manages payments for software developers. Swipe manages a set of APIs so developers can abstract payment processes for their web service and transfer control over to Swipe’s APIs to manage payment security and finance handling (contacting credit card providers, etc.)

Your database system should store a collection of API keys with their attributes, as shown in the example below:

<pre tabindex="0" class="chroma"><code>{
 &#34;API_key&#34;:{
 &#34;Transaction_list&#34;:{
 &#34;User_ID&#34;:{
 &#34;Name&#34;:{
 &#34;f_name&#34;: 
 &#34;l_name&#34;: 
 }
 &#34;Card_Details&#34;:{
 &#34;Bank&#34;:
 &#34;Card_No&#34;:
 &#34;Expiry&#34;:
 &#34;CV&#34;:
 } 
 }
 }
 &#34;Curr_balance&#34;:
 }
}
</code></pre>

What are the functional and non-functional requirements for this database?

Determine the requirements for designing a database system to store payment APIs

Swipe Payment API

Given a table called <code>employees</code>, get the largest salary of any employee by department

Example:

Input:

<code>employees</code> table

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>department</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>salary</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
Output:

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>department</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>largest_salary</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>

Write a query to get the largest salary of any employee by department

Largest Salary by Department

Sysdig Data Engineer Interview Questions

Design, build, maintain and optimize the enterprise cloud data warehouse platform. Candidates must be located in Phoenix, Arizona. Essential Duties: Design and develop highly scalable and resilient infrastructure and pipelines for data processing and services using modern cloud data technologies Manage overall orchestration of the enterprise data warehouse platform Optimize performance, costs, and scalability of cloud data services Train and deploy machine learning models Other duties as assigned. Qualifications: Degree in Computer Science, Engineering, Analytics, or technology related discipline 4 years' experience working as a data engineer with exposure to cloud data platforms like AWS, Azure, GCP, Snowflake, Databricks Expert knowledge of the following: Spark, Python, Scala, Hive, SQL Extensive experience of building and maintaining large scale ETL pipelines and in-depth knowledge of various big data frameworks and architectures Must be flexible and willing to work the demands of the department which may be subject to evenings, weekends, and holidays. Corporate Summary: At Shamrock Foods Company, people come first - our associates, our customers, and the families we serve across the nation. A privately held, family-owned and -operated Forbes 500 company, Shamrock is an innovator in the food industry and has been since being founded in Arizona in 1922. Our Mission: At Shamrock Foods Company, we live by our founding family's motto to "treat associates like family and customers like friends." Why work for us? Benefits are a major part of your overall compensation, and we believe offering them at an affordable cost is not only the right thing to do, but it helps keep you and your family healthy. That's why Shamrock Foods pays for the majority of your health insurance, allowing you to take home more of your paycheck. And it doesn't stop there - our associates also enjoy additional benefits such as 401(k) Savings Plan, Profit Sharing, Paid Time Off, as well as our incredible growth opportunities, continued education, wellness programs. Equal Opportunity Employer At Shamrock Foods Co all qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, disability, veteran status, sexual orientation, gender identity or any other basis protected by applicable law. Equal Opportunity Employer/Protected Veterans/Individuals with Disabilities The contractor will not discharge or in any other manner discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay or the pay of another employee or applicant. However, employees who have access to the compensation information of other employees or applicants as a part of their essential job functions cannot disclose the pay of other employees or applicants to individuals who do not otherwise have access to compensation information, unless the disclosure is (a) in response to a formal complaint or charge, (b) in furtherance of an investigation, proceeding, hearing, or action, including an investigation conducted by the employer, or (c) consistent with the contractor's legal duty to furnish information. 41 CFR 60-1.35(c)

We are seeking a skilled and collaborative Data Engineer to join our team and drive the development of data pipelines and analytics tooling to support the growth of our energy business, including Industrial, Residential, Supercharger, and Solar products. As a key member of our team, you will design, build, and maintain large-scale data systems to extract insights from various data sources, including deployed fleets, internal applications, and other data warehouses. Your work will contribute to our understanding of key business performance indicators, informing data-driven decisions to optimize our strategies and drive business growth. You will be responsible for leading the development of data efforts to identify and resolve fleet-wide performance, operation, and maintenance challenges across our growing fleet of energy products. You will work closely with cross-functional teams, including Industrial Energy, Residential Energy, and Solar Product and Service Engineering, to maximize product health of our fleet. Responsibilities Derive an overall strategy of data management, within an established information architecture that allows us to scale data processing effectively Gather and processes raw, structured, semi-structured, and unstructured data using batch and real-time data processing frameworks Develop aggregate and summary tables for various engineering teams that span across multiple product lines and geographies Mentor and lead data engineers providing technical guidance and oversight Design and develop new systems and tools to enable cross functional teams to consume and understand data faster Research and incorporate emerging software infrastructures, tools, and technologies, especially pertaining to big data processing Requirements Degree in Computer Science, Engineering, Physics, or related quantitative field or equivalent experience Real work experience in building, deploying, and maintaining Py spark-based data pipelines Experience with SQL and familiarity with database performance concepts like indices, segmentation, projections, and partitions Knowledge of various data communication protocols (e.g., REST API, GRPC) Strong dev-ops capabilities and bias for automation Proficiency in Python or Scala Experience with Linux Compensation and Benefits Benefits Along with competitive pay, as a full-time Tesla employee, you are eligible for the following benefits at day 1 of hire: Aetna PPO and HSA plans > 2 medical plan options with $0 payroll deduction Family-building, fertility, adoption and surrogacy benefits Dental (including orthodontic coverage) and vision plans, both have options with a $0 paycheck contribution Company Paid (Health Savings Account) HSA Contribution when enrolled in the High Deductible Aetna medical plan with HSA Healthcare and Dependent Care Flexible Spending Accounts (FSA) LGBTQ care concierge services 401(k) with employer match, Employee Stock Purchase Plans, and other financial benefits Company paid Basic Life, AD&D, short-term and long-term disability insurance Employee Assistance Program Sick and Vacation time (Flex time for salary positions), and Paid Holidays Back-up childcare and parenting support resources Voluntary benefits to include: critical illness, hospital indemnity, accident insurance, theft & legal services, and pet insurance Weight Loss and Tobacco Cessation Programs Tesla Babies program Commuter benefits Employee discounts and perks program Expected Compensation $80,000 - $162,000/annual salary  cash and stock awards  benefits Pay offered may vary depending on multiple individualized factors, including market location, job-related knowledge, skills, and experience. The total compensation package for this position may also include other elements dependent on the position offered. Details of participation in these benefit plans will be provided if an employee receives an offer of employment.

Overview This is a hybrid role - 2 days remote and 3 days in the Malvern, PA office. CubeSmart is currently seeking a Senior (Sr) Data Engineer to join the Information Technology team at our corporate office in Malvern, PA. The Sr Data Engineer will be responsible for the design, development, and implementation of data products and services used to build and enrich the Cloud Data Platform at Cubesmart. Who we are: At CubeSmart, we're intentional about culture. You can experience it everywhere from our mission statement of "genuine care" to our "It's What's Inside That Counts" tagline to calling each other "teammates" rather than employees. This spirit fosters a fun and collaborative environment that has resulted in our rapid growth and being recognized amongst the top in our industry. CubeSmart's award-winning team is made up of people who genuinely care. Teammates care about our customers and the life events and/or business needs they are facing. Teammates are passionate, responsible and understanding. The CubeSmart team is made up of people who have a can-do attitude, are committed to their own success and the success of the company, and lead by example. If this sounds like a team and culture that matches your personal values and motivations, we want to hear from you. Responsibilities This role will also be responsible for the development and maintenance of the data products and services used to build and enrich the Cloud Data Platform. The role will serve as a technical expert to analyze the needs of CubeSmart's applications and business data requirements and produce optimal solutions according to their needs and specifications, and in alignment with CubeSmart's design standards. This individual will be part of the team building a data and analytics platform leveraging the Databricks Data Intelligence Platfrom in the Azure cloud. This will include designing and building the data ingestion and ETL pipelines and processes to create and manage the cloud Lakehouse environment. In addition, this person will be engaged with the Revenue Management and Analytics teams to help created optimized solutions for the Cloud. Responsibilities <ul><li>Collaborate with program manager(s), application development, and data consumers to develop data ingestion and ETL pipelines required to enable CubeSmart data consumers to onboard processes to the Cloud Data Platform</li><li>Build new or enhance existing Data Products and Services used within the Cloud Data Platform</li><li>Provide support as needed to other projects or programs that are consumers of the Cloud Data Platform</li><li>Perform code reviews and mentor junior team members as needed and provide mentorship to analytics users of the Cloud Data Platform</li><li>Collaborate with 3rd party vendors to understand externally sourced data and incorporate into CubeSmart's data environment, in adherence with CubeSmart's data design standards</li></ul> Qualifications <ul><li>Bachelor's degree in Computer Science, Information Technology or a related discipline</li><li>7+ years of experience in Data Engineering, Cloud Data Warehousing, Real-time Data Processing including Streaming and Transactional Systems</li><li>Strong skills in SQL, Python, and PySpark</li><li>Experience with Azure data services (Azure Databricks, Azure SQL, Azure Data Factory and Databricks ML Flow) or equivalent is preferred</li><li>Full understanding of the Software Development Life Cycle (SDLC) and experience with Agile software development processes</li><li>Solid understanding of cloud DevOps capabilities, preferably Azure DevOps</li><li>Experience creating and managing CI/CD pipelines and processes</li><li>Experience with MongoDB (or similar non-relational database platform) or PostgreSQL is a plus</li><li>Experience with Microsoft .Net is a plus</li></ul> Knowledge, Skills, Abilities and Personal Characteristics <ul><li>Service orientation towards business-focused 24x7 support and service mentality</li><li>Ability to work within a team environment, showing an openness to collaborate with technical and business peers, with the ability to step up and take on new challenges in response to changing business conditions</li><li>Ability to learn quickly, adapt work processes to adhere with best practices</li><li>Ability to clearly convey technical information to both technical and non-technical audiences</li><li>Collaborative individual who creates open channels of communications and encourages technical dialogue across the department and who is capable of mentoring junior team members</li><li>Well-developed analytical and problem-solving abilities</li><li>Ability to work on multiple tasks and projects at once, with the ability to properly prioritize one's own work and the work of others</li></ul> We are an Equal Opportunity Employer, Minority/Female/Veteran/Individuals with Disabilities/Sexual Orientation/Gender Identity. #LI-MT1

Zions Bancorporation's Enterprise Technology and Operations (ETO) team is transforming what it means to work for a financial institution. With a commitment to technology and innovation, we have been providing our community, clients and colleagues the best experience possible for over 150 years. Help us transform our workforce of the future, today. We are seeking a Master Data Management (MDM) Data Engineer to join Zions Bancorporation. The MDM Data Engineer plays a key role in the analysis and design of new and enhanced digital capabilities for our master data management platform that will enable simple and delightful experiences for our partners and customers. The Master Data Management (MDM) Data Engineer will: <ul><li>Partner with data & integration product teams to implement master data management solutions.</li><li>Conduct discovery work for new attributes and sources being added to MDM system, understanding what the impacts will be from a probabilistic matching engine (PME)/deduplication standpoint, and analyzing the results.</li><li>Coordinate with Solution Architects & Data Solutions Designers to utilize reusable solution design patterns that meet enterprise standards.</li><li>Conduct research and discovery in support of upgrades, cloud enablement, continuous delivery, and other modern engineering practices.</li><li>Participate in root cause analysis with appropriate subject matter experts to identify and address root causes to production issues.</li><li>Demonstrate strong analytical, organizational, and problem-solving skills.</li><li>Strong focus on long-term strategy, application stability, refactoring and re-usability.</li><li>Other duties as assigned.</li></ul> Qualifications: <ul><li>6+ years of experience supporting, designing and developing applications using IBM Master Data Management (MDM) Advanced Edition (Physical), with expertise in master data management concepts, including PME, deduplication, and survivorship.</li><li>Experience with development of data integration solutions with expertise in data modeling, data quality and data profiling.</li><li>Working knowledge of Data-as-a-Service (DaaS) and API management concepts and how to use these with different types of integration technologies. Experience with REST APIs, JSON, SOAP Web Services, XML, XSD, WSDL, Python, and Kafka.</li><li>Ability to elicit, gather and analyze user requirements.</li><li>Strong leadership experience.</li><li>Ability to communicate well with business stakeholders.</li><li>Expertise in Banking, Insurance and financial services.</li><li>Requires a Bachelor's degree in Computer Science, Computer Engineering, Information Systems or related field. A combination of education and experience may meet requirements.</li></ul> Location: This position can be located 100% remote within the United States or will be a hybrid work from schedule with a minimum of three days per week in the office if you are located within 50 miles of the new Zions Technology Center in Midvale, UT. Pay Range: $115,000 - $160,000 (Based upon relatable skills/experience) Benefits: <ul><li>Medical, Dental and Vision Insurance - START DAY ONE! </li><li>Life and Disability Insurance, Paid Parental Leave and Adoption Assistance </li><li>Health Savings (HSA), Flexible Spending (FSA) and dependent care accounts</li><li>Paid Training, Paid Time Off (PTO) and 11 Paid Federal Holidays</li><li>401(k) plan with company match, Profit Sharing, competitive compensation in line with work experience </li><li>Mental health benefits including coaching and therapy sessions</li><li>Tuition Reimbursement for qualifying employees </li><li>Employee Ambassador preferred banking products</li><li>Employees may, at the company's discretion, be eligible for a bonus.</li></ul> Apply now if you have a passion for impactful outcomes, enjoy working collaboratively with co-workers, and want to make a difference for the clients and communities we serve.

Key Responsibilities•Ability to breakdown complex problems using appropriate design patterns and best practices demonstrating a strong knowledge foundation and a sound approach to problem-solving.•Ability to build automated pipelines that integrate and transport software through reliable secure and governed channels from conception to production. Ability to The Selected candidate will be responsible for the design development and administration of data management solutions that align with the business needs of the organization.•Design relational databases to support business enterprise applications and physical data modeling according to project requirements for data acquisition and security as well as customer-defined deliverables.•Create build and lead the implementation of strategies and roadmaps for enterprise and department level data projects. Collaborate with internal and external personnel including senior management system architects' software developers database administrators design analysts and information modeling experts to determine project requirements and capabilities and strategize development and implementation timelines.•Work alongside data engineers to develop comprehensive automated ELT/ELT workflows that align with database design.•Manage source data access security configure authentication and authorization manage and enforce data policies and standards.•Document and organize findings including data retrieval stand up libraries.•Collaborate with Business Analysts to gather business requirements for Analytics and Reporting.•Build and modify tables views and optimize SQL queries indexes as needed.•Responsible for implementing security on developed Dashboards in accordance with security and access requirements.•Good to have Insurance Domain language understanding.Required Skills•Has at least 10+ years of relevant experience in Data Engineering especially ADF Python PySpark SparkSql Sql and Databricks programming and cloud data ingestion data pipelines handling structured and semi-structured data formats like JSON XML. Must have experience in building ETL/ELT Data warehouses/lakes for Real time event driven scenarios.•Good to have Insurance Domain language understanding. Experience in development of Power BI backend development includes Sql relevant experience•Engage in prototyping designing developing publishing Power BI Reports.•Develop Power BI Data models and recommend best practices for report creation and visualizations.•Implement security in reports and dashboards.•Troubleshooting and resolving issues related to performance environments monitoring and operability of Power BI reports Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.

Job Description This position is responsible for designing, developing, and maintaining robust data solutions to support business operations and decision-making processes. This role involves working with cutting-edge technologies, including Microsoft Azure Cloud, Snowflake, and Power BI, to build and manage data lakes, data warehouses, and reporting systems. The Database Engineer ensures data accuracy, system reliability, and high-quality deliverables by performing thorough quality assurance testing and providing user support. Collaboration with IT and business teams is essential to identify opportunities for operational efficiency and align data solutions with organizational goals. Experience is Everything: At LCS, experience is everything. We provide you the opportunity to use your talents in a progressive, growing organization that makes a positive difference in the lives of the seniors we serve. If you are seeking an organization that gives back, you'll love working here. Our principles and hospitality promise define our company culture. LCS employees can be found participating in volunteer activities, getting involved in our committees or collaborating with team members in our innovative workspace. You'll find several opportunities to grow as a professional, serve the community, and enhance the lives of the seniors. The Role: <ul> <li>Design, develop, and maintain scalable data lakes, data warehouses, and data storage solutions.</li> <li>Create and optimize ETL/ELT pipelines for efficient data processing and integration.</li> <li>Implement data models to support reporting, analytics, and operational workflows.</li> <li>Build, manage, and enhance Power BI dashboards and reports for actionable insights.</li> <li>Collaborate with business stakeholders to gather reporting requirements and ensure data accuracy.</li> <li>Leverage Microsoft Azure Cloud services (e.g., Data Factory, Synapse Analytics, Data Lake Storage) for data integration and processing.</li> <li>Utilize Snowflake to manage and optimize data warehouse performance.</li> <li>Perform thorough QA testing to ensure data accuracy, system reliability, and high performance.</li> <li>Provide user support and troubleshoot issues with data systems, pipelines, and reporting tools.</li> <li>Work closely with cross-functional teams to align data solutions with organizational objectives.</li> <li>Identify and implement operational efficiencies in data workflows and system designs.</li> <li>Ensure compliance with data governance, privacy, and security standards.</li> <li>Follow best practices for version control, documentation, and CI/CD processes.</li> <li>Stay updated on emerging technologies and trends to enhance data strategies and tools.</li> <li>Recommend improvements to existing data processes and tools to drive innovation.</li></ul> Experience: <ul> <li>4 year degree in Computer Science, MIS, or equivalent work experience.</li> <li>2-4 years of experience in developing, implementing, and supporting enterprise solutions.</li> <li>Experience with database concepts, advanced SQL, and reporting/Business Intelligence software.</li></ul> Knowledge & Skills: <ul> <li>Expertise in data architecture, data modeling, and building scalable data solutions.</li> <li>Proficiency with Microsoft Azure Cloud services (e.g., Azure Data Factory, Synapse Analytics, Data Lake Storage).</li> <li>Strong experience with Snowflake for data warehousing and optimization.</li> <li>Advanced skills in Power BI for creating reports, dashboards, and data visualizations.</li> <li>Proficient in SQL and scripting languages like Python for data integration and analysis.</li> <li>Solid understanding of ETL/ELT pipelines and data integration workflows.</li> <li>Strong problem-solving skills and attention to detail for troubleshooting data systems.</li> <li>Excellent communication and collaboration abilities to work effectively across teams.</li></ul> Why LCS Industry leader. The Nation's third-largest senior living operator ranked number one in customer satisfaction among senior living communities. Inclusive and collaborative culture. We're dedicated to diversity, equity, and inclusion and have an engaged Diversity and Inclusion Council focused on creating awareness and educating employees on inclusivity. In addition, LCS creates a collaborative culture that provides an exceptional experience for every employee. Top Workplace USA: LCS has earned the Top Workplaces USA award for several consecutive years and is recognized for our strong company culture and engaged workforce. In addition, LCS earned ten culture excellence awards in areas such as DE&I practices, top managers, professional development, and clued-in leaders, to name a few. Top Workplace Iowa: LCS employees truly believe we are an employer choice. This recognition, for 4 years running, is in large part due to the culture of excellence that our employees help deliver every single day. Competitive pay, great benefits and vacation time. We are an equal opportunity employer with benefits including medical, dental, life insurance, disability, 401(k) with company match and paid parental leave. Charity and community involvement. We are recognized as a national team for the Alzheimer's Association and consistently a top contributor to United Way. We also support our employee's individual community contributions and provide opportunities to get involved at our corporate locations and in our communities. Outstanding advancement opportunities. LCS is growing and we think you should too. Our company growth allows for internal growth opportunities across all of our business lines. Ongoing career development. Onsite education opportunities, education assistance, and continuing education credits allow LCS employees to keep their knowledge of current industry changes relevant. LCS creates living experiences that enhance the lives of seniors. You'll see this commitment in our people. They're talented, dedicated professionals who truly care about residents, with each conducting his or her work with integrity, honesty, and transparency according to the principles of LCS. We strive to help every community succeed-strengthening available resources, establishing proven practices that lead to long-term growth and creating lasting value for those living in, working for and affiliated with the community. Check us out on our website: www.lcsnet.com Travel Frequency: 0-10% Estimated Salary Range: $110,400 - $138,000 The actual title and salary offer will carefully consider a wide range of factors, including your skills, qualifications, experience, and other relevant factors. A POST-OFFER BACKGROUND CHECK, INCLUDING REFERENCES, IS REQUIRED LCS IS AN EQUAL OPPORTUNITY EMPLOYER

ALTA IT Services is looking for a Data Engineer to work for a large government contractor in Washington DC Metro area. This is an exciting opportunity to join a great team and bring your skills to the table. Duration: Six-month temp to perm Pay Rate: 75-85/hr W-2 Clearance: Active DoD Secret - would love higher Work Model: Hybrid - 3 days on-site in Arlington VA The ideal candidate will have a strong background in data engineering and experience in designing, implementing, and maintaining robust pipelines. This role requires deep understanding of data architecture, data warehousing, ETL processes and big data technologies. The Data Engineer will work closely with data scientists, analyst, and other stakeholders to ensure the seamless flow and integrity of data across the organization. Responsibilities : Build, maintain, and optimize data pipelines for extracting, transforming, and loading (ETL) data from various sources into our data warehouse. Integrate large, complex data sets that meet functional and non-functional business requirements. Design and implement scalable and reliable data warehousing solutions. Extract data from various sources, transform it into a usable format, and load it into Azure data storage solutions such as Azure Data Lake Storage, Azure SQL Database, or Azure Synapse Analytics Develop and maintain data pipelines using Azure Data Factory and other relevant Azure technologies Manage and optimize databases, ensuring data quality, integrity, and security. Automate manual processes, optimize data delivery, and re-design infrastructure for greater scalability. Collaborate with data scientists, analysts, and other stakeholders to understand data needs and deliver high-quality data solutions. Document data flows, processes, and system architecture to ensure clarity and knowledge sharing within the team. Required : An active DoD Secret clearance. BA/BS in Computer Science, Information Technology, Data Analytics, or related field. 5 years of related experience. Strong experience in data extract transform load (ETL) processes using Azure services, such as Azure Data Factory, Azure Databricks, Azure Data Lake Storage, Azure SQL Database, or Azure Synapse Analytics. 4 years experience and skill writing coding languages (such as SQL, Python, R, SAS, and Java Scripts). 2 years experience working with projects involving machine learning, natural language processing, robotics process automation, artificial intelligence, text and/or data mining, as well as statistical and mathematical methods. Experience with using SQL to conduct complex database queries. Experience with AI/ML. Desired : 2 years of experience working within the DoD or IC Community. Master's degree in Mathematics, Statistics, CS, or Data Analytics. Clearance: TS with SCI Eligibility. The project is specifically focusing on managed identities and cyber threats. You will be dealing with very large data sets of audit logs that will need to be cleaned and processed to use for data modeling. We are looking for someone who is "jack of all trades" and can do data engineering, analysis and science. System One, and its subsidiaries including Joul, ALTA IT Services, CM Access, TPGS, and MOUNTAIN, LTD., are leaders in delivering workforce solutions and integrated services across North America. We help clients get work done more efficiently and economically, without compromising quality. System One not only serves as a valued partner for our clients, but we offer eligible full-time employees health and welfare benefits coverage options including medical, dental, vision, spending accounts, life insurance, voluntary plans, as well as participation in a 401(k) plan. System One is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, age, national origin, disability, family care or medical leave status, genetic information, veteran status, marital status, or any other characteristic protected by applicable federal, state, or local law.

Description Summary:The Senior Data Engineer will be responsible for designing, developing, and maintaining the data warehouse and analytics architecture to fulfill the organization's business analysis and reporting requirements. This role also involves mentoring team members and providing strategic recommendations to support the organization's goals. Duties to include, (but not limited to): <ul><li>Create and maintain optimal data pipeline architecture.</li><li> Assemble data sets that meet business requirements.</li><li> Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, APIs, and Infor/AWS data lake tools.</li><li> Focus on gathering and cleaning data to ensure its quality and accuracy.</li><li> Build analytics tools that utilize the data pipeline to provide actionable insights to business customers.</li><li> Run tests on designs to isolate errors.</li><li> Identify and implement internal process improvements.</li><li> Collaborate with business and technology stakeholders to develop and maintain data warehouse and data analytics platform architecture and utilization.</li><li> Collaborate with Business Analysts to develop analysis, reports, dashboards, visualizations, and metrics as business needs dictate.</li><li> Utilize source control and change management practices for documentation and collaboration.</li><li> Mentor and guide others on the team to help them develop their skills and knowledge. </li><li> Create data models to support analysis and reporting.</li><li> Participate in developing data strategies and aligning them with the organization's goals. </li><li> Provide recommendations on critical technical decisions regarding data, architecture, infrastructure, and systems. </li><li> Provide occasional after-hours support on an as-needed basis.</li><li> Perform other duties as assigned.</li></ul> A Data Engineer will possess and/or demonstrate the ability to: <ul><li>Provide first-rate IT assistance and solutions.</li><li> Collaborate closely with stakeholders from across the entirety of WSC to understand their data needs and deliver insights. </li><li> Remain highly organized, flexible, and able to react well to immediate issues with attention to detail and deadlines.</li><li> Be a self-initiator and work with minimal supervision independently and/or with teammates.</li><li> Lead data engineering projects, overseeing their execution from start to finish. </li><li> Write and debug complex SQL/Transact-SQL/ OLAP queries/code.</li><li> Learn innovative technologies and keep abreast of the latest technological advances.</li><li> Be motivated by the desire to make a difference in the business through collaboration with IT and business stakeholders.</li></ul> Education / Experience: <ul><li>Possess a minimum of a two-year degree from an accredited college or university in information technology, mathematics, computer science or a related field and a minimum of 5 years of related experience.</li><li> Strong analytical ability, judgment, and problem analysis techniques.</li><li> Strong written and verbal communication skills.</li><li> Strong technical communication skills with a variety of audiences.</li><li> Experience developing reports and visualizations in Power BI and other tools.</li><li> Experience with Infor CSDE/M3, Microsoft Dynamics 365, Phocas, or Birst Analytics a plus, but not required.</li><li> Project participation experience, including understanding of application lifecycle management.</li><li> Proficient with source control and change management practices.</li><li> A successful history of manipulating, processing, and extracting value from disconnected datasets.</li></ul> Working Demands / Physical Requirements: <ul><li>General office conditions with continuous sitting with heavy computer usage.</li><li> Occasional short-term out-of-town travel for educational purposes.</li><li> Occasional after-hours support.</li><li> Ability to lift up to 50 pounds.</li></ul> Reports to: Business Intelligence Director Conclusion: The Senior Data Engineer will exemplify the high standards that have come to be associated with WSC. As an ESOP owner, acting with honor, integrity, and positivity is imperative.

Summary: As a Data Engineer at Meta, you will shape the future of people-facing and business-facing products we build across our entire family of applications (Facebook, Instagram, Messenger, WhatsApp, Reality Labs, Threads). Your technical skills and analytical mindset will be utilized designing and building some of the world's most extensive data sets, helping to craft experiences for billions of people and hundreds of millions of businesses worldwide.In this role, you will collaborate with software engineering, data science, and product management teams to design/build scalable data solutions across Meta to optimize growth, strategy, and user experience for our 3 billion plus users, as well as our internal employee community.You will be at the forefront of identifying and solving some of the most interesting data challenges at a scale few companies can match. By joining Meta, you will become part of a world-class data engineering community dedicated to skill development and career growth in data engineering and beyond.Data Engineering: You will guide teams by building optimal data artifacts (including datasets and visualizations) to address key questions. You will refine our systems, design logging solutions, and create scalable data models. Ensuring data security and quality, and with a strong focus on efficiency, you will suggest architecture and development approaches and data management standards to address complex analytical problems.Product leadership: You will use data to shape product development, identify new opportunities, and tackle upcoming challenges. You'll ensure our products add value for users and businesses, by prioritizing projects, and driving innovative solutions to respond to challenges or opportunities.Communication and influence: You won't simply present data, but tell data-driven stories. You will convince and influence your partners using clear insights and recommendations. You will build credibility through structure and clarity, and be a trusted strategic partner. Required Skills: Data Engineer, Product Analytics Responsibilities: Conceptualize and own the data architecture for multiple large-scale projects, while evaluating design and operational cost-benefit tradeoffs within systems Create and contribute to frameworks that improve the efficacy of logging data, while working with data infrastructure to triage issues and resolve Collaborate with engineers, product managers, and data scientists to understand data needs, representing key data insights visually in a meaningful way Define and manage Service Level Agreements for all data sets in allocated areas of ownership Determine and implement the security model based on privacy requirements, confirm safeguards are followed, address data quality issues, and evolve governance processes within allocated areas of ownership Design, build, and launch collections of sophisticated data models and visualizations that support multiple use cases across different products or domains Solve our most challenging data integration problems, utilizing optimal Extract, Transform, Load (ETL) patterns, frameworks, query techniques, sourcing from structured and unstructured data sources Assist in owning existing processes running in production, optimizing complex code through advanced algorithmic concepts Optimize pipelines, dashboards, frameworks, and systems to facilitate easier development of data artifacts Influence product and cross-functional teams to identify data opportunities to drive impact Mentor team members by giving/receiving actionable feedback Minimum Qualifications: Minimum Qualifications: Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent 7 years of experience where the primary responsibility involves working with data. This could include roles such as data analyst, data scientist, data engineer, or similar positions 7 years of experience with SQL, ETL, data modeling, and at least one programming language (e.g., Python, C++, C#, Scala or others.) Preferred Qualifications: Preferred Qualifications: Master's or Ph.D degree in a STEM field Public Compensation: $173,000/year to $242,000/year bonus equity benefits Industry: Internet Equal Opportunity: Meta is proud to be an Equal Employment Opportunity and Affirmative Action employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law. Meta participates in the E-Verify program in certain locations, as required by law. Please note that Meta may leverage artificial intelligence and machine learning technologies in connection with applications for employment. Meta is committed to providing reasonable accommodations for candidates with disabilities in our recruiting process. If you need any assistance or accommodations due to a disability, please let us know at accommodations-extfb.com.

What We Do The CERT Cyber Mission Readiness (CMR) directorate fortifies the U.S. and its partners against cyber threats. We design large and small-scale exercises, facilitate tabletop scenarios, run cybersecurity contests, and create open-source tools to bolster cyber defense capabilities. Actively partnering with sponsors, CMR identifies and addresses vulnerabilities to advanced cyber adversaries. Central to CMR is a culture of collaboration and dedication. Committed to arming our nation and allies with the tools to combat cyber threats, we seek passionate individuals who are technologically inclined and value teamwork. The right attitude, coupled with a thirst for knowledge, is paramount. We foster continuous professional growth through extensive training, mentorship, and cross-learning. Within the CMR directorate is the Operational Readiness and Evaluation (ORE) team. ORE intensifies the Cyber Workforce's defense posture. Collaborating with SEI teams and governmental units, we assess operational capabilities against established benchmarks, shaping strategies for maximum mission assurance. Leveraging innovative simulation technology, we develop high-end virtual training arenas and design challenging scenarios, striving to be at the pinnacle of cyber readiness. ORE champions a unified and progressive culture. Driven to uplift the national cybersecurity workforce, we invite individuals with a zeal for technology, collaboration, and ongoing learning. Our ethos revolves around comprehensive training, mentorship, and skill-sharing. Position Summary As a Senior Data Engineer, you will be integral to our efforts in developing data architectures and analytics frameworks that support complex analysis and operational missions. Your role involves not only maintaining robust data solutions but also directly engaging with mission partners to assess and enhance their data engineering capabilities. Your expertise will be vital in defining new work roles and creating tailored training programs to empower our government partners. Requirements: <ul><li>You have BS in Computer Science, Engineering, Mathematics, or a related discipline with ten (10) years of experience; OR MS in the same fields with eight (8) years of experience; OR PhD in the same fields with five (5) years of experience. </li><li>Willingness to travel up to 25%, including international travel, supporting SEI's mission. </li><li>Must pass a background check and be eligible for a Department of Defense security clearance. </li></ul> Duties: <ul><li>Design and implement effective data pipelines and architectures to support advanced data analysis and mission-critical operations. </li><li>Manage the data lifecycle from modeling to database architecture, ensuring integrity and accessibility of data. </li><li>Develop and optimize tools for data extraction, transformation, and loading (ETL). </li><li>Utilize data science algorithms and machine learning to address specific challenges faced by Mission Partners. </li><li>Work collaboratively with Mission Partners to evaluate and improve their data engineering and analysis programs. </li><li>Define new work roles and develop customized training, exercise, and simulation programs to enhance the skills and capacities of individual team members and groups. </li><li>Ensure data systems comply with cybersecurity controls and policies tailored to the defense sector. </li></ul> Knowledge, Skills, and Abilities: Joining ORE provides a chance to enhance your skills. While prior experience with the following is advantageous, it's not mandatory: <ul><li>Strong proficiency in object-oriented programming languages such as Python, Java, and SQL. </li><li>Expertise in cyber data processing, management, and visualization, including experience with tools like Elasticsearch, Apache NiFi, Kafka, and Logstash. </li><li>Comprehensive knowledge in data science and machine learning, both supervised and unsupervised. </li><li>Demonstrable experience with Big Data platforms and tools such as Jupyter Lab, ROCK NSM, and ALEX. </li><li>Excellent communication skills with the ability to work effectively in a team environment and engage with government Mission Partners. </li><li>Commitment to ongoing learning and professional development in data engineering and cybersecurity. </li></ul> Desired Experience: <ul><li>Previous roles in government or military sectors dealing with cybersecurity, including identifying and eliminating cyber threats, large-scale cyber data collection via log and sensor instrumentation, and cyber data management. </li><li>Experience in educational roles, particularly in instructional design, delivering training and instructional programs in data-related fields, instrumenting, collecting, and analyzing learner experience data. </li></ul> Why work here? <ul><li>Join a world-class organization that continues to have a significant impact on software. </li><li>Work with cutting-edge technologies and dedicated experts to solve tough problems for the government and the nation. </li><li>Be surrounded by knowledgeable staff with broad expertise across AI/ML, cybersecurity, software engineering, risk management, and policy creation. </li><li>Get 8% monthly contribution for your retirement, without having to contribute yourself. </li><li>Get tuition benefits to CMU and other institutions for you and your dependent children. </li><li>Enjoy a healthy work/life balance with flexible work arrangements and paid parental and military leave. </li><li>Get access to university resources including mindfulness programs, childcare and back-up care benefits, a monthly transit benefit on WMATA, free transportation on the Pittsburgh Regional Transit System. </li><li>Enjoy annual professional development opportunities; attend conferences and training or obtain a certification and get reimbursed for membership in professional societies. </li><li>Qualify for relocation assistance and so much more. </li></ul> Joining the CMU team opens the door to an array of exceptional benefits available to eligible employees. Those employees who are benefits eligible have the opportunity to experience the full spectrum of advantages from comprehensive medical, prescription, dental, and vision insurance to an enticing retirement savings program offering a generous employer contribution. You can also unlock your potential with tuition benefits and take well-deserved breaks with ample paid time off and observed holidays. Finally, rest easy knowing you are covered by life and accidental death and disability insurance. Other perks include a free Pittsburgh Regional Transit bus pass, our Family Concierge Team to help navigate childcare needs, fitness center access, and so much more! For a comprehensive overview of the benefits that may be awaiting you, explore our Benefits page. At Carnegie Mellon, we value the whole package when extending offers of employment. Beyond just credentials, we consider the role and responsibilities, your invaluable work experience, and the knowledge gained through education and training. We acknowledge and appreciate your unique skills and the diverse perspective you bring. Your journey with us is about more than just a job; it's about finding the perfect fit for your professional growth and personal aspirations. Are you interested in an exciting opportunity with an exceptional organization?! Apply today! Location Arlington, VA, Pittsburgh, PA Job Function Software/Applications Development/Engineering Position Type Staff - Regular Full Time/Part time Full time Pay Basis Salary More Information: <ul><li>Please visit "Why Carnegie Mellon" to learn more about becoming part of an institution inspiring innovations that change the world.</li><li>Click here to view a listing of employee benefits</li><li>Carnegie Mellon University is an Equal Opportunity Employer/Disability/Veteran.</li><li>Statement of Assurance</li></ul>

Sysdig Data Engineer Interview Guide

Overview

Sysdig Data Engineer Salary

Sysdig Data Engineer Interview Process

Sysdig Data Engineer Interview Questions

Sysdig Data Engineer Jobs