Role - MS FabricExperience- 6 to 12 yrsLocation- Pan India JD-Primary Responsibilities:<ul><li>Design and build Data Warehouse structures (databases and SSAS cubes);</li><li>Develop unique, simple solutions to meet the immediate and long-term needs of information customers and stakeholders;</li><li>Develop data architecture, data modelling, and ETL mapping solutions;</li><li>Support the development and validation required throughout the entire lifecycle of the data warehousing and business intelligence systems;</li></ul> Additional tasks and duties are assigned;<ul><li>Support the data warehouse technical ecosystem, including SharePoint for the delivery of reporting and analytic applications;</li><li>Ability to present complex topics to key stakeholders, business analysts and senior executives;</li><li>Ability to document business, functional, and technical requirements;</li><li>Lead cross-functional teams in developing innovative client solutions;</li><li>Understanding of cloud data warehouse platforms and modern data warehousing techniques;</li><li>Troubleshooting the reporting database environment and reports;</li><li>Provide quality assurance of imported data, working with quality assurance analyst if necessary;</li><li>Commissioning and decommissioning of data sets;</li><li>Managing and designing the reporting environment, including data sources, security, and metadata;</li><li>Generating reports from single or multiple systems.</li></ul> Requirements:<ul><li>10+ years of experience as a Data Warehouse architect and ETL developer;</li><li>At least three years of data modelling experience, and knowledge of 3NF, Star-Schema and other data warehousing modelling techniques;</li><li>Experience with BI reporting tools</li><li>Ability to communicate information clearly, concisely, and effectively both orally and in writing;</li><li>Five or more years of experience with data analysis and data profiling using SQL and other tools;</li><li>Extensive experience in T-SQL development, MS SQL Server preferred, with strong knowledge in query optimization;</li><li>Very strong written and verbal communication skills;</li><li>Big Data and Advanced Analytics Experience;</li><li>Ability to analyse existing tools and databases and provide software solution recommendations;</li><li>SQL Server analysis services. Tabular modelling;</li><li>Azure cloud-based services, good to have.</li><li>MS fabric Knowledge , exposure certification</li></ul>

We are looking for a talented Data Engineer with experience in ETL processes, database administration, and scripting, as well as expertise in tools such as Snowflake, Azure Data Factory, DBT, and Talend. The ideal candidate will have a strong technical background and be capable of working on various aspects of data engineering, from building data pipelines to optimizing data storage and retrieval. Key Responsibilities:ETL Development and Maintenance:<ul><li>Design, develop, and maintain scalable ETL processes to extract, transform, and load data from various sources into data warehouses and data lakes.</li><li>Utilize Talend and Azure Data Factory to create and manage ETL workflows that ensure data accuracy and efficiency.</li></ul>Database Administration:<ul><li>Administer and optimize databases, ensuring high performance, availability, and security of data assets.</li><li>Manage Snowflake data warehouse, including schema design, data partitioning, and performance tuning.</li></ul>Data Engineering and Scripting:<ul><li>Develop and maintain Python scripts for data processing, automation, and integration tasks.</li><li>Implement and optimize data transformation processes using DBT (Data Build Tool) to ensure data quality and consistency.</li></ul>Data Integration and Management:<ul><li>Integrate data from multiple sources and ensure seamless data flow and accessibility.</li><li>Collaborate with data analysts and data scientists to understand data requirements and deliver relevant solutions.</li></ul>Performance Optimization:Monitor and optimize data pipelines and database performance to ensure efficient data processing and retrieval.

Project descriptionDevelop scalable data collection, storage, and distribution platform to house data from vendors, research providers, exchanges, PBs, and web-scraping. Make data available to systematic & fundamental PMs, and enterprise functions: Ops, Risk, Trading, and Compliance. Develop internal data products and analytics Responsibilities<ul><li>Web scraping using scripts/APIs/Tools</li><li>Help build and maintain greenfield data platform running on Snowflake and AWS</li><li>Understand the existing pipelines and enhance pipelines for the new requirements.</li><li>Onboarding new data providers</li><li>Data migration projects</li></ul> SkillsMust have<ul><li>Streamlit Expertise</li><li>SQL</li><li>Python</li><li>Linux</li><li>Containerization(Docker, Kubernetes)</li><li>Good communication skills</li><li>AWS</li><li>Strong on Dev ops side of things(K8s, Docker, Jenkins)</li></ul> Nice to have<ul><li>Market Data Projects/ Capital markets exp</li><li>Snowflake is a big plus</li><li>Airflow</li></ul>

Job Title: Data Engineer Organization: National e-Governance Division (NeGD), Ministry of Electronics and Information Technology, Government of India Location: New Delhi, India Job Summary:The National e-Governance Division (NeGD) is seeking a highly skilled and motivated Data Engineer to join our technical team. The ideal candidate will be responsible for building and maintaining the data infrastructure required for optimal extraction, transformation, and loading of data from various sources to support e-governance initiatives under the Digital India program. This role is crucial in ensuring the reliability, scalability, and efficiency of data systems that enable data-driven decision-making and enhance public service delivery. Key Responsibilities: Data Pipeline Development:<ul><li>Design, develop, and maintain scalable data pipelines and ETL processes to handle large volumes of data.</li><li>Ensure the timely and accurate movement of data from various sources to data warehouses or data lakes.</li></ul> Data Integration:<ul><li>Integrate data from a variety of sources, including databases, APIs, and third-party systems.</li><li>Collaborate with cross-functional teams to understand data requirements and ensure seamless data flow.</li></ul> Data Warehousing:<ul><li>Develop and maintain data warehouse architectures to support business intelligence and analytics needs.</li><li>Optimize database schemas and queries for performance and scalability.</li></ul> Data Quality and Governance:<ul><li>Implement data validation and cleansing processes to ensure data accuracy and consistency.</li><li>Enforce data governance policies and maintain documentation of data models and processes.</li></ul> Performance Optimization:<ul><li>Monitor and improve the performance of data systems.</li><li>Troubleshoot and resolve issues related to data pipelines and infrastructure.</li></ul> Collaboration and Communication:<ul><li>Work closely with data scientists, analysts, and other stakeholders to support their data needs.</li><li>Provide technical guidance and support to team members on data engineering best practices.</li></ul> Security and Compliance:<ul><li>Ensure data systems comply with government regulations and data protection laws.</li><li>Implement security measures to protect data integrity and privacy.</li></ul> Qualifications and Experience: Education: <ul><li>Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.</li><li>A Master's degree or specialization in Data Engineering or related disciplines is an added advantage.</li></ul> Experience:<ul><li>Minimum of 5-7 years of experience in data engineering or related roles.</li><li>Proven experience in building and maintaining large-scale data pipelines and data architectures.</li><li>Experience with big data technologies and cloud platforms is highly desirable.</li></ul> Certifications:<ul><li>Relevant certifications in data engineering, cloud technologies, or big data platforms are advantageous.</li></ul> Skills and Competencies: Technical Proficiency:<ul><li>Strong programming skills in languages such as Python, Java, or Scala.</li><li>Proficiency with data processing frameworks like Apache Spark, Hadoop, or Flink.</li><li>Experience with ETL tools and workflow schedulers like Apache Airflow or NiFi.</li><li>Knowledge of SQL and NoSQL databases (e.g., MySQL, PostgreSQL, MongoDB).</li><li>Familiarity with cloud platforms (AWS, Azure, or Google Cloud) and their data services.</li></ul> Data Modeling and Warehousing:<ul><li>Strong understanding of data modeling concepts and data warehouse design.</li><li>Experience with data warehousing solutions like Amazon Redshift, Google BigQuery, or Snowflake.</li></ul> Problem-Solving Skills:<ul><li>Excellent analytical and troubleshooting abilities.</li><li>Ability to handle complex data-related challenges and provide effective solutions.</li></ul> Communication and Collaboration:<ul><li>Strong communication skills to work effectively with cross-functional teams.</li><li>Ability to explain technical concepts to non-technical stakeholders.</li></ul> Organizational Skills:<ul><li>Ability to manage multiple tasks and projects simultaneously.</li><li>Strong attention to detail and commitment to delivering high-quality work.</li></ul> Understanding of E-Governance:<ul><li>Familiarity with the Digital India initiative and e-governance frameworks.</li><li>Understanding of government data policies, standards, and regulatory requirements.</li></ul>

Greetings from TCS!TCS has proudly created an exceptional leadership community with leaders like you to drive change, foster innovation and retain TCS as the top player in the business market.As a Technical Professional you will provide expertise in specific technology / methodology / business / application / product in terms of technical / solution architect. Job skill: GCP Data EngineerExperience: 5 to 10 yearsLocation: Bangalore/Mumbai/Chennai/Delhi Required Technical Skill Set:<ul><li>GCP,</li><li>PySpark</li><li>Python</li><li>HDFS</li><li>Hadoop</li><li>SQL</li></ul> Must-Have:Good Hands-on knowledge on GCPShould have worked on Data Migration projects, from On-prem to CloudShould have Cloud Storage Knowledge, Big Query, Cluster KnowledgeSound programming knowledge on PySpark & SQL in terms of processing large amount of semi structured & unstructured dataAbility to design data pipelines in end-to-end mannerKnowledge on Avro, Parquet formatKnowledge on working on Hadoop Big Data platform and ecosystemStrong debugging and troubleshooting capabilities.Have experience to guide the Technical Team for attaining the delivery milestones Disclaimer:We encourage you to register at www.tcs.com/careers for exploring an exciting career with TCS. At the time of your application to TCS, the personal data contained in your application and resume will be collected by TCS and processed for the purpose of TCS's recruitment related activities. Your personal data will be retained by TCS for as long as TCS determines it is necessary to evaluate your application for employment as per our retention policy. You have the right to request TCS for temporary/permanent exclusion of your candidature from any recruitment related communication. For any such request you may write to careers.tcs.com.

Role- Azure Data EngineerLocation- Pan IndiaExperience- 5 TO 8 YRS JD- <ul><li>3+ years of relevant experience in Pyspark and Azure Databricks. </li><li>Proficiency in integrating, transforming, and consolidating data from various structured and unstructured data sources. </li><li>Good experience in SQL or native SQL query languages. </li><li>Strong experience in implementing Databricks notebooks using Python. </li><li>Good experience in Azure Data Factory, ADLS, Storage Services, Serverless architecture, Azure functions. </li><li>Advanced working knowledge in building robust data pipelines using ADF </li><li>Experience in building processes supporting data transformation, data structures, metadata, dependency and workload management. </li><li>Exposure to any cloud based Datawarehouse like Snowflake. </li><li>Exposure to basic issues in working within a Cloud (Azure) environment. ·</li><li>Advanced working SQL knowledge and experience working with relational/non-Relational databases, query. </li><li>Exposure to Jira and Confluence required.</li><li> Experience in working in an Agile project management environment. </li><li>A successful history of manipulating, processing and extracting value from large, disconnected datasets. </li><li>Build data pipeline infrastructure for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, cloud based relational or non-relational databases and/or scripting languages like Perl/Python </li><li>Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.</li></ul>

Data Engineer Jobs in Delhi, India

Etl Data Engineer Ms Fabric
November 15, 2024

Data Engineer
November 15, 2024

Data Engineer For Market Data Projects With Streamlit Expertise
November 12, 2024

Data Engineer
November 8, 2024

Gcp Data Engineer
November 5, 2024

Azure Data Engineer
October 27, 2024

Quick filters

Top companies

Data Engineer Jobs in Delhi, India

Etl Data Engineer Ms FabricNovember 15, 2024

Data EngineerNovember 15, 2024

Data Engineer For Market Data Projects With Streamlit ExpertiseNovember 12, 2024

Data EngineerNovember 8, 2024

Gcp Data EngineerNovember 5, 2024

Azure Data EngineerOctober 27, 2024