Pl note candidate should have overall experience of 7+ yrs in Bigdata along with Hadoop and Java / scala . He/ She should be available to join in 30 days notice period Experience - 8-15 yrsLocation : Bangalore or ChennaiWork Mode : Hybrid (2 days in a week should be ready to work in office and 3 days from home)Skills Required- Bigdata along with Hadoop , Hive, Java/ scala Notice period- Immediate to 30 days only (Not going ahead with 45-90 days notice period) Project Description We've been engaged by a large UK Based Investment Bank in the Corporate, Commercial and Institutional Banking (CCIB) portfolio to work on the projects for managing analytics support for all the key initiatives of all segments/products .• Ensuring quality service output, submitted in a timely manner and communicated effectively• Continuous Improvement in Productivity to the standards prescribed from time to time• Manage data & information support for all the key initiatives across Retail Bank supporting countries & group teams.• Work closely with technology on solutions to resolve identified production issues which are impacting existing infrastructure /solution covering data quality, data assurance, refresh timeliness, data security governance• To work closely with Business & Technology to review the integration of new functionality / technology implementation projects initiated• Extend analytics support spanning across entire gamut of business (Customer Acquisition, Portfolio Management, New Product Development)• Manage analysis & reporting across all Retail products/segments• Interact with metrics owners across various functions and get end to end understanding of the metrics and applicability of the same across various MI dashboards• Provide development opportunities through regular engagement, ensuring that all staff is aware of the opportunities available in order to help them succeed within their role• Create and maintain optimal data pipeline architecture, assemble large, complex data sets that meet functional / non-functional business requirements.• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. Mandatory Skills• 7+ years of relevant technology experience• Hands on Hadoop and Bigdata tech. e.g. HIVE, HBase, Kafka, Spark• Strong analytic skills related to working with unstructured datasets.• Strong experience on SQL• Good Experience on Data Base & Data Warehouse building and management• Java / Scala programming experience• Exposure to data pipelines, architectures and data sets• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.• Build processes supporting data transformation, data structures, metadata, dependency and workload management.• Working knowledge of message queuing, stream processing and highly scalable 'big data' data stores. Experience - 8-15 yrsWork Location- Bangalore/ ChennaiWork Mode - Hybrid (2 days in office and 3 days at home)

Project descriptionProvide End to End Development solution to one of the largest investment banking products by scaling up the latest technology stack by adopting the cloud-based services under Payment Settlement stream. Responsibilities<ul><li>Write effective, scalable code using Python.</li><li>In-depth working knowledge of data management activities like data extraction, data mining, data analysis, data validation, and data storage.</li><li>Migrate legacy shell scripts to Python using optimized code standards to improve performance and error handling.</li><li>Test and debug the programs.</li><li>Managing the pipelines in AWS/GCP using Python code.</li><li>Coordinate with internal teams to understand user requirements and provide technical solutions.</li><li>Understand the current logic of the scripts and migrate it to Python.</li></ul> SkillsMust have<ul><li>4-6 Years of Experience working on Python Development projects</li><li>Deep understanding of Python's language, ecosystem, features, and experience in designing and developing data pipelines</li><li>The ability to program SQL databases with Python</li><li>Experience in interfacing with Resilient Distributed Datasets (RDDs) in Python</li><li>Solid experience in building queries in various SQL/NoSQL databases (Postgres/Cassandra/Elastic Search/MangoDB)</li><li>Experience in building a Data Quality Framework on various types of Datasets</li><li>Experience in deploying the MicroService in Docker/Kubernetes/OpenShift</li><li>Understanding of CI/CD Pipelines through Jenkins/Git/Github/Github Actions</li></ul> Nice to have<ul><li>Certification in Data Engineering or ML is an added advantage</li><li>Knowledge or working experience in building AI or ML pipeline</li><li>Experience in investment banking-related projects</li></ul>

Project descriptionWe are hiring for projects in the Middle East. The team comprises front-end and back-end developers, data analysts and data scientists, architects, analysts, and project managers. You will work with modern technologies and be an integral part of continuing to evolve our high-performing platforms. If you are serious about being a part of this fast-paced and exciting journey, this project should be top of the list! This role will be primarily responsible for closely collaborating with a business SME to implement data validation rules.Responsibilities<ol><li>Collaborating with a business SME to implement data validation rules.</li><li>Understand business requirements and actively provide input from a data perspective.</li><li>Understand the underlying data and flow of data.</li><li>Understand overall platform architecture and capabilities and how to load data effectively.</li><li>Recognize and adapt to the changes in processes.</li></ol> SKILLSMust have<ol><li>7 years + of relevant experience.</li><li>Proficient hands-on experience with PySpark-based data platforms: Databricks, Cloudera, Apache Spark, etc. (3y+).</li><li>Proficiency in Python and PySpark (5y+).</li><li>Advanced knowledge in SQL (5y+).</li><li>Should be able to analyse and understand complex data.</li><li>Knowledge of Azure data/delta lake, Azure DevOps, CI/CD, and Git is essential.</li><li>Should clearly understand the data platform lifecycle and contribute to preparing Design documents, Unit Test plans, and Code review reports.</li><li>Knowledge of data Lake house and business intelligence architecture.</li><li>Experience working in an Agile environment (Scrum, Agile) is a plus.</li><li>Interpersonal/communication skills (both oral/written) with the ability to communicate at various levels with clarity & precision.</li><li>Bachelor's degree in IT/Computer Science or related field.</li></ol> Nice to haveKnowledge of the financial markets, portfolio theory, and risk management is a plus.
#J-18808-Ljbffr

Project DescriptionOur Data Strategy and Operations team is focused on implementing and evolving clients' investment data strategy to enable the full range of business objectives; data for minimizing risk, data for efficient processes, and data for growth and innovation. It also delivers a set of capabilities to source, automate, and curate data assets to enable and enrich core investment and distribution processes, quantitative analytics, reporting and analysis, and advanced analytic efforts such as artificial intelligence.Responsibilities<ol><li>Design and develop Tableau reports and dashboards.</li><li>Provide operational support for existing Tableau reports/dashboards.</li><li>Provide subject matter expertise for designing of Tableau workbooks.</li><li>Participate in analysis, design, development, testing, deployment, and support of dashboards and reports.</li><li>Perform impact analysis and validation of existing production reports and universes after application and database upgrade.</li><li>Interface with business analysts and/or users to clarify requirements. Work closely with different business groups to cater to their reporting and dashboard needs.</li><li>Troubleshoot dashboard and report problems independently, determining the root cause and fixing the problem.</li><li>Understand reporting database schema/model, identify joins/tables, and perform data analysis.</li><li>Experience building complex dashboards using Tableau.</li><li>Implement industry BI standards and best practices.</li><li>Proactively identify problems/issues/risks and provide solutions.</li><li>Develop reports & dashboards using Business Objects and Tableau and provide status updates on a regular basis, committing to aggressive delivery timelines.</li></ol>SkillsMust Have:<ul><li>Data (SQL, ETL).</li><li>Python - does not need to be very deep knowledge, but aspects specific to work with data are important e.g. pandas.</li><li>BI/data analytics tools such as Tableau or PowerBI.</li></ul>Nice to Have:<ul><li>Alteryx.</li></ul>
#J-18808-Ljbffr

A candidate should have Streamlit Expertise! Project Description:Develop scalable data collection, storage, and distribution platform to house data from vendors, research providers, exchanges, PBs, and web-scraping. Make data available to systematic & fundamental PMs, and enterprise functions: Ops, Risk, Trading, and Compliance. Develop internal data products and analyticsResponsibilities:<ul><li>Web scraping using scripts/APIs/Tools</li><li>Help build and maintain greenfield data platform running on Snowflake and AWS</li><li>Understand the existing pipelines and enhance pipelines for the new requirements.</li><li>Onboarding new data providers</li><li>Data migration projects</li></ul>Mandatory Skills Description:<ul><li>SQL</li><li>Python</li><li>Linux</li><li>Containerization(Docker, Kubernetes)</li><li>Good communication skills</li><li>AWS</li><li>Strong on Dev ops side of things(K8s, Docker, Jenkins)</li></ul>

Job Description Project Description:Our Data Strategy and Operations team is focused on implementing and evolving clients' investment data strategy to enable the full range of business objectives; data for minimizing risk, data for efficient processes, and data for growth and innovation. It also delivers a set of capabilities to source, automate, and curate data assets to enable and enrich core investment and distribution processes, quantitative analytics, reporting and analysis, and advanced analytic efforts such as artificial intelligence. Responsibilities:Design and Develop Tableau reports and dashboards• Provide operational support for existing Tableau reports/dashboards• Provide subject matter expertise for designing of Tableau workbooks• Participate in analysis, design, development, testing, deployment and support of dashboards and reports• Perform impact analysis and validation of existing production reports and universes after application and database upgrade• Interface with business analysts and/or users to clarify requirements. Work closely with the different business groups to cater to their reporting and dashboard needs.• Troubleshoot dashboard and report problems independently, determining the root cause and fixing the problem.• Understand reporting database schema/model, identify joins/tables, and perform data analysis.• Experience building complex dashboards using Tableau• Implement industry BI standards and best practices• Proactively identify problems/issues/risks and provide solutions• Develop report & dashboards using Business Objects and Tableau and Provide status updates on regular basis and be committed to aggressive delivery timelines Mandatory Skills Description:1. Data (SQL, ETL).2. Python - does not need to be very deep knowledge, but aspects specific to work with data are important e.g. pandas.3. BI / data analytics tools such as Tableau or PowerBI Nice-to-Have Skills Description:Alteryx Languages:English: C1 Advanced

Project descriptionAs a Data Engineer you'll be working alongside data architects to take data throughout its lifecycle - acquisition, exploration, data cleaning, integration, analysis, interpretation and visualization. You will be creating the pipeline for data processing, data visualization, and analytics products, including automated services, and APIs. Responsibilities<ul><li>Use and create data pipelines, analyze data, tune query performance, perform advanced analytics all within the Azure Ecosystem.</li></ul> Mandatory Skills:<ul><li>Proficient in ADF, ADB, PySpark, Python</li><li>Strong SQL knowledge ( capable of writing and creating complex SQL queries)</li><li>7+ years of relevant experience</li></ul> Nice-to-Have Skills<ul><li>Good to have Azure Synapse knowledge.</li><li>Good to have PowerBI, ThoughtSpot knowledge.</li><li>CICD (GIT or Azure DevOps)</li><li>Proficient in query tuning and Performance optimization.</li></ul>

Data Engineer Luxoft Jobs

Hiring Senior Big Data Engineer 815 Yrs Bangalore Chennai Hybrid Model Immediate Joiners
December 17, 2024

Data Engineer Python
December 14, 2024

Senior Pyspark Data Engineer
December 11, 2024

Data Engineer
December 7, 2024

Data Engineer
December 5, 2024

Senior Pyspark Data Engineer
November 24, 2024

Data Engineer
November 24, 2024

Azure Data Engineer
November 24, 2024

Quick filters

Top companies

Data Engineer Luxoft Jobs

Hiring Senior Big Data Engineer 815 Yrs Bangalore Chennai Hybrid Model Immediate JoinersDecember 17, 2024

Data Engineer PythonDecember 14, 2024

Senior Pyspark Data EngineerDecember 11, 2024

Data EngineerDecember 7, 2024

Data EngineerDecember 5, 2024

Senior Pyspark Data EngineerNovember 24, 2024

Data EngineerNovember 24, 2024

Azure Data EngineerNovember 24, 2024