Senior Data Engineer with over 12 years technical proficiency in designing and building data piplelines for big data analytics and data warehouses with full life cycle Implementation on cloud platforms and on premises. Involved in the direct interaction with the clients.
Skills:
Apache Stack: Nifi, Airflow ,Hadoop, Hive, Spark, Kafka,
Programing: SQL, Python, T-SQL, PL/SQL, C#, Core Java, VB.net, VBA, Shell Script,Powershell
Azure Stack: Data Factory, Stream Analytics, CosmosDB, PowerBI and Azure Blob
ETL: Nifi, AirFlow, SSIS
RDBMS: SQL Server, MySQL, Postgres, Oracle
NoSQL: Cassandra, MongoDB
DevOps: Git, , TFS, GitHub ,Docker, Kubernetes, Azure Devops, Terraform
Storage Server: Azure blob storage, MinIO and AWS S3
• Skilled in engineering cloud data pipelines to load and transform data from multiple sources in Azure/GCP/Apache/MinIO.
• Customized Apache Nifi for building own data pipelines.
• Created Airflow DAGs for scheduling and workflow orchestration in Python.
• Integrate ELK Stack to Airflow and ETL framework for real time log aggregation, analysis and querying
• Experienced in building, publishing and administering reports with exposure to dashboards such as Power BI and SSRS.
• Collaborate with Testers, PM’s, Librarians, Release Management, Batch Management, Environment Management, Change Management teams for End to End transition of the project.
• Work with Run the Bank (Prod-Support) teams to resolve production issues.
• Gook knowledge of cloud platforms including Azure, AWS.
• Skilled in writing complex T-SQL code, Query optimization, performance tuning, data migration.
• Sound knowledge of all Windows and SQL utilities like Powershell, ULS, Profiler
• Working Experience in India and Singapore.
• Skilled in the development of applications using various Project Management Practices including AGILE Methodologies and Waterfall Models.
• Experienced Information Technology Specialist with a demonstrated history of working in the industrial automation industry.
•Skilled in web technologies like asp.net , javascript, jquery, HTML and Angular 4