Wagdy El-Sherbiny,科威特哈瓦利省Salmiya的开发商
Wagdy is available for hire
Hire Wagdy

Wagdy El-Sherbiny

Verified Expert  in Engineering

Data and Database Developer

Location
Salmiya, Hawalli Governorate, Kuwait
Toptal Member Since
April 18, 2022

Wagdy is a senior data and database engineer with over eight years of experience with data-related and real-time mega projects. Highly proficient in SQL, databases, and data warehouses, including designing and implementing, 他管理过软件项目,并为重要客户工作过, such as ADNOC, Emarat, KNPC, and KOC. Wagdy is an expert in data pipelines, data analysis, and Python and focuses on understanding clients' needs and designing and developing state-of-art solutions.

Portfolio

Susquehanna Technology Management, Inc. (STMI) - Main
Python, SQL, PL/SQL, Linux, C#.NET, Oracle, C#, Microsoft SQL Server, Shell...
SIG - Main
SQL, Python, PL/SQL, Oracle, Linux, C#.NET, C#, Data Warehousing...
Diyar United Company
Azure SQL, C#, C#.. NET,数据库,数据工程,数据分析,敏捷...

Experience

Availability

Full-time

Preferred Environment

Data Engineering, Databases, Data Warehouse Design, Python, Data Analysis, NoSQL, SQL, PostgreSQL, MongoDB, ETL Tools

The most amazing...

...我开发的是ADNOC在阿联酋使用的自助支付和操作站, the world's 12th largest oil company by production.

Work Experience

Python and PL/SQL Developer

2023 - 2024
Susquehanna Technology Management, Inc. (STMI) - Main
  • 实现了一个Python程序来替换遗留系统, drastically reducing processing time for calculating trading volumes per symbol and exchange messages. 将处理时间从7到10小时减少到大约20到30分钟.
  • 设计了一个Python脚本从url中提取数据, parse JSON information, 并将其无缝存储在SQL server数据库中,以实现高效的数据管理.
  • 创建了一个shell脚本,用于从FIDO交易系统中检索数据. 实现过滤,提取重要信息,保存到CSV文件. 利用外部表方便地将数据加载到Oracle表中.
  • 将最初在Python 2中编写的所有遗留Python脚本迁移到最新的Python 3.7版本,确保当前开发标准的兼容性和优化.
  • 使用Jira管理bug跟踪和解决过程, ensuring seamless production support and maintenance.
  • 编写了一个健壮的数据库日志记录和调试实用程序,以简化故障排除. Also, designed a Python module for easy integration into scripts, bolstering debugging capabilities.
  • Revamped and reimplemented outdated systems prone to failure when processing large files with efficient and swift Python programs.
Technologies: Python, SQL, PL/SQL, Linux, C#.NET, Oracle, C#, Microsoft SQL Server, Shell, Bash, Workflow Automation, Data Pipelines, ETL, Data Engineering, Big Data, PySpark

Python and PL/SQL Developer

2022 - 2023
SIG - Main
  • Developed a regression testing program that validates the outcomes of one of the leading trading solutions at SIG. 该应用程序便于高层管理人员验证数据结果.
  • Built a real-time data pipeline that monitors three trading symbols' order trades and extracts data as of the ACT messages protocol. 该脚本提取和解析数据,然后将其加载到Oracle数据库中.
  • Participated in PySpark script development, converting legacy reports into new ones that optimized execution time and processed the vast amount of daily data, 500+ million transactions.
  • 开发了一个ETL,用于解析XML文件将数据加载到Oracle数据库表中.
  • 使用SSRS创建了许多报告,并使用qliksense创建了一个现代可视化仪表板.
  • Converted a legacy Perl program into Python. The scripts aim to process trading orders and calculate the five-minute sliding average for each symbol.
技术:SQL, Python, PL/SQL, Oracle, Linux, c#.NET, C#, Data Warehousing, Data Engineering, Database Modeling, Data Visualization, Data Reporting, Optimization, Data Pipelines, PySpark, Big Data, Qlik Sense

Senior Data Engineer

2014 - 2022
Diyar United Company
  • Collaborated with the project management team to analyze and consolidate request requirements to improve the decision-planning process and meet project deadlines.
  • 领导中型项目从范围定义到执行和实施.
  • 与跨职能团队合作解决问题, identify appropriate data sources, extract data, 并开发集成信息传递解决方案.
  • Designed, implemented, 维护各种数据库管理系统(dbms), such as Oracle, Microsoft SQL Server, Microsoft Azure SQL, and PostgreSQL.
  • Analyzed and compiled an itemized list of components needed for data migration or integration with various systems, such as Oracle Enterprise Resource Planning (ERP), SAP, and Comarch Loyalty Management (CLM).
  • 参与设计和实现一个数据仓库, data marts for analysis, data mining projects, and implementing changes to existing BI systems.
  • 使用SSIS工具从各种资源中收集数据, 遵循数据整理周期,存储和交付清理后的数据以供数据分析, 并开发了一个定制的管道,用于从外部系统获取和加载数据.
  • Built a fast and reliable invoicing system that efficiently processes millions of transactions and integrates with the Oracle ERP or SAP system. 发票生成时间从6小时减少到20分钟.
  • Identified the key reports and dashboards to satisfy the client's requirements and developed different kinds of reports like parameterized reports, ad hoc reports, 并使用SSRS和Power BI对报告进行深入研究.
  • Troubleshooted data issues, analyzed and corrected error and performance logs, created backup routines, and prepared system deployments and upgrades. 培训各个业务单元团队有效地使用流程、工具和资源.
Technologies: Azure SQL, C#, C#.. NET,数据库,数据工程,数据分析,敏捷, SQL Server Reporting Services (SSRS), SQL Server Integration Services (SSIS), Microsoft Power BI, Qlik Sense, Oracle, PL/SQL, T-SQL (Transact-SQL), Microsoft SQL Server, PostgreSQL, Python, Data Pipelines

Full-stack Developer

2012 - 2014
Management Development Center (MDC)
  • 参与了Rhazes解决方案的开发,包括所有的医疗, clinical, and administrative modules needed to manage and operate a hospital in a smooth and user-friendly way.
  • Developed SCHOOLIZER software used to efficiently manage all the systems and processes related to running any educational institute.
  • 排除故障并修复与称为MUST的电子学习系统相关的生产问题.
Technologies: Active Server Pages (ASP), ASP.NET, Windows Services, Web Services, Crystal Reports, SQL, Microsoft SQL Server, HTML, CSS, C#, Audacity

Teaching Assistant

2012 - 2014
Misr University for Science and Technology
  • Managed, planned, and prepared the learning materials and courses related to computer science for different classes and levels.
  • Used different educational methods to promote a positive learning environment and organized group work projects and brainstorming sessions.
  • 为不同的评估设定要求,并记录学生的发展情况.
  • 与计算机科学领域的变化和发展保持同步.
Technologies: C#, C++, Data Structures, Operating Systems, Education, Communication, University Teaching, Algorithms, Applied Mathematics, Statistics, Databases

ADNOC Smart Wallet

http://smart.adnocdistribution.ae/
ADNOC分销公司是阿联酋领先的零售加油站运营商. ADNOC钱包使ADNOC加油站支付快捷方便. 它是完全安全的,并跟踪ADNOC加油站的日常支出. 它包括加油、购物、更换润滑油和洗车. 此外,客户可以在一个账户中注册多辆车或受益人.

I was the software engineer who participated in and developed all objects related to the data area, including databases such as triggers, stored procedures, jobs, data warehouse, pipelines, and reporting. I created a set of systems, including the following:

• A card management system (CMS).
• A comprehensive centralized software solution providing our clients with a user-friendly interface that integrates tightly with other back-end and external systems, such as customer management, beneficiary management, token management, transaction management, system settings, BI reports and dashboards, differential pricing, notification management, ERP integration, loyalty system, e-vouchers, coupons, data migration, 以及个性化智能卡的卡片打印系统, vehicle depot system, and invoicing.

Emarat Petroleum Company Station Management Service

http://www.emarat.ae/businesses/fleet-payment-solutions
Emarat is a multi-channel power and energy company with a network of service stations and fuel depots across Dubai and the Northern Emirates. In this project, I worked on the following:

• Implementing and customizing a turnkey solution to meet the company's station management and retail services needs.
•参与系统需求收集和分析阶段.
• Analyzing and compiling components needed for data migration and integration with various systems, such as Oracle ERP, NCR, and NAMOS.
•开发数据迁移工作流,将数据迁移到使用SSIS的新系统.
•建立差异化定价体系和NCR折扣.
•使用SSRS和Power BI开发各种报告和仪表板.

Kuwait E-passport

I collaborated with a team to design and implement a set of reports and dashboards for executives and operation users of Kuwait’s Ministry of the Interior e-passport system. I performed requirements analysis, made system designs, oversaw implementation, and drafted documentation. Also, I created numerous processes using Microsoft BI tools to develop and implement a data warehouse and designed an SSIS package that extracts citizen data from an SQL Server database using lookup transformation and the Foreach Loop container. Finally, I built dashboards to demonstrate the overall performance of the passport printing center.

KNPC Petrol Filling Stations

科威特国家石油公司(KNPC)是科威特的国家炼油公司. As a software engineer, I participated in building a turnkey solution for retail service stations from scratch to meet KNPC's business needs and operations. I participated in all of the systems development lifecycle (SDLC) to build a centralized solution that manages customers' profiles, consumption, transactions, invoicing, loyalty points, e-vouchers, pre-printed vouchers, 与Oracle ERP和NCR系统的数据集成, in addition to building modules related to managing users permissions and system setting and data backup routines.

Kuwait Digital Oil Field Project

我在科威特参与了第一个数字油田(DOF)项目, using Petroleum Experts (PETEX) technology to enhance the oil well production and operations by providing frameworks and dashboards for decision-makers. 我是PETEX解决方案的一线支持人员, 协助石油工程师进行日常作业, overseeing troubleshooting data issues, 并帮助用户解释和理解报告数据. 我使用以下软件解决方案:集成可视化管理(IVM), Integrated Production Modeling (IPM), IFM, PROSPER, and Model Catalogue.

Kuwait National Guard's Fleet Management

这个web应用程序管理所有科威特国民警卫队的车队操作. As a software engineer, I oversaw gathering requirements, analyzing the client's needs, and providing detailed design documents to the team. I worked closely with the development team to design and implement the database's objects and the Windows Communication Foundation (WCF) service to provide the front-end developers with the required info for a web app. Also, I managed the production deployment activities and troubleshooting issues and conducted training and user acceptance testing (UAT) sessions for end-users.

The Ford GoBike System

http://github.com/wagdySamy/DataAnalysis
The Ford GoBike System dataset contains information about individual rides made in a bike-sharing system covering the greater San Francisco Bay area. There are approximately 183,数据集中有400条记录,这些记录具有描述乘车的属性, including start station, end station, and duration.

WeRateDogs Analysis

I illustrated the steps and procedures used for wrangling a real-world dataset known as WeRateDogs. WeRateDogs是一个幽默的推特账号,用评论给狗打分. I collected data from different resources, such as ZIP and CSV files, and queried the Twitter API to fetch data.
2008 - 2012

Bachelor's Degree in Computer Science

埃及吉萨科技大学

MARCH 2021 - PRESENT

Spark and Python for Big Data with PySpark

Udemy

NOVEMBER 2020 - PRESENT

HCIA-Big Data Course

Huawei Academy

JULY 2020 - PRESENT

AWS Machine Learning

Udacity

MAY 2020 - PRESENT

Data Analysis Professional Nanodegree Program

Udacity

MAY 2020 - PRESENT

Data Engineer with Python

DataCamp

FEBRUARY 2020 - PRESENT

MongoDB Basics

MongoDB University

JANUARY 2020 - PRESENT

Data Analysis Advanced Nanodegree Program

Udacity

AUGUST 2019 - PRESENT

Career Track Data Analyst with SQL Server

DataCamp

MARCH 2019 - PRESENT

Big Data Introduction Course

Coursera

JUNE 2017 - JUNE 2020

MCPS: Microsoft Certified Professional

Microsoft

SEPTEMBER 2014 - PRESENT

Computer Science Diploma

Mansoura University

Libraries/APIs

Pandas, Matplotlib, NumPy, Twitter API, PySpark

Tools

Microsoft Power BI, Qlik Sense, Flink, Flume, Apache ZooKeeper, SQL Server BI, Oracle ERP, Crystal Reports, Shell, Audacity

Storage

Databases, SQL Server Reporting Services (SSRS), PL/SQL, PL/SQL Developer, Oracle PL/SQL, SQL Server Integration Services (SSIS), PostgreSQL, Data Pipelines, NoSQL, MongoDB, Azure SQL, Microsoft SQL Server, HBase, Apache Hive, Database Administration (DBA), Database Modeling, SQL Server Analysis Services (SSAS)

Languages

SQL, Python, C#, C++, C#.. NET, T-SQL (Transact-SQL), Scala, Active Server Pages (ASP), HTML, CSS, Bash

Platforms

Oracle, Apache Kafka, Linux

Paradigms

ETL Implementation & 设计、敏捷、商业智能、MapReduce、ETL、用户验收测试(UAT)

Frameworks

Hadoop, Spark, ASP.NET

Other

Data Warehouse Design, PL/SQL Tuning, Data Engineering, Data Analysis, ETL Tools, ELT, Data Wrangling, Big Data, Azure Data Factory, Software Engineering, Data Structures, Operating Systems, Algorithms, Windows Communication Foundation (WCF), Artificial Intelligence (AI), Neural Networks, Streaming, Machine Learning, Data Visualization, PETEX, DAX, Data, IIS, Analysis, Teamwork, Team Management, Data & Backup Management, Data Modeling, Production, Deployment, Software Troubleshooting, Troubleshooting, Education, Communication, University Teaching, Applied Mathematics, Statistics, Windows Services, Web Services, Linear Regression, Multidimensional Expressions (MDX), Star Schema, Big Data Architecture, Data Warehousing, Data Reporting, Optimization, Workflow Automation

Collaboration That Works

How to Work with Toptal

Toptal matches you directly with global industry experts from our network in hours—not weeks or months.

1

Share your needs

在与Toptal领域专家的电话中讨论您的需求并细化您的范围.
2

Choose your talent

在24小时内获得专业匹配人才的简短列表,以进行审查,面试和选择.
3

Start your risk-free talent trial

与你选择的人才一起工作,试用最多两周. Pay only if you decide to hire them.

Top talent is in high demand.

Start hiring