The document provides details about Dimple Dhok's work experience and projects. It outlines her experience as an Engineer in Delivery and Practices and describes 7 projects related to cloud, healthcare, databases, data science, data engineering, and data migration.
The document provides details about Dimple Dhok's work experience and projects. It outlines her experience as an Engineer in Delivery and Practices and describes 7 projects related to cloud, healthcare, databases, data science, data engineering, and data migration.
The document provides details about Dimple Dhok's work experience and projects. It outlines her experience as an Engineer in Delivery and Practices and describes 7 projects related to cloud, healthcare, databases, data science, data engineering, and data migration.
The document provides details about Dimple Dhok's work experience and projects. It outlines her experience as an Engineer in Delivery and Practices and describes 7 projects related to cloud, healthcare, databases, data science, data engineering, and data migration.
Engineer in Delivery and Practices June 2020 - Present
Konverge.ai | Nagpur, India ● 3.8 years of experience in Data Science & Engineer, and a wealth of knowledge in Python, API Development, and ETL Tools, as well as leveraging expertise to deliver data-driven solutions tailored to specific needs.
PROJECTS
1).Project 1->Cloud based
Developed an automated, scalable, and resilient data pipeline for seamlessly migrating on-premise data to the cloud. Transformed and stored the data in a cloud-based SQL database, ensuring real-time accessibility for further analysis. ● Set up an Azure pipeline for on-premise and cloud integration using Integration Runtime. ● Moved on-premise files to the cloud through Azure Data Factory, employing a copy activity with a linked service to store raw data in a storage container. ● Processed unstructured data with Azure Databricks Notebook and saved the output.Set up a logic app for pipeline failure notifications.
2).Project 2-(Health Care)
The project successfully consolidated cannabis-related government data from
various U.S. states into a single application, offering users comprehensive information on one platform.
● Checked websites for scraping, addressing security and authentication concerns.
● Effectively gathered data from multiple websites. ● Automated web data extraction by navigating internal pages. Extracted information from CSV sheets and saved it in MongoDB. 3).Project 3-(Database related)
The project was divided into two phases. In the first phase, public profile data from LinkedIn was scraped and then recorded in a database. The second phase included scraping public profile URLs from the Sales Navigator.
● Retrieved public profile data through web scraping.
● Automated the scraping process using Selenium. ● Incorporated the nohup process to concurrently execute two processes. ● Managed data in a MongoDB database and utilized Flask to create APIs.
4).Project 4-(Data science)
The project involved creating a Risk Quantification Engine that calculated the risk for all nodes at the FAIR level, utilizing a Bayesian network.
● Researched efficient libraries for Bayesian networks, devised methods for
calculations at all levels, and managed data creation and storage in MongoDB. ● Built APIs using Flask, integrated SonarQube with the project, and authored test cases to ensure comprehensive code coverage reflected on SonarQube.
5).Project 5-
Description:- (Data science)
Our project focused on reducing fuel and tire management costs in fleet operations. By leveraging data-driven insights and targeted measures, we aimed to optimize these expenses, enhancing overall cost efficiency and financial performance for the fleet.
● Collected relevant data from Palantir Foundry or other databases, then filtered to remove unnecessary or redundant information. ● Conducted data sampling and preprocessing by selecting a representative sample for prediction. Implemented necessary preprocessing steps to ready the data for analysis and predictive modeling. 6).Project 6-
Description:- (Data Engineering)
The objective was to establish a complete data pipeline for a product, simplifying the report creation process by monitoring a folder, detecting changes in files and data, and generating Directed Acyclic Graphs (DAGs) using Apache Airflow.
● Designed a data pipeline utilizing Airflow to reduce manual intervention in the
report creation process. Introduced a file watcher functionality to alert users of specified events in various scenarios. ● Explored Python libraries for monitoring file modifications. ● Initiated the file watcher functionality using the FastAPI framework. ● Recorded the modified event for the specified file, using the Dag ID associated with that file to trigger the Airflow Dag. ● Generated a new report based on the triggered Dag and added it in a table (Postgres).
7).Project 7-
Description:-(Data Migration)
The project's objective was to migrate from an on-premise infrastructure to a
cloud-based solution. The tasks undertaken and achievements include:
● In Databricks, I evaluated the functionality of shell code compared to Python.
Conducted end-to-end testing for all translated code using real data. Translated shell code to Python and Perl scripts were converted to Python. ● Converted Scala code to a PySpark-compatible format and tested SQL scripts for data extraction and loading during the project. ● Addressed production issues and applied necessary bug fixes. EDUCATION
Bachelor of Engineering (B. E.) - Computer Science Engineering JUNE, 2015 - JUNE, 2019
-Basic Python for beginners. Data structure and Algorithm for a Python developer. -Selenium Webdriver with PYTHON from Scratch + Frameworks(udemy course)
