Hi, I'm Harshdeep
a Data Engineer. a Software Engineer. a lifelong learner.

My experience spans developing robust software solutions and building data systems that drive meaningful insights.

Personal Details & My Background

About Me

I'm a computer science graduate student at the University at Buffalo with two years of experience in the software industry. Recognized for delivering high-quality results, I consistently received 'Outstanding' performance ratings. I'm now seeking full-time roles in both data engineering and software engineering starting in December 2024.

Name on my passport: Harshdeep Harikesh Mishra
Since 10th November 1999: Evolving One Day at a Time
Operating Out of: New York
Get in Touch: harshdeepmishra82@gmail.com
Dial in: +1-716-426-8682

VIEW MY RESUME

Over 2 years of experience

My Experience

Haver AI Inc

June 2024 - August 2024

Data Engineer Intern

Developed monthly data ingestion jobs using AWS Lambda (Python) and AWS Eventbridge to fetch claims data from BCDA API. Enhanced system reliability with real-time notifications via AWS SNS
Utilized AWS Glue and PySpark for preprocessing and integrating JSON data from various sources into AWS Redshift.
Designed and executed backend functionality for serving REST API requests using AWS API Gateway and Lambda.
Developed a scalable MLOps pipeline that automates the execution of a 6-step machine learning workflow in AWS SageMaker, triggered by user interactions on the frontend. Utilized AWS SageMaker AutoML to generate predictions as part of the pipeline.

Skuad

Dec 2022 -May 2023

Data Engineer

Implemented business KPIs for a telecommunications client leveraging PySpark on Google Cloud Platform (GCP) and Amazon Web Services (AWS) with Customer 360 data.
Collaborated with product manager to understand client requirements and develop highly efficient data processing pipelines with AWS Glue, S3, Athena, and GCP's Dataproc ephemeral clusters and BigQuery, resulting in a 40% increase in data processing speed and enabling timely insights for decision-making.
Provided comprehensive training to 5+ developers on a range of AWS services.

Quantiphi

Nov 2021 - Nov 2022

Data Engineer

Spearheaded creation of a cloud solution on GCP with Apache Airflow and Cloud Vision API for parallel processing of 300,000 PDF documents. Achieved a 65% optimization in processing time, reducing it from 28 to 10 days.
Implemented a comprehensive logging strategy with Cloud Logging leveraging structured log formats, routing logs to BigQuery via log sinks for seamless integration. Exported 100 logs per minute, enabling detailed analytics.
Led end-to-end migration of critical workflow orchestration projects from Hadoop ecosystem to GCP Composer (Airflow) and Dataproc, improving overall resource utilization by 40%.

Quantiphi

Nov 2021 - Nov 2022

Data Engineer Intern

Utilized a Google Cloud Virtual Machine (VM) to execute Python scripts for testing real-time and batch endpoints on Vertex AI, Google Cloud's AutoML platform.
Created a custom Cloud Dataflow template with Apache Beam for data transformations and employed Airflow to automate daily ETL jobs.
Gained extensive training and hands-on experience with AWS, GCP, big data, advanced SQL, Informatica, Tableau, and Snowflake, enhancing overall technical proficiency and versatility.

Academic Background and Qualifications

Education

August 2023-December 2024

University at Buffalo, SUNY

Masters in Computer Science and Engineering

Relevant Coursework : Algorithms, Data Intensive Computing, Operating System, Introduction to ML, Computer Security, Network concepts, Database Systems

July 2017 - June 2021