Uday Bhaskar

Migrating to Cloud Infrastructure/Big Data tools - Python, Scala, Spark, Parquet, Flask, Sql; Developing an application for the product teams to analyze customer's click, rest services data which helps in figuring out anomalies by computing required statistics using Spark jobs; Design/Developing multiple pipelines to shift entire existing infrastructure to OCI resources like Object Store, ADW, DataFlow(Spark); Shifted the existing python aggregations to scala spark aggregations by storing logs as parquets and aggregating them using scala spark for customer usage analytics; Developing a generic metadata model using SparkSql to merge different pipelines which helps in config driven pull, parse & aggregation jobs; Created Rest Services, UI to fetch the data from ADW(Oracle SQL database) using Flask; Application Maintenance - OJet(Js), Flask, MongoDB, PyMongo, Logstash; Actively maintaining frontend and backend services of existing applications using OJet, Flask, MongoDB; Fixed critical bugs in Rest Services, parsing/aggregation jobs which reduced the downtime of the tool; Tracking multiple VM's devOps, health, pull/parse/aggregation jobs for smooth running of the application.

The abstract is to Develop a generic automated pipeline for evaluating 4 diagnostic products of human fluids. Made a yaml config driven code flow for users to provide query for extracting data, thresholds for bucketing. Extracted predictions, ground-truths from MongoDB using PyMongo with respective query, path provided by the user. Generated metrics,plots using pandas, scikit, imb after config driven elimination,transformation,bucketing of predictions.