There is a particular kind of clarity that only arrives in retrospect. When I look back at the trajectory of my career — from a software engineer optimising terabytes of telecom data at Hewlett Packard, to a researcher unravelling the mathematics of tumour growth at Banaras Hindu University, with several unexpected detours in between — what strikes me most is not how different each chapter was, but how deeply each one informed the next. This is an attempt to trace that thread.
I write this not as a polished résumé narrative, but as an honest account of what each role actually felt like from the inside — what I learned, what surprised me, and what I would carry forward into any room I walked into next.
Chapter One: Learning to Think at Scale — Hewlett Packard Globalsoft
My first professional encounter with data was not glamorous. It was a Teradata Enterprise Data Warehouse space remediation project for a large telecom client — over 14 terabytes of data spread across more than fifty interfaces, accumulated over years of ETL processes that had quietly grown inefficient. My job was to help fix that.
At the time, I had no idea this was where I would learn one of the most lasting lessons of my career: the discipline of understanding data before touching it. We spent weeks analysing ETL usage patterns, identifying objects that had not been queried in months, tracing staging layers that existed only out of historical inertia. Before a single line of SQL was rewritten, we had mapped what was truly being used and what was simply taking up space.
"The data you do not need is just as important to understand as the data you do — because it is consuming resources, creating noise, and slowing down everything else."
The redesigned SQL processes and delta-based data loading patterns we implemented reduced storage requirements significantly. But the deeper takeaway was methodological: before you optimise anything, you must first understand it completely. That principle has followed me through every project since.
Chapter Two: The Precision of Recognition — ISI Kolkata (2013–14)
After completing my M.E. at Jadavpur University, I joined the Indian Statistical Institute, Kolkata, on a project aimed at developing a robust document image analysis and recognition system for printed Indian script. The challenge was deceptively specific: how do you efficiently identify and correct character recognition errors in a dataset of printed Bengali, Devanagari, and other Indian scripts?
The solution I worked on centred on the use of a splay tree data structure — a self-adjusting binary search tree that moves recently accessed elements closer to the root, making repeated lookups progressively faster. For the problem of word-level error detection, where the same characters and words appear with high frequency, this was an elegant fit. The splay tree's amortised efficiency meant that the more the system processed, the faster it became at catching errors it had seen before.
It was a short project — barely five months — but it sharpened something important: the instinct to match algorithmic structure to the natural structure of the problem. Document recognition is not just a pattern matching task; it is a frequency task, and frequency calls for data structures that reward recency.
Chapter Three: Predicting What Moves — IIT Kharagpur
Traffic on Indian highways is not random — but it is close. The IIT Kharagpur project on AI-driven traffic prediction and signalling placed me squarely in one of the hardest problems in applied machine learning: real-time anomaly detection in an environment defined by uncertainty, heterogeneity, and incomplete data.
My work involved analysing historical traffic data to identify statistical signatures of incidents — the subtle shifts in speed distributions, vehicle headways, and flow rates that precede an accident or breakdown, often minutes before it becomes visible to human observers. Statistical inference drawn from historical data was then used to build models that could flag anomalies in real time.
What made this project technically rich was the mismatch between the stationarity assumptions embedded in classical statistical models and the deeply non-stationary nature of Indian highway traffic — affected by weather, festivals, lane discipline, and the ever-present presence of mixed vehicle types from bicycles to heavy goods vehicles. Every model we built had to be robust to conditions it had never been explicitly trained on.
"A model that works only on clean, stationary data is not a model for the real world. The real world is messy, shifting, and indifferent to your assumptions."
This experience gave me a deep respect for the gap between a model that works in a notebook and one that works on a highway. Bridging that gap requires not just technical skill but a kind of epistemic humility — a willingness to question your own assumptions before the data does it for you.
Chapter Four: When Data Saves Lives — ISI Kolkata (2015–17)
If the IIT Kharagpur project taught me about real-time uncertainty, this one taught me about something more urgent: what happens when the cost of a wrong prediction is measured not in revenue, but in lives.
The project — Development of Risk Analytics towards Multidisciplinary Big Data Study of Humanitarian Logistics for Disaster Response — was concerned with one of the most pressing applied problems in a country like India: how do you allocate shelter and distribute relief supplies efficiently during a flood, when roads are submerged, communication networks are degraded, and the affected population is scattered and mobile?
Our work was grounded in the flood-prone zones of Kolkata and its surrounding districts. We built a cost optimisation model for shelter allocation and relief distribution, formulated as a geographic optimisation problem with strict capacity constraints, multiple demand sites, and a discrete feasible domain. Evolutionary algorithms were used to search the solution space efficiently — classical linear programming being too brittle for the irregular, constraint-heavy nature of real flood scenarios.
Alongside this, we developed a visual analytics dashboard for a flood decision support system — integrating geographical map visualisation to allow emergency coordinators to see risk zones, resource allocation, and population displacement at a glance. In a crisis, every second of decision time matters. The dashboard was designed to collapse complex multi-dimensional data into a form that a coordinator under pressure could act on immediately.
We also explored how social media mining could augment official data sources in post-disaster situations — a then-emerging idea that has since become standard practice. Geo-tagged social media posts, when processed for sentiment and location, could reveal where affected populations were concentrating faster than any ground survey.
"The best analytics tool in a disaster scenario is the one that puts the right number in front of the right person at the right moment — and then gets out of the way."
Chapter Five: Machine Learning at Industrial Scale — PricewaterhouseCoopers
After completing my PhD at BHU — where my world had been the mathematics of tumour growth, agent-based models, and fractals — returning to industry at PwC was a deliberately chosen recalibration. I wanted to understand what applied data science looked like at the scale of a global consulting firm, working across sectors and clients simultaneously.
My work centred on demand forecasting and supply chain management — applying machine learning to help clients across diverse industries anticipate what they would need, when they would need it, and how to position resources proactively. The problems were deceptively similar across industries but technically distinct in their particulars: a pharmaceutical distributor's demand signal looks nothing like a consumer electronics retailer's, even if both can be framed as time-series forecasting problems.
The full pipeline, in practice
What distinguished the PwC environment from research was the end-to-end nature of the work. Every project began not with data, but with business objective definition — long conversations with cross-functional teams to understand what the client actually needed to make better decisions. This translation layer, from business language to machine learning problem formulation, is where most real-world projects succeed or fail.
From there, the technical work was comprehensive. Exploratory data analysis to understand distributions, relationships, outliers, and anomalies. Multicollinearity assessment among predictor variables — evaluating correlation coefficients, applying VIF (Variance Inflation Factor) analysis to ensure the independence assumptions of regression models were not silently violated. Feature selection and scaling to prepare inputs that would generalise reliably beyond the training window.
Model training was followed by rigorous hyperparameter tuning — grid search, random search, and Bayesian optimisation depending on the problem's complexity and time budget. Equally important was performance evaluation: not just accuracy on a held-out test set, but assessment of reliability across different demand regimes, and scalability under production data volumes.
Scaling to production
One of the most technically demanding aspects was scaling models to handle large datasets efficiently. We used model parallelism and data parallelism to distribute the computational load across available resources — particularly important when training deep learning architectures on demand signals with long historical windows and many hierarchical levels (SKU → category → region → national).
"A model that takes three weeks to retrain is not a production model — it is a research prototype wearing a suit. Real deployment demands that the pipeline is as robust as the model itself."
What PwC gave me, beyond technical depth, was an understanding of the organisational context of data science. The best model is worthless if the team cannot explain it to the client, if the client cannot integrate it into their workflow, or if the business stakeholders were not part of the problem definition from the beginning. I came away from those two years with a much more complete picture of what applied machine learning actually requires in the field.
The Thread Running Through
Looking back across these roles, what I notice is that each one contributed something the others could not. HP gave me rigour and the discipline of understanding data at scale. ISI's document project sharpened my instinct for algorithmic elegance. IIT Kharagpur taught me humility in the face of real-world messiness. ISI's humanitarian work reminded me that data analysis is not abstract — it lands somewhere, affects someone. And PwC showed me what it takes to build systems that actually survive contact with the world.
In between all of this, a PhD in Mathematical Oncology at BHU — building mathematical models of tumour growth, cancer invasion, and angiogenesis — taught me something none of the industry roles could: how to hold a problem at its deepest level, to resist the temptation of approximation, and to follow the mathematics wherever it leads, even when the destination is uncomfortable.
I do not think of my career as a series of pivots. I think of it as a single, continuous act of attention — each chapter asking a different question about how data, mathematics, and computation can be made to serve human needs. That question has not changed. Only the context has.
— Sounak Sadhukhan, Kolkata, June 2025