Judgement Calls: Why Data Science Demands More Than Technical Skills

Ariel K
Oct 4, 2023
2 min read

This article is about the judgement calls data scientists must make during projects and why business understanding is critical.

Data science projects demand far more than coding proficiency and statistical chops - they require seasoned judgement and discretion. From problem framing to result interpretation, data scientists must make multifaceted judgement calls relying on business acumen as much as technical skills.

Framing the Problem

Data projects start with translating business needs into analytical problems. This involves discernment on defining key questions, metrics of success, and constraints. Judgement is needed to focus efforts on high-impact issues versus following unproductive tangents.

Choosing Data Sources

Myriad data touchpoints exist, but which are most relevant? Navigating political firewalls to get access requires judgement on pushing just enough. Creative thinking helps fill gaps by blending internal and external data sources for a 360-degree perspective.

Preparing Data

Deciding how much data cleaning and preparation is "good enough" is more art than science. You want to flag true anomalies without scrubbing characteristic quirks that enrich understanding. Judgement is applied to balance data quality with speed.

Selecting Features

Feature engineering is an iterative cycle of hypothesizing, testing, and deciding which attributes are most predictive. Judgement guides winnowing down endless potential features into a lean set providing maximal insight into key relationships.

Picking Models

With dozens of algorithms and approaches available, there are generally multiple valid options. Judgement weighs factors like accuracy, interpretability, and scalability to select models aligning with business needs - not just maximizing technical metrics.

Tuning and Testing

Endless tuning is possible but risks overfitting. Judgement navigates balancing underfit versus overfit to arrive at robust models. Similarly, you must determine when model testing against enough scenarios signals readiness versus danger of missing edge cases.

Interpreting Results

Statistical significance alone does not guarantee practical usefulness. Judgement contextualizes outputs into actionable decisions and next steps for the organization. Explaining complex insights simply but accurately relies on creative discretion.

In summary - judgement calls and business sense ultimately drive impact

Leveraging technical data science skills requires equal parts art and science. While technical expertise is mandatory, judgement and business sense ultimately drive impact. Data scientists must weigh many factors beyond pure data technique to deliver value. Trusting one's seasoned intuition remains an indispensable skill.