From data analysis to a decision: what to check before taking action

A data-driven decision uses analysed information to compare alternatives or justify a course of action, but does not in itself eliminate uncertainty. Having well-organised records, dashboards or numerical estimates does not guarantee that a course of action will solve the anticipated problem, nor that its costs will be outweighed by the expected benefits. Before turning a result into a recommendation, it is worth asking: what makes a data analysis sufficiently reliable and relevant to justify a decision?

Before analysing the data, we need to know what we want to decide

A common mistake when starting an analytical project is to ask questions that are too open-ended. Questions such as ‘how can we improve the service?’ lack a sufficiently defined objective to directly guide an analysis. By contrast, investigating which factors contribute to delays in certain repairs and which operational changes could reduce those delays allows us to identify observable outcomes.

This initial clarification requires defining the problem, the indicators that would enable a change to be assessed, the available courses of action, and the budgetary or logistical constraints affecting the project.

For example, a network of electronic device repair shops may notice that many of its jobs involve replacing batteries and consider purchasing more units to reduce waiting times. However, it still needs to check whether the availability of these parts is actually what is causing the delays.

Exploratory analytics is also useful when no definitive decision has yet been made. It enables the discovery of unexpected relationships and the formulation of initial hypotheses.

In both cases, the usefulness of an analysis depends on its ability to answer a relevant question, not on the volume of records accumulated.

The available records do not necessarily represent the whole picture

The data collected by an organisation does not always cover the full scope of the phenomenon under study: it reflects only what its systems have recorded.

If a network of garages examines only completed orders, it may overlook essential information: quotes rejected due to high costs, repairs cancelled because of long waiting times, or incidents that certain branches have not classified correctly.

This situation raises three important concepts:

• Representativeness: the extent to which the available data reflects the situations or the population from which we wish to draw conclusions.
• Data quality: whether the information is sufficiently accurate, complete, consistent and suitable for the intended analysis.
• Selection bias: the distortions that arise when certain cases are more likely to be included in the analysed data than others.

Having a large sample does not guarantee that its results can be generalised. Nor is it enough simply to remove duplicate records or fill in missing values: it may be necessary to check how variables such as supplier waiting times or the actual duration of a repair have been measured.

Understanding how software, data and artificial intelligence interact within a digital product allows us to gain a deeper insight into the technical origins of that information and the dependencies that determine its availability and reliability.

Finding a correlation does not prove that we have identified the cause

Verifying that two variables change in tandem can provide valuable information, but does not in itself prove that one is the cause of the other.

Suppose that workshops with the highest volume of battery replacements also have longer average lead times. A hasty interpretation might conclude that these repairs are causing the delays and that simply purchasing more batteries would be enough to resolve them.

However, the association allows for alternative explanations. The repairs may be concentrated on more complex devices, coincide with supply issues, or take place in garages that already have a heavy workload.

Correlation and causation are not the same thing.

Statistical correlations enable us to investigate relationships, detect anomalies and develop predictive models. The problem arises when they are used as definitive causal explanations without considering other factors that might be involved.

To obtain evidence of a possible cause, appropriate comparisons, statistical control methods or evaluations designed to study the effect of an intervention may be used. It will not always be possible to identify the cause with certainty, and this limitation must be acknowledged in the conclusions.

Responsible analysis distinguishes between what the data show, the hypotheses they suggest, and what cannot yet be asserted.

The chosen method must answer the question, not be driven by the appeal of the technology

The soundness of a project depends on choosing a methodology proportionate to the need at hand, not on using the most sophisticated algorithm.

Depending on the objective, different approaches may be appropriate:

• Describing what has happened: finding out how many repairs have been delayed, which garages have the highest number of incidents, or how times are distributed. Descriptive statistics and visualisation can provide sufficient answers.
• Investigating relationships: checking which variables are associated with delays and whether the observed patterns stand up to more detailed analysis.
• Anticipating scenarios: estimating future demand for components or identifying repairs most likely to be delayed using statistical techniques or machine learning models.

These tasks are not mutually exclusive and can be combined within a single project.

Artificial intelligence can be useful when it provides capabilities that justify its complexity. However, not every prediction requires AI, nor does every data problem require training a model.

Incorporating complex solutions when a simpler procedure meets the requirements can increase development costs, make it harder to interpret the results and complicate maintenance.

Professional judgement involves assessing which method best answers the question, given the data, resources and constraints actually available.

Before relying on a model, it must be compared with an alternative

The performance of a predictive model should not be judged in isolation.

An algorithm designed to estimate weekly demand for batteries may perform well during testing and yet offer no significant advantages over a simpler operational procedure.

Therefore, the evaluation must include a benchmark or baseline: a basic rule, a method based on historical data, or the solution the organisation previously used.

Comparing the model with this baseline allows us to determine whether its complexity translates into a sufficiently significant improvement.

It is also necessary to check how the evaluation was carried out. In supervised learning, performance must be assessed using appropriate evaluation data that is separate from the information used to train the model, thereby avoiding information leakage that could produce artificially favourable results.

A high metric does not guarantee a useful solution when it does not accurately represent the problem.

For example, the percentage of correct predictions may be relevant for certain classification tasks, but it is not a universal measure for evaluating any model.

Validating a model means checking what errors it makes, under what conditions they occur, and whether its results are an improvement on a relevant alternative for the project.

Even then, one question remains: what consequences will those errors have when the predictions are used to make decisions?

Two errors can have completely different consequences

Technical metrics typically summarise a model’s performance using aggregated figures. However, two prediction errors of similar magnitude can have very different operational consequences.

In our example of the workshops, an incorrect estimate of demand can lead to two situations:

• Overestimating requirements. The model forecasts demand higher than the actual figure, and the organisation purchases too many batteries. This can lead to tied-up capital, storage costs and the risk of the stock deteriorating or becoming obsolete.
• Underestimating requirements. The system estimates demand to be lower than it actually is, and the workshop runs out of sufficient parts. This can lead to repairs being held up, longer waiting times and dissatisfied customers.

Both outcomes represent deviations from actual demand, but their consequences are not equivalent.

The organisation will need to assess the financial and operational costs of each situation before deciding what margin of error is acceptable.

This reasoning also explains why there is no single suitable metric for evaluating all analytical projects.

In certain classification problems, the differences between false positives and false negatives are significant. In a quantitative inventory forecast, however, it is important to assess the consequences of estimating quantities that are either higher or lower than required.

The evaluation must be tailored to the problem and its actual risks.

A technically accurate solution may prove impractical if it gets it wrong precisely in those situations where the error has the most significant consequences.

A useful conclusion has yet to be turned into action

Obtaining a valid analytical result is not the same as having a recommendation ready for implementation.

In practice, it is useful to distinguish between three related concepts:

• Result: the analysis identifies a pattern, estimates a quantity or provides evidence of a relationship.
• Recommendation: a possible course of action is proposed, taking these results into account alongside constraints and alternatives.
• Decision: those responsible choose which course of action to take, weighing up the evidence, costs, objectives and risks.

These activities may be carried out iteratively and do not necessarily always have to be performed by different professionals.

A defensible recommendation must explain what is proposed to be changed, what information it is based on, what limitations it has, and what other possibilities have been considered.

This professional dimension also features in the UDIT interview with Ángel Galán, director of the Official Master’s Degree in Data Analysis, in which he highlights the importance of analysing data methodically and knowing how to communicate the results.

A technically rigorous result loses some of its usefulness if those who need to take action cannot understand its scope, the uncertainties it entails or the consequences of the available alternatives.

The decision does not end when it is put into practice

Implementing a measure does not necessarily bring the analytical project to a close. It may also mark the start of assessing its actual effects.

If the network of garages changes its ordering policy, it will need to monitor whether waiting times are reduced, how costs evolve and whether any unforeseen consequences arise.

Here it is important to distinguish between:

• Analytical performance: how the model or method performs against the criteria used during its evaluation.
• Operational outcome: what happens when the information is used to guide actual action.

If deliveries improve after implementing a measure, that result cannot automatically be attributed to the intervention. Seasonality, changes in demand or variations amongst suppliers may also play a part.

Depending on the project, it will be necessary to use appropriate evaluation methods to determine what proportion of the change can be linked to the intervention.

When AI models are used on an ongoing basis, there may also be requirements for monitoring, traceability, data protection and performance tracking. The <escape><</escape>NIST risk management framework<escape>></escape> does not replace the specific evaluation of each system, but provides a reference for identifying and managing risks throughout its use.

The assessment does not end once it has been verified that the model works: it is also important to consider what happens when its results are turned into decisions.

What a project must be able to explain before recommending a decision

Before justifying an organisational intervention based on data analysis, the team responsible should be able to explain five key points:

This summary does not represent a universal methodology or a rigid sequence of steps.

Its purpose is to show what a recommendation must be able to justify before it becomes a course of action.

What you learn when specialising in Data Science, Data Analysis and Artificial Intelligence

Working in these fields requires training in statistics, programming, analytical methods and the evaluation of technological solutions. At UDIT, these skills are developed through degree programmes offering varying levels of depth and academic objectives.

Foundation programme: Data Science and Artificial Intelligence

The Official Bachelor’s Degree in Data Science and Artificial Intelligence combines mathematical foundations, statistics, programming, databases and machine learning.

Its approach enables students to understand how information is obtained and analysed, how models are developed, and what technical knowledge is required to build data-driven solutions.

Specialisation in Data Analysis and Interpretation

The Official Master’s Degree in Data Analysis delves deeper into statistical modelling, forecasting techniques, processing large volumes of data, visualisation and data governance.

These skills are particularly relevant to the ability to evaluate results, communicate conclusions and provide analytical support for decision-making processes.

Specialisation in Intelligent Systems Development

The Official Master’s Degree in Artificial Intelligence covers the development and evaluation of advanced models, including machine learning, deep learning, computer vision and natural language processing.

Its focus allows for an in-depth exploration of the technical requirements and responsibilities involved in designing, implementing and evaluating artificial intelligence solutions.

The three programmes share analytical foundations, but are not equivalent. The bachelor’s degree provides a basic university education, whilst the master’s programmes offer advanced specialisations across different areas of application.

Good analysis does not eliminate the need for a decision: it enables it to be well-founded

Data analysis can provide verifiable evidence to act with greater discernment, but it does not replace the responsibility of making a decision. It helps to narrow down uncertainty, compare alternatives and anticipate consequences; the final choice also requires weighing up objectives, resources and priorities.

A recommendation is more defensible when it explains where the data comes from, why a particular method has been chosen, what errors may arise and how the results will be verified.

The value of analysis lies not merely in reaching a conclusion, but in understanding how far that conclusion takes us before we act.